CI/CD - CircleCI - Part One

   
   
 git  ssg  deploy  
 devops  
Article Series

CI/CD for SSG Articles.

Where to Discuss?

Local Group

Table of Content

Preface

Goal: Deploy SSG in github using CircleCI.

As usual, explanation using figures should be sufficient as a starter. However, explanation using .circleci/config.yml is must, as CircleCI config is usually longer than Travis counterpart. This a good resource for YAML engineer.

When to use Worktree ?

For CircleCI+Github tier using NodeJS based SSG. Most work will simply be done using gh-pages. It is an NPM based app to manage github pages.

For the rest other than this tier, we are going to utilize git worktree to reduce build time. We can see how useful git worktree in this CircleCI situation.

Official Site

1: Prepare Your Repository

Just like Travis, CircleCI is a third party CI/CD. CircleCI can read from github/bitbucket repository, then build your favorite SSG, and write back to repository, or do something else, such as Amazon S3 thing.

Unlike Travis counterpart, who has tight couple with github pages, CircleCI is more flexible, but yet more complex.

Sign In

Your first attempt should be, creating account

CircleCI: Sign-in

Supported Repository

CircleCI support these two cvs repositories:

I have successfully build in github. But I never try CircleCI for bitbucket. I will explore CircleCI+Bitbucket tier later.

Gitlab have their own internal CD/CD. So gitlab user do nt need either Travis nor CircleCI.

What happened when you sign in to an account with empty jobs? Consider have a look at my bitbucket account.

CircleCI: Sign-in with bitbucket

For the rest of this guidance, I’m going to use my github account. This would looks different, because I already have some Travis jobs over there.

CircleCI: Sign-in with github

Practice

Now you can make repository, with any name, for example:

  1. Jekyll:

  2. Eleventy:

  3. Hexo:

  4. Pelican:

  5. Hugo:

Github: Project Page Repository

This is just an example, you should use your username account. and you should use your own repository

2: Managing SSH Keys

As said before, CircleCI can write to repository. This privilege require security access, that can be granted with SSH keys.

How do I set-up SSH keys 🤔?

Checkout Menu

Unlike Travis, CircleCI use different keys for each project. SSH keys can be automatically generated by CircleCI. By default, it is read-only. To enable read-write we have to configure.

CircleCI: Checkout SSH Keys

Generate User Key

For most my SSG project, I remove the read only access, and generate new user key with re write privilege.

CircleCI: Generated User Key

Github User Keys

If you do not need the key anymore, you can delete each keys manually from github.

CircleCI: List of User Keys in Github

You can spot the orphaned keys by the color.

3: Prepare Your Configuration

Allright. I have a ready to use SSG, let’s say eleventy, that I have put in circleci-11ty repository on github account. What to do next 🤔?

Workflows

As a starter, examine this YAML configuration below:

version: 2

workflows:
  version: 2
  build:
    jobs:
      - buildsite:
         filters:
            branches:
              only: master

For the sake of simplicity, I only make one job called buildsite.

Each SSG have a different configuration. But the basic is pretty similar.

Save this in directory called .circleci and name it as config.yml. Add file, commit and push to travis-11ty repository.

There is a detail for this as we will explain it later. For a while consider skip the detail, and fast forward to the result as below:

CircleCI: Eleventy Build Time

You can spot the buildsite job in figure above.

Jobs

The build process itself has skeleton as below:

jobs:
  buildsite:
    docker:
      - image: node:8.10.0
    steps:
      - checkout
      - add_ssh_keys:
          ...
      - run:
          name: Prepare Git Initialization
          ...
      - run:
          name: Install and Configure Dependencies
          ...
      - run: 
          name: Install Eleventy Site Dependencies
          ...
      - run: 
          name: Generate Eleventy Static Files
          ...
      - deploy:
          ...

The keyword run and deploy are interchangeable_, it means you can replace one with another.

Again, skip the detail and preview the skeleton of the process build

CircleCI: Eleventy Build Time

You can see each steps in figure above.

Docker Image

The most precedence part that should be done, before doing anything is choosing proper docker image.

jobs:
  buildsite:
    docker:
      - image: node:8.10.0

Each SSG has their own images.

CircleCI: Docker Image

Docker image process comes under section: Spin Up Environment. Choosing proper docker image is not just saved you lines of code, but also affect build minutes. In some case, it takes more than image of docker build. More docker image means more build time. Some docker images is so old that might not having feature you require.

SSH Keys

The next step is to write down your generated user image in configuration.

jobs:
  buildsite:
    ...
    steps:
      - checkout
      - add_ssh_keys:
          fingerprints:
            - "44:96:a5:95:1e:86:90:cf:e6:ef:67:e7:33:23:3b:0f"

Do you have any security issue? Or do you want to put the fingerprints in environment variable? You may desire to read the explanation here.

Ypu can put this fingerprints anywhere, no need in top most step. Just make sure that fingerprints is placed before writing to gh-pages.

Run

The rest is so self explanatory. They are all just git command.

jobs:
  buildsite:
    ...
    steps:
      ...
      - run:
          name: Disable jekyll builds
          command: touch ~/.nojekyll
      - run:
          name: Prepare Git Initialization
          command: |
            git config user.email "someone@somewhere"
            git config user.name "someone"            
      - run:
          name: Install and Configure Dependencies
          command: |
            npm install -g gh-pages@2.0.1
            npm install -g @11ty/eleventy            
      - run: 
          name: Install Eleventy Site Dependencies
          command: npm install
      - run: 
          name: Generate Eleventy Static Files
          command: eleventy --pathprefix="/circleci-11ty/"

First thing first. We need to setup git.

git config user.email "someone@somewhere"
git config user.name "someone"

You can add any bash command, such as adding ~/.nojekyll, so that github treat it as plain static html.

Deploy

If everything is fine we can continue to deploy section.

jobs:
  buildsite:
    ...
    steps:
      ...
      - run:
          ...
      - deploy:
          name: Deploy Static Files to gh-pages Branch
          command: gh-pages --dotfiles --message "[skip ci] Updates" --dist dist

This gh-pages is an NodeJS based module.

We have previously install gh-pages with this command.

      ...
      - run:
          name: Install and Configure Dependencies
          command: |
            npm install -g gh-pages@2.0.1
            ...

This gh-pages do the magic, reading from your build directory in master branch, such as _site or public or dist, and writing to gh-pages branch.

This method works good with NodeJS based application in github. But be aware that we require different approach for other case.

Avoid Unnecessary gh-pages Build.

By default any changes in repository will trigger CircleCI. Just add [skip-ci] in commit message to avoid build in gh-pages branch.

CircleCI: Docker Image

4: Summary: NodeJS based SSG

As a summary here is the complete config for NodeJS based SSG.

1: Eleventy

Consider rewrite all above in one configuration.

version: 2

workflows:
  version: 2
  build:
    jobs:
      - buildsite:
         filters:
            branches:
              only: master

jobs:
  buildsite:
    docker:
      - image: node:8.10.0
    steps:
      - checkout
      - add_ssh_keys:
          fingerprints:
            - "44:96:a5:95:1e:86:90:cf:e6:ef:67:e7:33:23:3b:0f"
      - run:
          name: Disable jekyll builds
          command: touch ~/.nojekyll
      - run:
          name: Prepare Git Initialization
          command: |
            git config user.email "someone@somewhere"
            git config user.name "someone"            
      - run:
          name: Install and Configure Dependencies
          command: |
            npm install -g gh-pages@2.0.1
            npm install -g @11ty/eleventy            
      - run: 
          name: Install Eleventy Site Dependencies
          command: npm install
      - run: 
          name: Generate Eleventy Static Files
          command: eleventy --pathprefix="/circleci-11ty/"
      - deploy:
          name: Deploy Static Files to gh-pages Branch
          command: gh-pages --dotfiles --message "[skip ci] Updates" --dist dist

2: Hexo

Configuration for Hexo is pretty similar to Eleventy.

version: 2

workflows:
  version: 2
  build:
    jobs:
      - build:
         filters:
            branches:
              only: master

jobs:
  build:
    docker:
      - image: node:8.10.0
    steps:
      - checkout
      - run:
          name: Disable jekyll builds
          command: touch ~/.nojekyll
      - run:
          name: Prepare Git Initialization
          command: |
            git config user.email "someone@somewhere"
            git config user.name "someone"            
      - run:
          name: Install and Configure Dependencies
          command: |
            npm install -g gh-pages@2.0.1
            npm install -g hexo-cli            
      - run: 
          name: Install Hexo Site Dependencies
          command: npm install
      - run: 
          name: Generate Hexo Static Files
          command: hexo generate 
      - add_ssh_keys:
          fingerprints:
            - "34:65:f3:a6:58:55:cc:fa:29:96:44:73:e2:6a:f9:64"
      - run:
          name: Deploy Static Files to gh-pages Branch
          command: gh-pages --dotfiles --message "[skip ci] Updates" --dist public

What is Next ?

We are done with basic stuff.

Consider continue reading [ CI/CD - CircleCI - Part Two ]. A more complex example, applying git worktree in CircleCI.