CI/CD - CircleCI - Part Three

   
   
 git  ssg  deploy  
 devops  
Article Series

CI/CD for SSG Articles.

Where to Discuss?

Local Group

Table of Content

Preface

Goal: Deploy SSG in bitbucket using CircleCI.

For you that feels like gitlab.io and github.io is not enough, you may consider bitbucket.io.

With CircleCI, you have more build time, than the default bitbucket’s pipeline.

10: Problem Definition: The SSH Issue

Unlike github, I never get any luck, in using SSH fingerprints with buckbet.

CircleCI+Bitbucket: Fingerprint Issue

My solution is to use my own SSH as environment variable in CircleCI.

I have to setup in three places

  • Local PC: Create my own SSH

  • Bitbucket

  • CirleCI Environment Variables.

Generate RSA Key

In my local PC

$ ssh-keygen -o -t rsa -b 4096 -C "epsi.nurwijayadi@gmail.com"

Now I can see my public RSA key.

CircleCI+Bitbucket: id_rsa.pub

Setting Up SSH in Bitbucket.

There are two places to setup SSH in Bitbucket. Global setting, and each repository. The global setting need only public RSA key, while for repository require pair, both private and public. For this example, I use global setting.

Copy then paste, the public RSA key.

CircleCI+Bitbucket: Setup SSH in Bitbucket

You can use any name.

Testing SSH

You may use this command to test whether it works or not.

$ ssh-keyscan bitbucket.org

CircleCI+Bitbucket: ssh-keyscan bitbucket.org

CircleCI Environment Variable

Again, copy then paste, the public RSA key.

CircleCI+Bitbucket: CircleCI Environment Variable

Name the variable, for this example DEPLOY_KEY.

YAML Configuration

We will use this variable later

      - run: 
          name: Prepare Deploy Key
          command: |
           echo "${DEPLOY_KEY}" > ~/.ssh/id_rsa
           chmod 600 ~/.ssh/id_rsa
           ssh-keyscan bitbucket.org >> ~/.ssh/known_hosts

I know, it looks strange that it does not write to ~/.ssh/id_rsa.pub.

11: Prepare Your Repository

This time we are going to define new model for our repository.

  1. branch develop: source code
  2. branch master: generated site

Push Both

For each branch, you have to git push, one by one.

$ git push -u origin master

then later

$ git push -u origin develop

If you git push for master branch only. The develop branch would be left behind, unpushed.

Also note that, recent github recently using main instead of master.

Now it is a good time to practice.

Create Master Branch

$ git checkout -b master
Switched to a new branch 'master'

$ echo "Test Master Branch" > MASTER.md
$ git add MASTER.md
$ ls
MASTER.md

$ git commit -m "Master: First Commit"
 create mode 100644 MASTER.md

$ git branch | cat
* master

Create Develop Branch

$ git checkout -b develop
$ ls
README.md
$ rm MASTER.md

$ echo "Test Develop Branch" > DEVELOP.md
$ git add DEVELOP.md
$ ls
DEVELOP.md
$ git commit -m "Develop: Second Commit"
 create mode 100644 DEVELOP.md

$ git branch | cat
* develop
  master

Bitbucket: Prepare Both Branches in Bitbucket

Push Both

As a reminder, do not forget to push both.

$ git push -u origin master
$ git push -u origin develop

Bitbucket: Push Both Branches in Bitbucket

Branches in Repository

In your bitbucket repository, your branches would looks similar to this:

Bitbucket: Branches in Bitbucket Repository

You can check with your terminal

$ git branch -a -l | cat
* develop
  master
  remotes/origin/develop
  remotes/origin/master

Bitbucket: Branches in Terminal

Notice that in local repository, and remote, might have different content.

SSG

The next step is to set-up SSG. You can use previous article to setup SSG.

12: Configuration

I provide two CircleCI configuration. Both Works with Bitbucket.

The header is the same for both:

version: 2

workflows:
  version: 2
  build:
    jobs:
      - buildsite:
         filters:
            branches:
              only: develop

But the jobs have different commands.

Using Refspec

jobs:
  buildsite:
    docker:
      - image: circleci/python:3.7.1
    working_directory: ~/source
    steps:
      - checkout
      - run: 
          name: Prepare Deploy Key
          command: |
           echo "${DEPLOY_KEY}" > ~/.ssh/id_rsa
           chmod 600 ~/.ssh/id_rsa
           ssh-keyscan bitbucket.org >> ~/.ssh/known_hosts           
      - run:
          name: Prepare Git Initialization
          command: |
            git config user.email "someone@somewhere"
            git config user.name "someone"            
      - run: 
          name: Install and Configure Dependencies
          command: |
            python3 -m venv venv
            . venv/bin/activate
            pip install --user --upgrade pip
            pip install -r requirements.txt            
      - run: 
          name: Generate Pelican Static Files
          command: |
            . venv/bin/activate
            make html            
      - deploy:
          name: Precheck Output 
          command: |
            mkdir ~/public
            cd ~/public
            mv ~/source/output/* .
            ls -lah            
      - deploy:
          name: Deploy Release to GitHub
          command: |
            cd ~/public
            git init
            git config user.email "someone@somewhere"
            git config user.name "someone"
            git remote add origin git@bitbucket.org:epsi/epsi.bitbucket.io.git
            git add --all
            git status
            git commit --allow-empty -m "[skip ci] Update at $(date)"
            git push origin master:refs/heads/master --force
            echo "[Deployed Successfully]"

Using Worktree

jobs:
  buildsite:
    docker:
      - image: circleci/python:3.7.1
    working_directory: ~/source
    environment:
      BUILD_DIR: ~/public
    steps:
      - checkout
      - run: 
          name: Prepare Deploy Key
          command: |
           echo "${DEPLOY_KEY}" > ~/.ssh/id_rsa
           chmod 600 ~/.ssh/id_rsa
           ssh-keyscan bitbucket.org >> ~/.ssh/known_hosts           
      - run:
          name: Prepare Git Initialization
          command: |
            git config user.email "someone@somewhere"
            git config user.name "someone"            
      - run: 
          name: Install and Configure Dependencies
          command: |
            python3 -m venv venv
            . venv/bin/activate
            pip install --user --upgrade pip
            pip install -r requirements.txt            
      - run: 
          name: Generate Pelican Static Files
          command: |
            . venv/bin/activate
            make html            
      - deploy:
          name: Precheck Output 
          command: |
            git worktree add -B master $BUILD_DIR origin/master
            git worktree list
            cd $BUILD_DIR
            ls -lah
            find . -maxdepth 1 ! -name '.git' -exec rm -rf {} \;
            mv ~/source/output/* .
            ls -lah            
      - deploy:
          name: Deploy Release to GitHub
          command: |
            cd $BUILD_DIR
            git add --all
            git status
            git commit --allow-empty -m "[skip ci] $(git log develop -1 --pretty=%B)"
            git push --set-upstream origin master
            echo "[Deployed Successfully]"

Also note that, recent github recently using main instead of master.

Live Site Preview

The bitbucket will render the site based on the master branch.

Bitbucket: Live Site Preview

Also notice, that since the source code relies on the develop branch, we always push to develop branch instead of master.

$ git push -u origin develop

Real Live Example

In repo perspective: Backend ➕ DevOps 🔜 Frontend.

  1. Source Backend: SSG (Pelican): 🕷 https://bitbucket.org/epsi/epsi.bitbucket.io/src/develop/

  2. Source Frontend: Site (HTML+CSS+Assets): 🕷 https://bitbucket.org/epsi/epsi.bitbucket.io/src/master/

  3. Source Devops: CircleCI (Workflows+Jobs): 🕷 https://bitbucket.org/epsi/epsi.bitbucket.io/src/develop/.circleci/config.yml

13: Prevent Push: To

You should always push, only to develop branch.

❯ git push -u origin develop

Using develop branch can be error prone, if your muscle memory tend to push to master branch.

If you mistakenly push to remote master branch while in local develop branch. the push will be rejected, because your HEAD is behind.

But you are going to be in trouble, if you are already in master branch, checking out on something, and accidentaly push to remote master branch.

You can prevent pushing to master by using .git/hooks/pre-push.

#!/bin/sh

current=$(git rev-parse --abbrev-ref HEAD)

if [ "$current" = "master" ]; then
  echo "Thou should not push to master!"
  exit 1
fi

exit 0

Now evertyime you mistakenly push from master branch:

❯ git push -u origin master
Thou should not push to master!
error: failed to push some refs to 'git@bitbucket.org:epsi/epsi.bitbucket.io.git'

Your master branch is safe now.

Conclusion: Branch Name

Hypotethically, this method can be utilized in any CI/CD, not just CircleCI, because it is just pure git command line. This means that we can use any repository other than github without gh-pages branch. Or still using github, but with different branch strategy, for example:

  • branch develop: source code
  • branch master: generated site

After all, the limit is your creativity.

What is Next ?

We are done with Bitbucket. How about going back to Github? Consider continue reading [ CI/CD - Github Workflows ]. This time we are going to explore Github Actions.