Preface
Goal: Deploy SSG in github using CircleCI.
We are going to utilize git worktree
to reduce build time.
We can see how useful git worktree
in this CircleCI situation.
5: Problem Definition
Once, I was looking for a way to do CircleCI.
And this article below got me a good attention.
Although, I can’t get to work, the git worktree
command got me curious.
Then I found another article, that is too complex.
I do not use Amazon S3
either.
And I finally find good snippet by megrxu. This guy is smart. And I finally build my SSG using CircleCI. I should be thankful.
The only issue is build time.
After examining the code.
I found that this configuration utilize two docker images.
The first docker image for building the site,
and the second docker image only to run gh-pages
.
version: 2
workflows:
version: 2
build:
jobs:
- build
- deploy:
...
jobs:
build:
docker:
- image: cibuilds/hugo:latest
...
deploy:
docker:
- image: node:8.10.0
...
- run:
name: Deploy docs to gh-pages branch
command: gh-pages --dotfiles --message "[skip ci] Updates" --dist public
I was triggered to replace this nodejs gh-pages
application with pure git
command.
And I found this also good article below. The only issue is it has external script.
After a long thinking, I goes back to the first artikel from Will Schenk.
Using git worktree
.Lucky me, I have learnt worktree
a month before.
That you can read my long article here:
Now the next challenge. How do I apply worktree in CircleCI. And of course, get it to work, until my site, live.
6: Branch Issue
Before we doing this cool git worktree
feature,
we have to face a fundamental issue.
gh-pages Existence
Before we rewrite this origin/gh-pages
in every build,
we need to create origin/gh-pages
for the first time.
And also not making it over and over again.
This means we have to push something using gh-pages
branch in our first build.
Do you get, the issue we face here 🤔? Or should I just keep on writing this blog?
Bash Approach
After a few trial and error, I finally manage to check if the branch exist.
$ git branch -a -l
* master
remotes/origin/HEAD -> origin/master
remotes/origin/gh-pages
remotes/origin/master
The complete oneliner command is as below:
$ git branch -a -l | grep gh-pages | wc -l
You can try yourself in your comfortable PC, or notebook.
This command below will result 1
.
$ git branch -a -l | grep master | wc -l
Now here the complete bash script. that we are going to use in CI/CD.
if [ $(git branch -a -l | grep gh-pages | wc -l) -eq "0" ]; then
echo "[Create gh-pages for the first time]"
git checkout -b gh-pages
git commit --allow-empty -m "Create gh-pages for the first time"
git push --set-upstream origin gh-pages
git checkout master
fi
7: Run: Manage Bash Command in YAML
The next step is to puth the order of bash script,
into ./circleci/config.yml
.
We are going to use Pelican SSG as an example. And we will have both Jekyll example and Hugo example, as complementary config.
The Skeleton
Let me remind you about the skeleton. So that we can undertand the bigger structure.
- run:
name: Prepare Git Initialization
...
- run:
name: Install and Configure Dependencies
...
- run:
name: Generate Pelican Static Files
...
- deploy:
name: Precheck Output
...
- deploy:
name: Deploy Release to GitHub
...
We are going to put the script in each section. The same script, but in YAML format.
Prepare Git Initialization
Consider rewrite, the line commands above, in a YAML fashioned.
- run:
name: Prepare Git Initialization
command: |
git config user.email "someone@somewhere"
git config user.name "someone"
git branch -a -l | cat
if [ $(git branch -a -l | grep gh-pages | wc -l) -eq "0" ]; then
echo "[Create gh-pages for the first time]"
git checkout -b gh-pages
git commit --allow-empty -m "Create gh-pages for the first time"
git push --set-upstream origin gh-pages
git checkout master
fi
This script part is the same for Jekyll, Hugo or Pelican.
Install and Configure Dependencies
Here, wince Pelican is Python based. We are going to install Pelican using PIP.
- run:
name: Install and Configure Dependencies
command: |
python3 -m venv venv
. venv/bin/activate
pip install --user --upgrade pip
pip install -r requirements.txt
Which the requirement.txt
in my case is:
Jinja2 ~= 2.10.1
pelican ~= 4.2.0
Markdown ~= 3.1.1
This part does not have anything to do with git
.
And this part is different for each SSG (Jekyll, hugo, Pelican)
Generate Pelican Static Files
This section is about SSG.
And it is also doesn’t have anything to do with git
.
- run:
name: Generate Pelican Static Files
command: |
. venv/bin/activate
make html
The next steps would be similar for each SSG (Jekyll, Hugo, Pelican)./
Only pure git
.
8: Deploy: Using Worktree
Now we still have these section left.
- deploy:
name: Precheck Output
...
- deploy:
name: Deploy Release to GitHub
...
Be aware that this is the most complex part in this article. You might need to read twice.
Precheck Output
Actually, there are two purpose here.
-
🕷 Working with worktree itself.
-
🕷 Debugging process. Since this is a complex parts. I need to be sure, that it works as I want. Especially for first time build.
- deploy:
name: Precheck Output
command: |
git worktree add -B gh-pages $BUILD_DIR origin/gh-pages
git worktree list
cd $BUILD_DIR
ls -lah
find . -maxdepth 1 ! -name '.git' -exec rm -rf {} \;
mv ~/source/output/* .
touch .nojekyll
ls -lah
First of all, we use worktree
,
to a directory that we have prepared before.
For example:
-
Source Directory:
~/Source
(master branch) -
Build Directory:
~/Public
(gh-pages branch)
The next step is to purge all the build directory in ~/Public
,
except .git
if exist.
So that we have a clean directory to start over.
Then we need to copy build files,
depend on SSG the name could vary, for example from ~/Source/output
, or ~/Source/_site
,
or ~/Source/dist
, or ~/Source/Public
.
We need to copy build files to $BUILD_DIR
,
that we define in config.yml
.
That is all.
Environment Variable
We can either set here
jobs:
buildsite:
docker:
- image: circleci/python:3.7.1
working_directory: ~/source
environment:
BUILD_DIR: ~/public
Or in dashboard, for example a Hugo
environment variable below:
Deploy Release to GitHub
Finally the last section.
- deploy:
name: Deploy Release to GitHub
command: |
cd $BUILD_DIR
git add --all
git status
git commit --allow-empty -m "[skip ci] $(git log master -1 --pretty=%B)"
git push --set-upstream origin gh-pages
echo "[Deployed Successfully]"
Again we switch directory to $BUILD_DIR
.
Then as usual: commit
and push
.
But be aware of these two additional git commit
tips.
-
Use
--allow-empty
. If yo don’t, thecommit
command will returnexit code
as1
, that leads tobuild failed
. -
Add
[skip-ci]
incommit message
. So thatpush event
from github does not triggerCircleCI
to processbuild
. That leads to unecessary multiplebuild
process.
Preview
If everything is fine,
the Pelican site will be served in github pages
soon.
Summary: Pelican, Jekyll, Hugo
As a summary, here is the complete configuration for each SSG.
Notice that we already show,
both Eleventy
and Hexo
configuration in previous article.
3: Pelican
version: 2
workflows:
version: 2
build:
jobs:
- buildsite:
filters:
branches:
only: master
jobs:
buildsite:
docker:
- image: circleci/python:3.7.1
working_directory: ~/source
environment:
BUILD_DIR: ~/public
steps:
- checkout
- add_ssh_keys:
fingerprints:
- "b8:0d:46:75:5d:be:c4:2b:bd:fc:74:8d:d7:0c:4b:c1"
- run:
name: Prepare Git Initialization
command: |
git config user.email "someone@somewhere"
git config user.name "someone"
git branch -a -l | cat
if [ $(git branch -a -l | grep gh-pages | wc -l) -eq "0" ]; then
echo "[Create gh-pages for the first time]"
git checkout -b gh-pages
git commit --allow-empty -m "Create gh-pages for the first time"
git push --set-upstream origin gh-pages
git checkout master
fi
- run:
name: Install and Configure Dependencies
command: |
python3 -m venv venv
. venv/bin/activate
pip install --user --upgrade pip
pip install -r requirements.txt
- run:
name: Generate Pelican Static Files
command: |
. venv/bin/activate
make html
- deploy:
name: Precheck Output
command: |
git worktree add -B gh-pages $BUILD_DIR origin/gh-pages
git worktree list
cd $BUILD_DIR
ls -lah
find . -maxdepth 1 ! -name '.git' -exec rm -rf {} \;
mv ~/source/output/* .
touch .nojekyll
ls -lah
- deploy:
name: Deploy Release to GitHub
command: |
cd $BUILD_DIR
git add --all
git status
git commit --allow-empty -m "[skip ci] $(git log master -1 --pretty=%B)"
git push --set-upstream origin gh-pages
echo "[Deployed Successfully]"
The Pelican build time looks good.
4: Jekyll
version: 2
workflows:
version: 2
build:
jobs:
- buildsite:
filters:
branches:
only: master
jobs:
buildsite:
docker:
# Choose Image that Support Git Worktree Command
- image: circleci/ruby:2.4
working_directory: ~/source
environment:
BUILD_DIR: ~/public
steps:
- checkout
- add_ssh_keys:
fingerprints:
- "dc:18:73:7a:1a:a7:5f:92:31:67:cb:20:eb:0f:77:31"
- run:
name: Prepare Git Initialization
command: |
git config user.email "someone@somewhere"
git config user.name "someone"
git branch -a -l | cat
if [ $(git branch -a -l | grep gh-pages | wc -l) -eq "0" ]; then
echo "[Create gh-pages for the first time]"
git checkout -b gh-pages
git commit --allow-empty -m "Create gh-pages for the first time"
git push --set-upstream origin gh-pages
git checkout master
fi
- run: bundle install
- run: bundle exec jekyll build
- deploy:
name: Precheck Output
command: |
git worktree add -B gh-pages $BUILD_DIR origin/gh-pages
git worktree list
cd $BUILD_DIR
ls -lah
find . -maxdepth 1 ! -name '.git' -exec rm -rf {} \;
mv ~/source/_site/* .
touch .nojekyll
ls -lah
- deploy:
name: Deploy Release to GitHub
command: |
cd $BUILD_DIR
git add --all
git status
git commit --allow-empty -m "[skip ci] $(git log master -1 --pretty=%B)"
git push --set-upstream origin gh-pages
echo "[Deployed Successfully]"
The Jekyll Section shown here.
5: Hugo
version: 2
workflows:
version: 2
build:
jobs:
- buildsite:
filters:
branches:
only: master
jobs:
buildsite:
docker:
- image: cibuilds/hugo:latest
working_directory: ~/source
environment:
BUILD_DIR: ~/public
steps:
- checkout
- add_ssh_keys:
fingerprints:
- "af:ff:21:08:af:e3:94:5d:ae:be:9a:d2:00:e4:9d:5e"
- run:
name: Prepare Git Initialization
command: |
git config user.email "someone@somewhere"
git config user.name "someone"
git branch -a -l | cat
if [ $(git branch -a -l | grep gh-pages | wc -l) -eq "0" ]; then
echo "[Create gh-pages for the first time]"
git checkout -b gh-pages
git commit --allow-empty -m "Create gh-pages for the first time"
git push --set-upstream origin gh-pages
git checkout master
fi
- run: HUGO_ENV=production hugo
- deploy:
name: Precheck Output
command: |
git worktree add -B gh-pages $BUILD_DIR origin/gh-pages
git worktree list
cd $BUILD_DIR
ls -lah
find . -maxdepth 1 ! -name '.git' -exec rm -rf {} \;
mv ~/source/public/* .
touch .nojekyll
ls -lah
- deploy:
name: Deploy Release to GitHub
command: |
cd $BUILD_DIR
git add --all
git status
git commit --allow-empty -m "[ci skip] $(git log master -1 --pretty=%B)"
git push --set-upstream origin gh-pages
echo "[Deployed Successfully]"
Hugo Comparation
Here is the comparation.
-
Hugo Build Time with Node Image
-
Hugo Build Time with only One Image
What is Next ?
We are done with Github. How about Bitbucket ? Consider continue reading [ CI/CD - CircleCI - Part Three ].