Twelve-Factors in practice - Part I - Codebase

One codebase tracked in revision control, many deploys

Triangle

Twelve-Factors in practice - Part I - Codebase

One codebase tracked in revision control, many deploys

The first of a 12 part series on how to use Twelve-Factor App in practice. This entry is written in collaboration with my good friend and (twice/double) former coworker Mycha de Vrees.

We will be implementing https://12factor.net/codebase based on https://nvie.com/posts/a-successful-git-branching-model/

There is a TL;DR at the bottom but please take your time to read this document!

Gitconfig

First things first, setup a proper config, the one below states the following;

  • user; who is working
  • filters; main for LFS i.e. large files
  • aliases, shorten a bunch of long commands we can’t and don’t want to remember
  • branch and push config, simple push method and prefer rebase instead of merge, more on that later

This config usually lives at ~/.gitconfig, but you can also use a project-specific config which can be usually found in project_root/.git/config

[user]
	name = Patrick van der Leer
	email = pat.vdleer@gmail.com
[filter "lfs"]
	clean = git-lfs clean -- %f
	smudge = git-lfs smudge -- %f
	process = git-lfs filter-process
	required = true
[alias]
  bw = blame -w -M
  c = commit
  b = rev-parse --abbrev-ref HEAD
  commend = commit --amend --no-edit
  cc = commit --all --amend --no-edit
  ca = commit --all
  co = checkout
  cb = "!f() { git checkout `git log --until=\"$*\" -1 --format=%h`; } ; f"
  s = status --short
  d = diff
  dc = diff --cached --word-diff=color
  dw = diff --word-diff=color
  l = log
  a = add
  af = add -f
  au = add -u # stages modified and deleted, without new
  p = push
  pf = push -f # force push, useful after using rebase
  ss = show -1 --format=%B--stat
  sw = show -1 --format=%B--stat --word-diff=color
  whatis = show -s --pretty='tformat:%h (%s, %ad)' --date=short
  lg = log --graph --pretty=format:'%Cred%h%Creset %C(yellow)%an%d%Creset %s [%N] %Cgreen(%ar)%Creset' --date=relative
  lgd = log --graph --pretty=format:'%Cred%h%Creset %C(yellow)%an%d%Creset %s [%N] %Cgreen(%ar)%Creset' --date=default
  lgm = log --graph --pretty=format:'%Cred%h%Creset %C(yellow)%an%d%Creset %s [%N] %Cgreen(%ar)%Creset' --date=relative --author=pat.vdleer@gmail.com
  abbr = "!sh -c 'git rev-list --all | grep ^$1 | while read commit; do git --no-pager log -n1 --pretty=format:\"%H %ci %an %s%n\" $commit; done' -"
[color]
  # ui always will result in colors being in patch (git diff > diff.patch) and then when using git apply it will throw errors.
  # ui auto will show colors in terminal but not when written to file
  # ui = always
  ui = auto
[branch]
  autosetuprebase = remote
[push]
  default = simple

Branching

Master and develop

In our repository we have the master branch as main branch. Master is used to deploy to staging, demo and production. In the link posted above they also mention a develop branch, because we have a gitlab pipeline based on master it is not really necessary for us to use develop branch.

Feature, bug and maintenance branches

During development, we’d like to refrain from working directly on the master branch. Instead, we’d like you to use feature branches. When a ticket is assigned to you and you are going to start working on it, please make sure master is up to date with the latest commits and create a feature branch first (git checkout -b) using this convention: feature/123-short-ticket-description. This indicates it’s a feature, the ticket number and a description on what it is about. Bug branches have a similar convention, but then use bug/123-short-ticket-description instead of feature/. For maintenance the same applies. Do not use bare numbers! This offers a few advantages:

  1. If you need help on a ticket, the person who’s helping you can easily find the branch and check it out.
  2. It is easier to get an overview of branches based on ticket type: git branch --list "feature/*" or git branch --list "bug/*".

You can create a branch from a ticket by using the dropdown on the right - just above the comments section. It initially says Create merge request and branch, just change to to Create branch.

Committing your work

For commits, please make sure you reference the ticket number (refs #123) and add a clear description on what the commit is about: refs #123 - implementation of specific feature. If you need to commit multiple things which are related, please use git commit --amend --no-edit (or git commend if you use our gitconfig) - this adds the change to the latest commit (HEAD) without creating a new commit and commit message. Please note that you change history this way and it requires a git push --force. sidenote: force pushing to master is not enabled, only in feature branches

Rebasing vs merging

To get a clear overview of the differences, please check the following link: https://www.atlassian.com/git/tutorials/merging-vs-rebasing

Merge explanation

Merging is nice because it’s a non-destructive operation. The existing branches are not changed in any way. This avoids all of the potential pitfalls of rebasing (discussed below). On the other hand, this also means that the feature branch will have an extraneous merge commit every time you need to incorporate upstream changes. If master is very active, this can pollute your feature branch’s history quite a bit.

Rebase explanation

The major benefit of rebasing is that you get a much cleaner project history. First, it eliminates the unnecessary merge commits required by git merge. Second, rebasing also results in a perfectly linear project history. You can follow the tip of feature all the way to the beginning of the project without any forks. This makes it easier to navigate your project with commands like git log

Rebase is the way to go

As we’d like to keep everything clean and readable, rebase is the choice of having a clean history in the git log. In the beginning it will take some getting-used-to.

Interactive rebasing

A cleanup of your commit history (interactive rebasing)

Seeing the example above you will probably notice that so many commits without a decent commit message makes no sense: you don’t know what it is about and what has changed. There are a few ways to prevent this from happening and if it happened - how to clean it up (using git rebase -i).

Amend without editing the latest commit message

To prevent new commits being added all the time (note: only if they are basically the same, or the addition is not big enough to have its own commit) you can use git commit --amend --no-edit. This adds your staged change to the latest commit (HEAD) without changing the commit message. Congratulations, you’ve rewritten history! Don’t forget to force push your change (git push -f).

Merge requests

For the contributor

So you’ve finished your ticket, all tests pass and you decide to create a merge request to master? Good. Here’s a few key things about merge requests:

  1. Only create a merge request for functionality that is in your perspective done and matches the Acceptance Criteria of the ticket (so please no WIP: merge requests as this makes no sense, ticket progress can be monitored in the ticket itself).
  2. Make sure the latest master (checkout to master and git pull first) is in your featurebranch by doing a git rebase master in your feature branch. You will either see your commit(s) being placed on top of master (rewriting history and you should git push --force) or it will notify you it is already up to date with master. In the latter, no more action is required. Worst case, you’ll get a conflict which you will have to resolve first. Follow the onscreen steps if that’s the case (or contact Mycha with a request for help :))
  3. A default set of reviewers/approvers is added when you create a merge request. Please check if all the appropriate users are on there, and add or remove where necessary.

Merge request approvals

Before a merge request should be merged, please make sure everybody who should approve also approves.

Merging a merge request

Merge requests can be handled manually by the user via commandline, or via the gitlab interface itself. Both will be done in a fastforward way.

If you decide to do it manually: As the to-be-merged feature branch only contains new changes on top of master, you must use git merge to merge to master! Do not use git rebase to merge the feature branch. Master is a protected branch so you cannot force push. You will not create a new merge commit as only the feature branch commits are placed on top!**

Tagging

When preparing for a demo or production release, a tag of the master branch should be created. Tags are used to mark a point in the commit history as important. A release is an important milestone, so in our git history it should be marked as such. It can also be used as a reference point (if a developer is looking for code from a previous release, this tag as a reference will limit the time spent on searching)

Tags should follow the semantic versioning convention, where it is always MAJOR.MINOR.PATCH.

  • MAJOR means it is an incompatible / non backwards capable change.
  • MINOR is when changes are added but they are (backwards) compatible
  • PATCH is used for compatible bug fixes

Note: MAJOR version 0.y.z is used for development only. Please check SymVer specification #4 .

In the full example below I also added a section on how to create a tag.

Examples

Cleaning up your commit message mess

So you were not able to contain the mess beforehand, or you work in somebody else’s feature branch, you need to clean up the history. This can be achieved using git rebase -i. A step by step plan:

  1. pull the latest feature branch changes (git pull on your feature branch)
  2. Fetch the git log (git log OR if you want a clear overview: git log --graph --pretty=format:'%Cred%h%Creset %C(yellow)%an%d%Creset %s [%N] %Cgreen(%ar)%Creset' --date=relative)
  3. Determine from which commit hash you’d like to start rebasing
  4. Let’s say we want to start from bd1b4d1: git rebase -i bd1b4d1

We now get an overview which looks like:

$ pick 9010820 refs #591 added a reset option to every store module and created a clearstate method in the vuex root index.js so we can use each module mutation seperately
$ pick 0bb292e refs #591 fetch current user permissions and store them in account, handle front interactions based on these permissions
$ pick d74b147 refs #591 made decent use of store getters, removed RoleUtils as everything is done through getters now
$ pick 3915b49 refs #591 squash later: projectmembers api integration and temporarily disabled fetchTasks for quarantine
$ # Rebase bd1b4d1..3915b49 onto bd1b4d1 (4 commands)
$ #
$ # Commands:
$ # p, pick = use commit
$ # r, reword = use commit, but edit the commit message
$ # e, edit = use commit, but stop for amending
$ # s, squash = use commit, but meld into previous commit
$ # f, fixup = like "squash", but discard this commit's log message
$ # x, exec = run command (the rest of the line) using shell
$ # d, drop = remove commit
$ #
$ # These lines can be re-ordered; they are executed from top to bottom.
$ # If you remove a line here THAT COMMIT WILL BE LOST.
$ # However, if you remove everything, the rebase will be aborted.
$ # Note that empty commits are commented out

First thing you will notice is that the list is turned around: your latest commit is now at the bottom. Rules are executed from top to bottom. Please read all the commands carefully when executing this.
In this case we’d like to fixup (fixup because we can disregard its commit message) the latest commit, so in our rebase it will look like this:

pick 9010820 refs #591 added a reset option to every store module and created a clearstate method in the vuex root index.js so we can use each module mutation seperately
pick 0bb292e refs #591 fetch current user permissions and store them in account, handle front interactions based on these permissions
pick d74b147 refs #591 made decent use of store getters, removed RoleUtils as everything is done through getters now
f 3915b49 refs #591 squash later: projectmembers api integration and temporarily disabled fetchTasks for quarantine

Save this and you will see git applying the results, which will show refs #591 made decent use of store getters, removed RoleUtils as everything is done through getters now as your latest commit message (but containing the fixed up commit). Don’t forget to force push your changes.

Example from start to finish

$ git branch
* master
$ git checkout -b feature/123-new-cool-feature
$ git add path/to/file
$ git commit -m 'refs #123 added file to handle expected functionality'
$ git push -u origin feature/123-new-cool-feature

We are on branch master and from the current tree, we checkout to a new branch called feature/123-new-cool-feature. Now we edited a file and we commit that file, push that to origin. (-u equals to --set-upstream because the feature branch does not exist on remote) Let’s say in the meantime master was updated with new commits and we need to update our feature branch

$ git branch
* feature/123-new-cool-feature
$ git checkout master
$ git pull
Updating 63ee230..d761996
Fast-forward
 .dockerignore               |  1 +
 .gitlab-ci.yml              |  2 +-
$ git checkout feature/123-new-cool-feature
Switched to branch 'feature/123-new-cool-feature'
Your branch is up to date with 'origin/feature/123-new-cool-feature'.
$ git rebase master
First, rewinding head to replay your work on top of it...
Applying: refs #123 added file to handle expected functionality
$ git push -f

We are on our feature branch, now we are going to update the feature branch with the latest master changes by using git rebase (Please note that for the reviewer / person who merges to master, this is required otherwise you will get merge conflicts on master). After this you’ve updated history and a git push --force is required.

$ git branch
* feature/123-new-cool-feature
$ git pull 
Already up to date.
Current branch feature/123-new-cool-feature is up to date.
$ git checkout master
$ git pull
Already up to date.
Current branch master is up to date.
$ git merge feature/123-new-cool-feature
Updating 63ee230..d761996
Fast-forward
 path/to/file               |  1 +
$ git push

Now we’ve successfully merged the feature branch into master and now the feature branch can be closed / deleted. Note here we merge (and not rebase) to master. Because the feature branch is up to date with master, git performes a fastforward merge which will not create a merge commit.

Once we are done with all our changes, all tests are created and passed, the code is production-ready and we would like to deploy soon, we are going to create a tag of the current situation.

$ git branch
master
$ git tag -a 1.0.0 -m "First major release of frontend!"
$ git push origin --tags
Counting objects: 1, done.
Writing objects: 100% (1/1), 160 bytes | 0 bytes/s, done.
Total 1 (delta 0), reused 0 (delta 0)
To git@gitlab.0x43.nl:examples/frontend.git
 * [new tag]         1.0.0 -> 1.0.0

TL;DR

  • Example of a good commit message for a bug gives either a technical description of the bug solution or “fixes ”.
  • Example of a good commit message for a feature gives a technical description of the feature that has been realised or a part of the feature.
  • Bug or features in a branch, prefixed with ‘bug/’ or ‘feature/’
  • When committing, use commend (commit --amend --no-edit) whenever possible (if the gitlab issue is the same and the piece of code is related) - or, if you want to adjust the commit message but still keep everything in the latest commit: git commit --amend will prompt you to change the commit message.
  • After amend or rebase, you must push --force
  • After merging branches to master, you cannot change the latest commit on the feature branch (create a new commit on the (feature|bug) branch or just create a new branch), because you cannot push --force to master
  • Merging of feature branches to master require a succesful pipeline execution and necessary approvals
  • When preparing for a release and all merges are complete, please tag the latest commit used in the release.

See also