Custom Helm chart in GitLab auto devops -

The default auto devops is based on 2 major parts, the auto-deploy-image and the auto-deploy-app . We’ll go through both of them and take a look the standard possibilities they provide.

Goal

Deploying more than just our code and Postgresql; add Redis and more config/secrets.

In order to do this we need to set up our own chart.

How it works

First let takes a look at what we have and how it works;

In short; GitLab-CI is executed by the gitlab-runner, tests and builds, the usual and then the review part. This part loads the auto-deploy-image which preps the environment via a shell script and deploys the auto-deploy-app chart.

Let’s start with the pipeline itself and take a look at the deployment;

review:
  extends: .auto-deploy
  stage: review
  variables:
    CI_APPLICATION_REPOSITORY: $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME
    CI_APPLICATION_TAG: $CI_COMMIT_REF_SLUG
    PULL_POLICY: Always
  script:
    - auto-deploy check_kube_domain
    - auto-deploy download_chart
    - auto-deploy ensure_namespace
    - auto-deploy initialize_tiller
    - auto-deploy create_secret
    - auto-deploy deploy
    - auto-deploy persist_environment_url
  environment:
    name: review/$CI_COMMIT_REF_NAME
    url: http://$CI_COMMIT_REF_SLUG.$CI_PROJECT_PATH_SLUG.$KUBE_INGRESS_BASE_DOMAIN
    on_stop: stop_review
  when: manual
  artifacts:
    paths: [environment_url.txt]
  only:
    refs:
      - branches
      - tags
    kubernetes: active
  except:
    refs:
      - master
    variables:
      - $REVIEW_DISABLED

The command that is called in the script section is auto-deploy .

Looking at download_chart command we find the following;

function download_chart() {
  local auto_chart
  local auto_chart_name
  if [[ ! -d chart ]]; then
    auto_chart=${AUTO_DEVOPS_CHART:-gitlab/auto-deploy-app}
    # shellcheck disable=SC2086 # double quote variables to prevent globbing
    auto_chart_name=$(basename $auto_chart)
    auto_chart_name=${auto_chart_name%.tgz}
    auto_chart_name=${auto_chart_name%.tar.gz}
  else
    auto_chart="chart"
    auto_chart_name="chart"
  fi

  helm init --client-only
  # shellcheck disable=SC2086 # double quote variables to prevent globbing
  # shellcheck disable=SC2140 # ambiguous quoting warning
  helm repo add ${AUTO_DEVOPS_CHART_REPOSITORY_NAME:-gitlab} ${AUTO_DEVOPS_CHART_REPOSITORY:-https://charts.gitlab.io} ${AUTO_DEVOPS_CHART_REPOSITORY_USERNAME:+"--username" "$AUTO_DEVOPS_CHART_REPOSITORY_USERNAME"} ${AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD:+"--password" "$AUTO_DEVOPS_CHART_REPOSITORY_PASSWORD"}
  if [[ ! -d "$auto_chart" ]]; then
    helm fetch ${auto_chart} --untar
  fi
  if [ "$auto_chart_name" != "chart" ]; then
    mv ${auto_chart_name} chart
  fi

  helm dependency update chart/
  helm dependency build chart/
}

In short; if the directory chart exists we’re going to use that, else use gitlab/auto-deploy-app. So in our repository we’re going to create our own chart and base it on the gitlab/auto-deploy-app, the alternative would be to use helm to create a chart but that will cause a bunch of problems.

This should be a fairly simply git pull/download zip from auto-deploy-app and copy paste it over to our repository. So create a folder called chart in the root of your project and add the following files/folders;

Chart.yaml
values.yaml
templates

Optionally, if you have dependencies (like I do), add requirements.yaml and requirements.lock too.

One of the main reasons we base our chart on the auto-deploy-app rather than a fresh helm create one is that a bunch of names, labels and annotations are passed along from GitLab to the Chart in order to find it back. For example; GitLab’s auto-deploy-image expects the deployment to be named the same as the branch with a bunch of unique chars behind. The default helm chart uses “fullname” which is the same but with your project name behind it. This will deploy your chart but the job will fail and upgrades won’t work since it searches for a differently named deployment.

The other big issue is forgetting the releaseOverride in your annotations and labels, this causes Helm no to upgrade the deployment’s replicaSet and thus your pod will stay on the old version.

Now that’s done we can edit the setup to our likings but will run into a couple of things as listed below;

Extra helm chart repositories

Since I use the Bitnami Redis chart I need the repository added, all deployment steps extend on .auto-deploy so adding it there should fix it.

.auto-deploy:
  image: "registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image:latest"
  before_script:
    - helm init --client-only
    - helm repo add bitnami https://charts.bitnami.com/bitnami

Additional arguments for helm

In the gitlab-ci.yml we can add the variable HELM_UPGRADE_EXTRA_ARGS which is used in the auto-deploy-app.

helm upgrade --install \
    --wait \
    --set gitlab.app="$CI_PROJECT_PATH_SLUG" \
...
    --set application.migrateCommand="$DB_MIGRATE" \
    "${modsecurity_set_args[@]}" \
    "${helm_values_args[@]}" \
    $HELM_UPGRADE_EXTRA_ARGS \
    --namespace="$KUBE_NAMESPACE" \
    "$name" \
    chart/

variables:
  HELM_UPGRADE_EXTRA_ARGS: '--set redis.usePassword="false"'

Goal

How it works

Extra helm chart repositories

Additional arguments for helm

See also