The third of a 12 part series on how to use Twelve-Factor App in practice. This entry is, again, written in collaboration with my good friend and (twice/double) former coworker Mycha de Vrees. We will be implementing https://12factor.net/config based on my/our combined experience on how to deal with configuration.
Why
One of the things that’s isn’t made clear in the 12 factors is why you should use environment variables instead of config files, or worse, hard coded. The biggest reason is the ability to control your app without changing files which gives you a lot(/easier) control.
For example, if your app needs mysql and you use Gitlab CI/CD for testing you need to be able to tell your app where mysql is, what database to use and with which credentials. I’m not saying it is not NOT possible with config files, you could script something that writes it to a config file but reading it from env is a lot easier. Docker, including compose and swarm, and Kubernetes are build for the usage of environment variables, specifically how will become more apparent later on.
Languages
Each language has its own way of dealing with environment variables, where Python throws an exception, PHP might just give you a warning.
NodeJS
config = {
redis: {
enabled: process.env.REDIS_ENABLED || false,
host: process.env.REDIS || 'redis',
port: process.env.REDIS_PORT || 6379,
prefix: process.env.REDIS_PORT || ''
}
}
Python
import os
# Old style
os.environ['HOME']
# New style
os.environ.get('KEY_THAT_SHOULD_EXIST')
# New style with default
os.getenv('KEY_THAT_MIGHT_EXIST', default_value)
PHP
$myVar = getenv("KEY_THAT_SHOULD_EXIST");
Please note that yes PHP has getenv but this does not have to equal $_ENV Even putenv does not always ensure you will be able to getenv from the env.
DevOps
Docker
In Dockerfiles we use ENV to declare variables. ENV does exactly what you expect, set a global variable within the container (or image on build).
One you might often come across is setting DEBIAN_FRONTEND to noninteractive, when this is set you let (for example) apt-get know that this is run from within a script.
I.e. it should not prompt to ask since we expect that there will be no one there to answer.
FROM ubuntu:18.04
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get install xyz
Within MySQL images it is used to set versions and used within the script itself, this may seem a bit overkill but imagine extending on these images. Setting the version to a variable allows you use the vars instead of hard coding the versions and thus switching versions more easily. Secondly, if you run software within the container that can handle multiple version, checking the env is a lot easier than call the service to check what version is running.
In Dockerfile it looks something like this:
FROM debian:stretch-slim
# ...
ENV GOSU_VERSION 1.7
RUN set -x \
&& apt-get update && apt-get install -y --no-install-recommends ca-certificates wget && rm -rf /var/lib/apt/lists/* \
&& wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture)" \
&& wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture).asc" \
&& export GNUPGHOME="$(mktemp -d)" \
&& gpg --batch --keyserver ha.pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4
# ...
ENV MYSQL_MAJOR 5.7
ENV MYSQL_VERSION 5.7.27-1debian9
RUN echo "deb http://repo.mysql.com/apt/debian/ stretch mysql-${MYSQL_MAJOR}" > /etc/apt/sources.list.d/mysql.list
More on ENV in Dockerfiles can be found here
Docker-compose
Within docker-compose we have a couple of options, set it to a specific container, set it into a ‘global’ environment file or a specified one.
Container specific variables
version: '3'
services:
mysql:
image: mysql:5.7
expose:
- "3306"
environment:
MYSQL_ROOT_PASSWORD: "my_super_secret!"
MYSQL_DATABASE: "my_db"
MYSQL_USER: "my_user"
MYSQL_PASSWORD: "my_password"
Scripting and the env of your env
On of the problems I ran into with env files is in a situation where multiple apps use the same variable name but the values differ.
Let’s take a look at the following example:
MYSQL_DATABASE="my_app_a
version: '3'
services:
my_app_a:
image: myapp:1.0
environment:
# MYSQL_DATABASE: "my_app_a"
MYSQL_USER: "my_app_user_a"
MYSQL_PASSWORD: "my_password"
my_app_b:
image: myapp:1.0
environment:
# MYSQL_DATABASE: "my_app_b"
MYSQL_USER: "my_app_user_b"
MYSQL_PASSWORD: "my_password"
In this case using a .env file containing the variable names would result in both the values being the same.
One way to avoid this is using Bash variables in using the other side of the :.
MYSQL_DATABASE_A="my_app_a
MYSQL_DATABASE_B="my_app_b
version: '3'
services:
my_app_a:
image: myapp:1.0
environment:
MYSQL_DATABASE: ${MYSQL_DATABASE_A}
MYSQL_USER: "my_app_user_a"
MYSQL_PASSWORD: "my_password"
my_app_b:
image: myapp:1.0
environment:
MYSQL_DATABASE: ${MYSQL_DATABASE_B}
MYSQL_USER: "my_app_user_b"
MYSQL_PASSWORD: "my_password"
Expanding on this we could set defaults and make the environment file optional, Bash offer 2 options for this;
FOO=${FOO:'FOO'}
BAR=${BAR:-'BAR'}
The former,
:alone will merely check wetter or not the variable is set, the latter,:-will validate if it’s empty or not.
Putting this to good use would make the following;
version: '3'
services:
my_app_a:
image: myapp:1.0
environment:
MYSQL_DATABASE: ${MYSQL_DATABASE_A:-my_app_a}
MYSQL_USER: ${MYSQL_USER_A:-my_app_user_a}
MYSQL_PASSWORD: ${MYSQL_PASSWD_A:-my_password}
Please note, Bash variables within your
.envfile will not work, they will not be parsed
Native
Running your app native might now seem a lot more complex, having to set env variables to run it, but in this case the environment file is your friend. Since running the app native usually means your environment isn’t managed and thus isn’t going to change on the fly you can simply set the configuration in the environment file.