Logically the yml consists of 3 parts which coincides with the stage, each stage contains jobs;
If one of the jobs fail, the pipeline is stopped and the deployment is canceled. This should prevent me from releasing faulty code.
In practice we split up the repeating parts to keep the yml DRY.
In yml we can do
.test: &test_template which means we use the content of
.test as a variable, a copy paste if you will, within the yml.
Later on, the paste part, we call on the
test_template name and “paste” it into another section, optionally overriding parts.
To do this we use
<<: *test_template were it should be “pasted”.
Side note: properties within a yml are order based, if you use
<<: *test_templatefirst your variable,
stagefor example will override the value of
test_template. If you declare your
stagefirst the template will override it. Take a look at
deploy:test:for an example.
stages: - test - test-deploy - deploy cache: paths: - ~/.cache/pip/ .test: &test_template stage: test before_script: - pip install -r requirements.txt script: - python -m unittest discover test:PY3.6: <<: *test_template image: python:3.6 test:PY3.7: <<: *test_template image: python:3.7 test:PY3.8: <<: *test_template image: python:3.8 test:coverage: stage: test image: python:3.7 before_script: - pip install -r requirements.txt - pip install coverage script: - coverage run --omit="/*venv*,/*tests*" -m unittest discover - coverage report -m test:lint: stage: test image: python:3.7 before_script: - pip install pylint - pip install -r requirements.txt script: - pylint --max-line-length=120 --exit-zero solc .deploy: &deploy_template stage: deploy image: python:3.7 before_script: - pip install -r requirements.txt - pip install setuptools wheel twine - python setup.py sdist bdist_wheel deploy:test: <<: *deploy_template stage: test-deploy variables: TWINE_USERNAME: $TEST_USERNAME TWINE_PASSWORD: $TEST_PASSWORD script: - twine upload --repository-url https://test.pypi.org/legacy/ dist/* only: - tags deploy:live: <<: *deploy_template variables: TWINE_USERNAME: $LIVE_USERNAME TWINE_PASSWORD: $LIVE_PASSWORD script: - twine upload dist/* only: - tags
The above yml will result in a pipeline as displayed below, the 3 stages clearly split into columns. The testing of the deployment and the actual deployment are only ran on tags.
To publish to PyPi you need an account with username and password but you do not want these somewhere hardcoded and embeded in git for everyone to see. To handle these variable we get them, on run time, from the environment, in this case via the variables option from GitLab.
Note: We marked the passwords as masked to prevent them from showing up in the CI/CD pipeline/jobs logs.
In one of the jobs I check the coverage of the code, to get GitLab to check for the coverage we need to provide a regex to parse the output of the jobs.
We do not need to set a specific job to parse, as far as I know they are all parsed.
The regex I use,
^TOTAL\s+\d+\s+\d+\s+((?:\d+\%)|(?:\d+.\d+\%))$ is used for Python