176 Common Patterns E5a866ee

Common GitLab CI/CD Patterns

Collection of reusable GitLab CI/CD pipeline patterns and configuration examples.

Table of Contents

• Python Projects
• Node.js Projects
• Docker Build and Push
• Multi-Environment Deployment
• Monorepo Patterns
• Release Management
• Security Scanning
• Performance Testing

Python Projects

Basic Python Pipeline with uv

variables:
  PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
  UV_CACHE_DIR: "$CI_PROJECT_DIR/.cache/uv"

stages:
  - lint
  - test
  - build
  - deploy

cache:
  key:
    files:
      - pyproject.toml
      - uv.lock
  paths:
    - .cache/pip
    - .cache/uv
    - .venv/

.python_base:
  image: python:3.12-slim
  before_script:
    - pip install uv
    - uv venv
    - uv sync --frozen

lint:ruff:
  extends: .python_base
  stage: lint
  script:
    - uv run ruff check

lint:mypy:
  extends: .python_base
  stage: lint
  script:
    - uv run mypy .

test:
  extends: .python_base
  stage: test
  script:
    - uv run pytest --cov --cov-report=html --cov-report=xml
  artifacts:
    paths:
      - htmlcov/
      - coverage.xml
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage.xml
  coverage: '/(?i)total.*? (100(?:\.0+)?|[1-9]?\d(?:\.\d+)?)%/'

build:
  extends: .python_base
  stage: build
  script:
    - uv build
  artifacts:
    paths:
      - dist/

deploy:pypi:
  stage: deploy
  needs: [build]
  script:
    - uv publish --token $PYPI_TOKEN
  rules:
    - if: $CI_COMMIT_TAG

Python Matrix Testing

Test across multiple Python versions:

test:
  parallel:
    matrix:
      - PYTHON_VERSION: ["3.11", "3.12", "3.13"]
  image: python:${PYTHON_VERSION}-slim
  script:
    - pip install uv
    - uv venv
    - uv sync --frozen
    - uv run pytest

Node.js Projects

Basic Node.js Pipeline

variables:
  NPM_CONFIG_CACHE: "$CI_PROJECT_DIR/.npm"

stages:
  - install
  - lint
  - test
  - build
  - deploy

cache:
  key:
    files:
      - package-lock.json
  paths:
    - node_modules/
    - .npm/

install:
  stage: install
  script:
    - npm ci --cache .npm --prefer-offline
  cache:
    policy: pull-push

lint:
  stage: lint
  script:
    - npm run lint
  cache:
    policy: pull

test:
  stage: test
  script:
    - npm test
  coverage: '/All files[^|]*\|[^|]*\s+([\d\.]+)/'
  artifacts:
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml

build:
  stage: build
  script:
    - npm run build
  artifacts:
    paths:
      - dist/

deploy:
  stage: deploy
  script:
    - npm publish
  rules:
    - if: $CI_COMMIT_TAG

Docker Build and Push

Basic Docker Build

variables:
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: "/certs"

build:docker:
  stage: build
  image: docker:latest
  services:
    - docker:dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA .
    - docker tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
    - docker push $CI_REGISTRY_IMAGE:latest

Multi-Architecture Docker Build

build:docker:multiarch:
  stage: build
  image: docker:latest
  services:
    - docker:dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
    - docker buildx create --use
  script:
    - |
      docker buildx build \
        --platform linux/amd64,linux/arm64 \
        --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA \
        --tag $CI_REGISTRY_IMAGE:latest \
        --push \
        .

Docker Build with Kaniko

Kaniko builds without Docker-in-Docker:

build:kaniko:
  stage: build
  image:
    name: gcr.io/kaniko-project/executor:debug
    entrypoint: [""]
  script:
    - mkdir -p /kaniko/.docker
    - echo "{\"auths\":{\"$CI_REGISTRY\":{\"auth\":\"$(echo -n $CI_REGISTRY_USER:$CI_REGISTRY_PASSWORD | base64)\"}}}" > /kaniko/.docker/config.json
    - |
      /kaniko/executor \
        --context $CI_PROJECT_DIR \
        --dockerfile $CI_PROJECT_DIR/Dockerfile \
        --destination $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA \
        --destination $CI_REGISTRY_IMAGE:latest

Multi-Environment Deployment

Deploy to Multiple Environments

stages:
  - build
  - deploy:dev
  - deploy:staging
  - deploy:production

.deploy_base:
  image: alpine:latest
  before_script:
    - apk add --no-cache curl
  script:
    - curl -X POST $WEBHOOK_URL -H "Authorization: Bearer $DEPLOY_TOKEN"

deploy:dev:
  extends: .deploy_base
  stage: deploy:dev
  environment:
    name: development
    url: https://dev.example.com
  rules:
    - if: $CI_COMMIT_BRANCH == "develop"

deploy:staging:
  extends: .deploy_base
  stage: deploy:staging
  environment:
    name: staging
    url: https://staging.example.com
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

deploy:production:
  extends: .deploy_base
  stage: deploy:production
  environment:
    name: production
    url: https://example.com
  rules:
    - if: $CI_COMMIT_TAG
  when: manual

Environment-Specific Variables

.deploy_base:
  script:
    - deploy.sh $ENVIRONMENT

deploy:dev:
  extends: .deploy_base
  variables:
    ENVIRONMENT: development
    API_URL: https://api-dev.example.com

deploy:staging:
  extends: .deploy_base
  variables:
    ENVIRONMENT: staging
    API_URL: https://api-staging.example.com

deploy:production:
  extends: .deploy_base
  variables:
    ENVIRONMENT: production
    API_URL: https://api.example.com

Monorepo Patterns

Selective Job Execution

Run jobs only when relevant files change:

build:frontend:
  script:
    - cd frontend && npm run build
  rules:
    - changes:
        - frontend/**/*
        - package.json

build:backend:
  script:
    - cd backend && uv run build
  rules:
    - changes:
        - backend/**/*
        - pyproject.toml

build:shared:
  script:
    - cd shared && npm run build
  rules:
    - changes:
        - shared/**/*

Child Pipelines

Trigger separate pipelines for each component:

trigger:frontend:
  stage: trigger
  trigger:
    include: frontend/.gitlab-ci.yml
    strategy: depend
  rules:
    - changes:
        - frontend/**/*

trigger:backend:
  stage: trigger
  trigger:
    include: backend/.gitlab-ci.yml
    strategy: depend
  rules:
    - changes:
        - backend/**/*

Release Management

Semantic Versioning Release

release:
  stage: deploy
  image: node:latest
  before_script:
    - npm install -g semantic-release @semantic-release/gitlab
  script:
    - semantic-release
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

GitLab Release with glab

variables:
  GITLAB_HOST: $CI_SERVER_HOST

release:
  stage: deploy
  image: alpine:latest
  before_script:
    - apk add --no-cache curl bash
    - curl -sSL https://gitlab.com/gitlab-org/cli/-/releases/permalink/latest/downloads/glab_Linux_x86_64.tar.gz | tar -xz -C /usr/local/bin
  script:
    - |
      glab release create $CI_COMMIT_TAG \
        --notes "Release $CI_COMMIT_TAG" \
        --assets-links "[{\"name\":\"Package\",\"url\":\"$CI_PROJECT_URL/-/packages\"}]"
  rules:
    - if: $CI_COMMIT_TAG

Changelog Generation

changelog:
  stage: deploy
  image: node:latest
  script:
    - npx conventional-changelog-cli -p angular -i CHANGELOG.md -s
    - git add CHANGELOG.md
    - git commit -m "chore: update changelog [skip ci]"
    - git push origin HEAD:$CI_COMMIT_REF_NAME
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

Security Scanning

SAST (Static Application Security Testing)

include:
  - template: Security/SAST.gitlab-ci.yml

sast:
  stage: test
  variables:
    SAST_EXCLUDED_PATHS: "spec, test, tests, tmp"

Dependency Scanning

include:
  - template: Security/Dependency-Scanning.gitlab-ci.yml

dependency_scanning:
  stage: test

Secret Detection

include:
  - template: Security/Secret-Detection.gitlab-ci.yml

secret_detection:
  stage: test

Container Scanning

include:
  - template: Security/Container-Scanning.gitlab-ci.yml

container_scanning:
  stage: test
  variables:
    CS_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

Performance Testing

Load Testing with k6

load_test:
  stage: test
  image: grafana/k6:latest
  script:
    - k6 run --out json=results.json load-test.js
  artifacts:
    paths:
      - results.json
    reports:
      junit: results.xml

Lighthouse Performance Testing

lighthouse:
  stage: test
  image: cypress/browsers:node14.17.0-chrome91-ff89
  script:
    - npm install -g @lhci/cli
    - lhci autorun --config=.lighthouserc.json
  artifacts:
    paths:
      - .lighthouseci/

Reusable Configuration Patterns

Job Templates

.test_template:
  stage: test
  script:
    - npm test
  coverage: '/All files[^|]*\|[^|]*\s+([\d\.]+)/'
  artifacts:
    reports:
      junit: junit.xml
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml

test:unit:
  extends: .test_template
  script:
    - npm run test:unit

test:integration:
  extends: .test_template
  script:
    - npm run test:integration

Include External Configuration

include:
  - local: ".gitlab-ci-templates/*.yml"
  - project: "group/ci-templates"
    file: "/templates/python.yml"
    ref: main
  - remote: "https://example.com/ci-template.yml"

Dynamic Child Pipeline

generate-pipeline:
  stage: prepare
  script:
    - python generate_pipeline.py > generated-pipeline.yml
  artifacts:
    paths:
      - generated-pipeline.yml

child-pipeline:
  stage: execute
  trigger:
    include:
      - artifact: generated-pipeline.yml
        job: generate-pipeline
    strategy: depend

Best Practices

1. Use templates - DRY principle with .template and extends
2. Cache dependencies - Speed up pipeline with proper caching
3. Parallelize jobs - Run independent jobs concurrently
4. Use DAG with needs - Optimize job execution order
5. Conditional execution - Use rules to run only necessary jobs
6. Secure secrets - Use masked and protected variables
7. Set timeouts - Prevent hanging jobs
8. Version control configs - Track .gitlab-ci.yml changes
9. Monitor performance - Track pipeline duration and resource usage
10. Document patterns - Share common patterns with team