CI/CD Pipelines

Think of it as an

Assembly Line

for your code — automated, consistent, always running

The Factory Analogy

Imagine a car factory. Raw materials come in, pass through stations for assembly, quality checks, painting, and testing — then finished cars roll out. No human decides "should we test this one?" Every car goes through the same automated process. CI/CD does the same for your code.

👨‍💻

You Push Code

git commit && git push

🤖

Pipeline Runs

Build → Test → Deploy

🚀

Users Get Updates

Automatically, safely

The Problem It Solves

✗ Manual Deployment

# Friday 5pm deployment ritual...
1. "Did anyone run the tests?" 🤔
2. SSH into production server
3. git pull origin main
4. "It worked on my machine!"
5. Manually restart services
6. Hope nothing breaks over the weekend
7. It breaks. Phone rings at 2am.
8. Rollback... how? 😰

Tests skipped "just this once"
Different results on different machines
Deployments are scary (done rarely)
Rollback process unclear or manual
No record of what was deployed when

✓ Automated Pipeline

# Every git push triggers:
on: push
jobs:
  build: # ✓ Compiles
  test:  # ✓ 847 tests pass
  lint:  # ✓ Code quality OK
  deploy: # ✓ Prod updated

# Merge PR → Live in 3 minutes
# Sleep soundly 😴

Tests run on EVERY commit
Identical environment every time
Deploy 10x a day with confidence
One-click rollback to any version
Full audit trail of deployments

Core Concepts

🔄

Continuous Integration (CI)

Automatically build and test code every time someone pushes. Catch bugs early, merge confidently.

📦

Continuous Delivery (CD)

Code is always ready to deploy. One button to ship. Humans decide when to release.

🚀

Continuous Deployment

Fully automated — every passing commit goes straight to production. No manual gates.

🔗

Pipeline

A series of stages and jobs that run in order. Each stage must pass before the next starts.

📁

Artifacts

Output from builds — Docker images, binaries, test reports. Passed between stages or stored for deployment.

🖥️

Runners / Agents

Machines that execute your pipelines. Cloud-hosted or self-managed. Where your code actually runs.

💡

Delivery vs. Deployment — What's the Difference?

Continuous Delivery: Code is packaged and ready to deploy at any time, but a human clicks the button.
Continuous Deployment: Every passing build automatically goes live. No human in the loop.

Most teams start with Delivery (safer) and graduate to Deployment as confidence grows.

Pipeline Stages

Source

Trigger pipeline

Build

Compile code

Test

Run test suite

Package

Create artifact

Deploy

Ship to prod

CI/CD Tools

⚫

GitHub Actions

Built into GitHub. Workflows defined in YAML files under .github/workflows/. Free for public repos, generous free tier for private.

Hosted Runners

Ubuntu, Windows, macOS

Marketplace

15,000+ pre-built actions

Matrix Builds

Test across versions

Basic Pipeline Structure

A minimal pipeline that builds and tests your code on every push.

# .github/workflows/ci.yml
name: CI Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  build-and-test:
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      
      - name: Install dependencies
        run: npm ci
      
      - name: Run tests
        run: npm test
      
      - name: Build
        run: npm run build

# .gitlab-ci.yml
stages:
  - build
  - test

variables:
  NODE_VERSION: "20"

default:
  image: node:${NODE_VERSION}
  cache:
    paths:
      - node_modules/

build:
  stage: build
  script:
    - npm ci
    - npm run build
  artifacts:
    paths:
      - dist/
    expire_in: 1 hour

test:
  stage: test
  script:
    - npm ci
    - npm test
  coverage: '/Coverage: \d+\.\d+%/'

// Jenkinsfile
pipeline {
    agent {
        docker {
            image 'node:20'
        }
    }
    
    stages {
        stage('Install') {
            steps {
                sh 'npm ci'
            }
        }
        
        stage('Test') {
            steps {
                sh 'npm test'
            }
        }
        
        stage('Build') {
            steps {
                sh 'npm run build'
            }
        }
    }
    
    post {
        always {
            junit '**/test-results.xml'
        }
    }
}

Build & Push Docker Image

Build a Docker image and push to a container registry.

# .github/workflows/docker.yml
name: Build and Push Docker

on:
  push:
    branches: [main]
    tags: ['v*']

jobs:
  docker:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      
      - name: Login to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${${ secrets.DOCKERHUB_USERNAME $}$}
          password: ${${ secrets.DOCKERHUB_TOKEN $}$}
      
      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: myuser/myapp:$${${ github.sha $}$}
          cache-from: type=gha
          cache-to: type=gha,mode=max

# .gitlab-ci.yml
stages:
  - build
  - push

variables:
  IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA

build-docker:
  stage: build
  image: docker:24
  services:
    - docker:24-dind
  script:
    - docker build -t $IMAGE_TAG .
    - docker save $IMAGE_TAG > image.tar
  artifacts:
    paths:
      - image.tar
    expire_in: 1 hour

push-docker:
  stage: push
  image: docker:24
  services:
    - docker:24-dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  script:
    - docker load < image.tar
    - docker push $IMAGE_TAG
  only:
    - main
    - tags

// Jenkinsfile
pipeline {
    agent any
    
    environment {
        REGISTRY = 'docker.io'
        IMAGE = 'myuser/myapp'
        DOCKER_CREDS = credentials('docker-hub-creds')
    }
    
    stages {
        stage('Build Image') {
            steps {
                script {
                    dockerImage = docker.build("${IMAGE}:${env.BUILD_NUMBER}")
                }
            }
        }
        
        stage('Push Image') {
            steps {
                script {
                    docker.withRegistry("https://${REGISTRY}", 'docker-hub-creds') {
                        dockerImage.push()
                        dockerImage.push('latest')
                    }
                }
            }
        }
    }
}

Deploy to Kubernetes

Deploy your application to a Kubernetes cluster after successful build.

# .github/workflows/deploy.yml
name: Deploy to Production

on:
  workflow_run:
    workflows: ["Build and Push Docker"]
    types: [completed]
    branches: [main]

jobs:
  deploy:
    if: ${${ github.event.workflow_run.conclusion == 'success' $}$}
    runs-on: ubuntu-latest
    environment: production
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Configure kubectl
        uses: azure/k8s-set-context@v3
        with:
          kubeconfig: ${${ secrets.KUBE_CONFIG $}$}
      
      - name: Update image tag
        run: |
          sed -i "s|IMAGE_TAG|$${${ github.sha $}$}|g" k8s/deployment.yaml
      
      - name: Deploy to cluster
        run: |
          kubectl apply -f k8s/
          kubectl rollout status deployment/myapp

# .gitlab-ci.yml (continued)
stages:
  - build
  - test
  - deploy

deploy-staging:
  stage: deploy
  image: bitnami/kubectl:latest
  environment:
    name: staging
    url: https://staging.example.com
  script:
    - kubectl config set-cluster k8s --server="$KUBE_URL"
    - kubectl config set-credentials gitlab --token="$KUBE_TOKEN"
    - kubectl set image deployment/myapp myapp=$IMAGE_TAG
    - kubectl rollout status deployment/myapp
  only:
    - main

deploy-production:
  stage: deploy
  image: bitnami/kubectl:latest
  environment:
    name: production
    url: https://example.com
  script:
    - kubectl set image deployment/myapp myapp=$IMAGE_TAG
  when: manual  # Requires button click
  only:
    - tags

// Jenkinsfile
pipeline {
    agent any
    
    stages {
        // ... build stages ...
        
        stage('Deploy to Staging') {
            steps {
                withKubeConfig([credentialsId: 'kubeconfig']) {
                    sh '''
                        kubectl set image deployment/myapp \
                            myapp=${IMAGE}:${BUILD_NUMBER}
                        kubectl rollout status deployment/myapp
                    '''
                }
            }
        }
        
        stage('Deploy to Production') {
            when {
                branch 'main'
            }
            input {
                message 'Deploy to production?'
                ok 'Deploy'
            }
            steps {
                withKubeConfig([credentialsId: 'kubeconfig-prod']) {
                    sh 'kubectl apply -f k8s/production/'
                }
            }
        }
    }
}

GitOps & ArgoCD

🔄

What is GitOps?

GitOps is a way of managing infrastructure and deployments where Git is the single source of truth. Instead of running kubectl apply manually, you update a Git repo and an automated system syncs the cluster to match.

Push-based (Traditional CD)

Pipeline runs kubectl apply. CI system needs cluster credentials.

Pull-based (GitOps)

Agent in cluster pulls changes from Git. No external credentials needed.

🐙

ArgoCD

A Kubernetes-native GitOps controller. Installs in your cluster, watches Git repos, and automatically syncs your deployments. Includes a beautiful UI.

# Install ArgoCD in your cluster
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Get the initial admin password
kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d

# Access the UI
kubectl port-forward svc/argocd-server -n argocd 8080:443
# Open https://localhost:8080

# application.yaml — Tell ArgoCD about your app
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
  namespace: argocd
spec:
  project: default
  
  source:
    repoURL: https://github.com/myorg/myapp-manifests
    targetRevision: HEAD
    path: k8s/production
  
  destination:
    server: https://kubernetes.default.svc
    namespace: production
  
  syncPolicy:
    automated:
      prune: true       # Delete resources not in Git
      selfHeal: true    # Revert manual changes
    syncOptions:
      - CreateNamespace=true

📜

Complete Audit Trail

Every deployment is a git commit. Who, what, when — all in history.

⏪

Easy Rollbacks

git revert and ArgoCD auto-syncs. Deployment undone.

🔒

Improved Security

CI pipelines don't need cluster credentials. Pull model is safer.

🩹

Self-Healing

Someone manually changed something? ArgoCD reverts it automatically.

Real App Examples

🐍

Python Flask App Pipeline

# .github/workflows/python-ci.yml
name: Python CI/CD

on:
  push:
    branches: [main]
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.10', '3.11', '3.12']
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python $${${ matrix.python-version $}$}
        uses: actions/setup-python@v5
        with:
          python-version: ${${ matrix.python-version $}$}
          cache: 'pip'
      
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
          pip install -r requirements-dev.txt
      
      - name: Lint with ruff
        run: ruff check .
      
      - name: Type check with mypy
        run: mypy src/
      
      - name: Test with pytest
        run: |
          pytest --cov=src --cov-report=xml
      
      - name: Upload coverage
        uses: codecov/codecov-action@v4
  
  deploy:
    needs: test
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Deploy to Heroku
        uses: akhileshns/heroku-deploy@v3.13.15
        with:
          heroku_api_key: ${${ secrets.HEROKU_API_KEY $}$}
          heroku_app_name: "myflaskapp"
          heroku_email: "dev@example.com"

💚

Node.js Express App Pipeline

# .github/workflows/node-ci.yml
name: Node.js CI/CD

on:
  push:
    branches: [main]
  pull_request:

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${${ github.repository $}$}

jobs:
  test:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      
      - run: npm ci
      - run: npm run lint
      - run: npm run test:coverage
      - run: npm run build
      
      - name: Upload build artifact
        uses: actions/upload-artifact@v4
        with:
          name: dist
          path: dist/
  
  build-and-push:
    needs: test
    if: github.event_name == 'push'
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Log in to GHCR
        uses: docker/login-action@v3
        with:
          registry: ${${ env.REGISTRY $}$}
          username: ${${ github.actor $}$}
          password: ${${ secrets.GITHUB_TOKEN $}$}
      
      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ${${ env.REGISTRY $}$}/${${ env.IMAGE_NAME $}$}:$${${ github.sha $}$}

Best Practices

🧪

Testing

Run tests on every push, not just PRs
Fail fast — put quick tests first
Use matrix builds for multiple versions
Track code coverage trends

🔐

Secrets Management

Never commit secrets to code
Use platform secret stores (GitHub Secrets)
Rotate credentials regularly
Minimize secret scope (environment-specific)

⚡

Caching

Cache dependencies (npm, pip, maven)
Use Docker layer caching
Cache build artifacts between jobs
Faster pipelines = happier developers

🚀

Deployment Strategy

Deploy to staging first, then prod
Use environment protection rules
Implement health checks
Have a rollback plan (tested!)

When to Use CI/CD

✓ Use when:

You have more than one developer
You deploy more than once a month
Bugs in production are costly
You want confidence in your code
Manual deployments take >5 minutes
You value your weekends

⚠️ Maybe overkill if:

Solo project, just experimenting
No tests to run (but... write tests!)
Static site with no build step
Prototype with no users yet
Learning project (but... great practice!)

Trade-offs

Pros

Faster feedback on code quality
Consistent, reproducible builds
Deploy confidently, more often
Documentation through pipelines
Reduced "works on my machine" issues
Team can ship without DevOps bottleneck

Cons

Initial setup takes time
Another thing to maintain
Can become complex (yaml engineering)
Debugging pipeline issues is annoying
Cost for private repos / hosted runners
False sense of security if tests are bad

Key Takeaways

Automate the boring stuff

Let robots build, test, and deploy. Humans should be thinking, not clicking.

CI catches bugs early, CD ships them fast

Continuous Integration = always tested. Continuous Delivery = always deployable.

Start simple, iterate

A basic pipeline that runs tests is better than no pipeline. Add complexity as needed.

GitOps = Git as single source of truth

With tools like ArgoCD, your cluster state matches your repo. Rollback = git revert.

Your pipeline is code — treat it that way

Version control, code review, refactor. YAML files deserve the same care as your app.