CI/CD Pipeline Fundamentals

Introduction

CI/CD (Continuous Integration / Continuous Delivery or Deployment) is the backbone of modern software development. It automates building, testing, and deploying code, enabling teams to ship faster with fewer bugs.

The Problem CI/CD Solves

Before CI/CD (Integration Hell):

• Developers work in isolation for weeks
• "Big bang" integrations cause massive conflicts
• Manual testing takes days
• Deployments are scary, infrequent events
• Bugs found in production weeks after introduction

With CI/CD:

• Code integrated multiple times daily
• Automated tests catch issues immediately
• Deployments are routine, low-risk events
• Feedback loop measured in minutes, not weeks

CI vs CD: What's the Difference?

Continuous Integration (CI):

• Automatically build and test on every commit
• Merge code to main branch frequently (daily or more)
• Catch integration issues early
• Maintain a deployable codebase

Continuous Delivery (CD):

• Automatically prepare releases for deployment
• Every change is deployable (but manual approval to deploy)
• One-click deployment to production
• Release when business decides

Continuous Deployment (CD):

• Automatically deploy every passing change
• No manual intervention
• Requires robust automated testing
• Used by Netflix, Amazon, etc.

Anatomy of a CI/CD Pipeline

Typical Pipeline Stages:

1. Source — Code committed, pipeline triggered
2. Build — Compile code, create artifacts
3. Test — Run automated tests
4. Security Scan — Check for vulnerabilities
5. Deploy to Staging — Deploy to test environment
6. Integration Tests — Test in realistic environment
7. Deploy to Production — Release to users

Pipeline Visualization:

Commit → Build → Unit Tests → SAST → Package
                                        ↓
                    ← Staging ← Deploy Staging
                    ↓
              Integration Tests
                    ↓
              Deploy Prod → Smoke Tests → Monitor

Key CI/CD Concepts

Build Artifacts:

• Compiled binaries, Docker images, packages
• Versioned and stored in artifact repository
• Same artifact promoted through environments

Environment Promotion:

Dev → QA → Staging → Production

• Same artifact, different configs
• Increasing confidence at each stage

Feature Flags:

• Deploy code without enabling features
• Gradual rollout to users
• Quick rollback without deployment

Infrastructure as Code:

• Environment config in version control
• Reproducible environments
• Terraform, CloudFormation, Bicep

Types of Testing in CI/CD

Unit Tests:

• Test individual functions/methods
• Fast, run on every commit
• High coverage recommended

Integration Tests:

• Test components working together
• Database, API interactions
• Slower, run after unit tests

End-to-End (E2E) Tests:

• Test full user flows
• Browser automation (Selenium, Playwright)
• Slowest, run before production

Security Tests:

• SAST (Static Application Security Testing)
• DAST (Dynamic Application Security Testing)
• Dependency vulnerability scanning

Performance Tests:

• Load testing
• Run periodically or before major releases

Deployment Strategies

Rolling Deployment:

• Gradually replace old instances with new
• Zero downtime
• Rollback by continuing rollout with old version

Blue-Green Deployment:

• Two identical environments (blue and green)
• Switch traffic from blue to green
• Instant rollback by switching back

Canary Deployment:

• Deploy to small subset of users first
• Monitor for issues
• Gradually increase traffic
• Rollback affects only canary users

Feature Flags:

• Deploy code to everyone
• Enable feature for subset of users
• Not really a deployment strategy, but complementary

Popular CI/CD Tools

CI/CD Platforms:

• GitHub Actions — Built into GitHub
• GitLab CI — Built into GitLab
• Azure DevOps Pipelines — Microsoft ecosystem
• Jenkins — Self-hosted, highly customizable
• CircleCI — Cloud-native CI/CD
• AWS CodePipeline — AWS native

Artifact Repositories:

• Docker Hub, ECR, ACR, GCR (containers)
• npm, PyPI, NuGet (packages)
• Artifactory, Nexus (universal)

Deployment Tools:

• Kubernetes (kubectl, Helm, ArgoCD)
• Terraform (infrastructure)
• Ansible (configuration)

CI/CD for Different Platforms

Web Applications:

Push → Build → Test → Build Docker Image
    → Push to Registry → Deploy to K8s

Mobile Apps:

Push → Build → Test → Sign App
    → Upload to TestFlight/Play Console
    → Release to Store

Infrastructure:

Push → Terraform Plan → Review
    → Terraform Apply → Verify

Serverless:

Push → Build → Test → Package
    → Deploy Functions → Integration Test

Best Practices

Pipeline Design:

• Keep pipelines fast (under 10 minutes ideal)
• Fail fast (run quick tests first)
• Parallelize where possible
• Cache dependencies

Testing:

• Aim for high unit test coverage
• Don't skip tests to save time
• Test in production-like environments
• Include security scanning

Deployment:

• Use immutable artifacts
• Same artifact through all environments
• Automate rollbacks
• Monitor after deployment

Security:

• Scan dependencies for vulnerabilities
• Secrets in secure storage (not in code)
• Least privilege for CI/CD service accounts
• Audit pipeline changes

Common Pitfalls

Pipeline Issues:

• Too slow (developers bypass it)
• Flaky tests (ignored failures)
• Manual steps in "automated" pipeline
• Not testing in production-like environment

Cultural Issues:

• Blame culture around broken builds
• Not fixing broken builds immediately
• Treating CI/CD as "someone else's job"
• Skipping tests for "urgent" changes

Cloud CI/CD Services

Azure:

• Azure DevOps Pipelines (full platform)
• GitHub Actions (owned by Microsoft)
• Azure Deployment Center (simple apps)

AWS:

• CodePipeline (orchestration)
• CodeBuild (build)
• CodeDeploy (deployment)
• CodeCommit (source control)

Google Cloud:

• Cloud Build
• Cloud Deploy
• Artifact Registry

Exam Tips

For DevOps Certifications:

• Know pipeline stages and their purpose
• Understand deployment strategies
• Security in CI/CD pipelines
• Testing types and when to use each

For Cloud Certifications:

• Cloud-native CI/CD services
• Integration with other cloud services
• Infrastructure as Code in pipelines
• Container deployment pipelines

Key Takeaway

CI/CD transforms software delivery from a risky, manual process to an automated, reliable pipeline. Continuous Integration ensures code works together; Continuous Delivery/Deployment ensures it reaches users safely. Start with CI, add automated testing, then progress to CD. The goal is fast, reliable, frequent releases that let you respond quickly to user needs.