Multi-Job Pipelines

Multi-job pipelines (also called DAG pipelines) allow you to define complex workflows where jobs can run in parallel, depend on other jobs, or execute conditionally. This is useful for separating concerns, running tests in parallel, and building sophisticated CI/CD workflows.

Important: Job Isolation

Each job runs in its own isolated environment — a fresh VM with a fresh repository clone. Files created by one job (build artifacts, installed dependencies, generated code) are not available in other jobs.

If your workflow requires passing files between steps (e.g., build → export → verify), use a single-job pipeline with sequential steps: instead.

Multi-job pipelines are best for independent workloads that don’t share filesystem state.

When to Use Multi-Job Pipelines

Use multi-job pipelines when you need:

• Parallel execution — Run multiple stages at the same time (e.g., unit tests and UI tests after build completes)
• Sequential gates — Run tests only after linting passes, deploy only after tests pass (each job rebuilds independently)
• Fan-out/fan-in — One job produces output that feeds into multiple parallel jobs, which then converge
• Conditional execution — Only deploy to production if on main branch and tests pass
• Separate failure handling — Handle test failures differently from build failures

Single-job pipelines (using top-level steps:) are simpler and sufficient for linear workflows.

When to Use Single-Job Pipelines Instead

Use a single-job pipeline (top-level steps:) when:

• Steps share build output — archive → export → verify workflows where each step needs files from the previous step
• Sequential steps in one environment — install → build → test → deploy in the same VM
• Simple workflows — most projects don’t need multi-job complexity

# Single-job: all steps share the same VM and filesystem
name: Build and Test
platform: ios

environment:
  xcode: "16.4"

triggers:
  - push

steps:
  - name: Install Dependencies
    run: bundle install && pod install

  - name: Build
    run: xcodebuild build -scheme MyApp

  - name: Test
    run: xcodebuild test -scheme MyApp -destination 'platform=iOS Simulator,name=iPhone 16'

  - name: Archive
    run: xcodebuild archive -scheme MyApp -archivePath build/MyApp.xcarchive

  - name: Export IPA
    run: xcodebuild -exportArchive -archivePath build/MyApp.xcarchive -exportPath build -exportOptionsPlist ExportOptions.plist

Linear Pipeline: Sequential Jobs

The simplest multi-job pattern: Job A → Job B → Job C. Each job installs its own dependencies (dependency caching makes this fast).

name: Gated Pipeline
platform: ios

environment:
  xcode: "16.4"

triggers:
  - push

jobs:
  lint:
    name: Lint
    steps:
      - name: Install Dependencies
        run: bundle install && pod install

      - name: Run SwiftLint
        run: bundle exec swiftlint

  test:
    name: Run Tests
    needs: [lint]
    steps:
      - name: Install Dependencies
        run: bundle install && pod install

      - name: Run Tests
        run: |
          xcodebuild test \
            -scheme MyApp \
            -destination 'platform=iOS Simulator,name=iPhone 16'

  build:
    name: Build Archive
    needs: [test]
    steps:
      - name: Install Dependencies
        run: bundle install && pod install

      - name: Archive
        run: |
          xcodebuild archive \
            -scheme MyApp \
            -archivePath build/MyApp.xcarchive \
            -destination 'generic/platform=iOS'

      - name: Export IPA
        run: |
          xcodebuild -exportArchive \
            -archivePath build/MyApp.xcarchive \
            -exportPath build \
            -exportOptionsPlist ExportOptions.plist

Execution order: lint → test → build (each job installs its own dependencies; cache hits make this fast)

Each job installs its own dependencies via pod install and bundle install. The needs: field creates gates — test won’t start until lint passes, build won’t start until test passes. Dependency caching makes repeated installations across jobs fast (typically under 10 seconds on cache hit).

Parallel Jobs: Fan-Out Pattern

Multiple jobs run after a dependency completes, in parallel. Each job is self-contained and installs its own dependencies.

name: Parallel Tests
platform: ios

environment:
  xcode: "16.4"

triggers:
  - push

jobs:
  build:
    name: Build
    steps:
      - name: Install Dependencies
        run: bundle install && pod install

      - name: Build App
        run: xcodebuild build -scheme MyApp

  test-unit:
    name: Unit Tests
    needs: [build]
    steps:
      - name: Install Dependencies
        run: bundle install && pod install

      - name: Run Unit Tests
        run: xcodebuild test -scheme MyAppTests

  test-ui:
    name: UI Tests
    needs: [build]
    steps:
      - name: Install Dependencies
        run: bundle install && pod install

      - name: Run UI Tests
        run: xcodebuild test -scheme MyAppUITests

  lint:
    name: Lint
    needs: [build]
    steps:
      - name: Install Dependencies
        run: bundle install && pod install

      - name: Run SwiftLint
        run: bundle exec swiftlint

Execution:

• build runs first
• test-unit, test-ui, and lint run in parallel after build completes
• Total time: ~10 minutes instead of ~18 (if running sequentially)

Each test job installs its own dependencies. Dependency caching makes these parallel installs fast (typically under 10 seconds on cache hit).

Diamond Pattern: Fan-Out then Fan-In

A classic pattern where one job feeds into multiple parallel jobs, which then converge into a final job. Each job is self-contained.

name: Diamond Workflow
platform: ios

environment:
  xcode: "16.4"

triggers:
  - push

jobs:
  build:
    name: Build App
    steps:
      - name: Install Dependencies
        run: bundle install && pod install

      - name: Build
        run: fastlane build

  test-unit:
    name: Unit Tests
    needs: [build]
    steps:
      - name: Install Dependencies
        run: bundle install && pod install

      - name: Run Tests
        run: fastlane test

  test-ui:
    name: UI Tests
    needs: [build]
    steps:
      - name: Install Dependencies
        run: bundle install && pod install

      - name: Run UI Tests
        run: fastlane test ui:true

  lint:
    name: Lint
    needs: [build]
    steps:
      - name: Install Dependencies
        run: bundle install && pod install

      - name: Run SwiftLint
        run: bundle exec swiftlint

  package:
    name: Package Build
    needs: [test-unit, test-ui, lint]
    steps:
      - name: Install Dependencies
        run: bundle install && pod install

      - name: Archive
        run: xcodebuild archive -scheme MyApp

  deploy:
    name: Deploy to TestFlight
    needs: [package]
    steps:
      - name: Upload to TestFlight
        if: branch == 'main'
        run: fastlane beta

Flow:

1. build runs
2. test-unit, test-ui, and lint run in parallel (each installs its own deps)
3. package waits for all three to complete and also installs its own deps
4. deploy runs if package succeeds and on main branch

This pattern is ideal for comprehensive checks before artifact creation. Each parallel job installs its dependencies independently.

Flutter Example: Multi-Platform Multi-Job

name: Flutter CI
platform: flutter

environment:
  flutter: "3.24.0"

triggers:
  - push
  - pull_request:
      branches: [main, develop]

jobs:
  analyze:
    name: Analyze
    steps:
      - name: Pub Get
        run: fvm flutter pub get

      - name: Analyze
        run: fvm flutter analyze

  test:
    name: Unit Tests
    needs: [analyze]
    steps:
      - name: Pub Get
        run: fvm flutter pub get

      - name: Run Tests
        run: fvm flutter test

  build-and-release-ios:
    name: Build and Release iOS
    needs: [test]
    steps:
      - name: Get Dependencies
        run: fvm flutter pub get

      - name: Build iOS App
        if: branch == 'main'
        run: fvm flutter build ios --release

      - name: Create iOS Release
        if: branch == 'main'
        run: |
          gh release create v1.0-ios \
            build/ios/ipa/*.ipa || true

  build-and-release-android:
    name: Build and Release Android
    needs: [test]
    steps:
      - name: Get Dependencies
        run: fvm flutter pub get

      - name: Build Android App
        if: branch == 'main'
        run: fvm flutter build apk --release

      - name: Create Android Release
        if: branch == 'main'
        run: |
          gh release create v1.0-android \
            build/app/outputs/flutter-app.apk || true

Flow: analyze → test → (build-and-release-ios + build-and-release-android in parallel on main branch)

Note: Each job includes necessary steps. Each job has its own isolated environment and must run all necessary steps (no file sharing between jobs). The Flutter SDK is automatically installed based on environment.flutter.

Android Example: Build and Deploy

name: Android Build
platform: android

environment:
  android_sdk: 34

triggers:
  - event: push
    branches: [main]
  - event: pull_request
    branches: [main]

jobs:
  test:
    name: Unit Tests
    steps:
      - name: Run Tests
        run: ./gradlew test

  lint:
    name: Lint
    steps:
      - name: Run Lint
        run: ./gradlew lint

  build-and-deploy:
    name: Build and Deploy to Google Play
    needs: [test, lint]
    steps:
      - name: Gradle Build
        run: ./gradlew assembleRelease

      - name: Upload to Google Play
        run: fastlane supply --aab build/app/outputs/bundle/release/app-release.aab

React Native Example: iOS and Android

name: React Native CI
platform: react-native

environment:
  node: lts
  xcode: "16.4"
  android_sdk: 34

triggers:
  - push
  - pull_request

jobs:
  test:
    name: Tests & Lint
    steps:
      - name: Install dependencies
        run: npm install

      - name: Lint
        run: npm run lint

      - name: Jest Tests
        run: npm test

      - name: TypeScript Check
        run: npm run type-check

  build-ios:
    name: Build & Sign iOS
    needs: [test]
    steps:
      - name: Install dependencies
        run: npm install

      - name: Install pods
        if: branch == 'main'
        run: cd ios && pod install

      - name: Build iOS Release
        if: branch == 'main'
        run: |
          cd ios
          xcodebuild \
            -workspace App.xcworkspace \
            -scheme App \
            -configuration Release \
            -derivedDataPath build \
            build

      - name: Collect artifacts
        if: branch == 'main'
        run: echo "Artifacts ready"

    artifacts:
      - ios/build/**/*.app
      - ios/build/**/*.dSYM

  build-android:
    name: Build & Sign Android
    needs: [test]
    steps:
      - name: Install dependencies
        run: npm install

      - name: Build Android Release
        if: branch == 'main'
        run: |
          cd android
          ./gradlew bundleRelease

    artifacts:
      - android/app/build/outputs/bundle/release/**/*.aab

How it works:

1. test job — Runs first, testing and linting the app
2. build-ios and build-android — Both wait for test to pass (needs: [test]), then run in parallel
3. Each build installs dependencies — Dependency cache is shared; subsequent installs are instant
4. Artifacts collected — Signed builds are available for download or deploy integrations
5. Conditional execution — Builds only happen on the main branch (if: branch == 'main')

Both jobs reinstall dependencies from npm, but because the package-lock.json hash is identical, they hit the cache and complete in seconds. Metro bundler cache is also shared across iOS and Android builds.

Tips and Best Practices

Job Key Naming

Keep job keys short and descriptive:

• Good: build, test-unit, lint, deploy
• Avoid: job1, my-pipeline-build-step, test_unit_and_integration

Meaningful Display Names

Use name: for clarity:

jobs:
  build:
    name: "Build iOS (Xcode 16.2)"  # Clear context in dashboard
    steps: [...]

  test-unit:
    name: "Unit Tests"
    needs: [build]
    steps: [...]

Avoid Circular Dependencies

This is invalid and will be rejected:

jobs:
  a:
    needs: [b]
    steps: [...]

  b:
    needs: [a]  # INVALID: circular dependency
    steps: [...]

Use Conditional Execution for Deployment

Only deploy on stable branches. Set the branch gate in the pipeline triggers block:

triggers:
  - event: push
    branches: [main]

jobs:
  deploy:
    name: Deploy to TestFlight
    needs: [test]
    steps: [...]

Alternatively, for more complex conditions, gate specific steps within the deploy job:

jobs:
  deploy:
    name: Deploy
    steps:
      - name: Build
        run: fastlane build

      - name: Deploy to TestFlight
        if: success()
        run: fastlane beta

Dependency Cache is Shared Across Jobs

RunnerHub’s automatic caching (CocoaPods, SPM, Bundler, npm) works across jobs in a pipeline. When build installs pods, the cache is saved. When test runs pod install, it gets a cache hit and restores instantly.

Important: This means dependency caches are shared, not build artifacts. Each job still needs to run pod install or bundle install — but with cache hits, this takes seconds instead of minutes.

build:
  name: Build
  steps:
    - run: pod install  # Creates cache

test-unit:
  name: Unit Tests
  needs: [build]  # Runs after build
  steps:
    - run: pod install  # Cache hit (instant)
    - run: xcodebuild test

test-ui:
  name: UI Tests
  needs: [build]  # Runs after build
  steps:
    - run: pod install  # Cache hit (instant)
    - run: xcodebuild test -scheme UITests

Keep Job Scope Focused

Each job should have a single responsibility:

• Good: one job for build, one for unit tests, one for UI tests
• Avoid: one job that builds, tests, lints, and archives

This makes logs easier to read and enables better parallelization.

Troubleshooting

Jobs not running in parallel

Check that jobs don’t have unnecessary needs: dependencies. Jobs without dependencies always run concurrently.

Pipeline fails when a job fails

This is expected. Dependent jobs are automatically cancelled. Use if: always() in a final notification job to run regardless of status.

Circular dependency error

Review your needs: fields to ensure no job depends on another that depends back on it.

Next Steps

• Learn about Conditional Steps for complex workflows
• Set up Matrix Builds to test multiple configurations
• Explore specific platform cookbooks: Fastlane, Flutter, Android, React Native