Mastering Matrix Builds for Multi-Platform Testing

In the world of modern software development, your code rarely runs in a single environment. A Java application might be deployed on Linux servers using OpenJDK 17, while developers work on macOS using Oracle JDK 21. Ensuring your code works across all these variations is critical. This is where GitHub Actions Matrix Builds come into play.

What is a Matrix Build?

A matrix build allows you to run a single job multiple times using different variables. Instead of writing ten separate jobs for ten different environments, you define a "matrix" of configurations. GitHub Actions then automatically creates a job for every possible combination of those configurations.

This approach is the backbone of efficient Continuous Integration (CI), allowing for massive parallel testing without duplicating workflow code.

How Matrix Strategy Works

The matrix strategy is defined under the strategy key in your workflow YAML file. GitHub takes the variables you provide and calculates the "Cartesian Product."

[OS Options: Ubuntu, Windows] x [Java Versions: 11, 17, 21]
Resulting Jobs:
1. Ubuntu + Java 11
2. Ubuntu + Java 17
3. Ubuntu + Java 21
4. Windows + Java 11
5. Windows + Java 17
6. Windows + Java 21
    

A Basic Matrix Example

Here is how you would define a matrix to test a Java application across multiple Operating Systems and JDK versions:

jobs:
  test:
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, windows-latest]
        java-version: [11, 17, 21]
    
    steps:
      - uses: actions/checkout@v4
      - name: Set up JDK
        uses: actions/setup-java@v4
        with:
          java-version: ${{ matrix.java-version }}
          distribution: 'temurin'
      - name: Run Tests
        run: ./gradlew test
    

Visualizing the Matrix Flow

To better understand the execution flow, consider this diagram representing the job spawning process:

Trigger (Push/PR)
      |
      v
Strategy Definition (Matrix)
      |
      +-----------------------+
      |                       |
[Job: Ubuntu / JDK 11]  [Job: Win / JDK 11]
[Job: Ubuntu / JDK 17]  [Job: Win / JDK 17]
[Job: Ubuntu / JDK 21]  [Job: Win / JDK 21]
      |                       |
      +-----------+-----------+
                  |
         Aggregate Results
    

Advanced Matrix Features: Include and Exclude

Sometimes, you don't want every single combination. Perhaps a specific library doesn't support Java 11 on Windows. You can use exclude to remove specific combinations or include to add unique configurations.

Using Exclude

This configuration will run all combinations except for Windows with Java 11.

strategy:
  matrix:
    os: [ubuntu-latest, windows-latest]
    java: [11, 17]
    exclude:
      - os: windows-latest
        java: 11
    

Using Include

You can also add extra variables to a specific combination. For example, adding an experimental flag only for the latest Java version on Linux:

strategy:
  matrix:
    os: [ubuntu-latest]
    java: [17, 21]
    include:
      - os: ubuntu-latest
        java: 21
        experimental: true
    

Real-World Use Cases

• Cross-Platform Libraries: If you are building a CLI tool or a library, you must ensure it compiles and passes tests on Linux, macOS, and Windows.
• Database Compatibility: Run your integration tests against multiple versions of PostgreSQL or MySQL using a matrix of Docker images.
• Dependency Testing: Test your project against different versions of a major dependency (e.g., Spring Boot 2.x vs Spring Boot 3.x).
• Browser Testing: For web projects, run Playwright or Selenium tests across Chromium, Firefox, and Webkit simultaneously.

Common Mistakes to Avoid

• Job Explosion: Be careful with the number of variables. A matrix of 5 OSs, 5 Java versions, and 5 Databases creates 125 jobs. This can quickly exhaust your GitHub Actions concurrency limits and billing minutes.
• Hardcoding Values: Always use ${{ matrix.variable_name }} in your steps. Hardcoding a version inside a matrix job defeats the purpose.
• Ignoring Failures: By default, if one job in the matrix fails, GitHub cancels all other in-progress jobs. Use fail-fast: false if you want to see the results of all combinations regardless of individual failures.

Interview Notes for Developers

• Question: How do you limit the number of parallel jobs in a matrix?
• Answer: You can use the max-parallel key under the strategy section to define the maximum number of jobs that can run at the same time.
• Question: What is the "fail-fast" strategy?
• Answer: It is a setting (enabled by default) that cancels all remaining jobs in a matrix if any single job fails. It saves runner minutes but can be disabled to debug multi-platform issues.
• Question: Can you add metadata to a specific matrix combination?
• Answer: Yes, using the include keyword, you can add specific key-value pairs to a particular combination of the matrix.

Summary

Matrix builds are an essential feature for robust CI/CD pipelines. They allow you to scale your testing horizontally across different environments with minimal configuration. By mastering include, exclude, and fail-fast, you can create a highly efficient testing suite that ensures your software is truly cross-platform compatible.

In the next lesson, we will explore how to optimize these builds further using Caching Dependencies in GitHub Actions to speed up our execution times.