Skip to content

QA Automation

This document outlines the blocking End-to-End (E2E) GitLab merge-request pipeline for the Android application. For local setup and debugging instructions, see Mobile E2E Testing.

graph TD
  subgraph E2E GitLab CI Pipeline
    A[Trigger: Merge request pipeline]
    B[Stage: test]
    A --> B

    subgraph Build
      B1[Job: build android app e2e]
      B1 -->|Generates APK| C1
    end

    subgraph E2E Testing
      C1[Job: run e2e tests with maestro]
      C2[Install Maestro CLI]
      C3[Start Android Emulator]
      C4[Install APK on Emulator]
      C5[Setup Backend]
      C5a[Reset DB from the e2e_seed template]
      C6[Run the ordered Maestro suite]

      C1 --> C2 --> C3 --> C4 --> C5 --> C5a --> C6
    end

    subgraph Docker Compose Services
      D[Start supporting services, including the Main API]
      C5 --> D
    end
  end

  C6 --> E[JUnit summary and diagnostic artifacts]

Pipeline Trigger

The Android E2E build and test jobs run automatically for merge_request_event pipelines. The E2E job is not allowed to fail, so it contributes a failed pipeline status when any flow fails. GitLab project settings must also enable Pipelines must succeed for that status to prevent merging.

The jobs remain available as optional manual jobs for the existing development, E2E feature, and tag workflows.

Build Stage

The first step in the pipeline is the build stage, where the Android application is prepared for testing. This involves generating an APK file that will be used in the next phase of testing.

End-to-End Testing Stage

Once the build is complete, the pipeline proceeds to end-to-end testing using Maestro, a UI automation tool. The following steps take place:

  1. Setting Up the Environment

    • The Maestro CLI is installed.
    • An Android emulator is started.
    • The APK is installed on the emulator.

  2. Backend Configuration

    • Supporting backend services are started using Docker Compose.
    • The base test database is recreated from the persistent e2e_seed template, which is initially loaded from e2e_seed.sql. Migrations and roles are then applied so services start from a deterministic state.

  3. Executing the Tests

    • The repository's .maestro/run-all.sh runner executes every supported flow in a fixed order.
    • Failure in one flow does not hide the remaining flow results.

  4. Publishing Results

    • JUnit output is published in GitLab's merge-request test summary.
    • Maestro debug output and test media are retained for one week.
    • Android logcat, Docker service state, and Docker Compose logs are captured even when the test job fails.

Supporting Services

To ensure a fully functional testing environment, essential backend services, including the main API, are started using Docker Compose. These services provide the necessary infrastructure for running the tests seamlessly.

Completion

Once all tests have executed, GitLab displays the JUnit result and makes the e2e-artifacts archive available from the job. A failed flow fails the job and pipeline; the artifacts provide the UI hierarchy, screenshots, service logs, and device logs needed for diagnosis.

By automating the build and testing process, this pipeline minimizes manual effort and ensures a consistent approach to validating the application's functionality before it progresses further in development.