4 of the 9 questions we have for this objective, shown with the answer and the reasoning.
Design and implement a testing strategy for pipelines
Hollins Cargo uses Azure Repos and a build validation pipeline that runs its .NET tests with the VSTest@3 task and codeCoverageEnabled: true, which publishes Visual Studio (.coverage) code coverage for every pull request. Overall coverage of the legacy codebase is 41 percent, and the team cannot raise it quickly. Management wants pull requests blocked when the new and changed lines in the pull request are less than 75 percent covered, without failing pull requests because of old uncovered code.
What should you do?
- AAdd a script step that fails the build when total line coverage is below 75 percentTotal coverage is 41 percent, so this would block every pull request regardless of how well the new code is tested.
- BSet failIfCoverageEmpty: true on PublishCodeCoverageResults@2This only fails when no coverage data is found; it does not enforce any percentage.
- CAdd a SonarQube Cloud quality gate condition on overall coverage greater than 75 percentA condition on overall coverage has the same problem as a total threshold; it measures the legacy code too.
- DAdd an azurepipelines-coverage.yml file to the default branch that sets coverage: status: diff: target: 75%, and add a status check branch policy that requires the resulting code coverage statusAzure Repos diff coverage measures only the lines changed in the pull request and posts a status, which a required status policy turns into a merge gate.
Why: Diff coverage, also called new-code coverage, is the practical way to raise quality on a legacy codebase: you hold new work to a high bar without punishing teams for history. In Azure Repos, an azurepipelines-coverage.yml settings file on the default branch defines the diff coverage target, the build posts a coverage status to the pull request, and a status check branch policy makes that status required. Pull request coverage validation currently works only with Visual Studio (.coverage) results, such as those from the VSTest task, the .NET Core test command, or the TRX option of Publish Test Results, so a Cobertura-only pipeline would not get the status. Thresholds on overall coverage are the trap, because they block every change until legacy coverage is fixed.
Design and implement a testing strategy for pipelines
Juniper Learning has a pull request pipeline that currently runs 1,200 browser-based end-to-end tests and takes 75 minutes, so developers wait more than an hour for feedback. Most failures are in business logic inside individual classes. The team wants fast, reliable feedback on every pull request while still verifying the full system before release.
Which testing strategy should you recommend?
- ARun the browser tests in parallel on more agents and keep them as the only test typeParallelism shortens the wall-clock time but keeps slow, brittle tests as the primary safety net and increases agent cost.
- BRun only load tests on pull requests because they exercise the most codeLoad tests measure performance under scale; they are slow, need a deployed environment, and do not pinpoint logic defects.
- CReplace all automated tests with manual exploratory testing before each releaseManual testing removes fast automated feedback entirely and cannot scale to every pull request.
- DMove business logic coverage to unit tests that run on every pull request, keep a smaller set of integration tests, and run end-to-end and load tests in later stagesThis follows the test pyramid: many fast unit tests give quick feedback where defects occur, and slower integration, end-to-end, and load tests run less often against deployed environments.
Why: A healthy testing strategy is shaped like a pyramid: a broad base of fast unit tests, fewer integration tests, and a small number of end-to-end and load tests. Fast tests run locally and on every pull request to shift defect detection left, while slower tests run after deployment to a test environment. Here, failures are in class-level logic, so unit tests catch them faster and more reliably than browser tests. Simply adding agents treats the symptom rather than the inverted pyramid.
Design and implement a testing strategy for pipelines
Dovetail Systems has 900 Selenium UI tests for a web client on Windows, run by the VSTest@3 task. The tests open a visible browser window and fail when no interactive desktop session exists. The full run takes 90 minutes on one agent, and the team must cut wall-clock time to about 30 minutes by spreading the tests across three self-hosted Windows agents in a pool named UITestPool.
Which two actions should you perform? Each correct answer presents part of the solution.
- ASet strategy: matrix with three identical entries named run1, run2, and run3A matrix creates copies of the job with different variables, so every copy would run all 900 tests rather than a slice.
- BMove the job to the windows-latest Microsoft-hosted image to get an interactive desktopMicrosoft-hosted agents support only headless browser testing, so tests that need a visible browser window fail there, and the scenario requires the self-hosted UITestPool anyway.
- CConfigure each agent in UITestPool to run interactively with autologon instead of as a Windows serviceUI tests that need a desktop session require an agent running interactively; an agent running as a service has no interactive desktop.
- DAdd continueOnError: true to the VSTest stepThis only lets the job continue after failures; it neither provides a desktop session nor reduces run time.
- ESet strategy: parallel: 3 on the test job so VSTest slices the tests across three agentsWhen a job uses the parallel strategy, VSTest distributes test cases across the job instances, so each agent runs about one third of the tests.
Why: Configuring test agents is part of implementing tests in a pipeline. Tests that drive a visible browser or desktop need an agent that runs in interactive mode, typically with autologon, because a service-mode agent runs in a non-interactive session. To reduce duration, the parallel job strategy with the VSTest task performs test slicing, distributing test cases across agents based on count or past run time. A matrix strategy is the classic trap: it multiplies jobs for different configurations, so each job would execute the entire suite.
Design and implement a testing strategy for pipelines
Granite Payments builds a .NET 8 solution on ubuntu-latest Microsoft-hosted agents. Unit tests run with DotNetCoreCLI@2 test. The team wants line and branch coverage displayed on the pipeline run's Code Coverage tab as an HTML report, using cross-platform tooling with no Windows-only collectors.
What should you configure?
- ARun the tests with the VSTest@3 task and codeCoverageEnabled: trueVSTest is a Windows task that produces .coverage files from the Visual Studio collector, which conflicts with the Linux agent and cross-platform requirement.
- BAdd arguments: --collect "XPlat Code Coverage" to the test step, then add PublishCodeCoverageResults@2 with summaryFileLocation: $(Agent.TempDirectory)/**/coverage.cobertura.xmlThe coverlet XPlat collector emits Cobertura XML on any OS, and PublishCodeCoverageResults@2 turns that file into the Code Coverage tab report.
- CAdd PublishTestResults@2 with testResultsFormat: CoberturaPublishTestResults handles test outcomes, and Cobertura is a coverage format, not a supported test result format.
- DAdd --collect "XPlat Code Coverage" and publish the coverage folder with PublishPipelineArtifact@1The report would only be downloadable as an artifact; it would not appear on the Code Coverage tab.
Why: Code coverage in Azure Pipelines has two halves: a collector that measures coverage during the test run, and a publish step that renders the result. For .NET on Linux, the coverlet XPlat Code Coverage data collector writes coverage.cobertura.xml, and PublishCodeCoverageResults@2 accepts Cobertura, JaCoCo, and other common formats to build the HTML report. Java projects follow the same pattern with JaCoCo output from Maven or Gradle. Mixing up test results and coverage results is the most common distractor.