4 of the 8 questions we have for this objective, shown with the answer and the reasoning.
Design and implement appropriate metrics and queries for DevOps
Wexford Capital has a GitHub Enterprise Cloud organization named wexford-cap with 140 repositories that have code scanning (GitHub Code Security), secret scanning (GitHub Secret Protection), and Dependabot alerts enabled. The CISO wants a single place to see open alert counts by severity across all repositories, filter to repositories owned by the payments team, and track how quickly alerts are remediated. Minimizing administrative effort is the deciding factor.
What should you use?
- AThe Security tab of each repository, reviewed one repository at a timeThe repository Security tab shows alerts for that repository only, so reviewing 140 repositories individually gives no organization-wide view and takes significant effort.
- BThe organization-level security overview, filtered by repository topic or custom property for the payments teamSecurity overview aggregates code scanning, secret scanning, and Dependabot alerts across the organization, supports filtering by repository topic, team, and custom property, and its overview dashboard shows alert trends and mean time to remediate.
- CAn organization webhook for code_scanning_alert events that writes alerts to an Azure SQL database for reportingA webhook pipeline into a database could support custom reports, but it requires building and running an integration that duplicates built-in reporting.
- DThe Insights tab of each repository, using the Dependency graph pageThe dependency graph lists a repository's dependencies; it does not summarize code scanning or secret scanning alerts across the organization.
Why: Security metrics are most useful when aggregated: how many open alerts exist, how severe they are, where they live, and how quickly they are fixed. GitHub security overview provides that view at the organization and enterprise levels, with filters for severity, tool, team, topic, and custom properties, and a dashboard that tracks alert age and remediation time. Code scanning and secret scanning are licensed as GitHub Code Security and GitHub Secret Protection, the products that replaced the single GitHub Advanced Security license for new purchases. Because security overview is built in, it meets the minimal-effort constraint. Repository-level pages lack the cross-repository view, and a webhook-based reporting store rebuilds a feature that already exists.
Design and implement appropriate metrics and queries for DevOps
Kingsbridge Telecom's site reliability team logs every production incident in an Azure DevOps project named NetCore as a Bug with the tag prod-incident. The operations manager wants a shared query, used by a Query Tile widget on the operations dashboard, that always counts prod-incident Bugs created in the last 30 days as a rolling window, regardless of their current state. The query must never need manual edits.
Which query clauses should you use?
- AWork Item Type = Bug, Tags Contains prod-incident, and Created Date >= @StartOfMonth@StartOfMonth resets on the first day of each calendar month, so early in a month the query covers only a few days rather than a rolling 30-day window.
- BWork Item Type = Bug, Tags Contains prod-incident, State = Active, and Created Date >= @Today - 30Adding State = Active drops incidents that were already resolved or closed, but the manager wants every incident regardless of current state.
- CWork Item Type = Bug, Tags Contains prod-incident, and Created Date >= @Today - 30@Today - 30 is evaluated each time the query runs, producing a rolling 30-day window on the creation date, and no state clause means all incidents are counted.
- DWork Item Type = Bug, Tags Contains prod-incident, and Changed Date >= @Today - 30Changed Date updates whenever anyone edits the Bug, so older incidents that receive a comment would be counted and the metric would no longer reflect incidents created in the window.
Why: Operations metrics such as incident counts depend on choosing both the right date field and the right macro. Created Date fixes each incident to the moment it was reported, while Changed Date moves with every edit. The @Today macro with an offset produces a rolling window that recalculates on every run, while @StartOfMonth is anchored to the calendar and resets abruptly. Leaving State out of the clauses keeps closed incidents in the count, and a Query Tile widget then displays the number with optional color thresholds for the operations dashboard.
Design and implement appropriate metrics and queries for DevOps
Saltmarsh Media deploys a streaming API named sm-playback with GitHub Actions to production environments. The VP of Engineering wants a delivery scorecard that follows the current DORA software delivery performance model and reports throughput, meaning how often and how quickly the team ships changes, separately from instability metrics such as change fail rate.
Which two of the following metrics belong in the throughput section of the scorecard? Each correct answer presents part of the solution.
- ADeployment frequency to productionDeployment frequency measures how often the team deploys to production and is one of the DORA throughput metrics.
- BLead time for changes, from commit to running in productionChange lead time measures how long a change takes to go from committed to version control to deployed in production and is a DORA throughput metric.
- CChange fail rateChange fail rate is the ratio of deployments that require immediate intervention, such as a rollback or hotfix, which DORA classifies as an instability metric, not throughput.
- DSprint velocity in story pointsVelocity measures planned effort completed per sprint; it says nothing about how often or how fast changes reach production.
- ECode coverage percentage of the main branchCode coverage is a quality indicator for tests and does not measure delivery speed or frequency.
Why: DORA groups software delivery performance into throughput and instability. Throughput is measured by change lead time, deployment frequency, and failed deployment recovery time, while instability is measured by change fail rate and deployment rework rate; recovery time moved to the throughput group when DORA replaced the original four keys with the current five-metric model. Keeping the groups separate shows whether speed is improving at the expense of reliability. Note that DORA lead time starts at commit, unlike the Azure Boards Lead Time widget, which starts when a work item is created. Velocity and code coverage are useful planning and quality signals, but they are not delivery metrics, and change fail rate belongs on the instability side of the scorecard.
Design and implement appropriate metrics and queries for DevOps
Corvane Energy runs a YAML pipeline named meter-service-ci in Azure Pipelines that publishes unit and integration test results on every run. The QA lead wants a tile on the team dashboard that shows the test pass rate and the count of failed tests across pipeline runs for the last 30 days, so trends are visible at a glance. The solution must use a built-in widget.
Which widget should you add?
- ABuild HistoryBuild History shows the outcome and duration of recent runs, but a run can fail for reasons other than tests and the widget does not report test pass rate.
- BChart for Test PlansChart for Test Plans reports on manual test points and test case authoring in Azure Test Plans, not on automated results published by pipeline runs.
- CRequirements QualityRequirements Quality shows test status for requirements linked to test cases, which does not give a pipeline-wide trend of automated test pass rate.
- DTest Results Trend (Advanced)Test Results Trend (Advanced) uses Analytics data to chart pass rate, passed and failed test counts, and duration for selected build or release pipelines over a chosen period, with optional filters for branch, test file, and owner.
Why: Automated test metrics in Azure DevOps come from the test results that pipelines publish, which feed both the Tests tab and Analytics. The Test Results Trend (Advanced) widget turns that data into a dashboard chart of daily pass rate and outcome counts over time for the selected pipelines, filtered by branch, test file, owner, or test run. For per-test investigation, the pipeline's test analytics report and flaky test detection complement the widget. Build History, Chart for Test Plans, and Requirements Quality are real widgets, but they measure pipeline outcomes, manual testing, or requirement coverage rather than automated test trends.