How can businesses measure whether their cyber security controls are working?

Cyber security controls are working when they consistently perform their intended task, reduce the risk they were introduced to address and produce evidence that can be independently checked. Measurement should therefore combine control coverage, operating quality, security outcomes and the speed and completeness of improvement, rather than relying on whether a policy exists or whether a tool is installed.

Start with a measurable control objective. Each control should have a clear statement of what it must achieve. For example, an access control objective might be that only authorised users can reach a defined business application, while a backup objective might be that required data can be restored within an agreed recovery requirement. The objective should identify:

• the asset, process or information being protected;
• the threat or failure the control is intended to address;
• the people or systems responsible for operating it;
• the evidence that demonstrates performance; and
• the condition that would trigger investigation or corrective action.

This prevents measurement becoming a collection of disconnected statistics. A firewall rule count, for instance, says little about protection unless it is linked to an approved network design, reviewed for unnecessary access and checked against actual traffic and business requirements.

Measure coverage before judging effectiveness. A control may work correctly wherever it is deployed but still leave important gaps. Coverage measures should show which users, devices, applications, sites, suppliers and data stores fall within its scope. Examples include:

• the proportion of business accounts subject to the required authentication policy;
• the proportion of supported devices receiving security configuration and endpoint protection;
• the proportion of critical applications included in access reviews;
• the proportion of important data sets included in protected backup arrangements; and
• the proportion of identified weaknesses assigned to an owner and tracked to resolution.

Coverage must be compared with an authoritative asset and identity inventory. Otherwise, a dashboard can report excellent compliance for the devices or accounts it knows about while excluding unmanaged assets.

Assess whether the control operates as designed. Operational measures examine the quality and consistency of day-to-day performance. Suitable measures depend on the control, but may include the age of security configurations, the frequency of access reviews, the proportion of alerts assessed according to procedure, or the number of exceptions that remain open beyond their approved expiry date. For change controls, examine whether changes were authorised, tested, recorded and reversible. For email protection, examine how malicious messages are handled and whether users can report suspicious content through an established process.

It is important to distinguish a control being enabled from it being effective. Multi-factor authentication can be switched on while exempt accounts remain unprotected. Security monitoring can collect data while important events are omitted or alerts are not investigated. Backups can complete successfully while restoration has never been proven. Measurement should test the outcome of the process, not just the presence of a setting.

Use both leading and lagging indicators. Leading indicators show whether the organisation is maintaining the conditions that support protection. They can include completion of scheduled reviews, closure of exceptions, staff participation in relevant training, timely application of approved security changes and the proportion of critical assets with current ownership information.

Lagging indicators show what has already happened. They can include confirmed incidents, unauthorised access attempts that succeeded, malware detections, data-loss events, repeated control failures and findings from independent assessments. Neither type is sufficient alone. A low incident count may indicate good protection, weak detection or poor reporting. Conversely, a rise in detected events may show that monitoring has improved rather than that security has deteriorated.

Track trends and exceptions, not isolated results. A single compliant review provides limited assurance. Repeated measurements reveal whether performance is stable, improving or declining. Reports should show movement over an agreed period, recurring failure types, assets repeatedly excluded from controls and exceptions approaching or exceeding their review date. Where possible, results should be segmented by business unit, location, system criticality or user group so that weaknesses are not hidden by an overall average.

Exceptions deserve separate attention. Every exception should have a documented reason, a named owner, compensating measures where appropriate, an expiry or review date and a decision about whether the remaining risk is accepted. An increasing number of extensions may indicate that the control is unrealistic, under-resourced or poorly designed. Treating exceptions as a permanent alternative to compliance makes the reported control position misleading.

Test controls independently. Evidence supplied by the control owner is useful, but it should be supported by sampling, technical validation or a separate review. Testing might involve checking a sample of access permissions against current job responsibilities, attempting to restore selected data, reviewing whether security alerts follow the documented escalation path, or confirming that an old account is disabled after a leaver process. The method should be proportionate to the risk and should avoid disrupting production systems.

Testing should also include realistic failure scenarios. A backup test that confirms only that a job completed does not establish that usable data can be recovered. A phishing exercise does not by itself establish that email security, reporting procedures and account protections would contain a real compromise. The test should define what success looks like, record limitations and capture any remedial action required.

Set thresholds and decision rules in advance. A useful metric has a defined target, tolerance and response. For example, a control owner may need to investigate any critical asset without an assigned owner, any privileged account without a current business justification or any unresolved high-risk exception past its review date. Thresholds should reflect business impact and risk appetite rather than arbitrary dashboard preferences.

Each measure should also have a clear source, calculation method, reporting frequency and accountable owner. Consistent definitions matter: “closed” vulnerability, “active” account, “successful” backup and “resolved” incident should each mean the same thing across reports. Without this discipline, apparent improvement may result from changing the way data is classified.

Connect measurements to risk and business impact. Results should help decision-makers determine whether further investment, a different control or formal risk acceptance is required. Prioritise findings affecting critical services, sensitive information, privileged access or dependencies that could interrupt operations. A control with minor technical defects may require less urgent action than a control protecting a key business process, even if both produce the same number of findings.

Management reporting should be concise but traceable. A useful report can show the control objective, current coverage, recent test result, trend, material exceptions, associated risks, action owner and due date. Supporting evidence should be retained so that conclusions can be checked later. Review the measurement approach after incidents, major technology changes, new regulatory obligations or changes to the organisation’s operating model.

Finally, treat measurement as a continual improvement cycle. Record the result, investigate failures, identify the underlying cause, assign and verify corrective action, then retest the control. If the same issue returns, address the process or design rather than repeatedly recording the symptom. This approach gives a more reliable view of cyber security effectiveness than a one-off audit score and helps a growing organisation direct effort towards the controls that matter most.