Tools & Techniques - Key Performance Indicators


To date, I've facilitated senior level reporting on the performance of security driven activity in almost every position I've held. For the best part, this has been a green-field requirement which has meant that I've been able to set out and make a case for reporting that drives improvement in the real-world security posture (RWSP) of my charge from the get-go. I believe, no matter what else is going on, that if you can inform the changes and behaviours that lead to your organisation protecting itself, you've had a good day. Secondly to driving improvement, KPIs and metrics provide your leadership team with the insight and visibility into the efforts and valuable work of your security team which might otherwise be hidden from them. 

There is no shortage of materials and musings about the importance of security KPIs however, despite this, many organisations struggle in a number of ways to define and/or employ them. I thought then I'd share some thoughts on the subject and set out some simple parameters for defining KPIs.

What is a Key Performance Indicator (KPI)?

Lets stars with a definition: 
  • Key Performance Indicator (KPI) is a measurable value that demonstrates how effectively your organisation is achieving key objectives. To this end then, organisations can use KPIs to evaluate their success at reaching targets.
  • A KPI is not the turn-key reporting from every product or service managed by the security team or that the security team has an interest in...
  • A KPI is not a dashboard representing the collective outputs from every client, server and device across your estate (aka dashboard fever)...
Understanding what a KPI should be and why/how it should be used compared with the raw data that might inform it is vital. KPIs should be specific to your organisation and some time and effort should be needed to design them. For a lot of organisations though, the need for KPIs has become confused with the need for teams, products and services to demonstrate their own intrinsic value. Vendors too recognise the need to demonstrate their value and typically include 'Executive' or 'Management' reporting or dashboards which typically become presented as KPIs. 

Getting Started - Defining your Objectives

To develop effective KPIs you first need to identify your objectives. The amount of steer and support you receive from your leadership team is fundamentally important. A lack of management commitment will not only effect your ability to define KPIs but also your ability to make the necessary changes in their pursuit. KPIs should reflect specific business objectives. If you don't have the support of the business then KPIs, and the work that informs them just becomes another *thing* security tries to get people to do. 

KPI Parameters 

KPIs should comply with these parameters:
  • KPIs should be aligned with the strategic goals of the business. They should have a clear, documented objective (i.e. "Lowering non-compliant hosts below 1% total hosts will reduce our vulnerable surface area in line with the Executive's risk appetite").
  • KPIs should be attainable and their pursuit achievable. KPIs should not be aspirational or beyond the reach of your current capabilities.
  • KPIs should be acute and should help keep everyone on the same page and moving in the same direction. They should be approved by stakeholders before being tracked and reported on.
  • KPIs should be accurate, based on trustworthy and reliable data sources. 
  • KPIs should be actionable and should provide insight and information into the business it's processes.
  • KPIs should be alive and reviewed, updated or amended periodically in line with the business objectives or emerging situations.

Developing KPIs

In addition to being customised, KPIs need to be developed over time as your organisation's objectives change and as the pursuit of these effects how security considerations and requirements adapt. Pursuing poorly conceived KPIs, or KPIs that are impossible, unrealistic or outdated can result in significant cost and disruption. Pursuing the wrong KPIs is detrimental and can even instigate 'bad' behaviours. If you don't receive feedback the relevance of KPI sets, ask...

Thoughts on Presentation 

Executive level updates are likely to form a small part in a larger set of cross-business updates. To this end, brevity and impact are essential when considering how to effectively communicate with business leaders. Visual aids can be used to convey large amounts of information quickly. Traffic lights, scales and binary measures have always served me well. Know your audience...

Popular posts from this blog

Splunk Security Cheat Sheet

Developing Leeds Scene

PCI-DSS: Your CDE may be getting bigger