Tools & Techniques - Key Performance Indicators
IntroductionTo date, I've facilitated senior level reporting on the performance of security driven activity in almost every position I've held. For the best part, this has been a green-field requirement which has meant that I've been able to set out and make a case for reporting that drives improvement in the real-world security posture (RWSP) of my charge from the get-go. I believe, no matter what else is going on, that if you can inform the changes and behaviours that lead to your organisation protecting itself, you've had a good day. Secondly to driving improvement, KPIs and metrics provide your leadership team with the insight and visibility into the efforts and valuable work of your security team which might otherwise be hidden from them.
There is no shortage of materials and musings about the importance of security KPIs however, despite this, many organisations struggle in a number of ways to define and/or employ them. I thought then I'd share some thoughts on the subject and set out some simple parameters for defining KPIs.
What is a Key Performance Indicator (KPI)?
Lets stars with a definition:
- A Key Performance Indicator (KPI) is a measurable value that demonstrates how effectively your organisation is achieving key objectives. To this end then, organisations can use KPIs to evaluate their success at reaching targets.
- A KPI is not the turn-key reporting from every product or service managed by the security team or that the security team has an interest in...
- A KPI is not a dashboard representing the collective outputs from every client, server and device across your estate (aka dashboard fever)...
Understanding what a KPI should be and why/how it should be used compared with the raw data that might inform it is vital. KPIs should be specific to your organisation and some time and effort should be needed to design them. For a lot of organisations though, the need for KPIs has become confused with the need for teams, products and services to demonstrate their own intrinsic value. Vendors too recognise the need to demonstrate their value and typically include 'Executive' or 'Management' reporting or dashboards which typically become presented as KPIs.
Getting Started - Defining your Objectives
To develop effective KPIs you first need to identify your objectives. The amount of steer and support you receive from your leadership team is fundamentally important. A lack of management commitment will not only effect your ability to define KPIs but also your ability to make the necessary changes in their pursuit. KPIs should reflect specific business objectives. If you don't have the support of the business then KPIs, and the work that informs them just becomes another *thing* security tries to get people to do.
KPI ParametersKPIs should comply with these parameters:
- KPIs should be aligned with the strategic goals of the business. They should have a clear, documented objective (i.e. "Lowering non-compliant hosts below 1% total hosts will reduce our vulnerable surface area in line with the Executive's risk appetite").
- KPIs should be attainable and their pursuit achievable. KPIs should not be aspirational or beyond the reach of your current capabilities.
- KPIs should be acute and should help keep everyone on the same page and moving in the same direction. They should be approved by stakeholders before being tracked and reported on.
- KPIs should be accurate, based on trustworthy and reliable data sources.
- KPIs should be actionable and should provide insight and information into the business it's processes.
- KPIs should be alive and reviewed, updated or amended periodically in line with the business objectives or emerging situations.
In addition to being customised, KPIs need to be developed over time as your organisation's objectives change and as the pursuit of these effects how security considerations and requirements adapt. Pursuing poorly conceived KPIs, or KPIs that are impossible, unrealistic or outdated can result in significant cost and disruption. Pursuing the wrong KPIs is detrimental and can even instigate 'bad' behaviours. If you don't receive feedback the relevance of KPI sets, ask...