Essential Metrics to Boost Support for Your Cybersecurity Learning Program – Part 1

Cybersecurity learning is a critical part of any cybersecurity program. Cybersecurity professionals must continuously expand their knowledge and upgrade their skills to cope with the rapid evolution of cyber threats. New skills enable organizations to confidently deploy emerging technologies related to cloud computing, remote work, frictionless e-commerce, artificial intelligence, and other innovations that power competitive advantage. And at a time when the shortage of skilled cybersecurity professionals has reached near-crisis proportions, upskilling existing employees may be the only way to fill vital security positions.

But if you are a CISO or a cybersecurity manager in charge of the learning your team gets, can you improve the effectiveness of your cybersecurity learning program without measuring its results? Can you get leaders to fund your activities without showing how training contributes to revenue? Almost certainly not.

That’s where metrics come in. Selecting and tracking the right metrics enables you to make optimal use of your resources and opportunities, channel everyone’s efforts toward the right objectives, and gain the support of your management and peers.

In this and a subsequent blog post, we’ll outline organizational and individual metrics you can use to boost support for your cybersecurity learning program. We will explore the value of four essential categories of cybersecurity learning and development metrics:

1. Metrics to optimize training activities
2. Metrics to measure the impact of learning on cyber readiness and business performance
3. Metrics to track individual development and performance
4. Metrics to assess job satisfaction and improve employee retention

1. Metrics to Optimize Training Activities

These include metrics that track training activities collect data about enrollments in training programs, courses, and other content, about completion rates, and about the number of employees who pass assessments or tests or receive certifications or other validation. Some obtain figures that reflect user engagement (such as hours spent online or in classrooms) and satisfaction with learning programs or courses. Useful statistics include both absolute numbers – such as the number of programs, courses, or modules completed – and percentages – such as completions as a percentage of activities started, or the percentage of eligible employees who finish program segments. (See table).

How these metrics are used:

Among other purposes, these metrics enable organizations to:

• Assess demand for training on specific topics
• Determine how well training activities are meeting learner expectations
• Track improvements in enrollments, completions, pass rates, and learner satisfaction. 

Of course, these metrics also allow CISOs and cybersecurity managers to understand the training patterns of security professionals in their group and new skills being acquired. This can be especially valuable if they are seeking to move existing employees into roles where recruiting external candidates is difficult. Which, come to think of it, includes most cybersecurity positions today.

These training activity statistics also provide data and baselines for many of the other metrics we will discuss below. For example, if you want to compare the performance of individuals or group that have received a certain type of training with the individuals or groups that haven’t, you need to know who has completed the relevant modules, learning paths, or courses.

How these metrics are collected:

Metrics about enrollments and completions, pass rates, and engagement can be collected in spreadsheets, learning management systems (LMS), or from the training and content provider’s administrative console. 

Learner satisfaction ratings should be obtained from surveys given throughout the learning process or when certain objectives are achieved, to ensure continuous tracking. 

2. Metrics to Measure the Impact of Learning on Cyber Readiness and Business Performance

Today, many CIOs and CISOs meet regularly with CEOs and discuss cyber readiness with boards of directors. Yet they still find it challenging to come up with data that connects security-related activities with business objectives, and even more so to devise learning metrics that would interest non-technical executives.

Some of the other metrics we discuss in these blog posts, such as those related to job productivity and employee retention, can show that cybersecurity learning reduces costs, and cost reduction is always a popular management goal. But what else?

One approach is not to devise any new metrics. Instead, to get the attention of management, show them how training influences the key performance indicators (KPIs) or objectives and key results (OKRs) they have already been assigned (the ones that affect their bonuses). 

Obviously, organizational KPIs and OKRs will differ by industry. However, you should be able to find some that are affected directly, or through a reasonably simple chain of events, to changes in the productivity and effectiveness of the security organization. 

You can then link these to cybersecurity training in two ways: By comparing performance before and after the training of individuals, teams, or departments, or by comparing the performance of teams or departments that have received significant amounts of training with comparable groups that have not. 

(See table)

For example, if a security team finishes training and then reduces website downtime from DDoS attacks, that will have a direct effect on revenue. If a team that received training is able to quickly contain an attack that caused data breaches at competitors or in other business units of the same organization, you can estimate the savings in terms of breach notification costs, regulatory fines, and legal expenses. 

Training can also affect organization-level objectives in other areas, such as:

• Employee diversity (helping professionals advance to management and higher-level technical positions)
• New product launches (rolling out secure new applications or secure new IT infrastructure faster)
• Entering new markets (upgrading applications to meet additional security and privacy regulations)

Obviously, it is not possible to calculate the precise effect of learning and some of these outcomes, but in many situations showing a significant correlation is enough to persuade management that something important has been achieved.

How these metrics are used:

These metrics can help cybersecurity leaders and business executives:

• Justify the current investment in cybersecurity learning
• Plan for additional training to support upcoming business and technology initiatives

Most often, organizational metrics determine additional investment in a program, but in the case of cybersecurity learning programs, metrics can play another critical role as well. In part 2 of this series we will examine how metrics can help you track the development of individual cybersecurity professionals, improve their skillfulness, and protect your investment in hard-to-replace experts.

To address the specific learning and skills development needs of the enterprise, we offer a Learn Enterprise subscription. As OffSec’s answer to cybersecurity skills development at scale, Learn Enterprise can help you attract, assess, develop, and retain top cybersecurity talents through continuous skills development and knowledge acquisition.

Book a meeting with an OffSec’s learning and skills development expert to get started with Learn Enterprise.