May the Indicators be with You

May the Indicators be with You  ...  or it it "may the force be with you?"

In any case, the sentiment expressed by the phrase made famous by the Star Wars movies is also expressed by the saying, "you should pay attention to what's going to matter most, and ignore everything else."

In the Star Wars movies the narrative explanation was evidenced in scenes where Luke Skywalker ignored the jet fighter's guide-by-wire systems and went instead with his gut instinct and intuition and used joy-stick controls without guidance-controls.

The dichotomy between yin and yang, white and black, and loss and success plays itself out daily for cyber operations where we are used to measuring almost everything digital that can be measured.

We find ourselves drowning in too much information.

Let's find out why and what can be done about it.

Why We Will Always Have TOO MUCH Data

We've spent the past few decades instrumenting security relevant events. And, we continue instrumenting security relevant events in the hope the latest tech experiments - here's looking at you RPA, AI, Big data, Blockchain, IoT, etc. - can be tamed or at least made known.

We are drowning in information now, and most of our talk about key cyber indicators focuses on “performance” indicators is an attempt to focus on just the "right" set of information from among the sea of data.

But we find the interpretation of what the words "key", “performance” and "metrics" means depends on who’s looking at it and what’s “key” for one audience is clearly not "key" not for another.

The beauty of KPIs are in the eye of the beholder

What's Ketchup to you is Catsup to someone else.

The same is true for KPIs.

For example, key performance indicators for CFOs include such things as debt free cash flow, return on assets, cost of capital, and billing to expense ratios among others.

CEOs  listen to their CFO's but are more focused on price to earnings, earnings per share, and a slate of upcoming customer meetings.

COOs and operational line managers focus on other key metrics, those relevant to the business. These might be new admissions and returns (education), customer retention, same store sales (retail), refinery utilization (petroleum), peak demand load (utilities), bookings (any industry), freight on board (trucking), and average revenue per user (telecommunications and software publishers) among others.

One of the factors to selecting successful key performance indicators (KPIs) that drive organizational behavior is determining how the information will used, how it will be communicated and with whom.

Without effective communication, there really is no reason to go to the trouble of collecting information and then presenting it as key indicators.

Recommendations about KPIs

A few of recommendations to make sure your KPIs are useful include:

  • Relevance: Link your KPIs to your core digital business strategy
  • Clear: Clearly and simply define the KPIs for everyone
  • Explain: Make sure people know why the KPI is useful and what its purpose is
  • Trends: Use trends to provide meaning and context for everyone
  • Timescales: Use time slices for easier comparison - from past to the future trends
  • Benchmarks: Make comparison meaningful using benchmarks
  • Source: Document the sources, limitations and assumptions
The Alphabet Soup in IT

When it comes to IT, there are numerous performance indicators, and the value of indicators being used in IT depend on who’s looking at the metrics, just as with other functions in the organization.

Most CIOs we interact with say they want to see five to eight KPIs - and no more.

Anything more than eight KPIs is a waste of their time according to the many CIOs.

There were a few outliers looking for more than this, but they are the exception, not the norm.

Reducing what's going on in IT to eight KPIs does not leave a lot of room for explanation (detailed insight comes from drill-downs).

Instead the five-to-eight high-level KPIs are the summary of where IT is today, and where IT is headed given the current trajectory.

KPIs commonly cited by CIOs include:

  • Percentage of the IT portfolio allocated to delivering value
  • Cost per revenue
  • Ratio of the IT portfolio allocated to change the business versus run the business
  • Percentage of first call resolutions completed
  • Ratio of unresolved to resolved security incidents
  • Percentage of risks accepted without controls
  • Project cost to budget

In addition, we also find other indicators being used to manage different functional responsibilities.

These include:

  • Key Value Indicators (KVIs) cited by CIOs
  • Key Risk Indicators (KRIs) cited by CISOs
  • Key Controls Indicators (KCIs) cited by IT auditors, compliance, information security and business continuity managers

These additional indicators are being used to quickly identify where the organization is and where its headed - given the trajectory - for value being delivered by IT, the level of risk being incurred by the organization, and whether the controls are adequate to the risks.

Advice from Others

Whatever indicators you select, realize that you'll be measuring sources that make up the indicator, and depending on the indicator the frequency of measurement could vary from daily to monthly.

One of the sagest pieces of advice we received is that if you’re not going to report on it, don’t measure it.

Also, think through the impact of indicator selection and updates to indicators.

Experience says that explaining why an indicator is being used is easier than explaining why it’s being updated or changed.

No one we talk with with claims to have silver-bullet lists of indicators: although a many of the people we talked with claim some indicators are worth more than others because of their acceptance and utility.

What’s really interesting about KPIs is that some organizations use indicators to almost make decisions by drive-by-wire; whereas others use indicators but appear to be very cautious about ceding too much intelligence to “indicators” and instead reserve the right to get up and walk around the organization and talk with people, and then make intuitive judgments - that only people can make - to guide decisions.

Whether it’s guide-by-wire, walkabouts or some combination of how your organization uses indicators, may the indicators be with you.

Related Research

Why Managing Risk Does Not Compute

How Mature is Your Risk Profile

Cyber Security Metrics and Measures

Featured Research

2018 Cyber Security Spend Report

The 2018 Cyber Security Spend Report focuses on global spend today through the coming five years. Did you know that global spend on cyber security is: More than $103 billion today Will exceed $130 billion by or before 2022 Today’s top two spend categories account for almost 50 percent of spend by enterprises today. The

Innovative Digital Business Models

Is your company making money using some of the innovative digital business models of the future, today? If not, it is important to understand what some of the new digital business models are, what they are best used for, who should consider using them, why they are important, and when they should be used. Not