Refining the Five Percent Club

A guest post today from Aale Roos:

Knowing how Rob loves the CMDB ;) I thought I should publish my latest survey results here instead of ITSM Portal. I do a few surveys every year on different subjects. My method is to try to compress the survey in three questions (see reasons for that here). I have a small group of Finnish IT-people who read my e-mails, visit my website and also sometimes answer to my surveys, probably about 500 people. All surveys are open, I know who has answered and I can combine different surveys if the same person has answered both. This being a small country, I usually also know the person who has answered.

This time I asked about configuration management and CMBD. The report is here but it is in Finnish, sorry. My three questions were:

1) Do you have a fully operating configuration management activity (I gave a fairly demanding explanation of this in the introduction)
2) How much work it causes
3) Future plans

The first question had a number of choices of which the respondents could choose any combination. The choices were:
a) no, we do not have any system, experts take care of their domains
b) we do not have a naming standard
c) data is updated automatically without change control
d) the data does not contain or has only limited relationships
e) the system covers only assets
f) the data has not been verified or is outdated
g) we have a good, working configuration management

The results show that 13% do not have configuration management, 65% have some deficiencies and 22% have a good, working system. The most common deficiency is the lack of relationship information (50% of those who report deficiencies.)

The 22% is more than the 5% Rob estimates as the real number that truly get benefit from CMDB (the Five Percent Club) but this result is probably not representative of Finland as those who are interested enough to answer my survey are likely to be more active in trying to use ITSM methods. Compared to US, my guess is that Finland is at roughly the same level of adaption, at least this was true during the ten years I worked with HDI. At this moment 15% might be the right number but it is a moving target so there will be (has been) a time when that number is correct.

The respondents said that configuration management causes work, but not too much and 75% think they need to put more effort in the future. Only a very small minority thought it was wasted effort.

I did a similar study on Change management earlier this year, which I have published in English. I combined the results of these two surveys and it shows that having a good configuration management systems correlates strongly with not having change related interruptions in the service.

The message seems to be that having a CMDB is possible and worth it, but requires a lot of effort. Another quite important message from both of these surveys is that you get the benefits only when your processes are fairly mature.



how good does a cmdb need to be to deliver value

Fwiw, I had a customer who looked at the basic concepts behind CMDBs in the following way, which I think is the sensible approach. Whatever the CMDB is in a particular situation, it provides data that is used to drive the ITIL processes that depend on it (whether that be simple billing, change management, to prioritise the event management process based on impact of the failure of some subcomponent or some other ITIL (sub) process. In order for the CMDB to be useful, the data within it must be of sufficient quality that the supported processes 'work', on balance.

The gut estimate was that unless the relevant data needs to be >97% accurate for the process that it supported to 'work'. If the quality drops below this level, it ceases to be trusted and people just run around checking the data. In this context, 100% is the union of the set of data that represents the real world and the data that's contained in the CMDB. 97% represents 1.5% false positives (a record for an asset or relationship that doesn't really exist) and 1.5% false negatives (missing records for an asset/relationship).

If this view is correct, then it's pretty easy to ensure that the data quality is managed for the specific processes that are being supported and then to measure how much it costs to keep the data quality up.

I've measured the data quality of many CMDBs, and >90% is really tough and expensive, except to support the very simplest of processes. I've also seen plans for building CMDBs where just managing the data schema (let alone the data in the schema) would cost a comparable amount to managing the services that the CMDB models.

Most of my customers in this space started with the vision of building some sort of bottom up view of how their IT supported their business processes. That's really hard and expensive.

normalising the results I'm still at 5%

I think 22% in your survey is highly compatible with 5% as the "true" number who need a CMDB.

If we normalise that number we can reduce it because:

- as you say those who you know are already a self-selecting sample of those interested and active in ITSM
- it is also a self-selecting sample in that those with a positive experience with CMDB, with pride in their CMDB, will be more inclined to answer
- even those who did not get a good result from the CMDB will be inclined to give a more positive assessment of the outcome

Finally 22% say that have a good working CMDB but that says nothing about
a) did they actually get a nett positive value returned on their investment?
b) even if they got positive VOI, was the right project portfolio decision made? i.e. did the tech bias for cool toys skew them to work on a CMDB when there were better uses of the money that were left begging?

Taking all those into account, I'm still at 5%

Define the population PLEASE

Rob, if your population of interest is all economic entities from single person proprietorships to the Fortune 25, I think 5% is probably overstated if anything.

If your population of interest is the economic entities big enough to have IT specialists who would find your blog of interest (or mine, or Aale's), I think Aale's results are probably a good conservative read.

The 5% figure is meaningless until you put more parameters around it. Global 5000? IT spend larger than $100m? You could define the population as "organizations big enough to have full time professionals who self-identify as 'IT specialists.'" Another alternative would be "what % of self-identified IT specialists work in organizations big enough to have a CMDB."

That % I would expect to be much higher, and would have more explanatory power as to why it's such a popular topic, your efforts notwithstanding :-).

And ROI is not the point. What is the ROI on the general ledger? The HR database? The legal matters management system? The records retention system?

Charles T. Betz

A few comments

I did think about the ROI but it is really not realistic to measure it. Or put it in other words. There is no ROI in measuring the ROI of ITSM. Proper measuring can be harder than testing. Many activities are related. My results indicate that the CMDB supports good change management and reduces disruptions to the service, but it does not show that it is is necessary to have a perfect CMDB to get good results from change management.

The people who have gone through the process of implementing it think that it was worth the effort. Nearly all, who are in the process of implementing it, want to put more effort to it. I think they know what they are doing.

About the size of companies, this is Finland. The largest Finnish financial group is about 5% of Wells Fargo so we are not talking about Fortune 500.


PS Thanks Rob for the guest post.

The lower limit

Incidentaly I checked ITIL Small Scale Implementation written by Ivor Macfarlane (co-author of ITIL V3 Service Transition which defines CMDB/CMS) and Sharon Taylor (Chief Architect of ITIL V3) and it says (p47)

In many small organisations... the overheads of establishing and maintaining the configuration links will be too great

and they ought to know. Regular readers will know that this blog uses "establishing and maintaining the configuration links" as the formal definition of a CMDB/CMS, and it is hard to imagine what else Ivor and Sharon were referring to.

If ITIL should only be applied to larger organisations then let's please have that in large red letters on the cover of every book. if ITIL is to be more general, then let's make clear which components are essential and which depend on factors of scale and complexity.

The boundary

I wonder if that old "general ledger has no ROI" is a myth. Try running a business with no GL. the VOI is clearly identifiable in terms of reduced losses, controlled risks, and the decision-making value of information. Note i carefully said "value" not "return". VOI is a standard term for trying to quantify value other than direct dollar value. And i think every artifact and procedure a business builds has, or should have, VOI. That's clear and that's what i referred to. But I even think that so long as you are willing to accept avoided losses as a real $ return, then all business constructs have a ROI.

As for the scope of the population, yes I do think when we talk about ITSM we should talk about more than just the Fortune 500. Is there a lower limit beyond which the discussion is absurd? Of course. Can i crisply define that limit? No. But I have two clients who have less than fifty employees in the whole organisation and zero staff whose sole responsibility is IT. They both have clearly defined IT practices including configuration, incident and change, and a clearly defined function of IT Management. And they both - bless them - are prepared to spend real money on ITSM advice. I don't think they are below the "absurdity limit".

Syndicate content