An ITIL V3 certification exam multiple-choice question debated

Should ITIL certification exam multiple-choice questions test one's knowledge of the holy writ of the ITIL books or one's knowledge of the principles of ITIL or one's skills in mental manipulation and logic? I'm told by someone with photographic memory that one of the ITIL Foundation exam questions looks something like the following, which is quite similar to a sample question we discussed before.

Order the following CSI implementation steps in the correct order of the PDCA (Plan-Do-Check-Act) model:

1) Assign roles and responsibilities to work on CSI activities
2) Measure and review CSI plan objectives are being met
3) Identify CSI requirements, objectives and scope
4) Implement CSI enhancement

a) 1-3-2-4
b) 2-3-4-1
c) 3-1-2-4
d) 3-4-2-1

Now remember I'm relying on someone's memory here so this whole discussion may be totally off beam. Maybe someone else can confirm spotting this particular question? And I have in fact jumbled the answers in case some idiot decides to just memorise "a)". It is the principle we are discussing here.

Also bear in mind that a whole new syllabus and set of questions came out on May 1st. Reminder: make sure your training provider is using the updated samples and teaching to the updated syllabus!! The inside word is that they are better - we'll see just how much better. I'd love to know what adult-education expertise was engaged to consult in the design of each version of the syllabus and the exams - I have my guess.

I have two comments:

First, "Assign roles and responsibilities to work on CSI activities" is an extremely confusing and ambiguous thing for someone to work out after a couple of days of Foundation blast.
Remember the issue is should ITIL certification exam multiple-choice questions test one's knowledge of the holy writ of the ITIL books or one's knowledge of the principles of ITIL?
If it is a test of basic principles then I have to say I could place (1) in either PLAN or DO or ACT. I would eventually get it right by a process of elimination after working out the other three (see my technique for multiple choice exams).
If it is a test of rote learning, then the sacred book on page 113 says

PLAN:
“Framework of management roles and responsibilities"

DO:
“Documenting roles and responsibilities”
“Allocation of roles and responsibilities to work on CSI initiatives”

ACT:
“Updating… roles and responsibilities”

so yes the answer is there but I challenge any student except my friend with the photographic memory not to get confused.

My second comment: what an over-complicated question. Are we testing knowledge of PDCA or the ability to mentally map sequences to each other. Here's my rewrite:

Assign the following CSI implementation steps to the correct steps of the PDCA (Plan-Do-Check-Act) model:
1) Assign roles and responsibilities to work on CSI activities
2) Measure and review CSI plan objectives are being met
3) Identify CSI requirements, objectives and scope
4) Implement CSI enhancement

a) PLAN-CHECK-ACT-DO
b) ACT-PLAN-DO-CHECK
c) CHECK-PLAN-DO-ACT
d) ACT-CHECK-PLAN-DO

Why make it a test of mental gymnastics?

I'll be writing on this in greater depth soon

Comments

I am in total agreement that

I am in total agreement that all the ITIL exams are an exercise in passing a fairly useless exam.

In fact the general failure in the industry to actual successful ITIL adoption is surely not helped by this, I mean it's understandable that consultants who have an existing deep practical knowledge are ploughing through this process just to get certified, it's mundane and could be improved, but to be honest I think I see that in all industries when a new certification comes out.

What’s worse is that when embarking on an ITIL adoption one of the first recommendations made to senior management is: "you should take an ITIL foundation course" where you will spend 3 days learning how to take and pass an ITIL foundations exam - Just what your average CEO needs!

You then have a whole level of management who have not had ANY basic training on ITIL and think they have, or you have a furious team of people who have seen through this charade and really don't trust the entire process.

I think we need to seriously look at why adoption success rate is so bad, this shoddy training and certification program should come under the microscope.

ITIL Lifecycle exams & courses.

I've been in IT Service Management for over 10 years mainly in Operational roles, Incident, Problem, Change and Release management - for very large global companies. I completed my foundation certificate in 2000, bridging last year and am currently completing the Lifecycle exams - I've passed Service Transition and Continual Service Improvement but failed Service Operations on two occasions.

I was surprised to receive exactly the same exam questions for SO!!

My thoughts on the exams....

There a very, very few practice intermediate exams.

During the Service Operations course we were given a 'Capability' practice exam because there were no exams available! - not helpful really!!

What really annoys me is the lack of feedback from the ISEB - Someone complained during the CSI exam that one of the questions was 'a trick', the other 10 attendees, me included, thought one of the choices was 'odd'. A note was made by the official but I doubt we will get feedback.

Surely, ITIL is based on CSI - but there doesn't seem to be any as far as ISEB is concerned!!

Courses...

The courses are 3 days in length but I'm sure could be condensed into two - as I would estimate 4 or 5 hours is taken up with the trainers 'passed triumphs'. Some are relevant most are not!!

I'm dreading getting the same trainer for another Lifecycle course - it was bad enough hearing their stories the first time!

T

Skep, this is the most

Skep, this is the most common complaint that comes from mgmt/exec participants in foundations-level courses I'm connected to. Thankfully they still see the value of the framework in spite of the idiosyncrasies of the exam (their term for this isn't as gentle). Frankly, your example question isn't even one of the bad ones.

ITIL used to be (and still retains this value beneath the shiny new veneer) a framework to develop common language around core ITSM terms like incident, problem, etc... These days, that language litmus test is extended into bookish questions like the one you mentioned above, as well as others that are PROFOUNDLY worse. I would defend the question you reference above, but would also suggest that 5-10% of the foundations level questions I've been exposed to are indefensible. You've taken this exam now, so I expect you have some first-hand experience with this. The new syllabus has not brought improvement, and is arguably worse due to the 'newness' of the questions ...

The intermediate exams are so rife with first-level errors (mis-marked answer numbering/lettering, inaccurate references from case study pages to exam pages, and glaring grammatical errors like missing/duplicate words, etc...) that I believe they weren't seriously evaluated on that level let alone the level you're talking about here.

I sure am glad the powers that be are focused on launching the new complementary certification courses and the sure-to-be-a-disaster ITIL tool certification programme. There is so much real and potential value in this guidance. It's tragic, but the good ship ITIL is at sea without a rudder.

P.S. The real world snickers when they're taught about the SKMS and takes notes and engages when discussing the Service-V Model.
Question - Which one do you think is important enough to still be part of the foundations syllabus and warrant a test question?
Answer - the one that someone wants to sell you.

Testing ITIL Understanding or How to Pass ITIL Exams?

Morning All Skeptics

I have to admit that the ITIL testing regime is something that has amazed me. In my last role I spent a year on and off teaching Foundations (v2). Well that's not quite correct, we used the framework of a Foundations course to teach the students how to pass the Foundations course exam. We had a list of exam hints on the tricks the examiners played with questions, and in the course of the three days we made sure the students saw as many of those tricks as we could. When I wanted to include - how ITIL really works segments I was told '..no focus on passing the exam..'.

I agree totally the exams have become a means by which the examiners prove how clever they are and can trick the students. It does not prove how well the students understand the core principles of ITIL.

You had to wait until you got to Practitioners or Manager to show that. At least then you could explain your thinking and provided you were not too far into reality and still within the guidelines of the holy writ, you became anointed. Funny I know people who could cite ITIL chapter and verse but they could not pass some of these more practical exams for 3 tries!

But wait, then there is ITIL v3 - they have applied the same mechanism to the Expert Bridging course - 20 multiple choice questions - on some of the most esoteric inane pieces of the framework!!! I would have expected a written exam - explain how this new stuff (the lifecycle) works? - but no they trotted out some of the inanities from Service Strategy and asked how this explained Gods place in the universe! - I did pass but I was so angry with this it took two tries!

I think they whole issue needs to be rethought - what are we training people to achieve? so what are we trying to test for?

We are now getting higher failure rates - this means more money from resits... so is it in the interests of the examiners to make it harder as there are more fees to be paid? I know one provider we use seems embarrassed by the higher failure rates - it could be argued that the results show that they have dropped their standards.

I could go on, but my skepticism is evident.

Miles N

Testing ITIL understanding or how to pass exams

Simple but sad, the ITILV3 exams do try to test ITIL but do not test how to manage services; throughout their lifecycle or anywhere else for that matter.
ITILV3 exams are almost entirely ITIL exams -by design or by default.
ITILV3 exams (despite what you may be told) do not provide the rigour you may expect from a Service Management exam based on ITIL. They are unlike the ITIL V2 Managers or Practitioners which (mostly) are ITIL based Service Management exams.

A little story
Over the last 14 months I have experienced all twelve 12 ITIL V3 exams. I passed ten first time, failed gloriously two of them; re-took them and even passed one with perfect scores –why I passed or failed I do not know nor do I care.
Some exam questions were really very good, well written and did make me think. The majority were banal; in many cases I did not need any knowledge just the common sense of a dim glow-worm.

To prepare for the 12 exams I read the syllabus and read just the relevant bits of the books -mostly just scanned them. I doubt whether I spent more than 20 hours in total over 14 months doing this. So much for the suggested 250 hours minimum (23 credits from 6 exams)

I did not take a course -I had "Gods rights" although sadly, not “Gods place”. Like many others in a similar situation, we were not mandated to attend a course. Sad to say, I doubt very much whether a training course would have helped me if I had done so. Mainly because the intermediate and lifecycle exams (from my experience) do not focus on the “why do it, what to do and how to do it” –despite what the syllabus suggests.

Good training providers prefer to get people to think about “why do it, what to do and how to do it” but whilst the exams are not focussed on this outcome , it will soon be yet another course on “what did the books chapter writer say should be done in Chapter 6. 4.5.1 third sentence” ‘because that is what the question setter has based the question on –and nothing else

Like Miles I was “angry” and like “visitor” I also suffered poorly written questions with “Scenarios that seemed to have little relevance”. Seems that the foundation faults have propagated to the intermediates

An Exam Hint:
For most questions (where I passed first time) I just answered the question first and then checked against the scenario. I made very few changes and finished all exams in less than 60 minutes.
If any of you are taking exams soon, you could try this approach as it makes everything so much simpler.

Based on my experience I think must be a better justification to state why people must attend an accredited course; the exams do not examine anything much more than simple regurgitated book knowledge applied to a basic scenario. They are little more than Foundation on small dose testosterone

I know there was a lot of pressure to release the exams (sounds like Release Control and Validation) and “not much time for testing” and the less experienced question setters are still learning the trade and subject, however whilst the questions are just set and reviewed by their peer group and not reviewed by those who need to teach, learn, apply and understand the concepts we are likely to remain in this state. I am encouraged by proposed initiatives to resolve this

At least in the old V2 exam format (if you could string some sentences together) you did have to “prove” you knew what you could do, would do and how to do it. Shame the new ITIL V3 ones are based on “If Janet and John do this today what should Sharon and Goldilocks do on a sunny day?”

In summary,
I think that the question setters need to go back to first principles and actually review the way they are applying Blooms taxonomy to the questions –it looks like the question setters don’t understand this very well.

If I were short-listing a candidate for a job and a qualification was important I think the ITIL expert would be the goal, however someone with a V2 Managers exam or V2 practitioners would always be ahead of anyone with a V3 intermediate. S/he can always read up on the other areas. I say V2 because there is some testing of the practicalities –not a lot, but some.

Perhaps an Oral exmination might help, but who would take it, who would examine it?

Would anyone seriously base their Service Strategy wholly on the ITIL book guidance -good though the guidance may be? Is it those who want to see another major (Government?) project disaster but can say we "followed process, procedures and guidlelines" so cannot be blamed

That’s my rant for a while –not a big rant, perhaps I will get wilder and rantier after another good refreshing sleep
Nairda

Five glaring ITIL exam issues persist

For those who have had to suffer my blogging on this previously, a small apology. For the rest of you who either sit exams, create exams, or perhaps deliver classes including exams - stand up for your students and your profession!

ITIL exams suffer from five glaring issues that have persisted over their entire life:

1) the exams lack 'control questions' - these are questions that are being tested, do not score, but are included without the knowledge of the student to QA, repair, or sample student methods and know-how

2) the exams lack the number of questions required to properly test a student's knowledge - especially at the Foundation level. Skep - can you help me point back to an earlier blog on 'somethings wrong with ITIL math here?' If there are 35 subject areas (processes by another name as listed in ITIL Qualification Scheme explanation), and as many as 10 topical areas within each subject area (goals, activities, inputs, outputs, problems, benefits, key artifacts owned etc).... and Blooms level 1 is assumed - (what is this, where does it belong), we should be much nearer 150-200 questions. Why aren't we?

3) the length of study time is more aligned with the (previous) business needs of the education providers (ATOs) than the student. ITIL V3 effectively tripled the scope yet originally reduced the study time by 2 hours, only to add those back on after the first review. The Foundation class is a four day event - well 3.5 plus exam prep. In this format the student will truly get a better grounding. The shorter the duration the more it must become an exam coaching exercise.

4) The subsequent levels of certification require those involved (trainer and trainee) to revisit Foundation level concepts at length in class - why? I would much prefer for my students to be required to grab their old Foundation class book and review it, and be prepared to attend a class that adds INCREMENTAL knowledge, not one that fills in gaps. The result of the current approach is more exam coaching.

5) There is no third-party test of exams that I know of - I for one have volunteered my own time and that of my clients under non-disclosures to sit planned (live) exam papers and offer a critique or guidance. There is a distinct feeling that exams are written and tested by persons who lack the experience of designing exams.

Meanwhile, training companies who should know better - or demonstrate they care more about their student welfare than revenue. It seems we still find ourselves apologizing in class for 'awkward' samples that bear no resemblance to the real (no I am not asking for a simpler exam for US folks - just consistency and quality!), and after the exam for a handful of questions that were off syllabus, required grammatical gymnastics, or were deliberately so deep in the paragraph text of a core book as to be impossible to prepare for...

As I started - small apologies for comments that smack of deja vu of years gone by....

So, what are we to do here to help the student - my #1 concern - the CUSTOMER and where is the representation (ATO User Group / itSMF) in all this?

Where are the ATOs and itSMF in all of this.

First of all I agree with many of your points Ian.

Your final question concerns the ATOs and itSMF.

The ATOs are an uneven bunch with a lot of different interest both locally and globally. In some countries they are fighting to keep up the quality by ensuring the right length and the right quality of instructors. In others like the US it seems to be very market driven and I tend to say that the customers gets what the pay for. I've seen examples of classes delivered in even less than three days, classes with too many students and classes delivered by non qualified instructors.

The ATOs are also struggling to get influence. Up till now all qualification issues has been handled by the ITIL Qualification Board (OGC, TSO, APMG, all the EIs, the chief examiners and itSMFI). Two sub groups has now been formed, one for the EIs and one for the ATOs. Time will tell if it will make any difference.

All exam development is done by the APMG examiners panel (http://www.itil-officialsite.com/Qualifications/ITIL3Examiners.asp).

I think the problem is lack of governance rather than an people issue. Who is doing what, who decides etc isn't clear.

itSMF is also struggling to get influence. For that purpose a sub committee has been formed called IQC (International Qualification & Certification) with representatives from all local chapters. In general the problem for itSMF is that it is an organisation based to volunteers with limited time for this work. I can only encourage all to participate on the local level in support of the representatives in any itSMFI committee (currently IPESC and iDaSC besides IQC). Local committees can make a difference to the work in itSMFI.

So long and thanks for the fish (Douglas Adams)

Thank you all for this hitchhikers guide to passing ITIL intermediates. I'll do an attempt to pass all 9 intermediates (excluded MALC) next week in 3 days time. Keep you posted on my experiences.

Thank you all too for the rich English you all use. This tought me a few words I couldn't find in the dictonairy ;-)

With regards to our dear beloved ITIL. I think it's time that OCG (as ITIL's "owner") kicks AMPG out of this game. Their performance is way below acceptable! I already was astonished that OCG accepted a complete newcomer in the ITIL world and just ignored the exam institutes ISEB and EXN who had years of experience creating ITIL exams and did a quit well job in the end (including multi language support)

The plans that APMG has set out for "switchting" PRINCE2 (2005) to PRINCE2 (2009) in 3 months time proves that AMPG are anything but a learning organism.

I've already urged in other fora for a complete redesign of IT Service Management Exams (based on ITIL) which could consist of:
- Foundation level for "techies" that gets them to grasp the advantages of ITIL and makes them enthousiast about ITIL !
- Foundation level for "managers" that makes them understand the benefits of ITIL for their company !
- a series of applicated courses and certificates for distinct jobs in IT Service Management
- a consultants/projectmanagers training that prepares consultants/project managers to assist companies to implement ITIL
- a train the trainers series that really prepares trainers to conduct proper ITIL trainings (including didactical training ! )

Well, so far my 2 cents worth.....

Syndicate content