Kill the CAB

ImageRequiring permission from a Change Advisory Board after a change is complete is absurd. A system has to be broken to need any discussion by that stage. Fix the system and kill the CAB. Its possible.

Building on my post about how Change goes away, let's look at its worst manifestation, the CAB. Maybe it's because I'm at DOES17 UK and I'm in fighting mood but I say kill the CAB, or at the very least make it fight for its existence.

Phil Green on Facebook said

    "Approval Central" - what change management can look like in large organisations. A complex series of approvals spanning multiple groups, functional areas and CABs, where everyone is empowered to say no and the large number of approvals makes no-one accountable. Yet these same organisation are keen to to DevOps! Where do you start?

Louis Stanford replied

    I once did a quick calculation of a client CAB which involved ~6 Execs and between 20-30 senior technical staff waiting for an hour (or more) for their turn to Explain Your Change. One of the most striking things about the meeting was how many people were engrossed in their smartphones until their name was called.

Or as Jon Hall put it (in an excellent discussion of this, well worth a read):

    a key goal for DevOps teams is the establishment of a high cadence of trusted, incremental production releases. The CAB meeting is often seen as the antithesis of this: a cumbersome and infrequent process, sucking a large number of people into a room to discuss whether a change is allowed to go ahead in a week or two, without in reality doing much to ensure the safe implementation of that change.

When you are ready for real change:

  1. Make development accountable. Teams should self-certify. The only people accountable for a failed change should be the people who made it.
    "People" plural because all work should be peer reviewed.
  2. Stop building defects. One function of a CAB is to reluctantly accept the defect list of a project.
    Shift left. Bake in quality and security. Deliver quality and much of the rationale for the CAB does away. It is a defensive mechanism to fix symptoms of a low quality system.
  3. Release all the code together in cadenced cycles. Resolve the dependencies when planning a cycle, and prove it at development integration stage. Identifying dependencies when the build is finished
  4. Manually reviewing completeness of a change is inefficient and error prone. Automate workflow and approvals.
  5. Manage the impact. Deploy incrementally. We should be able to release "dark" then provision users as pulled by the business. Let them do canary releases to small populations before deploying more widely.
  6. In case of the rare occasions that a release or deploy fails, do it in work hours so everybody is there to fix it.
  7. Then fix it. In some organisations, a normal change is the fastest path to production. There is no need for urgent or emergency change paths. They can deploy a fix faster than you can assemble your emergency CAB meeting.

It's all about trust. Build trust, in build teams, the automation, quality, the system.

Then make the CAB survive its own process.

  1. Every quarter (other cadences are available) make the CAB apply for its own job. What is the data on current effectiveness, errors, cost, risks, impacts on value flow, benefits.
  2. Every member of the CAB has power of veto, and if any one member says it shouldn't go ahead then kill the CAB. Don't have one.
  3. Observe the result. Modify the proposed policy and mechanism. Resubmit next quarter until it passes every member and only then does it go live.
  4. Do a post implementation review after a month focused on benefit realisation.
  5. If things are going badly wrong, you can have an emergency CAB to decide you need the CAB back.
  6. Every quarter, everyone reviews data, reflects on CAB performance.
  7. Repeat.

I suggest that in many organisations​ the CAB will narrow in scope, focusing on a few changes which are high risk because of complex dependencies across teams and systems. Eventually as we improve systems architecture, it withers. One day we kill the CAB for good.

Syndicate content