PMI-ACP Perform Continuous Improvements: Retrospectives, Kaizen Experiments and Measuring Change

Performing continuous improvements means using evidence about how the team works to choose a small change, trying it in a controlled way, and checking whether it actually helped before keeping it. It is Task 5 of the Delivery domain in the PMI-ACP Examination Content Outline, whose domain description includes "performing continuous improvement to enhance project delivery and team performance". The task has three enablers, and they form a loop:

1. Obtain metrics and feedback to drive continuous improvements. Decide what to improve from shared data and from the people the work affects, not from memory or opinion.
2. Implement improvement actions. Turn the insight into a specific change and run it as an experiment with an expected result.
3. Evaluate the effectiveness of process improvement. Compare the result with a baseline, look for side effects, and then keep, adapt or drop the change.

The outline's Process Improvement tools that belong here are Kaizen, retrospectives and intraspectives, process tailoring and hybrid models, control limits and the pre-mortem. The plan-do-check-act cycle is the simplest way to hold them together: plan a change with an expected result, do it for a fixed period, check the outcome against what you expected, then act by standardising the change or adjusting it and trying again.

Neighbouring tasks cover related ground. Choosing and radiating metrics is Manage agile metrics; finding root causes with Five Whys or a fishbone diagram, and diagnosing special causes on a control chart, belong to Facilitate problem resolution; building an improvement habit and capacity in the team is Promote agile mindset principles and practices; value stream mapping is Recognize and eliminate waste; and WIP limits as a flow technique are Optimize flow. This lesson uses those tools only where they serve the improvement loop itself.

The improvement focus should come from a combination of metric trends and feedback, because each catches what the other misses. A team that judges its process only by how the work feels can rate itself highly while the people it serves are unhappy; a team that watches only one metric can miss a problem showing up in another.

Look at several metrics together

A single healthy number can hide a deteriorating process. Suppose a platform team deploys just as often as it did last quarter, but the share of deployments that need a rollback has doubled and incidents take longer to resolve. The steady deployment count tells you the team is still shipping; the other two tell you it is shipping less safely. Read trends across several periods rather than one data point, and look at quality (escaped defects, rework, rollbacks), flow (cycle time, lead time, how much their values vary, the age of work in progress) and delivery (throughput, velocity) side by side.

Match the metric to the complaint

Feedback tells you which problem matters; metrics tell you where it comes from. A complaint about unreliable dates is a predictability problem, and predictability depends on how much individual items vary, not on how much the team finishes on average. A support team whose average ticket takes two days, but where one ticket in ten takes three weeks, cannot give a customer a dependable answer however good the average looks. The improvement belongs where the variation comes from, for example oversized items, work that waits on another team, or items that age unnoticed in progress. Fix the cause of the complaint rather than how it is reported.

Bring outside feedback in

Improvement inputs should go beyond the team's own impressions:

• Customer and stakeholder feedback from reviews, the product owner and users.
• Operational data such as support requests and production incidents after releases.
• Team health checks: short and run often enough to show a trend at the pace the team can act on, and anonymous where people are reluctant to speak up. Their value is the trend over time in how the team experiences its way of working, which gives early warning before frustration turns into conflict or attrition.

The practitioner's job is to bring this data into the team's own improvement conversation, usually the retrospective, so the team itself decides what to change. Improvement stays the team's responsibility; outside data informs it rather than replacing it.

A retrospective is the team's regular, time-boxed meeting to inspect how it worked and decide what to change, and the most widely used structure has five stages (from Esther Derby and Diana Larsen's Agile Retrospectives). Each stage exists because skipping it produces a predictable failure.

Gather facts and feelings

The data stage should combine objective data (metrics, a timeline of what happened) with subjective data about how people experienced the period. A common technique asks each member to mark their energy or mood at points along the timeline, producing a mood line. Low points that everyone shares often coincide with something no metric records, such as a tense handover or a week of late-night support, and they give the insight stage something real to work on.

Reading which stage a team is in or skipping

• Each person argues from their own recollection and the conversation cannot converge: the shared picture is missing, so the team needs gather data before anyone discusses causes.
• Every problem gets an immediate fix, the fixes are carried out, and the same problems reappear later: the team is jumping from data to action and skipping generate insights.
• Causes have been explored and the team is narrowing many ideas to a small number of owned experiments: it is in decide what to do.
• The team has just been through conflict, has changed membership, or has members who rarely speak: the session needs extra time in set the stage, whatever data is already to hand, because later stages depend on everyone contributing.

Retrospectives keep driving improvement only when they stay fresh and when each one checks whether the previous actions worked.

Vary the format

Any format repeated for long enough becomes routine: people predict what will be said, the same topics recur and energy falls. Varying the activities and the focus lets the team look at its work from new angles, for example a timeline, a sailboat or starfish exercise, a retrospective on one theme such as quality or collaboration, or one that starts from customer feedback. The aim is to renew the team's engagement in improving, so the variation should come with the same commitment to act on what is found.

Follow up on previous actions

Each retrospective should begin its data gathering by reviewing the actions from last time and asking whether they made a difference, not just whether they were done. This keeps evaluation inside the team's normal cycle, so the evidence about one experiment shapes the choice of the next.

Done is not the same as effective. Suppose a team set up a rota so a different member hosts each new joiner, aiming to cut the time before new joiners finish their first story. The rota ran all iteration, yet that time has not moved. The action is complete but it has failed, and doing more of the same will not help. The team should go back to the cause, because the original insight was probably wrong or incomplete: perhaps the delay comes from access requests, not from a lack of support.

Make actions specific

Vague intentions such as "be more careful" or "share knowledge more" cannot be seen happening, so they quietly fail. A good improvement action names a concrete, observable behaviour and a trigger: who does what, and when. "Be more careful with database migrations" cannot be checked; "run every migration against a copy of production data before it is merged" can. Keep the number of actions small enough that the team can actually carry them out.

An improvement action should be introduced as a time-boxed experiment with an expected result, not adopted straight away as a permanent rule. This is the Kaizen idea of small, continuous, team-owned change, run through plan-do-check-act.

Before starting, the team agrees:

• The hypothesis and expected result: what should change and roughly by how much.
• The measure and its current value (the baseline): how the effect will be measured, recorded before the change.
• The time box and evaluation date: for example, two iterations, then check.
• A stop condition: a guardrail that ends the trial early if crossed (for example, escaped defects above their usual range).
• A balancing measure: what the change might break.

The experiment belongs to the team. The people who do the work design it, run it together and judge it, because a change imposed from outside rarely has the commitment it needs, and a trial run by part of the team says little about how the whole process behaves. The decision to keep the change is made after the evidence is in, not promised beforehand.

Capture a baseline that is representative

No baseline means no evaluation. The baseline should describe the normal level and its normal range, taken from periods that represent ordinary working conditions; a period distorted by an outage, a hiring surge or a company event is a poor comparison point. Knowing the range matters as much as knowing the average. If merge time for pull requests normally falls anywhere between 4 and 9 hours, a trial result of 6 hours is ordinary; only a result consistently beyond that range is evidence that something changed.

Use a pre-mortem before a significant change

A pre-mortem asks the team to imagine that the change has already been made and has failed, then list every reason why. Gary Klein developed it to overcome the reluctance to criticise a plan everyone has agreed to. The PMI-ACP outline describes it as rule setting and failure analysis, and the list should be used before the change starts:

• turn the most likely causes into rules or safeguards now; and
• agree early warning signs to watch once the change is live.

The value of a pre-mortem comes from acting on it while the change can still be shaped: the safeguards go in before launch, and the warning signs give the team a reason to hold an early check if they appear. It differs from a retrospective (which looks back at work already done) and from Five Whys or fishbone analysis (which trace the causes of a problem that has already happened).

An intraspective is a short, focused inspection held during the iteration, when an experiment or a problem clearly needs attention now rather than at the next retrospective. Continuous improvement does not have to wait for the calendar.

Suppose a team has agreed to try a rule that every story gets a design check before development starts, and within days the designers are swamped and stories are piling up waiting for them. Waiting for the retrospective lets the harm continue; one person quietly changing the rule takes the experiment away from the team. The practitioner helps the team stop briefly, look at that one issue with the data it has, and adjust the experiment together, recording what it changed so the evaluation still makes sense.

Process tailoring means adapting the framework itself to the nature of the work, based on improvement evidence, and treating the tailored process as another experiment. The PMI-ACP outline lists process tailoring and hybrid models among its Process Improvement tools.

Two patterns are common:

• The cadence does not fit how work arrives. A data team that spends most of its time answering ad hoc requests from analysts, each needed within days, gains little from planning two weeks ahead, and its retrospectives keep saying so. When the evidence shows the framework and the work are mismatched, the response is to trial an approach that fits the work, for example continuous flow with explicit service classes for different kinds of request, and compare it with the current baseline. The team then keeps, adapts or drops the new approach on the evidence.
• One part of the work has a special constraint. A product team may build software that depends on a hardware test rig booked weeks ahead. The constraint affects only the stories that need the rig, so the tailoring should affect only those: for example, plan rig bookings on a longer horizon while everything else stays iterative. A hybrid model like this keeps agile practices where they fit and adds structure only where the constraint demands it.

The rule in both cases is the same: let evidence about the work drive the tailoring, change no more of the process than the evidence implicates, and evaluate the result like any other change.

Evaluating an improvement means comparing the outcome measure with its baseline and normal variation, checking for side effects, and ruling out other explanations, before deciding anything.

Measure the outcome, not the activity

Judge a change by the result it was meant to produce. Activity counts show only that the team did something. If a team started pairing to spread knowledge, the evidence is how often work now waits for one specialist, not how many hours were spent pairing. If it introduced a readiness check to cut mid-iteration clarifications, the evidence is how many stories stall for questions, not how many stories passed the check.

Separate signal from noise

One good data point inside the old range proves nothing. If lead time normally varies between 3 and 8 days and the first values after a change are 5, 4 and 6, the change has not yet shown an effect beyond normal variation, however pleasing the best of those values looks. Keep collecting data until results sit clearly and consistently outside the old range, or until the evaluation date arrives.

Check balancing metrics

Every improvement metric should be paired with a balancing metric that would show what the change could break, and it has to measure something other than the change itself. Raising a WIP limit to finish more items may lengthen how long each item takes, so watch cycle time alongside throughput. Releasing less often to save deployment effort may make each release riskier, so watch the size and impact of incidents. Asking only whether the intended benefit appeared cannot reveal the cost.

Account for confounding changes

If something else changed at the same time, the improvement cannot be credited to the team's change alone. A team that trialled a new estimation approach in the same month that an experienced engineer joined, or in a quiet holiday period, cannot tell which caused a better result. Record the other change as a possible cause and continue measuring once its influence has passed, so the effect of the team's own change can be seen on its own.

Expect a dip, but respect the stop condition

New practices often make performance worse before it gets better. This is the J-curve, which matches the chaos stage of the Satir change model; the model's stages run from late status quo through a foreign element and chaos to a transforming idea, practice and integration, and a new status quo. A team learning continuous integration may slow down for a while as it breaks work into smaller commits and fixes a fragile build. Judge the practice at the evaluation point agreed in advance and look for early signs of the intended benefit, rather than reacting to the first slow iteration. The dip never overrides a stop condition, though: a guardrail agreed before the trial is the team's own pre-commitment, so when it is crossed the trial stops and the team investigates why.

Control limits show the normal range of a measure such as cycle time, so after a deliberate process change they help the team tell a real shift from ordinary variation. On a control chart, a long run of consecutive points on one side of the mean (common run rules use around seven to nine in a row) signals that the process has changed, even if no single point breaks a limit.

Suppose a team starts splitting stories so that none is larger than two days of work, and from then on almost every story finishes faster than the old mean. Those points are not special causes to explain away; they are the improvement showing up. The team should:

• Treat it as a process shift, because the old limits describe a process that no longer exists.
• Recalculate the limits from data collected since the change, once there are enough post-change points to be meaningful. Control limits must describe the current process, because their job is to flag unusual behaviour in that process from now on.

Investigating unusual points as special causes is a problem-resolution activity covered in Facilitate problem resolution; here, control limits are used to confirm and lock in an intended improvement.

After evaluation, every experiment ends in one of three decisions, and a successful one must be recorded where the team keeps its rules so that it lasts.

Two points are easy to get wrong. First, standardise within the tested scope: if a readiness checklist helped on stories that integrate with a partner's API, adopt it for those stories, and treat any wider use (other kinds of story, other teams) as a new experiment that needs its own evidence. Other teams can learn from the result, but they decide for themselves. Second, a trial that meets its stop condition has given its answer. The team reverts, records what it learned and goes back to the cause, and any new idea starts as a fresh experiment with its own baseline.

Lock in proven improvements

Improvements decay when deadlines bite or attention moves on, and anything that depends on someone remembering or checking will fade with them. A proven improvement lasts when it becomes part of how the team defines its work:

• the definition of done, for quality steps every item must pass (for example, an accessibility check on every user-facing change); or
• the working agreement, for ways of working the team has chosen (for example, whoever breaks the build fixes it before starting anything new).

Once a step is in the definition of done, an item without it is simply not done, which keeps the improvement in place under pressure.