PMI-ACP Experiment Early: MVPs, Spikes, Hypotheses and a Culture of Innovation

Experimenting early means building something small, as soon as possible, to find out whether a solution works and whether anyone needs it, before the team commits large amounts of time and money to it. The PMI-ACP Examination Content Outline makes this Task 1 of the Mindset domain and gives it two enablers:

1. Build an increment of the product to validate solution and/or market need. "Solution" is feasibility: can it be built to the standard needed? "Market need" is value: will the intended users actually use it, pay for it and come back?
2. Create an environment to innovate, learn, and grow. Experiments only happen when the team has the capacity, permission and safety to run them, and when a failed experiment is treated as useful learning.

The reasoning behind both is the same. In complex work, the biggest risk is usually not building the product badly but building the wrong product, or discovering too late that a key part cannot be built. A cheap experiment early turns an assumption into evidence while changing course is still inexpensive. The rest of this lesson follows a simple loop: choose what to test, state how you will judge it, pick the cheapest test that answers the question, measure fairly, act on the result, and make the whole loop easy for teams to run.

Closely related material sits in sibling tasks: the mechanics of design thinking and lean startup, and involving stakeholders from day one, belong to Shorten feedback loops; a no-blame culture belongs to Foster psychological safety; and responding to changing requirements belongs to Embrace change.

The first experiment should target the assumption that is both most uncertain and most damaging if wrong. Every product idea rests on many assumptions; testing the comfortable ones first produces activity without reducing much risk.

A useful way to sort assumptions is by two questions:

• How much do we actually know? Evidence the team already holds, such as its own track record with a technology or customers already paying for something similar elsewhere, lowers uncertainty.
• What happens if it is wrong? Some wrong assumptions cost a tweak; others remove the reason the product exists. Ask which belief, if false, would make everything else irrelevant. A subscription home-repair service, for instance, assumes people will pay for maintenance before anything breaks; if they only ever call in an emergency, no amount of interface polish saves the model.

Assumptions fall into two broad kinds, and either can be the riskiest:

The riskiest assumption moves as evidence arrives. When an experiment settles one question, the ranking changes: whatever is now least known and most damaging comes next, and that is often the other kind of assumption. A team that has proved people value an outcome still has to prove it can deliver that outcome to the required standard, and the reverse. Re-ask "what could still sink this?" after every result.

An experiment needs a falsifiable hypothesis with a measurable success threshold, agreed before it runs. "Customers will find the new search easier" cannot be proved wrong, so it cannot be tested. A testable version names who, what behaviour, the baseline, the target and the time frame: for example, the share of shoppers' searches that end with an item added to the basket rising from its current level to an agreed target within a set number of weeks.

A good hypothesis has these parts:

• The change being tried.
• The metric that directly reflects the hypothesis: the behaviour the change is meant to move, such as completion, conversion or repeat use.
• A baseline and a threshold: the current value and the level that counts as success (and, where useful, a lower level that means stop).
• A time frame, which is also the experiment's timebox.

Agreeing the threshold before the test matters because afterwards almost any result can be argued into a success. It also lets the team and product owner decide quickly when the data arrives.

Measure the hypothesis, not vanity metrics. Cumulative totals and raw counts tend to grow with traffic, promotion or season whatever the change did, so they cannot show whether it worked. Ratings, delivery milestones and opinions are not the behaviour either. Report the metric the hypothesis names, against the threshold agreed for it.

Use the lowest-cost, lowest-fidelity artefact that still answers the question validly. Working software is often the most expensive way to learn something, and many questions can be answered in days without code.

Cheap must still be valid. A prototype tried away from the real setting shows whether people understand a design, not whether they will adopt it under real pressure. When the question is about behaviour in context, keep the context real and fake the expensive part. A travel company wondering whether customers will act on automatically generated itineraries could have its travel agents write the "generated" itineraries behind a real booking page: customers behave as they would with the finished product, and the company learns before building the planning engine. Progressive elaboration applies: start rough, and add fidelity only when the next question needs it.

A minimum viable product (MVP) is the least effort that produces validated learning about whether to build something. A minimal marketable feature (MMF), the term the PMI-ACP outline uses, is the smallest unit of functionality that customers will pay for and the business can actually sell; the release-sized version is often called a minimal marketable release. They answer different questions at different stages.

Two common mistakes:

• Treating a cut-down product as an MVP. An MVP is sized by the question it must answer, not by a feature list. If the planned "MVP" is a substantial build, ask what is the least the team could put in front of real users to answer the hypothesis; often a concierge, Wizard-of-Oz or single-workflow version will do. Re-planning the same scope in a different shape leaves the cost of learning unchanged.
• Using the wrong tool for the stage. Without evidence of demand, a sellable feature set is premature: test demand first, whoever is asking for the release. Once demand is clearly validated, repeating the demand test only delays value: move on to the MMF.

An MMF is defined by what makes it sellable, not by popularity alone. Popular requests matter, but so do the operational and regulatory pieces a customer-facing release cannot go live without, such as a way to take payment, account administration, or a compliance step the law requires. A release that leaves any of those for later cannot actually be sold.

A spike is a short, timeboxed investigation that answers one technical question so the team can estimate or decide. The term comes from Extreme Programming. A risk-based spike tests a specific uncertainty (will the reporting database return a large customer's full history within the response-time target?); an architectural spike tests a design approach or component the solution will depend on.

Use a spike when the team cannot size or plan a story because of a technical unknown. Any response that leaves the unknown in place, however it is dressed up, does not make the story plannable; only learning does.

Before a spike starts, the team agrees two things:

1. The question, stated precisely enough that the team will know when it is answered ("can it return the history in under two seconds?" rather than "look into the database").
2. The timebox: the most time the team will spend on it.

The output of a spike is knowledge, not production code. Spike code is usually written quickly and without tests; shipping it imports that debt. Keep the learning and build the real story to the team's definition of done. Spikes are also backlog work: they use team capacity, so they belong in the product backlog where the product owner can see and order them (see the last section).

Evidence is trustworthy when it reflects what target users actually do and when the effect of the change is separated from everything else that happened at the same time.

Behaviour beats opinion

Market need is validated by what target users do with a real increment: using it for their real work, committing money to it, coming back without being reminded. Statements of intent and approval are weaker, whoever they come from, because saying and doing differ. When stated enthusiasm and actual usage disagree, believe the usage, and then find out from the users what happened.

Use a concurrent, randomized control

To credit a result to a change, compare against a control group that does not get the change, with users assigned at random and both groups measured over the same period. This is what an A/B test does. Weaker designs fail for one of three reasons:

Run tests in representative conditions: an atypical period makes results hard to generalize. Interpret results against the threshold agreed beforehand. Improving on the current version is not the same as meeting the agreed criterion; when a result falls short of the bar, say so openly so the decision-makers decide with it in view.

When an experiment's timebox ends, the team and product owner decide what to do with what was learned. They do not extend the timebox by default. The main options are to persevere, pivot, stop, or run a new, narrower experiment.

• Inconclusive result: look at what was found and choose: a narrower follow-up, a different approach, or accepting the risk. Open-ended extensions defeat the purpose of a timebox.
• Result between agreed thresholds: the evidence has narrowed the question rather than answered it. Design the next timeboxed experiment around what is still unknown; an experiment with no end date is not a decision.
• Hypothesis disproved: a result that misses the agreed threshold is valid learning, and a cheap one. Make it visible to the people who own the decision and use it to decide whether to pivot or stop. Changing the success criteria after the fact, or adding scope in the hope of a different outcome, throws the learning away and teaches people that experiments are theatre.

Results should also feed the next hypothesis. Keep a record of what each experiment established and choose the next hypothesis from it, so the series builds on itself instead of circling back. This is the build–measure–learn loop of lean startup (its mechanics are covered with Shorten feedback loops).

When an early decision is both uncertain and expensive to reverse, explore more than one option with small experiments and commit at the last responsible moment: in Mary and Tom Poppendieck's words, the moment at which failing to make a decision eliminates an important alternative. This lean idea is often called set-based design or set-based concurrent engineering.

Examples: choosing between two mapping providers whose accuracy in rural areas is unknown, or between building and buying a scheduling engine when the volume it must handle is still uncertain. Small experiments on each option, ideally behind the team's own abstraction so a later switch stays cheap, turn the choice into an evidence-based one.

Committing early on familiarity or on one person's judgement locks in a guess. Halting work for a large up-front study, or waiting passively for a deadline, delays without learning. Keeping options open is different from both: the team is actively learning during the delay, and it decides as soon as the evidence is good enough or the last responsible moment arrives.

Teams experiment when experimenting is planned, visible, cheap and rewarded. The agile practitioner's job is to remove the barriers, not to control each experiment.

Make capacity, and make it visible

• Plan capacity for experiments, agreed with the product owner. Innovation that depends on spare time or goodwill rarely happens.
• Keep experiments and spikes in the product backlog, ordered by the product owner with everything else. Hidden experiments compete invisibly with committed work and erode trust when it slips.
• Use a risk-adjusted backlog, which orders risk-reducing work early. A spike or experiment on a key unknown ships no feature itself; its value is the risk it removes from later features. The practitioner coaches product owners to value it that way.

Remove friction and fix incentives

• Replace case-by-case approval with guardrails: agreed limits (exposure, data, spend, reversibility) inside which teams test freely, with formal review kept for larger or riskier changes. An approval queue of any kind slows learning, and counting or rewarding experiments does not remove the queue; it invites gaming.
• Remove structural delays at their source, such as slow access to test environments, usually by working with the group that owns them towards self-service.
• Watch for measures that punish exploration. Output metrics used to rank teams make any work that does not earn output look like a cost, so teams drop it even when capacity is formally allowed. Fix the incentive before tuning the allowance.

Make failure cheap and safe in production

Experiments can run with real users early when the cost of a bad result is contained:

Check which risks a release approach actually controls: some limit exposure but are slow to undo, and testing only away from real users is safe but says little about how customers behave.