From Pilot to Production: Making Vulnerability Management Work

A pilot is designed to answer questions. Can the tool see our systems? Do the findings make sense to the owners? Can the workflow close items without excessive effort? When the pilot answers those questions favourably, the temptation is to extend it across the estate immediately. That is often where difficulties begin. A pilot runs with close attention from a small team, on a carefully chosen set of systems. Production runs with ordinary staff, on everything, under pressure to change. The transition between the two is a project in its own right.

Define the pilot's purpose before it starts

A pilot should have a written purpose, a bounded scope and a short list of questions it must answer. Those questions might concern coverage, accuracy of validation, the time taken to route findings to owners, the effect on production systems and the usability of reports for leaders. Without them, the pilot becomes a demonstration, and its results are open to interpretation.

The scope should include at least one awkward case. A pilot run only on modern, well-documented systems will say little about legacy platforms, shared services or systems owned by teams with limited capacity.

Agree exit criteria in advance

Exit criteria determine whether the pilot succeeds. They should be specific and measurable, and they should be agreed by the people who will operate the programme afterwards, not only by the sponsors. Examples include a defined proportion of the pilot scope with confirmed owners, validated findings for an agreed sample, no unplanned disruption to production services and a completed workflow for a representative set of tickets.

Exit criteria should be reviewed at the end of the pilot, not adjusted to fit what was achieved. If a criterion was missed, the decision should be to address the gap, extend the pilot or accept a documented risk, rather than to declare success.

Plan the handover

Handover is where pilots most often fail. The team that ran the pilot understood the scope, the configuration and the exceptions. The operations team that inherits the programme may not. A handover plan should cover the configuration of scanning, the credentials and their rotation, the asset inventory and its maintenance, the prioritisation rules and the reasons behind them, and the list of known issues.

Handover should also include a period of shadowing. Operations staff should run the workflow with support from the pilot team before taking sole responsibility. That overlap reveals gaps that a document cannot.

Write the runbooks

Runbooks turn knowledge into repeatable action. For vulnerability management, the main runbooks cover scan scheduling and scope changes, triage of new findings, escalation of overdue items, raising and reviewing exceptions, handling a disputed finding and responding to a finding that requires urgent attention. Each runbook should name the role responsible, the expected inputs and outputs, and the point at which a decision must be escalated.

Runbooks should be short enough to use under pressure. A long document that nobody reads offers little protection when a weakness is disclosed in the middle of a busy week.

Set up the support model

Production needs a support model that answers two questions: who can help when something goes wrong, and how quickly will they respond? Define the first point of contact, the escalation path, and the arrangements for out-of-hours or urgent cases where they apply. Where external providers are involved, confirm how their support is delivered and how it is recorded.

The support model should also cover the programme's own health. Who checks that scans are still running, that credentials still work and that new asset types are being picked up? These tasks are easy to neglect once the programme is considered routine.

Manage the change deliberately

Rolling out to production in stages reduces risk. Choose the next group of systems or business areas carefully, apply the same exit criteria at each stage, and pause if disruption or confusion appears. A communications plan helps. Owners should know what is changing, why, when the first findings will arrive and where to get help.

Managing stakeholders through the rollout

A rollout changes how people spend their time, so it needs stakeholder management as well as technical planning. Owners who have never received vulnerability tickets will have questions about their workload, the priority rules and the expected response. Identifying the people most affected early, and explaining the change in their terms, reduces resistance more effectively than a general announcement.

Champions help. Choose a small number of respected people in each business area to test the workflow, give feedback and answer questions from colleagues. Champions should receive early access to new reports and additional information, and their feedback should be credited when it changes the process. Their support is particularly valuable when the rollout reaches teams that were not part of the pilot.

Training should be practical. Short sessions built around a real finding from the owner's own estate are more useful than a generic overview of the tool. Show owners how to find an item, record a decision, raise an exception and confirm a fix. Keep the materials available for reference, and review the questions that arrive in the first month, because they reveal where the process needs clearer guidance.

Expect some pushback from teams that feel the programme adds work without adding value. Listen to the concern, check whether it is valid and adjust where the process is genuinely unclear. Where the concern reflects a real difference in priorities, take it to the governance forum rather than resolving it informally.

Common mistakes

  • Treating the pilot's results as proof that production will behave the same way.
  • Adjusting exit criteria after the pilot has ended.
  • Handing over documentation without a period of shared operation.
  • Writing runbooks that are too long to use in practice.
  • Neglecting the programme's own monitoring once it becomes business as usual.

Next steps

Write the purpose, scope, questions and exit criteria for your next pilot before it begins. Ask the people who will run production to review the criteria, and agree the handover plan and runbook list at the same meeting.

Frequently asked questions

How long should a vulnerability management pilot run?

Long enough to answer its questions with evidence, including at least one difficult case. The duration should follow from the exit criteria rather than from a fixed calendar period.

What if the pilot succeeds technically but owners do not engage?

That is a process failure, not a tool failure. Review the ownership model, the quality of tickets and the feedback owners give before expanding the programme.

Who should sign off the move to production?

The sponsor should sign off, but the operational owners should confirm that the exit criteria have been met and that the handover is complete.

Related reading

If you are working through this topic, these related articles cover neighbouring questions:

Leave a Comment