CT-GenAI: Managing Change, Building GenAI Skills and Evolving Test Roles

Adopting GenAI in a test organization is a change-management problem because it changes what people need to know and how work is organized, not just which tools are installed. The CT-GenAI syllabus says successful implementation needs a structured approach to change management, built on two key aspects: building essential GenAI skills, and evolving traditional testing roles to embrace AI-enabled test processes. The transformation therefore has two sides:

The two sides depend on each other. Skills without updated roles leave open who owns the new work that GenAI creates, such as reviewing generated testware or maintaining shared prompts. Updated roles without skills give people responsibilities they cannot yet carry out. A complete change plan addresses both.

This section builds on the adoption roadmap, shadow AI, GenAI strategy and LLM selection described in syllabus section 5.1; here the focus is on the people side of adoption.

The syllabus names three skills that successful integration of GenAI into testing requires: mastering prompt engineering techniques, understanding model context windows, and developing test review methods. Each one protects a different part of the work:

All three are skills of using a model well for testing. The syllabus does not ask testers to build, train or host models; its list is about directing a model, respecting its input limits and checking its output.

Testers must combine domain and test expertise with AI skills to evaluate LLM-driven testing. The syllabus gives three example tasks where this matters: test case generation, defect report analysis and test data generation.

All three ingredients are needed together, because each covers a different gap:

The principle behind the table: the judgement of whether AI output is correct comes from human domain and test knowledge. Checks that stay inside the model, such as comparing its runs with each other or asking it to grade itself, can repeat the same mistake, so they support but never replace evaluation by someone who knows the domain and knows testing.

The syllabus lists three key competencies for testers working with GenAI, and one area of essential knowledge.

The three competencies follow the life of an LLM-based test task:

1. Assessing LLM capabilities: judging whether a model can do a given test task at all. It logically comes first, because effort spent on prompts or reviews is wasted if the model cannot handle the task.
2. Understanding prompt refinement techniques: improving a prompt iteratively when the output is not yet good enough.
3. Evaluating AI-generated testware: checking the test cases, scripts, data or reports the model produced before they are used.

The essential knowledge is understanding the inherent risks of GenAI, along with awareness of common mitigation strategies. Both halves matter: knowing that LLM output can contain errors is not enough without knowing how those risks are reduced, for example by reviewing output, grounding prompts in reliable context and protecting data. Hallucination, for instance, is an inherent risk, and human review of the output is its basic mitigation. Hallucinations, bias and other risks are taught in detail in chapter 3 of the syllabus.

Testers should understand the data security implications of sharing testware with LLMs, implement proper data sanitization, and follow data privacy-preserving prompt engineering practices.

Data sanitization, as the syllabus defines it, means removing or masking sensitive, personal or confidential information before it is shared. It concerns what data leaves the organization in a prompt, not the quality or format of the testware.

The key point is timing: protection happens before the prompt, not after the response. Once sensitive data has been sent to a model, it has been disclosed; nothing done to the prompt, the conversation or the model afterwards takes that back. Testware drawn from production, such as defect reports, logs or data extracts, is the usual place where personal or confidential values creep in, so it is sanitized before it is used in a prompt.

Privacy-preserving prompting extends the same idea to how prompts are written: include only the data a task needs, and use masked or synthetic values where real ones are not necessary. Data privacy risks and the regulations behind them are covered in syllabus section 3.2.

Environmental considerations are part of the essential knowledge for testing with GenAI. The syllabus lists three: optimizing model selection and usage patterns to reduce computational overhead, selecting right-sized models for test tasks, and balancing the benefits of GenAI automation with its impact on cost and energy consumption.

A right-sized model is one whose capability matches the task. Larger models cost more compute and energy per call, so using the most capable model for every job wastes resources on work a smaller model does just as well. The aim is balance: keep the automation benefit, and spend no more compute than the task needs. The overhead comes mainly from the model chosen and how often and how heavily it is called, so that is where the savings are.

Environmental and data-security knowledge are separate requirements, and one does not stand in for the other:

When a task carries both requirements, a complete approach meets each one with its own measure.

The syllabus says a hands-on approach is essential to strategically train test teams in GenAI for testing. Capability is built by doing, through these practices:

• Practicing with various LLMs and SLMs, so testers learn how different models respond and are not tied to the behaviour of a single model.
• Following structured learning paths, so skills build in a planned order.
• Gradually developing know-how through sharing within the organization.
• Guided exercises and peer learning, which focus training on practical skills.
• Gradual integration of AI into daily test tasks, rather than switching every task over at once.

The common thread is practical, guided, shared and gradual. Training that is mainly theoretical, a one-off event, or unstructured self-exploration misses one or more of those elements.

Prompting skill also develops in a set direction: team members progress from mastering basic prompt creation to more focused techniques, such as test-specific prompts. General prompting comes first; prompts tailored to test tasks come later.

A prompt pattern is a reusable template for crafting effective prompts that guides GenAI toward consistent and reliable outputs. The key words are reusable and consistent: a pattern captures a prompt structure that has worked, so others can apply it to the same kind of task. It differs from prompt chaining (chapter 2), which links several prompts into a sequence for one task.

Internal communities of practice support ongoing knowledge sharing. The syllabus describes:

• Regular meetings to highlight successful GenAI applications, discuss challenges and refine best practices.
• Promoting continuous improvement by sharing prompt pattern libraries and documenting lessons learned from GenAI for test implementations, across projects and domains.

A community of practice is a knowledge-sharing forum among practitioners. Decision-making and control functions, such as approving use cases or setting rules, sit with governance, which is a test management responsibility (see the test manager section below).

The two ideas work together: prompt patterns are what gets shared, and the community of practice is how it gets shared. A pattern library spreads what works across teams, so each team does not rediscover it separately and results stay consistent.

GenAI transforms the traditional test processes of both testers and test managers. For testers, the syllabus says they evolve from test design and test execution specialists to AI-assisted test specialists, combining their expertise in test techniques with skills to guide and verify AI-generated testware.

The tester's test tasks expand to include three things:

1. Review of the overall AI-based output.
2. Prompt refinement.
3. Maintenance of test-specific prompt libraries.

Test techniques stay with the tester. An AI-assisted test specialist still applies test design techniques and uses them to guide and judge the model; the expertise is combined with AI skills, not handed over to the model. The tester's new work is hands-on: prompts, output and prompt libraries. Strategy, risk management and governance sit with the test manager.

Test manager responsibilities are updated to include three additions: developing an AI-based test strategy, AI-based risk management, and monitoring and control of AI-based test processes.

The syllabus also names what test managers focus on:

• Balancing human and AI capabilities, using each for what it does best.
• Establishing AI governance frameworks for use cases, so teams work to shared rules for how GenAI is used. Governance addresses problems that skills alone cannot, such as teams applying inconsistent rules.
• Ensuring the team keeps both traditional testing competencies and AI literacy. AI literacy is added to, not substituted for, the ability to design and run tests without the model.

Finally, test managers will not only lead human testers but also coordinate with GenAI-powered test agents, which requires new management skills for overseeing hybrid teams of people and GenAI tools. In a hybrid team, people and agents are managed as one team, with work allocated by what each does best.

Use this split to tell the two roles apart: