CT-GenAI sample questions — 25 answered and explained

The context window is the amount of preceding text, measured in tokens, that the model can consider when generating a response.

Missing content from the start of a long input is the typical sign that the input exceeded the context window. The fix is to work with token counts: measure the input, then filter the log to the relevant lines or process it in chunks that each fit. Sampling settings, reruns and a different reasoning style do not change how much input the model can consider.

1. Recognise the symptom: only the later part of a long input was used.
2. Link it to the context window limit, measured in tokens.
3. Reject options that change sampling or reasoning but not input size.
4. Reduce or split the input so it fits.

Exam tip: If an LLM ignores part of a long input, count the tokens: the context window is the limit.

Embeddings are numerical vector representations of tokens that encode semantic, syntactic and contextual relationships.

Tokenization and embeddings together turn language into numbers the model can process. Tokenization splits text into tokens; embeddings then map each token to a vector in a high-dimensional space. Because similar meanings sit close together, the model can relate words, keep context and produce coherent answers.

1. Recall the two steps that turn text into numbers.
2. Tokenization splits; embeddings represent meaning as vectors.
3. Match the remaining options to hyperparameters and the context window.

Exam tip: Embeddings = meaning as vectors; close vectors = similar meaning.

Embeddings capture semantic relationships, so tokens with similar meanings have nearby vectors.

Matching different words with the same meaning depends on a numerical representation of meaning. That is what embeddings provide: vectors positioned so that similar meanings sit close together. Tokenization is the tempting near-twin, but it splits text without capturing meaning, and "sign-in" and "login" share no useful pieces.

1. Identify the need: match different wording with the same meaning.
2. Ask which concept represents meaning, not text form.
3. Choose embeddings; tokenization only splits text.

Exam tip: Same meaning, different words: embeddings make the match.

LLMs predict the next token based on learned relationships; their output is statistically plausible, but plausible is not necessarily correct.

Two clues rule out the near-twins. The wrong value repeats on every rerun, so random sampling (non-determinism) is not the cause. The specification is short and inside the prompt, so the context window is not the cause. What remains is the basic nature of transformer generation: it produces likely-looking text, which is why every generated expected result must be checked against the test basis. Sub-word tokenization is not the cause either: values are routinely split into several tokens and the model still uses them.

1. Use the repeat-on-rerun clue to rule out sampling randomness.
2. Use the short, included specification to rule out the context window.
3. Recall that generation favours plausibility, not correctness.
4. Conclude that fluent output still needs verification.

Exam tip: Fluent and consistent is not the same as correct: LLMs generate plausible text.

LLMs are based on the transformer architecture and generate text by predicting the next token.

LLMs are generative pre-trained transformers. Training teaches them how tokens relate to each other; at inference they use those relationships to predict the next token, again and again, until the response is complete. They do not retrieve documents, apply expert rules or retrain on each prompt.

1. Separate training (learning relationships) from inference (using them).
2. Recall that inference predicts the next token repeatedly.
3. Reject retrieval, rules and per-prompt retraining.

Exam tip: Inference = next-token prediction, one token at a time.

Prompt chaining decomposes a task into intermediate steps; the output of each step is checked (manually or automatically) and refined before the next prompt.

The defining features of prompt chaining are decomposition into several prompts and checking intermediate outputs. Examples in one prompt are few-shot prompting; having the LLM write prompts is meta prompting; repeating the same prompt and voting involves no decomposition at all.

1. Look for the option that splits one task into several prompts.
2. Confirm that intermediate outputs are checked between steps.
3. Map the other descriptions to few-shot and meta prompting.

Exam tip: Chaining = many prompts in sequence, each output checked before the next.

Prompt chaining breaks a complex test task into intermediate steps and checks each output before the next prompt.

The tester decomposes test analysis and design into three prompts and checks the result of each before moving on. That is the definition of prompt chaining. Zero-shot is tempting because no examples appear, but it describes a single prompt without examples, not the stepwise, reviewed workflow. Meta prompting would require the LLM to produce prompts, and few-shot would require examples.

1. Notice the work is split into several prompts in a fixed order.
2. Notice each output is checked before it feeds the next prompt.
3. Decomposition plus intermediate checks identifies prompt chaining.

Exam tip: A sequence of prompts with a review between each = prompt chaining.

Prompt chaining is particularly useful where tasks are complicated, need decomposition into subtasks, and benefit from systematic checking of intermediate LLM outputs.

Each option describes the natural home of a different technique. Chaining fits the complex feature because the work naturally splits into steps (conditions, then test cases) and the conditions should be checked before they drive design. Template matching from examples is few-shot, co-creating a prompt is meta prompting, and a single factual question needs no chain.

1. Recall what chaining offers: decomposition and checked intermediate results.
2. Find the situation that is complex and has an intermediate artefact worth reviewing.
3. Match the other situations to few-shot and meta prompting.

Exam tip: Complex, multi-step test tasks with reviewable intermediate results call for chaining.

Zero-shot gives no example, one-shot gives one, and few-shot gives more than one example of the expected result.

The syllabus distinguishes prompts by how many examples they include. One worked example followed by a new input is one-shot prompting. Few-shot is the near-twin but requires several examples to consolidate the desired behaviour. Zero-shot has none, and chaining is about a sequence of prompts.

1. Count the examples in the prompt: one.
2. Zero examples = zero-shot, one = one-shot, more than one = few-shot.
3. Rule out chaining because there is only one prompt.

Exam tip: Count the examples: 0 = zero-shot, 1 = one-shot, 2+ = few-shot.

The "shot" count is the number of examples given in the prompt: none for zero-shot, more than one for few-shot.

Few-shot prompting guides the model with several examples of input and expected output, so it can generalise the pattern; zero-shot provides no example and depends entirely on what the model already knows. The distractors redefine "few" as prompts in a chain, rules in a system prompt, or candidate prompts in meta prompting.

1. Recall that "shot" means an example in the prompt.
2. Zero-shot: no examples; few-shot: several examples.
3. Reject options that describe chaining, system prompts or meta prompting.

Exam tip: Shots are examples, not prompts or rules.

Few-shot prompting is particularly effective where examples can illustrate the required behaviour, helping the model generalise and produce consistent results.

The symptom is that a described format is not followed. Few-shot prompting addresses exactly this: several examples give a clear reference and keep results consistent with expectations. Chaining is workable but only detects drift after the fact; meta prompting improves the wording of a description that has already proven insufficient; a role cannot supply unseen conventions. Examples show the model the pattern instead of telling it.

1. Identify the failure: output does not match an existing, well-defined style.
2. Note that verbal description has already been tried and failed.
3. Ask which technique shows the pattern rather than describing it: few-shot.
4. Compare with chaining, which catches errors but does not prevent them.

Exam tip: When describing a format is not enough, show it: few-shot examples.

Meta prompting uses the LLM's ability to generate or refine prompts, which the tester then evaluates and refines in an iterative cycle.

The tester does not ask the model to analyse the reports yet; they ask it to produce the prompt for that analysis, and they co-create it. That is meta prompting, and the syllabus notes it is especially useful when the tester is unsure how to craft an effective prompt. The interaction is iterative, which can look like chaining, but chaining decomposes the test task itself.

1. Ask what the LLM is producing: a prompt, not test output.
2. LLM-generated or LLM-refined prompts define meta prompting.
3. Distinguish this from chaining, which decomposes the test task.

Exam tip: If the LLM's output is a prompt, the technique is meta prompting.

Bias detection reviews whether generated testware fairly represents the test strategy and coverage requirements, including underrepresented test types such as non-functional tests.

Two facts decide this item. First, a heavy skew toward functional tests with performance and accessibility missing is the syllabus example of bias related to test types. Second, the matching detection approach is a review of the output against the defined test strategy and coverage requirements. Cross-verification and output testing check each item that exists; they cannot reveal categories that were never produced. A domain expert checking expected results has the right label but the wrong check.

1. Note that every generated item is correct and traceable.
2. The defect is the distribution: non-functional test types are missing — bias.
3. Pick the detection approach that compares coverage with the test strategy.

Exam tip: Missing test types are a bias problem — compare the output with the test strategy, not item by item.

Cross-verification compares AI-generated output with existing documentation, requirements and known system behaviour, and can be automated to flag discrepancies.

Fields or error codes that do not exist in the specification are hallucinations. The syllabus lists cross-verification as a hallucination detection approach: compare the output with documentation, requirements and known system behaviour, with automated tools cross-referencing established sources. Logical validation and output testing target reasoning errors, and coverage review targets bias.

1. Recognise the defect: references to things not in the specification (hallucination).
2. The check compares output with a reference source.
3. Comparison with documentation = cross-verification.

Exam tip: Check the output against the spec = cross-verification.

Consistency checks verify that generated outputs are consistent with each other and with known information.

The syllabus lists three hallucination detection approaches: cross-verification (against documentation and requirements), domain expertise consultation, and consistency checks. Comparing several generated sets and flagging contradictions between them is a consistency check; a contradiction suggests at least one version contains invented content. Logical validation and output testing are the reasoning-error approaches.

1. Note what is compared: outputs with each other, not with a document.
2. Contradictions between outputs point to possible hallucinations.
3. Output-to-output comparison = consistency check.

Exam tip: Outputs compared with each other = consistency check; with the spec = cross-verification.

Domain expertise consultation: subject matter experts validate the accuracy of generated content, especially where nuance matters.

The deciding fact is that correctness depends on clinical knowledge the specification does not contain, so neither the tester nor a check against the documents can confirm the values. The syllabus lists domain expertise consultation as a hallucination detection approach because subject matter experts capture nuanced errors that automated checks and non-experts miss. Consistency checks and logical validation can expose some problems but cannot confirm that plausible-looking values are right, and temperature affects variability rather than accuracy.

1. Identify the risk: plausible-looking but possibly wrong content (hallucination).
2. Identify the constraint: correctness depends on knowledge that is not in the specification.
3. Choose the approach that adds the missing knowledge: a subject matter expert.

Exam tip: When correctness depends on knowledge outside the test basis, bring in a domain expert.

Output testing — running generated test cases or scripts against the test object — detects reasoning errors and can be partially or fully automated.

Faulty conditional logic is a reasoning error. The syllabus gives two reasoning-error detection approaches: logical validation (review of the reasoning flow; tools can help but complex cases need human judgment) and output testing (executing the generated testware against the test object). The stated constraint, maximum automation, favours output testing. Cross-verification targets hallucinations and coverage review targets bias.

1. Classify the risk: wrong branch logic = reasoning error.
2. List the reasoning-error approaches: logical validation and output testing.
3. Apply the constraint: executing scripts is the one that automates well.

Exam tip: To catch faulty logic in executable testware, execute it.

Reasoning errors are detected by logical validation of the reasoning flow and by output testing; only the first applies when nothing can be executed.

A double-counted step is a reasoning error in a test planning task, which the syllabus singles out as error-prone (its hands-on exercise uses effort estimation). The two reasoning-error detection approaches are logical validation and output testing. With no executable testware, output testing is not possible, so a review of the reasoning chain is what locates the faulty step. Comparing models is a mitigation that can expose a disagreement, but it does not locate the error.

1. Classify: a wrong intermediate calculation is a reasoning error.
2. Rule out output testing: there is nothing to execute.
3. Rule out model comparison for locating the error: it shows differences, not causes.
4. Logical validation reviews each step of the reasoning.

Exam tip: No executable output? Validate the reasoning step by step.

LLM-powered test infrastructure adds intelligent processing: the LLM reasons over testware, and the back end integrates multiple sources and post-processes output.

The syllabus lists what goes beyond client-server: the LLM is a smart processing component that interprets and reasons based on testware, insights are generated dynamically from context such as requirements, code or test results (unlike scripted rule-based bots), the back end integrates multiple sources such as relational and vector databases, and it post-processes output. A front end reading a relational database is plain client-server, and scripted rules are the opposite of dynamic reasoning.

1. Recall what a plain client-server system does: a server returns stored or computed data.
2. Find the option where the server-side component reasons over testware.
3. Reject options that describe ordinary interfaces, databases or rule-based bots.

Exam tip: The LLM is a reasoning component, not just a server.

In RAG, the user query is encoded and the most relevant chunks are retrieved by semantic similarity between the query embedding and the chunk embeddings.

Embeddings place text with similar meaning close together in a high-dimensional space. When the assistant encodes 'sign-in lockout' and compares it with the stored chunk embeddings, a chunk about authentication and account suspension is close in meaning even though it shares no words with the query. Keyword search, synonym tables and recency would not explain the match.

1. Notice that the query and chunk share no words.
2. Recall how RAG retrieves: encode the query, compare embeddings, rank by similarity.
3. Similar meaning, not shared words, drives the match.

Exam tip: RAG retrieval matches meaning through embeddings, not words.

RAG enhances LLMs by incorporating additional data sources into response generation, increasing the relevance and accuracy of outputs.

RAG combines a retrieval system with the language model: relevant content is retrieved from additional data sources and given to the LLM as context, so its response is grounded in that information rather than its training data alone. Retraining the model is fine-tuning, temperature controls variability, and prompt chaining structures a task.

1. Expand the acronym: retrieval, then augmented generation.
2. Retrieval supplies extra content; the model itself is unchanged.
3. Separate RAG from fine-tuning, temperature and prompt chaining.

Exam tip: RAG = retrieve relevant content, then generate with it as context.

The LLM chosen for a GenAI strategy must align with the test objectives, be compatible with existing test infrastructure, and meet scalability requirements.

Two facts combine here. The pilot showed the model performs well, so output quality is not the gap. The failures are integration (cannot connect to existing test management and CI tools) and load (slows with 200 users), which the syllabus covers as compatibility with existing test infrastructure and system scalability. The other pairs are real strategy or selection aspects that do not explain these symptoms.

1. Discard quality causes: the pilot output was good.
2. Map "cannot connect to existing tools" to infrastructure compatibility.
3. Map "slows with many users" to scalability.

Exam tip: A good pilot model can still fail on infrastructure fit and scale.

The effectiveness of LLM-powered testing depends on accurate, relevant input data; high-quality input data is key to results that can be trusted.

The stem rules out weak prompts and a weak model, leaving the input data. The syllabus says the effectiveness of LLM-powered testing depends on accurate, relevant input data, and that maintaining its quality is key to trustworthy results. A larger model and few-shot examples work on the wrong layer; a review gate is a valid control but treats the symptom rather than the cause.

1. Note what the stem rules out: prompt structure and model capability.
2. Trace outdated rules in the output back to the input.
3. Fix the root cause first; keep review as a later control.

Exam tip: Stale input in, stale testware out: fix data quality first.

Data quality plays a critical role in a GenAI test strategy: accurate, relevant input data protected by robust security procedures.

The syllabus states that LLM-powered testing is only as effective as its input data, which must be accurate and relevant and protected by robust security procedures. A larger model, anonymization or data minimization each address something else (capability, privacy) and do not make inaccurate data trustworthy.

1. Recall the two data properties: accurate and relevant.
2. Recall the protection: robust security procedures.
3. Reject options that substitute model size or privacy measures for quality.

Exam tip: Trustworthy GenAI results need accurate, relevant, secured input data.

A GenAI strategy for testing includes comprehensive training programs so test teams have the technical and ethical skills to use GenAI tools effectively.

The syllabus pairs two skill types in its strategy guidance, technical and ethical, and aims the training at the test teams that will use the tools. Training only specialists, showing leads the product's features, or training only the infrastructure administrators either leaves out the ethical skills or misses the test team as the audience.

1. Identify the audience: the test teams that use GenAI.
2. Identify the two skill types: technical and ethical.
3. Choose the option covering both for that audience.

Exam tip: Strategy training = technical + ethical skills for the test team.