CT-GenAI: Hallucinations, Reasoning Errors and Bias in LLM Testing

Hallucinations, reasoning errors and biases are the three defect types the CT-GenAI syllabus names for LLM output, and each has a different cause, a different symptom in testware and a different way of being caught. They reduce the quality of generated test cases, scripts, data and plans, so testers must identify them in the output and take measures to reduce them. The syllabus traces all three to the nature of the training data and the inherent limitations of the transformer model.

A useful way to keep them apart: a hallucination adds something that is not true or not there; a reasoning error takes correct inputs and draws the wrong conclusion from them; a bias produces output where each item may be fine but the set as a whole is skewed. Non-deterministic output, where the same prompt gives different responses, is not a fourth defect type: it is a property of LLMs that makes the three defects harder to fix (covered at the end of this lesson).

A hallucination is LLM output that appears factually incorrect or irrelevant to the task it was given. The output usually reads fluently and confidently, which is what makes it misleading: it can compromise the validity of test outputs if a tester takes it at face value.

The syllabus lists three ways hallucinations show up in software testing:

• Fictitious or irrelevant test cases, such as tests for a feature that is not in the release or not in the product at all.
• Incorrect or non-functioning test scripts, such as a script that calls a method, endpoint or library function that does not exist.
• Test cases that verify non-existent acceptance criteria, such as a test traced to a criterion number the user story never had, or one checking a rule nobody specified.

The test for "is this a hallucination?" is a comparison with the test basis and known system behaviour: does this element exist in the requirements, the specification, the API, the framework? If the element is invented, it is a hallucination. If every element exists but they have been combined or concluded wrongly, look at reasoning errors instead.

The syllabus hands-on exercise for hallucinations (HO-3.1.2a) has learners confront at least two LLMs with a situation where they invent irrelevant elements, such as unrelated criteria that are not in the given contextual data, and vary the prompt to see how prompting influences hallucinations.

A reasoning error occurs when an LLM misinterprets a logical structure, such as a cause-and-effect relationship, conditional logic or a step-by-step problem-solving process, and reaches an incorrect conclusion. The syllabus explains why: unlike humans, LLMs lack true logical reasoning and rely on pattern matching, which leads to faulty logic in tasks such as mathematical reasoning.

Typical forms in test work:

• Misapplied conditions: an exception introduced by "unless" ignored, an exclusive boundary ("more than") handled as inclusive, a rule applied to cases its condition excludes.
• Wrong ordering: a priority list that does not follow the risk levels the tester supplied.
• Arithmetic and step errors: a percentage applied to the wrong base, a unit conversion slipped in one step, or a later step built on a figure the earlier step never produced.

The syllabus names test planning and test case prioritization as test tasks that require logical reasoning and where LLMs make reasoning errors. Its hands-on exercise (HO-3.1.2b) uses a test planning problem, such as estimating test effort or prioritizing test cases, with input data complex enough to need real problem solving. The LLM's result is compared with the exact correct result, three model types are tried (an LLM, a small language model and a reasoning model), and prompt variations are used to try to improve the outcome.

The key signal of a reasoning error is that nothing has been invented. Every ID, risk level, criterion and number comes from the input; what is wrong is the logic applied to them.

LLM bias originates from the data the model was trained on, and it makes output favour certain types of information, approaches or assumptions. The syllabus example is a model trained mainly on English-language data, which may underrepresent non-English perspectives.

In software testing the syllabus names two tasks where bias influences the response: generating test data and refining acceptance criteria for test cases. Bias also shows up as a skew in the test types an LLM produces, for example mostly functional tests with non-functional types such as performance, security or accessibility underrepresented or missing.

Bias is a property of the set, not of a single item. One generated hotel guest who is a single adult is not bias; a set of guest profiles in which every booking is one adult in one standard room, with no families, groups or guests needing an accessible room, is. Inspect any single profile and nothing is wrong; only the set as a whole fails to represent the guests the test strategy says must be covered. That is why checks that look at items one by one miss it.

Because the cause is in the training data, rewording a prompt can reduce a skew in one response but does not remove its source, so the skew can return in later output.

To identify which defect an output shows, compare each element with the test basis and ask three questions in order: is anything invented (hallucination), is anything concluded wrongly from real inputs (reasoning error), and is the output as a whole skewed against what the strategy needs (bias)? One response can contain more than one defect, so classify each element separately.

Consider a hotel booking service with two rules: R1, a cancellation made more than 48 hours before check-in is free; R2, a later cancellation is charged 20% of the first night's rate. The booking screen offers Book, Modify and Cancel for a single booking. An LLM returns:

T1 (R1): cancel 72 h before check-in -> no charge
T2 (R1): cancel exactly 48 h before -> no charge
T3 (R2): 3 nights at 100/night, cancel 12 h before
         -> charge 60
T4: select several bookings, press "Cancel all"
    -> all cancelled, no charge

• T1 is correct.
• T2 treats "more than 48 hours" as if 48 hours itself qualified: the boundary is on the wrong side, a reasoning error.
• T3 uses the real rule and the real rate but takes 20% of the whole stay (300) instead of the first night (100), so the expected charge should be 20: another reasoning error.
• T4 tests a bulk-cancel function the screen does not have: a fictitious test case, so a hallucination.

If the same service's generated guest profiles were all single adults paying by card when the strategy required families, groups and several payment methods, that would be a bias: nothing invented, nothing miscalculated, but the coverage the strategy asks for is not represented. Arithmetic deserves its own check: recompute any figure the model derives rather than trusting a plausible-looking number.

Each defect type has its own detection approaches, applied through review or a combination of review and automated verification. Choosing the wrong approach means the defect is simply not looked for.

Two distinctions matter. First, cross-verification and output testing check items that exist; they cannot reveal categories of tests or groups of data that were never generated, so a skew passes them. Only a review against the strategy's coverage finds bias. Second, output testing judges the testware by running it, while logical validation examines the reasoning itself in review, so the two suit different kinds of output.

Depth follows risk. The syllabus states that the actual implementation of these methods depends on the estimated risk of hallucinations, reasoning errors or biases in the test task. Generated test data for a payment-settlement interface whose figures feed regulatory reports warrants deep, layered checking; a list of suggested names for test data files warrants a quick look. A uniform level of checking across all tasks, whether heavy or light, ignores this rule.

The syllabus gives five key techniques to reduce hallucinations, reasoning errors and biases, noting that these defects are more likely when prompts are not properly designed or relevant contextual input data is missing. Grouping them helps: three act on the prompt and two on the choice of model (a convenient grouping, not syllabus wording).

Match the technique to the cause: information gaps are closed in the prompt, ambiguous inputs are restructured, multi-step tasks are chained with a check after each step, and knowledge gaps call for a better-suited model. Changing an inference setting such as temperature addresses none of these causes; it affects how much output varies, not whether it is right.

Model comparison is a detection signal, not a vote. When models disagree, at least one is wrong, so the disagreement flags the item for cross-verification against the test basis. Agreement is weaker evidence than a check against the test basis, because models trained on similar data can make the same mistake.

The syllabus also mentions Retrieval-Augmented Generation and fine-tuning as complementary techniques for improving LLM results; they belong to chapter 4 and are covered in that lesson.

A hallucination, reasoning error or bias that seems fixed can reappear because LLMs are non-deterministic: the same input can produce different output. The variation comes from the probabilistic sampling used during inference, as the model chooses each next token from a probability distribution.

The syllabus draws the consequence directly: these defects are difficult to fix, because they may appear to be fixed for one LLM output and then reappear in another conversation with the same LLM. This applies to all three defect types. A refined prompt that removed an invented rule, or that produced a varied data set once, has shown that the fix can work, not that it always will.

Two practical conclusions follow. Earlier clean results do not certify later ones, so verification applies to each output. And reproducibility gets harder with length: long outputs carry a higher risk of variability than short ones.

Non-deterministic behaviour is reduced, never eliminated, by lowering the temperature and setting a random seed, and its remaining risk is managed by automating parts of output verification. The syllabus is explicit that complete reproducibility cannot be guaranteed.

Temperature is a choice per task. Tasks that need the same answer each time, such as expected results checked into a regression suite, suit a low temperature; tasks that value variety, such as ideas for usability sessions, suit a higher one. Using a low temperature everywhere trades away the variety such tasks need.

Keep the mechanisms apart: temperature shapes the distribution, a seed fixes the random sequence used to sample from it. Even with both set, the syllabus notes that long outputs carry a higher risk of variability, and consistent output is not the same as correct output, which is why verification stays in place.