LLM-Powered Test Infrastructure for Software Testing practice questions

The architecture has a front end (user interface), a back end (authentication, data retrieval, prompt preparation, LLM interaction, post-processing), external data sources and the LLM.

The back end processes user input and manages the critical functions: authentication, data retrieval, prompt preparation and interaction with the LLM. The front end only collects queries and shows results, the vector database is a data source the back end draws on, and the LLM generates responses from the prompts the back end prepares.

1. List the four parts: front end, back end, data sources, LLM.
2. Match each function to the part that owns it.
3. Authentication, retrieval and prompt preparation all sit in the back end.

Exam tip: Front end = interface; back end = authentication, retrieval, prompt preparation, LLM calls.

In LLM-powered test infrastructure, the back end handles authentication, data retrieval, prompt preparation and interaction with the LLM.

Both requirements map to back-end functions: restricting testers to their authorized projects is authentication and data retrieval, and adding the project's conventions to every request is prompt preparation. Doing either in the front end leaves it in the interface layer, post-processing runs too late to shape the request, and the LLM is a generation component rather than an access-control point.

1. Classify each requirement: access control, and content that must be in every prompt.
2. Recall which component owns authentication and prompt preparation: the back end.
3. Check that post-processing only acts on output after the LLM call.

Exam tip: Who may see what, and what goes into every prompt, are back-end jobs.

The LLM in a test infrastructure may be a third-party service accessed via an API or a custom in-house model; the back end prepares prompts and post-processes output.

Two facts combine. First, the LLM may be hosted as a third-party service accessed via an API or as a custom in-house model, and only the in-house option keeps confidential requirements inside the company. Second, the back end post-processes the LLM's raw output so it aligns with the test process before results move on, which is what produces a consistent import format. The third-party option fails the policy, the front-end-only design skips post-processing, and the in-house chatbot keeps the manual copy step.

1. Check the data constraint: where may the requirements be sent?
2. Choose the hosting model that satisfies it: in-house rather than a third-party API.
3. Check the format constraint: which component turns raw output into the required format?
4. Pick the option that has both an in-house LLM and back-end post-processing.

Exam tip: Data that must stay inside points to an in-house LLM; consistent output formats point to back-end post-processing.

The back end integrates multiple data sources: relational databases for structured test data and vector databases for semantic retrieval using embeddings.

A vector database stores embeddings, numerical vectors that encode meaning, so the back end can retrieve content that is semantically related to a query even when the wording differs. Structured records such as test cases sit in relational databases; authentication and model hosting are separate concerns.

1. Recall the two data sources the back end integrates.
2. Relational = structured data such as test cases.
3. Vector = semantic retrieval by comparing embeddings.

Exam tip: Vector database = find content by meaning, via embeddings.

Relational databases serve structured test data; vector databases serve semantic retrieval of related content using embeddings.

The syllabus gives the back end both kinds of source for a reason. A list of test cases with a given status in a given release is structured data, which a relational database returns exactly. Earlier defect reports that describe the same problem in other words are found by comparing embeddings in a vector database. Using one source for both, or swapping them, fails one of the two needs.

1. Classify the first need: exact filter on structured fields.
2. Classify the second need: match by meaning despite different wording.
3. Map structured to relational and meaning to vector.

Exam tip: Exact fields: relational. Similar meaning: vector.

Back-end post-processing enhances the LLM's raw output so that it aligns with the test process before reaching the front end.

Prompt preparation shapes what goes into the LLM, but LLM output can still deviate. The syllabus gives the back end a post-processing role: it enhances the raw output so that responses align with the test process before they are presented in the front end. Here that means checking each response against the template and correcting or rejecting the extra fields.

1. Note that the input side (the prompt) is already correct.
2. The remaining problem is in the output, so look for an output-side function.
3. Post-processing is the back-end step between the LLM and the front end.

Exam tip: Fix the prompt for the input; use post-processing for output that still deviates.