AIP-C01 AI Governance and Compliance: Decision Logs, Audit Trails, Lineage and Model Cards

AI governance and compliance is the set of controls that lets an organization prove what its foundation model (FM) applications did, on what data, under which policy, and that nobody quietly changed those controls. Task 3.3's four skills map onto six governance jobs, and each job has a natural AWS home:

A good governance design is mostly configuration rather than code: managed logging, managed policies and managed alarms beat custom Lambda scanners and daily comparison scripts whenever a requirement says "no code change", "least operational overhead" or "within minutes". Two boundaries help with scope. The guardrail policies themselves (content filters, denied topics, PII filters) are built in Task 3.1 and 3.2; fairness evaluations and explaining outputs to users belong to Task 3.4. This lesson is about recording, tracing, enforcing and watching.

The decision log for an Amazon Bedrock application is model invocation logging: a Bedrock setting that records the full request and response of every model call without any change to the application. Each JSON record holds the model ID, the operation, the input body, the output body, the caller's identity.arn, token counts and any requestMetadata the caller attached. That combination of content plus identity is what a regulator means by "show me the exact prompt and answer, and who asked".

The rules that matter in practice:

• Off by default. Nothing is recorded until someone enables it.
• Per account, per Region. A logging configuration only covers calls made in its own Region, so an application running in two Regions needs two configurations.
• Destinations: CloudWatch Logs, Amazon S3, or both. CloudWatch Logs suits querying with Logs Insights, metric filters and data protection policies; S3 suits cheap long-term retention and Athena.
• Same account, same Region. The log group or bucket must belong to the account and Region doing the logging. This is a property of the logging configuration, not a permissions question, so no resource policy changes it. To centralize records in a log archive account, deliver locally and then move them, for example with S3 Replication into an archive bucket protected by S3 Object Lock, or with log forwarding.
• Large bodies go to S3. With a CloudWatch Logs destination, request or response bodies larger than 100 KB (and binary data such as images) are written to an S3 location for large data delivery, and the log event holds a reference. If no large-data bucket is configured, those bodies are not kept. Forwarding the log group elsewhere cannot recover content that was never written.
• Modalities (text, image, embedding, video) choose which kinds of model output are logged; they do not change size handling.

Invocation logs are also the place to look when you need to know who triggered a safety event. On Converse, a blocked response carries stopReason guardrail_intervened in the output body, so a CloudWatch Logs Insights query that filters on that value and groups by identity.arn lists the principals behind the blocks.

AWS CloudTrail is the audit trail of API calls: it records who called which Amazon Bedrock API, when, from where and with what result, but it never stores prompt or response text. Bedrock APIs fall into two CloudTrail categories, and knowing which is which explains most "why is this call missing from my trail?" problems.

So a trail that logs management events shows Converse calls but shows no knowledge base retrievals until a data event selector for the knowledge base resource type is added. Pick the resource type that matches the API being audited; a selector for another Bedrock resource type covers different operations. CloudTrail Insights is not a substitute: it analyses the rate of management API calls and errors, not individual data events.

One data event is easy to miss. Amazon Bedrock records guardrail evaluations made during model invocation as ApplyGuardrail data events, not only calls to the standalone ApplyGuardrail API. Each event lists every guardrail in scope (guardrails passed on the request and guardrails enforced by account or organization policy) and the combined action. Auditors who want a per-request, per-principal record of guardrail outcomes therefore need a data event selector for AWS::Bedrock::Guardrail; the Converse management event does not carry the assessments.

For who-changed-what questions, management events are enough. A guardrail that suddenly stops blocking a topic is investigated through the UpdateGuardrail event in CloudTrail event history, which shows the userIdentity and eventTime. Invocation logs, guardrail metrics and guardrail traces describe inference traffic, not configuration changes.

Proving the audit trail itself is trustworthy

Regulators often ask for two separate guarantees about log files, and each has its own control:

• Integrity (prove nothing was modified or deleted): CloudTrail log file integrity validation. CloudTrail delivers a signed digest file every hour containing hashes of the log files it delivered, and validation detects any file that was changed or removed.
• Immutability (nobody may delete for N years): S3 Object Lock on the log bucket in compliance mode with a default retention period. In compliance mode no user, including the root user, can shorten the retention or delete a locked object version.

SSE-KMS encryption, S3 Versioning and lifecycle rules are good hygiene but provide neither guarantee on their own. An organization trail delivering to a bucket in a dedicated log archive account is the usual way to cover every member account.

Consistent oversight means the same safety policy applies to every FM call in every account, and teams cannot remove it. Amazon Bedrock offers three mechanisms, and they differ in whether the application must cooperate:

Enforced guardrails

Enforced guardrails apply in addition to any guardrail an application passes on its request; the result is the union, so teams can add stricter controls but cannot remove the central one. The rules to remember:

• Reference a numeric version, never DRAFT. DRAFT is the editable working copy, so enforcing it would let any edit change policy for the whole organization.
• When the guardrail lives in the management (or a central) account, attach a resource-based policy to it that allows bedrock:ApplyGuardrail for the organization's accounts; member roles that invoke models need that permission too.
• A guardrail exists in one Region. To enforce in another Region, create a guardrail in that Region, publish a numeric version, share it and reference it in the policy. Enforcement follows where the guardrail resource lives, and guardrail cross-Region inference is a capacity feature for running guardrail evaluations, not a way to cover another Region.
• AWS states that an Automated Reasoning policy is not supported in an enforced guardrail; including one causes runtime failures for every invocation the enforcement covers. Enforce a version without it and let the applications that need Automated Reasoning checks attach their own guardrail on their requests.

The bedrock:GuardrailIdentifier condition key

When a role already has an unconditional Allow for bedrock:InvokeModel, only an explicit Deny can override it. The working pattern is a Deny on the inference actions with a negated condition such as StringNotEquals on bedrock:GuardrailIdentifier. The condition value format decides which version is accepted:

• arn:aws:bedrock:<region>:<account>:guardrail/<id>:3 matches only published version 3.
• The bare ARN with no version suffix corresponds to the DRAFT working copy.
• A wildcard version suffix with StringNotLike would accept any version, including later unreviewed ones.

A request that names DRAFT when the policy expects :4 fails with AccessDenied; the fix is to send the published version. Two coverage details: Converse is authorized by bedrock:InvokeModel, but ConverseStream (and InvokeModelWithResponseStream) is authorized by bedrock:InvokeModelWithResponseStream, so a statement that lists only InvokeModel lets streaming calls through without a guardrail. And AWS cautions that roles which also call RetrieveAndGenerate or InvokeAgent can hit access denied under such a policy, because those services make internal model calls without the guardrail parameter. A Deny on bedrock:ApplyGuardrail governs only the standalone guardrail API, not model calls. The data-protection angle of these same controls is covered in the Task 3.2 lesson.

A service control policy (SCP) is the preventive control for governance settings: it caps what any IAM principal in a member account can do, including account administrators and roles created later, and it cannot be changed from inside the account. That makes it the right tool when a rule says "only the central platform role may change model invocation logging or guardrails anywhere".

The pattern is a Deny on the governance actions (for example the logging configuration put and delete actions and the guardrail create, update and delete actions) with a condition that exempts the platform role through aws:PrincipalArn. Compare the alternatives:

Region-restricting SCPs and cross-Region inference

Data residency SCPs usually deny actions when aws:RequestedRegion is outside an approved list. Cross-Region inference interacts with them because the SCP is evaluated against the Regions where the request may run:

• A geographic inference profile (for example an EU profile) can route a request to any Region listed in the profile, so the SCP must allow every destination Region of that profile for the Bedrock inference actions. Because those Regions are all inside the geography, residency is preserved.
• A global inference profile can route anywhere; it requires allowing aws:RequestedRegion with the value unspecified, which a strict residency rule should not permit.

No manual opt-in of destination Regions in the account settings is needed for cross-Region inference.

Changes to governance settings are Amazon Bedrock control-plane API calls, so CloudTrail records them as management events and Amazon EventBridge can react to them within seconds. That is the backbone of "alert within minutes" and "restore automatically" requirements.

• Alert: an EventBridge rule that matches the logging configuration delete or put event, or UpdateGuardrail/DeleteGuardrail, with an Amazon SNS target.
• Remediate: the same rule targeting an AWS Lambda function (or an SSM Automation runbook) that reapplies the approved configuration stored in code or Parameter Store. Notification alone is detection, not remediation.
• Prevent: an SCP (previous section) so the change cannot happen in the first place, keeping EventBridge as the safety net.

Slower patterns work but fail tight time limits: an AWS Config rule evaluated on a 24-hour schedule or a daily Lambda that compares settings against a baseline can leave a gap of a day. Watching usage metrics is not a proxy either: removing invocation logging does not change the Bedrock Invocations metric, because models keep being called.

Data source tracking means every dataset that feeds prompts, fine-tuning or knowledge bases is registered with its schema, location and ownership, and the path from raw to curated data is recorded. On AWS, the AWS Glue Data Catalog is the central technical metadata repository: databases and tables with schema, storage location and partitions, searchable by people and queryable by Athena, Redshift Spectrum, EMR and Glue.

• Crawlers keep it current. A scheduled Glue crawler infers schemas for structured and semi-structured data (JSON Lines, Parquet, CSV and similar), adds new partitions as they arrive and updates tables when schemas change.
• Sources other than S3 stay in place. Crawlers reach Amazon RDS, Aurora and other JDBC stores through a Glue connection, so a database can be catalogued alongside S3 data without copying it.
• Ownership and licence go into table properties (table parameters), for example owner, licence or classification, giving auditors one place to see who is accountable for each source.

Other services answer different questions: S3 Inventory lists objects, Amazon Macie finds sensitive data in S3, AWS Config and Resource Groups track AWS resources, and a Bedrock knowledge base is a retrieval index, not a register of governed sources.

Automatic lineage from AWS Glue

Lineage is captured automatically through OpenLineage and viewed in Amazon SageMaker Catalog (part of SageMaker Unified Studio, built on Amazon DataZone). Two paths exist, with distinct requirements:

A job on an older Glue version, or one built on DynamicFrames, produces no lineage even with the listener settings, so both conditions must be met. Glue job bookmarks track processed data for incremental runs; they are not lineage.

Source attribution means each generated answer can be traced to the documents that grounded it, using facts from your own records rather than the model's wording. In Amazon Bedrock Knowledge Bases this is done with document metadata and citations.

• Metadata sidecar files. For an S3 data source, each document can have a companion file named after it with the suffix .metadata.json, holding metadataAttributes such as source system, jurisdiction, document version, data owner or approval status. During ingestion these attributes are stored with every chunk in the vector store. S3 object tags and Glue table properties are not ingested this way.
• Citations. RetrieveAndGenerate (and Retrieve) responses include, for each cited passage, the retrieved references with their source location and metadata. The application displays those fields; asking the model to name its sources in the prompt depends on the model and is not a record.
• Metadata filters. A request can carry a filter in its retrieval configuration (for example an equals condition on approval_status), so one workflow grounds only on approved documents while another queries everything, from a single knowledge base. Prompt instructions and guardrails act after retrieval and cannot see document provenance.
• Sync after changes. The ingestion job is what reads documents and sidecars. Editing a .metadata.json file in S3 changes nothing in the index until the data source is synchronized again, so citations keep showing the old metadata until the next ingestion job (manual, scheduled or event-driven).

How evidence is presented to end users as part of transparency is covered in the Task 3.4 lesson.

Tagging for attribution means attaching consistent key-value labels so that every resource, invocation and charge can be tied to an owner, use case or data source. Different tagging mechanisms land in different places, so choose by where the attribution must appear:

Two practical points. Bedrock does not require requestMetadata, so setting it in a shared client library is what guarantees coverage. And foundation model ARNs cannot be tagged; when several business units share one IAM role and one gateway, the way to separate their spend is an application inference profile per unit, tagged and used as that unit's model ID, plus activating the tag key as a user-defined cost allocation tag in the Billing and Cost Management console so it appears in Cost Explorer. Tags on a log group label the whole group, not individual records.

An Amazon SageMaker AI model card is a structured, versioned record of a model's intended uses, risk rating, training details, evaluation results and additional information, which auditors can export to PDF. It is the compliance-framework artifact named in Skill 3.3.1.

• Programmatic creation. A CI/CD pipeline creates cards with the CreateModelCard API or the SageMaker Python SDK and updates them with UpdateModelCard; the content is a JSON document following the model card schema (model overview, intended uses, business details, training details, evaluation details, additional information).
• Models not trained in SageMaker AI. A third-party or Amazon Bedrock model can have a card, but nothing is auto-populated: you supply every section yourself. Auto-population applies to models SageMaker trained or registered.
• Structured evaluation results. SageMaker can parse evaluation reports produced by SageMaker Clarify or Model Monitor from S3. Results from other tools, such as an Amazon Bedrock model evaluation job, must be mapped by the pipeline into the evaluation_details metric groups (simple, bar chart, graph or matrix metrics); putting them in custom details leaves them as free text.
• Status workflow. Cards move through Draft, PendingReview, Approved and Archived, which supports a committee sign-off step.

Versioning is the tamper protection

Every edit to a card's content creates a new immutable version; a change of status alone does not. Content and status cannot be changed in the same UpdateModelCard call. Consequences for an approval process:

• Approved content cannot be silently overwritten. If metrics are edited after approval, the approved version still exists; retrieve it and export that version to PDF with CreateModelCardExportJob as audit evidence (for example into an Object Lock bucket).
• Approval attaches to the latest version. If a pipeline edits content after review and then sets the status to Approved, it approves content nobody reviewed. Approve the reviewed version with a status-only update first, then submit new content as a later version that goes back to review.

Model Registry integration. When releases are registered as model package versions in SageMaker Model Registry, each model package version has its own model card, a one-to-one link. Completing that card ties documentation to the exact released version in the catalog, with no separate mapping of documents to releases.

Continuous monitoring turns governance from a one-time review into automated detection and alerting. Three areas carry most of it.

Misuse and policy violations from guardrail metrics

Amazon Bedrock Guardrails publishes CloudWatch metrics in the AWS/Bedrock/Guardrails namespace, including Invocations, InvocationsIntervened and TextUnitCount, with dimensions such as GuardrailArn, GuardrailVersion, GuardrailPolicyType (for example TopicPolicy or ContentPolicy) and GuardrailContentSource (Input or Output). This enables configuration-only detection:

• A spike in blocked prompts for a denied topic: alarm on InvocationsIntervened filtered to the topic policy and input source, notifying SNS.
• Traffic that swings widely: alarm on metric math that divides the guardrail's InvocationsIntervened by the same guardrail's Invocations, giving a block rate that does not scale with volume. Dividing by text units or by the model's own AWS/Bedrock invocations (which include callers without the guardrail) gives the wrong denominator.
• No agreed fixed threshold, or a weekly pattern: a CloudWatch anomaly detection alarm learns the expected band, including seasonality.

The metrics have no principal dimension. To find who caused the blocks, query the invocation logs (see the decision-log section).

Token-level redaction of logs

Decision logs often contain personal data. A CloudWatch Logs data protection policy uses managed data identifiers (email address, phone number, credit card number and many more) to mask matching values in log events at ingestion, so most engineers can still search the logs. Principals granted logs:Unmask can view the original values, which suits a small audit team. Key behaviours:

• A policy can be attached to one log group or set at account level, which covers existing and future log groups without per-group automation.
• Detections are published as the LogEventsWithFindings metric in the AWS/Logs namespace, so an alarm can notify the security team when sensitive data starts appearing.
• Masking applies only to what is in CloudWatch Logs. Invocation bodies over 100 KB written to the S3 large-data bucket are outside its reach and need their own controls (tight bucket access, Macie discovery). KMS encryption controls access to the whole log group, not individual tokens, and a guardrail sensitive information filter changes the model's input and output rather than the logs.

Bias drift monitoring

Bias drift is a change over time in how outcomes differ between groups, for example the gap in refer or fast-track rates between age bands, compared with a baseline measured at launch. Amazon SageMaker Model Monitor, including its Clarify bias drift monitoring, is no longer open to new customers, and it targets SageMaker endpoints rather than Bedrock applications. For a new account the managed-light pattern is:

1. Record each decision with the segment attribute (for example in S3).
2. Run a scheduled job (EventBridge Scheduler invoking Lambda) that computes the per-segment outcome rates and publishes the gap as a custom CloudWatch metric.
3. Alarm on that metric: a static threshold at the launch gap when the baseline is fixed, or anomaly detection when the metric has a normal weekly pattern. Notify through SNS or start a remediation workflow.

Guardrail grounding checks and fixed-dataset model evaluations measure other properties, not drift in live decisions. Turning fairness policy into evaluations is covered in the Task 3.4 lesson; AI output policy filters themselves are Bedrock guardrails, covered in Task 3.1.