Managing Compute Engine, GKE, and Cloud Run: Day-2 Operations

You connect to a Linux Compute Engine instance with gcloud compute ssh INSTANCE_NAME --zone=ZONE, which generates and propagates an SSH key for you, and you list what is running with gcloud compute instances list. The list command shows every instance in the project across all zones in one call, with status, internal IP, and external IP. Narrow it with --filter= expressions, which can combine fields and labels — for example --filter="status=RUNNING AND labels.env=prod" — and use gcloud compute instances describe INSTANCE_NAME --zone=ZONE to see one instance's full configuration. (--zones= restricts the list to named zones; it does not filter by state or label.)

An instance with no external IP address needs a different path, and which one depends on the network you have:

Identity-Aware Proxy (IAP) TCP forwarding tunnels SSH through Google's infrastructure to the instance's internal IP, so you need neither a bastion host, a VPN, nor a public IP. Both prerequisites matter: without the firewall rule for IAP's source range the connection times out, and without the IAM role it is refused. --internal-ip alone does not help from a laptop on the internet — it only tells gcloud to use the private address, which your laptop cannot route to.

OS Login ties SSH access to IAM identities instead of SSH keys stored in metadata: enable it with the instance or project metadata key enable-oslogin=TRUE, then grant roles/compute.osLogin (or roles/compute.osAdminLogin when the user needs sudo). Access then follows the Google account — disable the account or remove the role and SSH access ends, with no keys left behind to clean up. One prerequisite is easy to miss: when the VM runs as a service account, OS Login users also need roles/iam.serviceAccountUser on that service account. For Windows instances, generate credentials with gcloud compute reset-windows-password and connect over RDP instead of SSH. (Configuring OS Login as part of launching instances is covered with compute planning in Section 2.)

Snapshots back up disks; images create instances. A snapshot is a point-in-time copy of a Persistent Disk or Hyperdisk, and after the first full snapshot, subsequent snapshots are incremental — they store only changed blocks, which keeps cost down. Snapshots are standalone, project-level resources: they survive the deletion of their source disk, so they keep costing money until you delete them. Create one with gcloud compute snapshots create SNAPSHOT_NAME --source-disk=DISK_NAME --source-disk-zone=ZONE (or gcloud compute disks snapshot), and restore by creating a new disk from it with gcloud compute disks create NEW_DISK --source-snapshot=SNAPSHOT_NAME. Snapshots can be taken while the disk is attached and in use.

To find and remove old snapshots, filter on the creation time and delete with the snapshot command: gcloud compute snapshots list --filter="creationTimestamp<'2026-03-15'" returns only snapshots created before that date, and gcloud compute snapshots delete SNAPSHOT_NAME removes one. --sort-by and --limit only reorder or truncate the list — they do not select by date — and gcloud compute images delete cannot remove a snapshot, because images and snapshots are different resource types.

To automate backups, attach a snapshot schedule — a resource policy — to the disk: gcloud compute resource-policies create snapshot-schedule POLICY_NAME --max-retention-days=14 --daily-schedule --start-time=04:00 --region=REGION, then gcloud compute disks add-resource-policies DISK_NAME --resource-policies=POLICY_NAME --zone=ZONE. Compute Engine takes the snapshots and its retention setting deletes old ones automatically. That is the lowest-effort option: a cron job or Cloud Scheduler job that calls the snapshot API also works, but you then own the code, the credentials, and the cleanup. Machine images capture a whole VM (configuration plus all disks) but are created on demand only — they have no schedule or retention policy of their own.

A custom image is a golden template: build it from a stopped instance's boot disk, a snapshot, or another image with gcloud compute images create IMAGE_NAME --source-disk=DISK_NAME --source-disk-zone=ZONE --family=FAMILY, then launch identical instances from it, usually through an instance template and a managed instance group. Put each build into an image family and reference the family (not a pinned image name) in templates: the family always resolves to its newest image that is not deprecated, so templates pick up each new build without edits. Mark an older build with gcloud compute images deprecate IMAGE_NAME --state=DEPRECATED --replacement=NEW_IMAGE; deprecated images drop out of family resolution, and you can roll back by un-deprecating or deprecating the bad newer build. To use images from another project without copying them, grant users roles/compute.imageUser on the image project (the least-privilege role for reading and using images); a managed instance group creates VMs through the Google APIs service agent (PROJECT_NUMBER@cloudservices.gserviceaccount.com), so that agent needs the same role. List images with gcloud compute images list (add --no-standard-images to see only your own) and remove one with gcloud compute images delete.

Decision rule: scheduled disk backups with retention → snapshots plus a snapshot schedule; a repeatable VM build or a template for managed instance groups → custom images in a family.

You inspect a GKE cluster with gcloud for the infrastructure and kubectl for the workloads. gcloud container clusters list shows clusters, versions, and node counts, and gcloud container clusters describe shows a cluster's configuration — but neither lists Pods or Services. gcloud container clusters get-credentials CLUSTER_NAME --region=REGION writes credentials into your kubeconfig so kubectl can talk to the cluster. From there, kubectl get nodes lists nodes, kubectl get pods --all-namespaces (or -A) lists Pods in every namespace, adding -o wide shows each Pod's IP and the node it runs on, kubectl get services lists Services with their cluster IPs and external endpoints, and kubectl describe pod POD_NAME surfaces events — the first place to look when a Pod is stuck in Pending or ImagePullBackOff.

GKE pulls container images from Artifact Registry through the kubelet on each node, and the kubelet authenticates as the node's service account. That account needs roles/artifactregistry.reader on the repository (or on the project that hosts it), and the node pool's access scopes must allow storage reads — cloud-platform or devstorage.read_only (included in the defaults). Same-project pulls with the default configuration generally work; the failure the exam probes is a cross-project repository, where Pods sit in ImagePullBackOff with a 403 until you grant the node service account Artifact Registry Reader in the project that owns the repository: gcloud artifacts repositories add-iam-policy-binding REPO --location=LOCATION --project=REGISTRY_PROJECT --member=serviceAccount:NODE_SA_EMAIL --role=roles/artifactregistry.reader.

Two near-misses are worth knowing. Workload Identity Federation for GKE gives Pods' application code an IAM identity, but image pulls are made by the kubelet, not by the Pod, so a reader grant to the Pod's Kubernetes ServiceAccount does not fix a pull failure — the grant must go to the node service account. And imagePullSecrets work only with an exported service account key stored as a Kubernetes Secret; when policy forbids service account keys, the IAM grant to the node service account is the keyless answer.

Reference images by their full Artifact Registry path, for example us-central1-docker.pkg.dev/PROJECT_ID/REPO/IMAGE:TAG, and confirm the tag exists before chasing permissions.

A node pool is a group of nodes in a Standard-mode cluster that share one configuration — machine type, disk, image type, accelerators, labels, and taints. Add one with gcloud container node-pools create POOL_NAME --cluster=CLUSTER_NAME --machine-type=e2-standard-4 --num-nodes=3 --region=REGION, list pools with gcloud container node-pools list --cluster=CLUSTER_NAME, and remove one with gcloud container node-pools delete POOL_NAME --cluster=CLUSTER_NAME — GKE drains its nodes so Pods reschedule onto the remaining pools.

Editing a pool is done with gcloud container node-pools update. Many settings change in place (labels, taints, autoscaling, upgrade settings), and current GKE versions can also change a pool's machine type, disk type, and disk size with flags such as --machine-type, --disk-type, and --disk-size — GKE recreates the nodes using the pool's upgrade strategy, so Pods are disrupted as nodes roll. What you cannot do is add GPUs to an existing node pool: create a separate GPU node pool instead, which also leaves the existing nodes untouched. When a change must not touch running nodes at all, a new pool plus migration is still the safe pattern.

Resizing sets a fixed count: gcloud container clusters resize CLUSTER_NAME --node-pool=POOL_NAME --num-nodes=5. In a regional or multi-zonal cluster, --num-nodes — on resize and on node-pools create — is per zone: in a two-zone multi-zonal cluster, --num-nodes=3 gives 6 nodes in total — a regional cluster multiplies the count by its number of zones the same way. Divide the total you want by the number of zones.

Node pool autoscaling is the cluster autoscaler applied per pool: it adds nodes when Pods are unschedulable for lack of capacity and removes underutilized nodes when their Pods fit elsewhere. Enable it with gcloud container clusters update CLUSTER_NAME --enable-autoscaling --node-pool=POOL_NAME --min-nodes=1 --max-nodes=10 --region=REGION. --min-nodes and --max-nodes are also per zone in multi-zone clusters; use --total-min-nodes and --total-max-nodes to bound the pool's total instead. Note what triggers it: pending Pods, not node CPU utilization. If Pods set no resource requests, the scheduler thinks everything fits and the autoscaler never acts. For load that grows in both Pods and nodes, pair an HPA (more replicas) with node pool autoscaling (room to schedule them).

In Autopilot clusters there are no node pools to manage — Google provisions nodes from your Pods' resource requests, which is why Autopilot questions pivot to requests (covered below) rather than pool sizing.

The Kubernetes objects the exam expects you to operate are Pods, Deployments, Services, and StatefulSets. A Pod is the smallest deployable unit — one or more containers sharing network and storage — but you rarely create bare Pods; a Deployment manages a ReplicaSet of identical, interchangeable Pods and handles rolling updates. Apply manifests with kubectl apply -f manifest.yaml, scale with kubectl scale deployment web --replicas=5, roll out a new image with kubectl set image deployment/web web=IMAGE:v2, and revert with kubectl rollout undo deployment/web.

A Service gives Pods a stable virtual IP and DNS name, selecting backends by label. Know the three types cold: ClusterIP (internal-only, the default), NodePort (opens a port on every node), and LoadBalancer (provisions a Google Cloud passthrough Network Load Balancer with an external IP). For HTTP(S) routing, an Ingress or Gateway resource provisions an Application Load Balancer instead. Expose quickly with kubectl expose deployment web --type=LoadBalancer --port=80 --target-port=8080.

A StatefulSet is for workloads where identity matters — databases, quorum systems. Unlike a Deployment, it gives each Pod a stable ordinal name (db-0, db-1), a stable network identity via a headless Service, and a dedicated PersistentVolumeClaim per replica that survives rescheduling. Pods are created and terminated in order. When a scenario mentions per-replica persistent storage or stable Pod names, the answer is a StatefulSet, not a Deployment.

The Horizontal Pod Autoscaler (HPA) changes the number of Pod replicas; the Vertical Pod Autoscaler (VPA) changes each Pod's CPU and memory requests; the cluster autoscaler changes the number of nodes. They answer different questions.

HPA utilization is a percentage of the container's request. A 60% CPU target means average usage near 60% of what each Pod requests, not 60% of the node. If the containers in a Deployment set no CPU request, the HPA has nothing to divide by: kubectl get hpa shows the target as <unknown> and it never scales, even at full CPU. The fix is to set requests in the Pod template, not to change the target or the node pool.

VPA must be turned on for Standard clusters before VerticalPodAutoscaler objects do anything: gcloud container clusters update CLUSTER_NAME --enable-vertical-pod-autoscaling. Autopilot clusters have it enabled by default. Its update modes differ in how they touch Pods:

The distinction between Off and Initial is the one people miss: Initial never evicts, but it still rewrites requests behind the manifest's back, so what runs no longer matches what was reviewed. The standard conflict rule: do not run HPA and VPA on the same workload using the same CPU or memory metric — VPA changes the requests that HPA's utilization is computed against, and they fight. Combining VPA with an HPA driven by custom or external metrics is fine.

In Autopilot clusters, resource requests do double duty: Google provisions nodes from them and, for general-purpose Pods, bills you for what your Pods request — not for limits, not for actual usage, and not for nodes. Consequences:

• Over-requesting costs money. Lowering requests toward observed usage lowers the bill without changing the replica count.
• Omitting requests does not switch to usage billing. Autopilot applies default requests (for general-purpose Pods, 0.5 vCPU and 2 GiB of memory per container) and bills those, which can cost more than small explicit requests.
• Values are adjusted at admission to fit allowed minimums, maximums, and CPU-to-memory ratios.
• Limits do not reduce the bill. A limit lower than its request is invalid in Kubernetes and is rejected.
• Some compute classes and specialised hardware (for example, certain accelerator workloads) use node-based billing instead of request-based billing.

HPA and VPA both work on Autopilot, so the operational skill is the same — set honest requests, then let the platform handle nodes.

Every deployment to a Cloud Run service creates an immutable revision. Ship one with gcloud run deploy SERVICE_NAME --image=us-central1-docker.pkg.dev/PROJECT_ID/REPO/IMAGE:v2 --region=REGION. Revisions are auto-named (SERVICE-00042-abc) unless you pass --revision-suffix=v2, which names the revision SERVICE-v2 — handy for commands that must refer to it. Old revisions are kept, so a rollback is a traffic change, not a rebuild. A revision that is currently serving traffic cannot be deleted; move its traffic first.

Where a new revision's traffic goes depends on the service's traffic setting, which is the part that trips people up:

Traffic splitting is how you run gradual rollouts and canaries. Deploy with --no-traffic (and optionally --revision-suffix), then shift a share: gcloud run services update-traffic SERVICE --to-revisions=SERVICE-v2=10. Revisions you do not list share the remainder in proportion to their previous shares — if the old revision had 100%, it now serves the other 90%. Watch error rates, then move to 100% or use --to-latest. A tag (--tag=preview) mints a dedicated URL such as https://preview---SERVICE-HASH.a.run.app for testing a revision directly; combined with --no-traffic, testers can reach the revision while production traffic stays put. The same revision-based traffic splitting applies to Cloud Run functions, since they run on Cloud Run.

On GKE, traffic splitting uses Kubernetes primitives, and the two options behave differently. Two Deployments behind one Service selector split traffic roughly by Pod count (4 old and 1 new Pod ≈ 20%), so the share drifts whenever an HPA changes either replica count. A Gateway API HTTPRoute with weighted backendRefs — one Service per version, weights such as 90 and 10 — fixes the percentage independently of replicas.

Autoscaling in Cloud Run is automatic — instances scale with request load, down to zero by default — and you shape it with three knobs:

• --min-instances keeps instances warm, so requests never wait for a cold start. Startup CPU boost (--cpu-boost) only shortens cold starts; it does not remove them.
• --max-instances caps scale-out to bound cost and protect downstream systems. Size it from the backend's budget: if a Cloud SQL instance can give the service 120 connections and each instance opens a pool of 8, the cap is 15 instances. Cloud Run can briefly exceed the cap during sudden spikes, so leave headroom.
• --concurrency sets how many simultaneous requests one instance handles. It does not cap the number of instances — lowering it makes Cloud Run run more instances for the same load (and so more connection pools). Lower it for CPU-heavy, request-scoped work; keep it high for I/O-bound services.

Accelerators attach differently per platform. On Compute Engine, add a GPU at instance creation with gcloud compute instances create VM_NAME --machine-type=n1-standard-8 --accelerator=type=nvidia-tesla-t4,count=1 --maintenance-policy=TERMINATE --zone=ZONE — GPU instances cannot live-migrate during host maintenance, so the maintenance policy must be set to terminate (A- and G-series accelerator-optimized machine types have GPUs built into the machine type instead). GPU availability is zonal and gated by GPU quota, which you often must request first.

On GKE Standard, GPUs live in a dedicated GPU node pool created with the same --accelerator flag — you cannot add GPUs to an existing pool, and a new pool leaves the other nodes untouched. Current GKE versions install the default NVIDIA driver on GPU nodes for you (selectable with gpu-driver-version in the accelerator flag). Pods request GPUs with an nvidia.com/gpu resource limit and target the right hardware with the node selector cloud.google.com/gke-accelerator: GPU_TYPE; GKE taints GPU nodes so ordinary Pods stay off them. On Autopilot, Pods request GPUs directly in the Pod spec with the same selector and Google provisions the nodes.

TPUs — Google's custom ML accelerators — come three ways. A TPU VM gives you direct access to the host the TPU is attached to: gcloud compute tpus tpu-vm create NAME --zone=ZONE --accelerator-type=v5litepod-8 --version=RUNTIME_VERSION, then gcloud compute tpus tpu-vm ssh NAME --zone=ZONE to run training by hand. GKE TPU node pools suit teams that already orchestrate with Kubernetes, and managed training services hide the hosts entirely. Choose the TPU VM when someone wants to SSH in with no cluster or managed service in between.

Agent Runtime on Gemini Enterprise Agent Platform (formerly Vertex AI Agent Engine) is the managed runtime for AI agents in production: you build an agent with a supported framework such as the Agent Development Kit, then deploy it with the platform SDK, and Agent Runtime handles hosting, scaling, and conversation sessions — no instances, clusters, or session store for you to operate. Do not confuse it with Agent Garden, which is a catalog of sample agents and tools to start from; Agent Garden does not host your deployed agent.

Identity changes at deployment. Locally the agent ran with your Application Default Credentials; once deployed it calls Google APIs as its runtime identity, never the developer's credentials. By default that is the project's Reasoning Engine service agent (service-PROJECT_NUMBER@gcp-sa-aiplatform-re.iam.gserviceaccount.com); you can instead set a custom service account (or, in preview, a per-agent IAM agent identity) at deploy time, and then the agent runs as that. So when a deployed agent's tool gets permission-denied while the developer still has access, grant the data roles to whichever identity the agent runs as — for BigQuery, roles/bigquery.dataViewer on the dataset and roles/bigquery.jobUser on the project that runs the queries.

For interactive data work there are two notebook options:

Workbench costs are controlled with idle shutdown: the instance stops itself after a period without notebook activity — kernel and notebook use, not CPU load. It defaults to 180 minutes, accepts 10 to 1,440 minutes, and is set with the idle-timeout-seconds metadata key, for example gcloud workbench instances update NAME --location=ZONE --metadata=idle-timeout-seconds=7200. A stopped instance keeps its disks (and the files on them), which are still billed; you can also stop, start, and change the machine type of an instance yourself.

Cloud Workstations are managed developer environments. Admins create a workstation cluster (in a VPC and region) and a workstation configuration (machine type, container image with the team's toolchain and IDE, persistent home disk, timeouts); developers then run personal workstations from the configuration and connect from a browser or local IDE, and their home directory persists between sessions. Key settings and roles:

• roles/workstations.workstationCreator on a configuration (or project) lets developers create their own workstations from it; the creator is automatically granted roles/workstations.user on the workstation they create. roles/workstations.user alone lets someone use an existing workstation but not create one.
• --idle-timeout stops a workstation after a period with no use; --running-timeout stops it after a fixed total running time even while in use — set both on gcloud workstations configs create or update.
• --disable-public-ip-addresses keeps workstation VMs off the internet; they then need Private Google Access or Cloud NAT to reach Google APIs and outside resources.

When the scenario is standardized, secure developer environments rather than data science notebooks, the answer is Cloud Workstations.