Choosing and Deploying Compute: Compute Engine, GKE, and Cloud Run

Choose by asking two questions in order: is the workload containerized, and who should manage the infrastructure? If you need OS access, custom kernels, GPUs attached to a specific machine shape, or you are migrating an existing VM as-is, use Compute Engine. If you run many containers that need orchestration — service discovery, rolling updates, stateful workloads, fine-grained scheduling — use Google Kubernetes Engine (GKE). If you have a stateless container that serves requests or processes events and you never want to think about nodes, use Cloud Run. If you have a single-purpose snippet of code that reacts to an event or an HTTP call, use Cloud Run functions. If you are deploying an AI agent you built (for example with the Agent Development Kit or LangChain), use Agent Runtime on the Gemini Enterprise Agent Platform (formerly Vertex AI Agent Engine).

Each platform has signals that point to it. Lift and shift, custom OS or sole-tenant licensing point at Compute Engine. Kubernetes, Pods or multi-container microservices with complex networking point at GKE. Scale to zero, pay only while handling requests or stateless HTTP container point at Cloud Run. Respond to a file upload or run code when a message arrives point at Cloud Run functions.

Agent Runtime versus self-hosting or a model endpoint

Agent Runtime is a managed runtime for agent code: you deploy the agent (its reasoning loop, tools and orchestration), and the platform hosts it, scales it and provides managed sessions and memory, with built-in observability. That is different from two near-twins. A model endpoint serves predictions from a single model — it does not run your agent logic or keep conversation sessions. Self-hosting the agent on Cloud Run or GKE works too, but you then build session storage, scaling and operations yourself; it fits when you need full control of the serving stack, and Agent Runtime fits when the requirement is the least operational effort for running agents.

Scenario: a retailer migrates a legacy inventory server that needs a specific Linux kernel module (Compute Engine), refactors its storefront into a dozen microservices its platform team already runs on Kubernetes on-premises (GKE), exposes a stateless recommendation API that spikes on weekends (Cloud Run), generates a thumbnail whenever a product image lands in a bucket (Cloud Run functions), and deploys a customer-service agent with conversation memory (Agent Runtime). One system, five different correct platforms.

One practice to avoid: gcloud compute instances create-with-container runs a container on a VM through the container startup agent, which Google has deprecated. For a new containerized workload, deploy it to Cloud Run (gcloud run deploy) or GKE; if it truly needs a VM, install a container runtime yourself (for example with a startup script or cloud-init).

You launch an instance with gcloud compute instances create, and the two launch settings that matter most are the availability policy and SSH key handling. The availability policy has two independent parts. On host maintenance: --maintenance-policy=MIGRATE (the default for most machine types) live-migrates the VM to another host with no reboot; --maintenance-policy=TERMINATE stops the VM instead. Automatic restart: --restart-on-failure (default) restarts the VM after a crash or a maintenance stop; --no-restart-on-failure leaves it stopped until someone starts it.

VMs with attached GPUs cannot live-migrate at all — they must use TERMINATE (optionally with automatic restart), so plan for the maintenance stop rather than expecting MIGRATE. The availability policy is mutable: change it on an existing VM with gcloud compute instances set-scheduling.

gcloud compute instances create web-1 --zone=us-central1-a --machine-type=e2-medium --maintenance-policy=MIGRATE --restart-on-failure

For SSH without OS Login there are metadata-based keys. Project-wide SSH keys in project metadata reach every VM in the project; instance-level keys in instance metadata reach one VM. Setting --metadata=block-project-ssh-keys=TRUE on a sensitive instance stops project-wide keys from working there — but it does not stop anyone who can set that instance's metadata (compute.instances.setMetadata) from adding their own key to the instance metadata. The simplest path is gcloud compute ssh web-1 --zone=us-central1-a, which generates a key and adds it to metadata for you. When access must be tied to identities and centrally controlled, use OS Login, covered next; enabling it makes the VM stop accepting metadata-based keys.

OS Login ties SSH access to IAM identities instead of scattered metadata keys, which is why Google recommends it for teams. Enable it with the metadata key enable-oslogin=TRUE at the project level (all VMs) or per instance: gcloud compute project-info add-metadata --metadata=enable-oslogin=TRUE. Instance metadata wins over project metadata, so an instance with enable-oslogin=FALSE keeps using metadata keys even when the project sets TRUE.

Access is then granted with IAM roles: roles/compute.osLogin for a standard user and roles/compute.osAdminLogin for sudo. If the VM runs as a service account, the user also needs roles/iam.serviceAccountUser on that service account — without it, login fails even with the OS Login role. Revoking a person's access becomes one IAM change instead of a hunt through metadata. OS Login also supports two-step verification through the user's Google identity.

Enforcing OS Login. Because anyone who can edit instance metadata could set enable-oslogin=FALSE and add their own key, project metadata alone is not a guarantee. The organization policy constraint compute.requireOsLogin makes it one: when enforced, requests that set enable-oslogin to FALSE in project or instance metadata are rejected, for new and existing VMs. Enforcing it also turns OS Login on for newly created projects.

VM Manager is the fleet-management suite for Compute Engine, built on the OS Config agent that ships in most Google-provided images. Its three capabilities map to three jobs: OS patch management runs on-demand or scheduled patch jobs and reports compliance; OS inventory management collects installed-package and OS-version data; and OS policies declaratively ensure a package or configuration is present (or absent) on matching VMs. VM Manager needs all of these before a VM reports in: the enable-osconfig=TRUE metadata key, a running OS Config agent, the OS Config API enabled in the project, a service account attached to the VM, and a network path to osconfig.googleapis.com — for VMs with only internal IPs, that means Private Google Access on the subnet (or Cloud NAT).

Choose the tool by the job: granting or revoking SSH through IAM is OS Login; patching, inventorying or enforcing configuration across many VMs is VM Manager. One governs who gets in, the other governs what runs on the machine.

Choose block storage by durability scope and performance model. A zonal Persistent Disk lives in one zone and is the default choice for boot disks and general data. A regional Persistent Disk synchronously replicates every write across two zones in the same region, so if the VM's zone fails you can force-attach the disk to a standby VM in the other zone — it exists for zonal failover, at higher cost and slightly higher write latency. Hyperdisk is the newest generation: IOPS and throughput are provisioned independently of capacity, so you no longer buy a bigger disk just to get more IOPS. The newest machine series support only Hyperdisk.

Within Persistent Disk: pd-standard (HDD-backed, cheap, sequential and backup workloads), pd-balanced (SSD, the general-purpose default), pd-ssd (higher IOPS for databases), and pd-extreme (provisioned IOPS on supported machine types). Hyperdisk variants follow the same logic, and their per-volume ceilings are far higher:

The VM's machine type also caps the performance it can drive, so a high provisioned value only helps on a machine series that supports it.

Synchronous versus asynchronous replication

Regional PD and Hyperdisk Balanced HA replicate synchronously across two zones of one region: no write is lost when a zone fails. Persistent Disk Asynchronous Replication copies a disk to a paired disk in another region, asynchronously: it protects against a regional outage, but the most recent writes may not have arrived when you fail over. Pick synchronous for zonal high availability with no data loss, asynchronous for cross-region disaster recovery.

Local SSD

Local SSD is physically attached to the host: extremely fast, but its data is lost if the host fails or the VM is deleted. Each standard Local SSD disk is 375 GiB (some machine series, such as Z3, attach larger fixed-size disks), so you add disks to get more capacity and performance. Data can be preserved across a VM stop or suspend when you ask for it (--discard-local-ssd=false), but nothing protects it against host failure, so Local SSD remains a place for caches and scratch data you can rebuild.

gcloud compute disks create data-disk --zone=us-central1-a --type=pd-ssd --size=200GB

gcloud compute disks create ha-disk --region=us-central1 --replica-zones=us-central1-a,us-central1-b --type=pd-ssd --size=200GB

Requirement mapping: surviving a zonal outage without restoring from backup means regional PD or Hyperdisk Balanced HA; surviving a regional outage means asynchronous replication or snapshots in another region; tuning IOPS separately from size means Hyperdisk; lowest-latency scratch data you can afford to lose means Local SSD; everything else defaults to zonal pd-balanced or Hyperdisk Balanced.

A managed instance group (MIG) keeps a set of identical VMs healthy and at the size you ask for, and it is built from an instance template — an immutable definition of machine type, image, disks, network and startup script. To change the fleet you create a new template and roll it out; you never edit a template in place:

gcloud compute instance-templates create web-template --machine-type=e2-small --image-family=debian-12 --image-project=debian-cloud --tags=http-server

gcloud compute instance-groups managed create web-mig --template=web-template --size=3 --region=us-central1

gcloud compute instance-groups managed set-autoscaling web-mig --region=us-central1 --min-num-replicas=3 --max-num-replicas=12 --target-cpu-utilization=0.65 --cool-down-period=90

To move existing instances to a new template, start a rolling update: gcloud compute instance-groups managed rolling-action start-update web-mig --version=template=web-template-v2 --region=us-central1. The MIG replaces VMs gradually, within the surge and unavailability limits you set (rolling-update strategy is covered in Section 3).

Autoscaling can key on CPU utilization, load-balancer serving capacity or Cloud Monitoring metrics, between the minimum and maximum you set. A regional MIG spreads instances across multiple zones, the standard high-availability pattern for VM-based web tiers; add an autohealing health check so the MIG recreates instances that fail application-level checks, not just crashed VMs.

Compute Engine cost has three levers that suit different usage patterns: Spot VMs for interruptible work, committed use discounts for steady always-on usage, and right-sized machine shapes for everything.

Custom machine types let you specify exact vCPU and memory (for example --custom-cpu=6 --custom-memory=24GB) on supported families. Each family allows a range of memory per vCPU; when an app needs more memory than that ratio allows, extended memory (--custom-extensions) adds memory beyond the ratio without buying extra vCPUs, billed at a higher per-GB rate. Extended memory is available on families such as N1 and N2, but not on E2 — E2 custom types top out at the standard ratio.

Right-sizing starts from measured utilization (for example Monitoring data or Compute Engine's machine-type recommendations): choose the smallest shape that still covers peak vCPU and memory use. A cheaper predefined shape is only a saving if it still fits — a highcpu type has about 1 GB of memory per vCPU, so dropping a VM that uses 20 GB onto an n2-highcpu-16 (16 GB) leaves it short of memory. When no predefined shape matches the measured need, a custom shape avoids paying for a size up the predefined ladder.

GKE offers two modes and several shapes, and you pick the combination from the requirements. Autopilot is the default and recommended mode: Google manages nodes, node pools and node-level security, you pay for Pod resource requests, and you cannot SSH into nodes. Standard mode gives you node pools to configure and pay for — the right choice when you need node-level control (custom node images, specific machine shapes, privileged system agents). A zonal cluster has its control plane in one zone; a regional cluster replicates the control plane and (in Standard) the nodes across multiple zones, so the Kubernetes API survives a zonal outage. Autopilot clusters are always regional.

A cluster with private nodes gives nodes only internal IP addresses, and access to the control-plane endpoint can be limited with authorized networks. Private nodes need two separate egress paths, which are easy to confuse:

gcloud container clusters create-auto prod-cluster --region=us-central1

gcloud container clusters create std-cluster --region=us-central1 --num-nodes=1 --enable-private-nodes --enable-ip-alias --master-ipv4-cidr=172.16.0.0/28

--num-nodes on a regional Standard cluster is per zone: three zones with --num-nodes=1 is three nodes in total.

To work with any cluster you need kubectl, installed as a gcloud component (gcloud components install kubectl), plus the GKE auth plugin (gcloud components install gke-gcloud-auth-plugin) that lets kubectl authenticate with your Google credentials. Without the plugin, kubectl commands fail with an error that the gke-gcloud-auth-plugin executable was not found. Then fetch credentials, which writes the cluster into your kubeconfig: gcloud container clusters get-credentials prod-cluster --region=us-central1. When kubectl has no credentials or points at the wrong cluster, get-credentials is the fix.

Deployment to GKE follows one pipeline: build an image, push it to Artifact Registry, create a Kubernetes Deployment that references the image, and expose it with a Service:

gcloud builds submit --tag us-central1-docker.pkg.dev/my-project/apps/web:v1

kubectl create deployment web --image=us-central1-docker.pkg.dev/my-project/apps/web:v1

kubectl scale deployment web --replicas=3

kubectl expose deployment web --type=LoadBalancer --port=80 --target-port=8080

In kubectl expose, --port is the port the Service listens on (what clients connect to) and --target-port is the port the container actually listens on; when they differ and you omit --target-port, traffic is sent to the wrong container port. A Service of type LoadBalancer provisions a Google Cloud passthrough load balancer with an external IP; ClusterIP keeps the Service internal to the cluster; and an Ingress (or the Gateway API) provisions an application load balancer for HTTP routing across services. In production you express all of this declaratively in manifests applied with kubectl apply -f deployment.yaml, which makes changes repeatable and reviewable.

Image pulls and permissions. GKE nodes pull images as the node service account, not as you. If Pods stick in ImagePullBackOff, check the image path for typos, then check that the node service account can read the repository — for an Artifact Registry repository in another project, grant roles/artifactregistry.reader to the node service account in that project. kubectl get pods and kubectl describe pod show the pull error. On Autopilot, also declare resource requests on every container: requests drive both scheduling and billing.

Cloud Run deploys a container image as an autoscaling HTTPS service in one command: gcloud run deploy api --image=us-central1-docker.pkg.dev/my-project/apps/api:v1 --region=us-central1 --allow-unauthenticated. Omit --allow-unauthenticated and callers must present IAM credentials with roles/run.invoker. Cloud Run functions put a function-as-a-service experience on the same infrastructure: you deploy source code with an entry point and a trigger, and Google builds and runs the container — for example gcloud functions deploy resize-image --runtime=python312 --trigger-bucket=uploads-bucket --entry-point=handler.

Services versus jobs

Work that runs longer than a request can last, or has no caller waiting for a response, belongs in a job (gcloud run jobs create), scheduled with Cloud Scheduler when it must run on a timetable.

Concurrency, minimum and maximum instances

Three settings trade cost against latency. Concurrency (--concurrency) is how many requests one instance handles at once (default 80); raising it lets fewer instances serve the same traffic, while setting it to 1 suits code that is not safe to run in parallel. Minimum instances (--min-instances) keep instances warm to avoid cold-start latency, but you pay for them while idle. Maximum instances (--max-instances) cap scale-out, which bounds cost and protects downstream systems such as a database with a limited number of connections.

Event processing has three patterns. Pub/Sub: a push subscription delivers each message as an HTTP POST to a Cloud Run service, or a function uses --trigger-topic — the standard pattern for decoupled, at-least-once message processing. Cloud Storage notifications: object finalize, delete, archive and metadata-update events trigger a function or, through Eventarc, a Cloud Run service — the pattern for reacting to uploads. Eventarc is the general routing layer: it delivers events from Google sources to Cloud Run and other destinations in CloudEvents format, for example gcloud eventarc triggers create new-file --destination-run-service=processor --event-filters=type=google.cloud.storage.object.v1.finalized --event-filters=bucket=uploads-bucket --location=us-central1 --service-account=eventarc-sa@my-project.iam.gserviceaccount.com.

Reacting to API calls: Eventarc audit-log triggers versus a hand-built sink

To run code when a specific API call happens (for example, a VM is created or a dataset is deleted), Eventarc can trigger on Cloud Audit Logs events, filtered by serviceName and methodName. Eventarc creates and manages the Pub/Sub plumbing for you. The alternative is to build it by hand: a Cloud Logging sink with a filter, a Pub/Sub topic, granting the sink's writer identity permission to publish, and a push subscription to your service. Both work; the Eventarc trigger is the one with the least to build and maintain, while a hand-built sink gives full control of the log filter and topic. Either way, the audit log must actually be written: Admin Activity logs are always on, but Data Access audit logs must be enabled for the service before data-access events (such as object reads) can trigger anything.

GPUs are the general-purpose accelerator: attach them to Compute Engine VMs, GKE nodes and Cloud Run for graphics, video transcoding, and training or serving models across a broad range of frameworks. TPUs are Google's custom ASICs built for large-scale tensor mathematics, the fit for very large model training or serving on frameworks with TPU support such as JAX, TensorFlow or PyTorch through PyTorch/XLA. TPUs are available as Cloud TPU VMs and as TPU node pools in GKE.

Remember the platform side effect: a Compute Engine VM with a GPU cannot live-migrate, so it uses the TERMINATE maintenance policy.