CCSP Secure Data Center Design: Tenant Partitioning, Location, HVAC, Connectivity and Resilience

Secure data center design is the set of decisions that make a facility protect the systems inside it: how tenants and administrators are separated logically, where the building sits and who owns it, how it is cooled and connected, and how its power, cooling and network survive failures. The CCSP outline groups these into four areas: logical design (tenant partitioning, access control), physical design (location, buy or build), environmental design (HVAC, multi-vendor pathway connectivity) and resilient design (power, HVAC, connectivity).

Who makes these decisions depends on where the workload runs. A provider building a cloud region makes all of them. A company building its own facility makes them too. A customer in colocation or public cloud makes almost none of the facility decisions, but still has to assess them: the provider's design becomes the customer's risk, verified through the contract, site visits where allowed and independent assurance such as SOC 2 reports, ISO/IEC 27001 certification and data center tier certifications. That is why CCSP questions often put you in the role of a customer reviewing a provider's design and asking what finding to raise.

Two habits run through the whole topic. First, find the root cause or the single point of failure a scenario describes and fix that, rather than adding more of something the design already has. Second, distinguish controls that prevent a failure (a diverse route, a separate network) from controls that only detect, record or compensate for it (an alarm, a session log, a service credit). The cloud infrastructure components themselves are covered in the platform components lesson, risk assessment of them in the infrastructure risk lesson, physical and environmental protection controls in the security controls lesson, and BC/DR planning in its own lesson.

Tenant network partitioning keeps each tenant's traffic in its own segment on shared switches and routers, so that no tenant can see or reach another's traffic. Three technologies do different jobs, and questions turn on picking the one that matches the constraint.

VLAN hopping. VLAN isolation holds only if the switch edge is configured correctly. In a double-tagging attack, a host sends a frame with two 802.1Q tags; a switch whose trunk native VLAN matches the outer tag strips it and forwards the frame into the VLAN named by the inner tag. In switch-spoofing, a host negotiates a trunk and receives every VLAN. The fixes are configuration: make tenant-facing ports access ports with trunk negotiation disabled, and set trunks to an unused native VLAN (or tag the native VLAN).

Segmentation is not encryption. A VLAN tag or VXLAN identifier tells the network which segment a frame belongs to; it does nothing to stop someone with access to the underlying links from reading it. When tenant segments cross a link the provider does not control, such as leased carrier fibre between two data centers, the traffic needs link or tunnel encryption (for example MACsec or IPsec). Only a control that actually encrypts gives confidentiality on that link; adding more encapsulation does not.

Compute and storage partitioning stop one tenant's workloads and data from being reachable through shared hosts and shared storage. The question to ask is always what must be isolated: traffic, memory, hardware or storage visibility.

In the normal multi-tenant model the hypervisor isolates virtual machines from each other on shared physical servers. When a requirement is about the hardware itself (no other tenant may run on the same physical server), the provider's answer is dedicated, single-tenant hosts: the customer's instances are placed on servers no other tenant uses, while self-service provisioning and scaling stay available. The principle is to match the control to the requirement's wording. Controls that protect data while sharing hardware, such as hypervisor hardening or memory protection, answer a confidentiality requirement, not a requirement about who shares the machine.

On a shared storage area network, two controls work together. Fabric zoning on the SAN switches decides which host initiators can talk to which storage ports. LUN masking on the array decides which logical units (volumes) are presented to each initiator. Zoning alone is not enough: a correctly zoned host that reaches a shared array port can see every volume on that port unless LUN masking restricts it. Zoning governs reachability at the fabric; masking governs visibility at the array.

The management plane (hypervisor management interfaces, switch consoles, server baseboard management controllers, usually reached over IPMI or Redfish) is the most powerful set of interfaces in a data center, so the design must keep tenants from reaching it at all. The standard design is an out-of-band (OOB) management network: a separate network carrying only administrative traffic, kept apart from tenant and production networks. In-band management, by contrast, shares production paths.

Three design points come up repeatedly:

• Remove exposure before hardening. Controls placed in front of an exposed management interface reduce the attack surface but leave the path open. The design fix is that no network a tenant can route to reaches the management interfaces at all, which is what a dedicated OOB network achieves.
• One controlled entry point. Every administrative session should pass through one hardened point, typically jump hosts (bastions) or a privileged access gateway, that enforces multifactor authentication and records the session. Concentrating access in one place means authentication and accountability are enforced together and cannot be bypassed by a path that skips that point.
• Physical independence for availability. A management VLAN on the production core switches is logically separate but shares hardware and uplinks, so it fails in exactly the outage that needs console access. Where management access must survive production failures, build a physically separate network with its own switches, console servers and an independent uplink.

Separating physical and logical access

Separation of physical and logical access means the people who can touch the hardware cannot also log in to the systems or data on it, and vice versa. A person with both can bypass controls on either side, for example by copying data through an administrator account and then removing the disk, or by pulling media that logical controls would never release.

In practice:

• Data center technicians who swap disks, reseat cables or replace servers get physical access only. Logical rights (array administration, OS logins) stay with the storage or system teams, who authorise and verify the work.
• Controls that strengthen or monitor a technician's logical access reduce or record the risk, but both kinds of access still sit with the same people. The root-cause fix is to separate the duty, not to watch it.
• In colocation, remote hands staff do physical tasks; the customer's own administrators do logical work through a remote console on each server's management controller, reached over an out-of-band network. Handing a technician administrator credentials to work on your behalf gives them full logical access, and nothing applied around that handover separates the duties again.

Encrypting drives is still good practice, because it protects media that leaves the facility, but it is a complementary control, not a substitute for separating duties.

Choosing a data center location is a risk decision with two halves: the hazards the site is exposed to, and the laws that reach the data held there.

Hazards. A site review covers natural hazards (flood plains, seismic zones, severe storms, wildfire) and man-made hazards (chemical plants, fuel depots and pipelines, transport corridors carrying hazardous goods, neighbouring industry, and dependence on one utility substation). Cheap land often prices in a neighbour's risk, so a review that rates only natural hazards is incomplete.

Hazard independence versus distance. A secondary site exists to survive what disables the primary. If both sites share a river valley, a regional power grid or a fault line, one event can take out both, however good each site is. Distance creates a tension, because synchronous replication (near-zero data loss) is only practical over short distances, while surviving a regional disaster needs distance. When both are required, the common design is two nearby sites replicating synchronously, plus a distant third site replicating asynchronously. Redundancy inside a site addresses equipment failure; only geographic separation addresses an event that affects the whole region. (Recovery objectives themselves belong to the BC/DR lesson.)

Jurisdiction. Where a data center sits decides which laws reach the data in it: data protection law, cross-border transfer rules and government access powers. When hazards, power and connectivity are comparable, the legal regime usually outweighs operating-cost advantages for a provider holding regulated data. Two refinements matter:

• The operator's jurisdiction travels with it. A provider headquartered in a country with extraterritorial disclosure laws (the US CLOUD Act is the usual example) can be compelled by its own authorities to produce data it controls anywhere. GDPR Article 48 says such foreign orders are not automatically recognised in the EU, but that creates a legal conflict for the provider; it does not remove the exposure. Location alone is not sovereignty.
• Residency covers every copy and every access path. A requirement that data stays in-country fails if backups replicate to a site across the border or support staff abroad can log in remotely. A control that protects or supervises a copy or a session held abroad changes how risky it is, not where it is, so it does not satisfy a residency rule.

The building and its layout

Inside the building, the data hall should sit where threats from every side are furthest away. The usual guidance is an interior space on a middle floor, away from exterior walls: basements risk flooding, top floors risk roof leaks and weather, and ground-floor exterior walls (especially near loading docks or roads) risk vehicle impact, observation and forced entry. The data hall should also not sit below water-carrying rooms such as kitchens, washrooms or plant rooms, because their plumbing runs overhead.

The general rule bends to the site. In a coastal storm-surge zone, for example, every level below the expected surge height is ruled out however well protected it otherwise is, and the choice moves to the interior spaces above it that still meet the other rules.

Layout also separates activities. Deliveries should arrive at a loading dock that leads to a staging (receiving) room where equipment is unpacked, inspected and burned in before it moves into the data hall. A dock that opens straight into the hall mixes untrusted people, dust, packaging and unverified hardware with production systems; controls added around that layout do not change what it lets into the hall. Other layout elements (perimeter, mantraps, zones of increasing security) are physical protection controls covered with the security controls topic.

Buy or build is the choice between constructing and operating your own facility and using capacity someone else operates. The trade-off is control against capital, time and flexibility, and the security consequence is which controls transfer to a provider.

Responsibility moves but accountability for the data does not. A colocation customer keeps logical access control, encryption and patching on its own servers, while perimeter security, power and cooling now rest mainly with the provider. Because the customer no longer operates those controls, it must verify them: contract terms and SLAs, the right to audit or visit, and independent reports such as a SOC 2 Type II report, ISO/IEC 27001 certification and the facility's tier certification.

Environmental design keeps equipment inside the temperature and humidity range its manufacturers support, as efficiently as possible. Servers draw cool air in at the front and exhaust hot air at the back, so the design goal is to deliver cool air to intakes and remove hot air without the two mixing.

• Hot aisle / cold aisle. Racks are arranged front-to-front and back-to-back, so intakes share cold aisles and exhausts share hot aisles. If all rows face the same way, each row's exhaust feeds the next row's intakes, causing hot spots. Adding cooling capacity compensates at a higher energy cost; arranging the rows correctly removes the cause.
• Containment. Enclosing the cold aisle or the hot aisle stops the remaining mixing over and around rows.
• Blanking panels fill unused rack spaces so hot exhaust cannot recirculate through the rack to the front.
• Thermal envelope. ASHRAE's thermal guidelines give a recommended inlet range (18–27 °C for most air-cooled equipment classes, A1 to A4; the high-density class H1 added in the 2021 guidelines has a narrower recommended range) and wider allowable ranges. Many halls run colder than needed; once containment is in place, raising supply temperature toward the top of the recommended range cuts chiller load and increases free-cooling (economiser) hours while staying within supported conditions.

Humidity has a window with a risk at each end:

Dew point is the better control variable than relative humidity, because condensation forms wherever a surface is colder than the air's dew point; relative humidity changes whenever the air temperature changes. Colder supply air or chilled pipework therefore needs a lower dew point limit. Airside economisers bring outside air in for free cooling, which means outdoor humidity becomes indoor humidity: cold, dry winter air heated to supply temperature has very low relative humidity, so an economiser in a cold, dry climate needs humidification to keep running safely.

Multi-vendor pathway connectivity means the data center's external network links come from more than one carrier over physically separate routes, so that no single carrier outage or cable cut isolates the site. Both kinds of diversity are needed:

• Carrier diversity protects against one carrier's outage. Several circuits from one carrier, however routed, fail together when that carrier fails.
• Path diversity protects against a single cut. Circuits from different carriers that share a conduit, a duct or a building entrance fail together when that shared element is cut.

Diversity has to be verified, not assumed. Carriers lease fibre from each other, so two carrier names can share one duct kilometres from the building. The control is to require documented physical routes for each circuit from the building entrance to the carrier's network, and to check them for shared segments.

Inside the property, diverse routes need separate building entrances: two or more entrance rooms (meet-me rooms) on different sides of the building, each with its own external route. Two carriers terminating in one entrance room still share a single point of failure: the room itself. Remember also that links crossing a carrier's network are outside your control: segmentation does not hide traffic from the carrier, so encrypt what crosses it.

Redundancy notation describes how much spare capacity a system has relative to the load it must carry, N:

Counting spares tells you which component failures are tolerated; the next step is to look for the shared element they all depend on. A generator bank with a spare that all feeds one set of paralleling switchgear survives a generator failure, but the switchgear is a single point of failure that no number of generators covers. The same holds for any redundant plant that converges on one shared component. Spare components protect capacity; only independent distribution paths remove the shared element.

The Uptime Institute Tier Classification is the widely used vendor-neutral benchmark built on that distinction:

Tier III is the lowest tier that allows planned maintenance without downtime; the difference between Tier II and Tier III is the number of independent distribution paths, so that is what moving from one to the other adds. When assessing a provider, check what was certified: the Uptime Institute's Tier Certification of Design Documents confirms the drawings, the Tier Certification of Constructed Facility confirms the built site performs to the tier, and a separate Operational Sustainability certification covers how it is run. Any assurance report answers only the question it was designed to test, so evidence about infrastructure topology comes from a tier certification, not from a general security attestation. ANSI/TIA-942 offers a comparable rating scheme (Rated-1 to Rated-4).

Resilient power design layers independent sources so the IT load never sees an interruption, and resilient cooling design makes sure heat removal survives the same events.

Power chain.

• Utility feeds should come from different substations over diverse routes. Two feeds from one substation are one source with two cables.
• The UPS carries the IT load the instant utility power fails and bridges the seconds to minutes generators need to start and stabilise. It is not sized for whole outages.
• The automatic transfer switch (ATS) moves the load between utility and generator.
• Generators carry the outage, for as long as their fuel lasts. In regions where storms can cut power for days and close roads, the gap is fuel endurance, not generator count: size on-site fuel storage for the longest credible outage and hold priority delivery contracts.
• A and B feeds to each rack let dual-corded servers survive the loss of either side. Single-corded equipment inherits one feed's single point of failure unless it is connected through a rack-mounted automatic (or static) transfer switch fed from both A and B; anything added on one side only still depends on that side.

Cooling resilience. A dense data hall overheats within minutes if cooling stops, even while power stays on, so cooling needs redundancy comparable to power. Common gaps:

• Chiller restart time. After a power event, chillers can take several minutes to restart on generator power while IT keeps producing heat. Chilled-water thermal storage, with pumps and air handlers on UPS power, keeps cooling flowing through that window.
• Single distribution path. N+1 chillers and air handlers on one supply and return loop fail together when the loop fails. Dual loops, or a valved ring that lets a section be isolated, remove that single point of failure.
• Cooling on one power side. Redundant power for IT load is wasted if the cooling plant hangs off only one of the power paths: lose that path and the servers keep running while the room heats up. Cooling (chillers, CRAH units, pumps) must be spread across the independent power paths, or fed through transfer switches, so that losing any one path leaves both power and cooling.

Connectivity resilience follows the same logic, as covered in the pathway section: independent carriers, independent routes, independent entrances. Every layer is only as resilient as its weakest shared element.