Table of Contents
Most Azure estates I’ve worked in didn’t start private. They started with storage accounts, Key Vaults and SQL servers on public endpoints, an IP allow-list on each one, and a plan to sort the networking out later.
Later arrives one service at a time. Someone turns on a service endpoint because it was a checkbox. Someone else deploys a private endpoint for the one workload that went through a security review. A year on, you have both, in different spokes, and nobody can say with confidence which path a given request actually takes.
This post is about getting that under control in a hub and spoke design. I’ll walk through what service endpoints and private endpoints actually do to your traffic, where DNS quietly breaks private endpoints, and the order I’d move an estate towards private networking. Securing and inspecting that traffic is a big enough topic for its own post, so that’s Part 2.
Two questions, not one
The mistake I see most often is treating service endpoints and private endpoints as two flavours of the same thing. They’re not. Each one answers two separate questions differently:
- Routing. Which path does the packet take from the client to the service?
- DNS. What IP address does the client get back when it looks up the service’s name?
| Service endpoint | Private endpoint | |
|---|---|---|
| What the client resolves | The service’s public IP; DNS unchanged | A private IP, when DNS is configured correctly |
| What changes on the network |
Routes to the service’s public prefixes, with next hop VirtualNetworkServiceEndpoint
|
A private IP on a managed NIC, with an InterfaceEndpoint route
|
| Scope of the endpoint | A service type enabled on a subnet | A particular resource and supported subresource |
| Access from on-prem over private connectivity | Not directly through the service endpoint | Yes, with routing and DNS in place |
| Public endpoint used by this path | Yes, subject to the service’s network rules | No; public access must still be disabled separately |
Keep those two questions in mind. Almost every private endpoint problem I’ve debugged was one of them answered wrongly, and it’s usually DNS.
Service endpoints: private source, public destination
A service endpoint is a setting on a subnet, not a separate endpoint resource. You enable Microsoft.Storage, Microsoft.KeyVault, Microsoft.Sql and so on per subnet, and two things happen.
First, traffic from that subnet to the service switches its source address from a public IP to the client’s private VNet address. Azure also passes the subnet identity, which is what lets you add a virtual network rule on the resource saying “allow this subnet”. The private IP on its own isn’t a unique identity: different VNets can use overlapping address ranges.
Any open TCP connections to the service drop during the switch, so don’t flip it on mid-batch-job. Get the resource’s network rules ready first.
Second, Azure adds system routes for the service’s public address ranges, with a next hop of VirtualNetworkServiceEndpoint. DNS doesn’t change at all. mystorage.blob.core.windows.net still resolves to a public IP; you’re just reaching it over an optimised path on the Microsoft backbone.
The route that ignores your firewall
This is the bit that catches hub and spoke designs out. Service endpoint routes take precedence over BGP and user-defined routes for the matching service prefixes.
So if you have the standard spoke route table with 0.0.0.0/0 pointing at the firewall in the hub, and someone enables a storage service endpoint on that spoke subnet, the matching storage traffic now leaves directly from the spoke. It never reaches the hub. Your firewall logs look clean because the firewall never saw it.
If inspection is required, there’s a different pattern: leave service endpoints off the workload subnet, route traffic through the hub NVA, and enable the service endpoint on the NVA’s subnet. The service’s network rules then allow that subnet. The appliance’s forwarding and source translation need to support that design; simply enabling an endpoint in the hub doesn’t make it apply to every spoke.
The important decision is where the endpoint is enabled. That’s what changes the path.
What service endpoints don’t do
- No direct on-prem path. An on-prem client doesn’t gain a service endpoint identity because it has ExpressRoute or a VPN. Without a separate proxy or translation design, it still needs access to the public service endpoint through the relevant network rules.
- No resource-specific outbound restriction by default. Enabling a storage service endpoint doesn’t restrict a subnet to your storage accounts. A compromised VM could still reach another account that permits access. Service endpoint policies can narrow the allowed destinations for supported Azure Storage scenarios.
- No private destination address. You’re still using the service’s public endpoint. Locking that endpoint to selected subnets is different from disabling public network access.
- Service-specific limitations. Azure SQL service endpoints, for example, require the VNet and logical server to be in the same region. Don’t assume every service behaves identically.
There’s also a newer standard service endpoint model that integrates with Network Security Perimeter and uses a public IP as a network identifier. That’s a separate access-control model I’ll come back to in Part 2. The subnet-rule discussion here is about classic service endpoints.
Private endpoints: a private address for a specific service
A private endpoint gives you a network interface in one of your subnets, with a private IP, connected over Private Link to a particular resource and subresource. Not “Azure Storage”, but the blob service of stdatalakeprod01. A storage account using blob and file needs separate endpoints for those subresources. ADLS Gen2 workloads can need both blob and DFS endpoints, so check what the application actually uses.
That scoping is the big difference from service endpoints. Each endpoint gives you a private path to its target. It doesn’t restrict all the client’s other outbound traffic, and it doesn’t replace authentication or authorisation.
With routing, DNS and network controls in place, clients can use that private IP from the VNet, connected VNets, or on-prem over VPN or ExpressRoute private peering.
What a private endpoint doesn’t do on its own
Creating a private endpoint does not close the public endpoint. The storage account, Key Vault or SQL server can still accept public connections, subject to whatever network rules it had before. You have to disable public network access separately.
That’s deliberate. It lets you build the private path alongside the public one and switch over when you’re ready. But it does mean “we’ve got private endpoints” and “our PaaS is private” are two different statements, and I’ve lost count of the number of times I’ve heard the first used to mean the second.
Which brings us to the part that does the real work: making clients actually use the private IP.
DNS is where private endpoints actually break
Your applications connect to stdatalakeprod01.blob.core.windows.net, not to an IP. Whether that name returns the private endpoint’s IP or the public one decides which destination they try to reach. Get it wrong and the app quietly goes over the public path, or fails once public access is off.
How the CNAME chain works
For Azure Storage, creating a private endpoint introduces a privatelink name into the public DNS chain. Simplified, it looks like this:
stdatalakeprod01.blob.core.windows.net
CNAME stdatalakeprod01.privatelink.blob.core.windows.net
A 10.20.4.5 (through your private DNS configuration)
or a public IP (through public DNS)
The application keeps using the normal service hostname. Your DNS resolution path needs to find the private record. In Azure, that normally means an Azure Private DNS zone called privatelink.blob.core.windows.net, holding an A record for the endpoint and linked to the VNet through which Azure DNS resolves the query.
Centralise the zones
In a shared hub and spoke design, I’d keep the zones in the connectivity subscription, created once and managed centrally. Don’t let every spoke create its own privatelink.blob.core.windows.net and expect the records to merge. They won’t. Using Azure Policy to deny private dns zone creation outside of the connectivity subscription is a useful way to prevent the next —> next —> deploy scenario that attempts to create the zone for you when you create the endpoint.
When a private endpoint is created, attach it to the central zone with a private DNS zone group. That manages the associated records with the endpoint’s lifecycle, instead of leaving someone to maintain them by hand.
Getting spokes to resolve the central zones
Peering doesn’t carry DNS. A private zone only answers for VNets linked to it, so you need to decide how a spoke’s query reaches one.
| Pattern | How it works | Watch out for |
|---|---|---|
| Link the zones to the spokes | Each spoke uses Azure-provided DNS and has links to the required private zones | Link sprawl; automate it |
| Centralised custom DNS | Spokes use the hub resolver’s inbound endpoint, or a central DNS service that forwards to it; zones are linked to the resolver’s VNet | Central DNS becomes a dependency for spoke name resolution |
| Forwarding ruleset | Spokes keep Azure-provided DNS; a linked ruleset forwards the required namespaces to the hub’s inbound endpoint | Requires an outbound endpoint for the ruleset; avoid forwarding loops and account for regional link limits |
If a ruleset forwards a namespace back to the resolver’s own inbound endpoint, don’t link that ruleset to the resolver’s own VNet, or you create a loop.
The right pattern depends on the DNS you already operate. The important thing is that it’s deliberate and consistent. A central zone with no working resolution path from the client is just a neatly organised outage.
On-prem resolution
On-prem DNS servers can’t query Azure Private DNS zones directly. They need a reachable Azure DNS Private Resolver inbound endpoint, or a DNS forwarder in Azure that can resolve the linked zones.
Microsoft’s Private Endpoint integration guidance recommends conditional forwarding for the service’s public namespace, such as blob.core.windows.net or database.windows.net, to that Azure resolution path. Azure can then resolve the original name and follow the CNAME chain using the private zone.
Forwarding a public namespace doesn’t mean creating a private authoritative copy of it. The latter can hide public records you haven’t recreated. With forwarding, names without a matching private override can still resolve publicly.
Check the recommended public forwarding namespace and private zone name for each service. They aren’t interchangeable.
Fallback to internet
There’s a classic failure in a central-zone design. A workload needs a partner’s storage account, and that account also has a private endpoint. Public DNS leads to partnerstore.privatelink.blob.core.windows.net, but your private zone has no record for it. Without fallback, it returns NXDOMAIN, even if the partner allows public access.
Azure now has a native option for this: set the private DNS zone’s VNet link resolution policy to NxDomainRedirect, shown in the portal as Enable fallback to internet. For supported Private Link zones, Azure retries public resolution when the private lookup returns NXDOMAIN.
Apply that setting where the query is actually resolved. In a central resolver design, that can be the zone’s link to the hub VNet rather than a link to each spoke.
It isn’t connection failover. An existing private record pointing at an unreachable endpoint doesn’t trigger public fallback. It also means a missing private record can resolve publicly, so keep public network access disabled wherever private-only access is required.
This matters during migration too. Linking an empty private zone can already change existing resolution. “We haven’t created any endpoints yet” doesn’t make a DNS change harmless.
Multi-region
One shared zone doesn’t automatically give clients the nearest healthy private endpoint. If you create endpoints for the same resource in multiple regions, decide how regional clients resolve and reach them, and how failover updates are managed. Don’t assume several A records provide regional steering or health checks.
Routing: the /32 you didn’t ask for
Private endpoints also change routing. An endpoint normally introduces a /32 route for its IP, with a next hop of InterfaceEndpoint, visible to clients in its VNet and directly peered VNets.
Under the default private endpoint network-policy configuration, your spoke’s 0.0.0.0/0 route to the firewall doesn’t catch that traffic. The endpoint has a more specific path.
You can inspect the routes on a client VM’s NIC:
az network nic show-effective-route-table \
--resource-group rg-app-prod \
--name nic-app-vm01 \
--output table
Look for the endpoint’s IP and the active next hop. Don’t assume traffic is inspected because the subnet has a default route to the firewall.
There is an exception. Enable route-table network policies on the private endpoint subnet and eligible UDRs can override the endpoint route. Those routes can use an aggregate prefix at least as specific as the endpoint VNet’s address-space prefix; they don’t always need a separate /32 per endpoint. A default route alone still doesn’t do it.
Placement matters as well. Endpoints in the hub normally have a direct path from peered spokes. Endpoints in a workload spoke may be reached through the hub NVA from another spoke, assuming suitable UDRs and forwarding settings and no direct peering. On-prem traffic arriving through the hub gateway can also have a direct path that bypasses inspection.
None of that guarantees a working inspected connection. The return path and source translation matter too. I’ll cover the placement choices, UDRs and routing symmetry together in Part 2, because splitting those decisions apart is how you end up with a design that looks right and drops packets.
Moving an estate towards private
If you’re starting from public endpoints and allow-lists, this is the order I’d go in. Build the private path alongside the public one, prove it works from every source, and only then take the public path away.
- Inventory the clients and existing DNS. Include on-prem systems, Azure workloads, partner integrations, build agents and administrative access. Find existing private endpoints and duplicate zones before introducing another shared namespace.
- Build and test the DNS foundations. Create the central zones and resolver path, then introduce links and forwarders in a controlled way. Check existing private records and external dependencies, and decide where fallback is appropriate before broad rollout.
- Put the automation in before scaling the endpoints. Policy should attach the appropriate zone groups, so the first endpoint and the hundredth get the same DNS treatment. Remediate existing endpoints as well.
- Create private endpoints alongside public access. Check the required subresources and connection approval. Once private DNS answers reach clients, new connections can start using the private path; allow for DNS caching and existing connections.
- Validate from every kind of client. Each spoke, the hub, on-prem, developer laptops on VPN, self-hosted build agents and Azure services accessing the resource. Check the normal service hostname resolves privately, then test an actual authenticated operation. Name resolution alone doesn’t prove the route, permissions or application work.
- Use logs to find what you’ve missed. Enable the relevant service diagnostics and review callers over a representative operating period, including scheduled jobs. Treat source IPs as supporting evidence: service endpoint traffic also uses private source addresses, and logging behaviour varies by service.
- Disable public network access and test again. Make the change per resource in a controlled window. Prove expected clients still work and the public path is blocked. Include less frequent operational tasks before calling the migration complete.
- Stop it regressing. Use service-specific Azure Policy controls to enforce the intended public-access settings, with an exception process for workloads that need one.
Where do service endpoints fit? They can be a useful stepping stone if you need to restrict a public endpoint to selected Azure subnets before the private DNS foundations are ready. They can also be a deliberate long-term choice when that access model meets the requirement.
But they don’t meet a requirement to disable public network access. On Azure Storage, for example, disabling it blocks service endpoint access through virtual network rules too, because that traffic still uses the public endpoint.
This is also where my usual point about governance comes in. Private networking isn’t a project you finish. It’s a set of guardrails that either stops the next public storage account being created or doesn’t.
Private endpoint vs service endpoint: which to use
For a hub and spoke estate that needs hybrid access, private destination addresses and public network access disabled, private endpoints are where I’d start. That’s a requirement-led choice, rather than a rule that service endpoints are wrong for every production workload.
| If you need… | Use |
|---|---|
| Access from on-prem over VPN or ExpressRoute private peering | Private endpoint, with routing and DNS configured |
| A private connection to a particular resource/subresource | Private endpoint |
| Public network access fully disabled | Private endpoint, where supported by the service |
| Selected Azure subnets accessing the public service endpoint without DNS changes | Service endpoint and resource network rules |
| A quick restriction while building the private networking foundations | Service endpoint can be a useful interim step |
| Central traffic inspection | Design the routing and return path explicitly for either option |
There is a cost and operating model decision here too. Classic service endpoints have no additional endpoint charge. Private endpoints add endpoint and data-processing charges, and a central resolver adds its own cost. You also take on DNS, IP allocation and endpoint lifecycle management. At scale, those are platform responsibilities, not details to leave to each application team.
Whichever you use, check the effective routes. Both can create paths that bypass the default route most hub and spoke designs rely on.
Next: who’s allowed to talk to it?
At this point you should know what your clients resolve, which path they take, and whether public access is actually closed. Private connectivity still needs access controls and, where required, inspection.
In Part 2 I’ll cover NSGs on private endpoint subnets, forcing traffic through an NVA without breaking routing symmetry, and Network Security Perimeter as another way to control access to supported PaaS resources.
Sources
- Azure virtual network service endpoints
- What is a private endpoint?
- Azure Private Endpoint DNS integration scenarios
- Azure Private Endpoint private DNS zone values
- Azure DNS Private Resolver endpoints and rulesets
- Fallback to internet for Azure Private DNS zones
- Manage network policies for private endpoints
- Use private endpoints for Azure Storage
- What is a standard service endpoint?