When You Actually Need a Service Mesh, and When It's Just Complexity

Where the problem comes from

As the number of services in a system grows, the same concerns get rewritten in every service: how many times to retry a failed call, what timeout to use, when to trip a circuit breaker, how to enforce mutual TLS, how to trace which request went where. If every team writes this by hand in its own HTTP client, behavior diverges across languages and libraries. If it's moved into a shared library instead, every service now has to upgrade that library in lockstep, which stops being realistic once an organization has more than a handful of teams.

A service mesh moves this responsibility out of application code and into the network layer. A proxy (sidecar) sits next to every pod, services talk to their local proxy instead of directly to each other, and traffic between proxies follows rules distributed by a control plane. Application code stops caring about retries, timeouts, or TLS; those live in the mesh's configuration instead.

This comes at a cost: an extra network hop, extra CPU and memory, and a new control plane to operate. Using a mesh well means being clear about what that cost is actually buying you.

Three different approaches

Istio uses an Envoy-based sidecar proxy and currently has the richest feature set of the major meshes: weighted traffic splitting, fault injection, detailed metrics, fine-grained authorization policy. The cost is the highest operational complexity and the highest resource footprint; every pod gets an Envoy sidecar, and those sidecars have their own memory limits and their own upgrade cadence to manage.

Linkerd uses its own lighter proxy (linkerd2-proxy, written in Rust). Its feature set is narrower than Istio's, but its installation and mental model are much simpler; for most teams, automatic mTLS, basic retries/timeouts, and the golden signals (latency, error rate, request volume) are already enough.

Cilium, being eBPF-based, can implement part of what a mesh does (mTLS, L3/L4 policy, observability) directly in the kernel, without a sidecar. The sidecarless approach removes the per-pod proxy container overhead, but its L7 traffic routing and fine-grained routing rules aren't as mature as Istio's yet.

When you actually need one

What justifies a mesh isn't the number of services, it's whether at least one of the following is genuinely true:

Multiple teams share a cluster and each wants to define its own traffic policy (retries, timeouts, circuit breaking) for its own services without a central platform team being in the loop for every change. A mesh makes this kind of autonomy possible because the policy lives in Kubernetes resources (CRDs) instead of application code.

Mandatory, automatic mTLS is required: certificate issuance, rotation, and verification need to be handled consistently across every service by the infrastructure, not by hand or in application code. In environments with compliance requirements, this is usually the strongest reason on its own.

Canary releases or A/B splits need to happen at L7, based on an HTTP header or user identity, not just weighted routing.

Per-service golden signals (latency, error rate, request volume) need to be collected automatically, without touching application code.

When you don't

In a system with ten or fifteen services owned by a single team, a mesh usually does more harm than good. If mTLS is the requirement, issuing certificates with cert-manager and letting the application terminate its own TLS, or terminating TLS at the ingress if the service graph is simple, is enough. If the only need is canary releases via weighted traffic, the Gateway API's HTTPRoute resource already solves this with weighted backendRefs, no mesh required; you can shift traffic gradually between two versions by adjusting weights alone. Retries and timeouts already exist in most modern HTTP client libraries; if the actual problem is just getting every team to configure them consistently, a shared configuration template solves that more cheaply than a mesh does.

In short, a mesh is not a scaling solution, it's an autonomy and automation solution. In small, single-team systems, the automation it provides doesn't cover its operational cost.

A simple traffic-splitting example

With Istio, weighted traffic splitting between two versions is done by defining subsets in a DestinationRule and assigning weights in a VirtualService:

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: payment-service
spec:
  host: payment-service
  subsets:
    - name: v1
      labels:
        version: v1
    - name: v2
      labels:
        version: v2
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: payment-service
spec:
  hosts:
    - payment-service
  http:
    - route:
        - destination:
            host: payment-service
            subset: v1
          weight: 90
        - destination:
            host: payment-service
            subset: v2
          weight: 10

Here, ten percent of traffic goes to the new version; if error rate and latency stay healthy, the weight is raised gradually.

Pitfalls

Sidecar startup ordering. The sidecar proxy starts at roughly the same time as the application container, but if the application sends its first request before the proxy is ready, the connection gets refused. Kubernetes' native sidecar container support has largely fixed this ordering, but on older clusters or setups that don't use it, occasional "connection refused" errors right at pod startup are usually this race condition.

Turning on strict mTLS in one step. A workload outside the mesh, such as kubelet doing a plain HTTP health check, or an old service that hasn't been onboarded yet, suddenly can't connect once mTLS becomes mandatory. The safer path is to roll it out in permissive mode first, observe which services are still using plain TLS or HTTP, clean those up, and only then switch to strict.

Not accounting for sidecar cost per pod. In a cluster with hundreds of pods, the proxy container added to every one of them adds up to a real chunk of CPU and memory demand. This is easy to leave out of capacity planning, and cluster resource usage jumps in a way nobody expected right after the mesh goes in.

Session consistency during canary rollouts. Weighted routing distributes each request independently, so consecutive requests from the same user can land on different versions. If that collides with cached session state or a feature flag decision, it shows up as inconsistent behavior. The fix is to define consistent-hash-based routing on a cookie or header inside the DestinationRule, so a given user stays on the same version for the life of the request flow.

Control plane and data plane version drift. Upgrading the cluster's Kubernetes version without checking the mesh's own supported version range can land you on an unsupported API version, and traffic management breaks silently. The mesh's supported Kubernetes version window should be checked before every cluster upgrade, not after.

Conclusion

Adopt a mesh because of a concrete automation or compliance need, not because the system "grew." If the only need is canary releases, the Gateway API's weighted routing is usually enough on its own. If the need is genuinely automatic, consistent mTLS and autonomy across teams, starting with Linkerd's simpler model and only moving to Istio's full feature set once a concrete need appears, such as rich L7 traffic management or fault injection, is the lower-risk path for most setups.