Achieving Traffic Mirroring in Kubernetes with Istio

Traffic mirroring, also known as shadowing, is a powerful feature in Istio that allows you to duplicate live traffic and send it to another version of your service without impacting the original traffic flow. This is especially useful for testing new versions of your application in real-world scenarios.

What is Traffic Mirroring?

Traffic mirroring involves duplicating real user traffic from a primary service and sending it to a secondary service. The primary service handles all user requests as usual, while the secondary service (often a new version or experimental setup) receives the same requests but does not impact the user experience.

This technique allows developers to test, monitor, and validate the performance and behavior of the new service version under real traffic conditions, without risking disruptions to the live application.

 

Why Use Traffic Mirroring?

Traffic mirroring is an effective strategy for:

• Testing New Versions: Validate the performance and correctness of a new service version in a production-like environment.
• Monitoring and Debugging: Identify potential issues or unexpected behavior in the new version by observing how it handles real traffic.
• Incremental Rollouts: Gradually introduce changes by mirroring traffic before rolling out the new version to actual users.
• Load Testing: Simulate real-world traffic conditions to ensure the new service can handle the expected load.

Advantages of Traffic Mirroring

1. Zero Impact on Users: Users are not affected since the mirrored traffic does not alter the primary service’s responses.
2. Real-World Validation: Test with actual production traffic instead of synthetic test cases.
3. Reduced Risk: Identify and fix issues in the new service version before it becomes the primary service.
4. Improved Observability: Gain insights into how the new version performs under real traffic.
5. Seamless Deployment: Prepare for smoother rollouts and minimize rollback risks.

In this blog post, we will explore how to achieve traffic mirroring in Kubernetes using Istio, step by step, with a practical example.

Prerequisites

Before you begin, ensure the following:

1. A Kubernetes cluster is set up and running.
2. Istio is installed and configured in your cluster.
3. You have a basic understanding of Kubernetes and Istio concepts.
4. kubectl is installed and configured to interact with your cluster.

Step 1: Set Up the Demo Application

Let’s deploy a sample application with two versions: v1 and v2.

Deploy Application Version v1

apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin-v1
  labels:
    app: httpbin
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpbin
      version: v1
  template:
    metadata:
      labels:
        app: httpbin
        version: v1
    spec:
      containers:
        - name: httpbin
          image: kennethreitz/httpbin
          ports:
            - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: httpbin
spec:
  ports:
    - port: 80
      targetPort: 80
  selector:
    app: httpbin

Apply the manifest:

kubectl apply -f httpbin-v1.yaml

Deploy Application Version v2

Create a similar manifest for httpbin-v2:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin-v2
  labels:
    app: httpbin
    version: v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpbin
      version: v2
  template:
    metadata:
      labels:
        app: httpbin
        version: v2
    spec:
      containers:
        - name: httpbin
          image: kennethreitz/httpbin
          ports:
            - containerPort: 80

Apply this manifest:

kubectl apply -f httpbin-v2.yaml

Step 2: Configure Istio VirtualService

Istio’s VirtualService allows us to define traffic routing rules. To mirror traffic, we need to route 100% of live traffic to httpbin-v1 and duplicate (mirror) it to httpbin-v2.

Create the following VirtualService:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
    - httpbin
  http:
    - route:
        - destination:
            host: httpbin
            subset: v1
      mirror:
        host: httpbin
        subset: v2

Apply this manifest:

kubectl apply -f virtualservice.yaml

Step 3: Define DestinationRules

To route traffic to specific versions (v1 and v2), we need to define DestinationRules.

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: httpbin
spec:
  host: httpbin
  subsets:
    - name: v1
      labels:
        version: v1
    - name: v2
      labels:
        version: v2

Apply this manifest:

kubectl apply -f destinationrule.yaml

Step 4: Verify Traffic Mirroring

Send traffic to the httpbin service using curl:

curl http://<EXTERNAL-IP>/status/200

Check the logs of both httpbin-v1 and httpbin-v2 pods:

• httpbin-v1 should show logs for all incoming traffic.
• httpbin-v2 should show logs for mirrored traffic.

To view logs:

kubectl logs -l app=httpbin -l version=v1


kubectl logs -l app=httpbin -l version=v2

Conclusion

Traffic mirroring in Istio is a valuable tool for testing new application versions under real traffic conditions without affecting users. By following the steps above, you can easily set up and verify traffic mirroring in your Kubernetes cluster.

Happy mirroring!