In modern cloud-native Kubernetes environments, security is paramount. One of the key challenges is ensuring secure communication between microservices while also managing secrets securely. Whether you're running a monolithic application or a set of microservices in Kubernetes, securing inter-service communication and secret management is crucial to prevent unauthorized access and ensure data privacy. Enter Istio Service Mesh combined with HashiCorp Vault Agent Injector, a powerful combination that allows you to secure communication between services and manage secrets without the need to modify your application code.
In this blog post, we'll explore how you can secure the communication of your workloads in Kubernetes using Istio service mesh while leveraging Vault Agent Injector for dynamic secret management, building upon the concepts we demonstrated in our previous posts.
Note: This article combines concepts from our previous posts about Kubernetes and Vault Agent Injector: Dynamic Secrets Management and extends them with service mesh security patterns similar to those shown in Secure Communication on HashiCorp Nomad with Consul Connect.
Istio is an open-source service mesh that provides a uniform way to secure, connect, and monitor microservices. It achieves this by enabling mutual Transport Layer Security (mTLS) between services, traffic management, observability, and policy enforcement. Each service gets its own identity, and Istio enforces that only trusted services can communicate with each other. By using Istio, service communication is authenticated and encrypted, ensuring that the network's security is maintained even in hostile or compromised environments.
Kubernetes provides excellent orchestration capabilities, but securing service-to-service communication and managing secrets requires additional tools. Istio provides the service mesh layer for secure communication, while Vault Agent Injector handles dynamic secret management. This combination gives you:
vault.example.com:8200First, install Istio in your Kubernetes cluster:
1# Download and install Istio
2curl -L https://istio.io/downloadIstio | sh -
3export PATH="$PATH:./istio-1.20.0/bin"
4
5# Install Istio with default configuration
6istioctl install --set values.defaultRevision=default -y
7
8# Enable Istio injection for the demo namespace
9kubectl create namespace demo
10kubectl label namespace demo istio-injection=enabled
11
12# Verify Istio installation
13kubectl get pods -n istio-system
Install the Vault Agent Injector as we demonstrated in our previous post:
1helm repo add hashicorp https://helm.releases.hashicorp.com
2helm repo update
3
4# Install only the Agent Injector (no Vault server)
5helm install vault-injector hashicorp/vault \
6 --set "global.externalVaultAddr=https://vault.example.com:8200" \
7 --set "injector.enabled=true" \
8 --set "server.enabled=false" \
9 --set "csi.enabled=false"
Set up the same Vault configuration as in our previous Kubernetes post:
1# Set Vault address environment variable
2export VAULT_ADDR="https://vault.example.com:8200"
3
4# Enable Kubernetes auth method
5vault auth enable kubernetes
6
7# Configure Kubernetes auth (same as previous post)
8KUBERNETES_HOST=$(kubectl config view --raw --minify --flatten -o jsonpath='{.clusters[].cluster.server}')
9KUBERNETES_CA_CERT=$(kubectl config view --raw --minify --flatten -o jsonpath='{.clusters[].cluster.certificate-authority-data}' | base64 -d)
10
11# Create service account and configure auth
12kubectl create serviceaccount vault-auth
13kubectl apply -f - <<EOF
14apiVersion: rbac.authorization.k8s.io/v1
15kind: ClusterRoleBinding
16metadata:
17 name: role-tokenreview-binding
18roleRef:
19 apiGroup: rbac.authorization.k8s.io
20 kind: ClusterRole
21 name: system:auth-delegator
22subjects:
23- kind: ServiceAccount
24 name: vault-auth
25 namespace: default
26EOF
27
28TOKEN_REVIEWER_JWT=$(kubectl create token vault-auth)
29
30vault write auth/kubernetes/config \
31 token_reviewer_jwt="$TOKEN_REVIEWER_JWT" \
32 kubernetes_host="$KUBERNETES_HOST" \
33 kubernetes_ca_cert="$KUBERNETES_CA_CERT"
1# Enable database secrets engine
2vault secrets enable -path=dynamic-app/db database
3
4# Configure MySQL connection
5vault write dynamic-app/db/config/mysql \
6 plugin_name=mysql-database-plugin \
7 connection_url="{{username}}:{{password}}@tcp(mysql-server.demo.svc.cluster.local:3306)/" \
8 allowed_roles="*" \
9 username="root" \
10 password="super-duper-password"
11
12# Create dynamic credentials role
13vault write dynamic-app/db/roles/app \
14 db_name=mysql \
15 creation_statements="CREATE USER '{{name}}'@'%' IDENTIFIED BY '{{password}}';GRANT ALL ON my_app.* TO '{{name}}'@'%';" \
16 default_ttl="1h" \
17 max_ttl="24h"
18
19# Create Vault policy
20vault policy write dynamic-app-istio - <<EOF
21path "dynamic-app/db/creds/app" {
22 capabilities = ["read"]
23}
24EOF
25
26# Create Kubernetes role
27vault write auth/kubernetes/role/dynamic-app-istio \
28 bound_service_account_names=dynamic-app-istio \
29 bound_service_account_namespaces=demo \
30 policies=dynamic-app-istio \
31 ttl=24h
1# mysql-istio-deployment.yaml
2apiVersion: apps/v1
3kind: Deployment
4metadata:
5 name: mysql-server
6 namespace: demo
7 labels:
8 app: mysql-server
9 version: v1
10spec:
11 replicas: 1
12 selector:
13 matchLabels:
14 app: mysql-server
15 version: v1
16 template:
17 metadata:
18 labels:
19 app: mysql-server
20 version: v1
21 spec:
22 containers:
23 - name: mysql
24 image: mysql:9
25 env:
26 - name: MYSQL_ROOT_PASSWORD
27 value: "super-duper-password"
28 ports:
29 - containerPort: 3306
30 name: mysql
31 resources:
32 requests:
33 cpu: 500m
34 memory: 1Gi
35 limits:
36 cpu: 500m
37 memory: 1Gi
38
39---
40# MySQL Service
41apiVersion: v1
42kind: Service
43metadata:
44 name: mysql-server
45 namespace: demo
46 labels:
47 app: mysql-server
48 service: mysql-server
49spec:
50 ports:
51 - port: 3306
52 name: mysql
53 targetPort: 3306
54 selector:
55 app: mysql-server
1# dynamic-app-istio.yaml
2apiVersion: v1
3kind: ServiceAccount
4metadata:
5 name: dynamic-app-istio
6 namespace: demo
7
8---
9apiVersion: apps/v1
10kind: Deployment
11metadata:
12 name: dynamic-app
13 namespace: demo
14 labels:
15 app: dynamic-app
16 version: v1
17spec:
18 replicas: 1
19 selector:
20 matchLabels:
21 app: dynamic-app
22 version: v1
23 template:
24 metadata:
25 labels:
26 app: dynamic-app
27 version: v1
28 annotations:
29 # Vault Agent Injector annotations
30 vault.hashicorp.com/agent-inject: "true"
31 vault.hashicorp.com/agent-inject-status: "update"
32 vault.hashicorp.com/agent-inject-vault-addr: "https://vault.example.com:8200"
33 vault.hashicorp.com/role: "dynamic-app-istio"
34 vault.hashicorp.com/agent-inject-secret-config.ini: "dynamic-app/db/creds/app"
35 vault.hashicorp.com/agent-inject-template-config.ini: |
36 [DEFAULT]
37 LogLevel = DEBUG
38 Port = 8080
39
40 [DATABASE]
41 Address = mysql-server.demo.svc.cluster.local
42 Port = 3306
43 Database = my_app
44 User = {{ .Data.username }}
45 Password = {{ .Data.password }}
46
47 [VAULT]
48 Enabled = True
49 InjectToken = True
50 Namespace =
51 Address = https://vault.example.com:8200
52 KeyPath = dynamic-app/transit
53 KeyName = app
54 spec:
55 serviceAccountName: dynamic-app-istio
56 containers:
57 - name: dynamic-app
58 image: ghcr.io/infralovers/nomad-vault-mysql:1.0.0
59 ports:
60 - containerPort: 8080
61 name: http
62 env:
63 - name: CONFIG_FILE
64 value: "/vault/secrets/config.ini"
65 - name: VAULT_ADDR
66 value: "https://vault.example.com:8200"
67 resources:
68 requests:
69 cpu: 256m
70 memory: 256Mi
71 limits:
72 cpu: 256m
73 memory: 256Mi
74 livenessProbe:
75 httpGet:
76 path: /health
77 port: 8080
78 initialDelaySeconds: 30
79 periodSeconds: 10
80 readinessProbe:
81 httpGet:
82 path: /health
83 port: 8080
84 initialDelaySeconds: 5
85 periodSeconds: 5
86
87---
88# Dynamic App Service
89apiVersion: v1
90kind: Service
91metadata:
92 name: dynamic-app
93 namespace: demo
94 labels:
95 app: dynamic-app
96 service: dynamic-app
97spec:
98 ports:
99 - port: 8080
100 name: http
101 targetPort: 8080
102 selector:
103 app: dynamic-app
Enable strict mTLS for the entire demo namespace:
1# demo-peer-authentication.yaml
2apiVersion: security.istio.io/v1beta1
3kind: PeerAuthentication
4metadata:
5 name: demo-strict-mtls
6 namespace: demo
7spec:
8 mtls:
9 mode: STRICT
Create authorization policies to control service-to-service communication:
1# mysql-authorization-policy.yaml
2apiVersion: security.istio.io/v1beta1
3kind: AuthorizationPolicy
4metadata:
5 name: mysql-server-policy
6 namespace: demo
7spec:
8 selector:
9 matchLabels:
10 app: mysql-server
11 rules:
12 - from:
13 - source:
14 principals: ["cluster.local/ns/demo/sa/dynamic-app-istio"]
15 to:
16 - operation:
17 ports: ["3306"]
18
19---
20# Dynamic App Authorization Policy
21apiVersion: security.istio.io/v1beta1
22kind: AuthorizationPolicy
23metadata:
24 name: dynamic-app-policy
25 namespace: demo
26spec:
27 selector:
28 matchLabels:
29 app: dynamic-app
30 rules:
31 - from:
32 - source:
33 namespaces: ["istio-system"] # Allow health checks from Istio
34 - to:
35 - operation:
36 ports: ["8080"]
1# dynamic-app-gateway.yaml
2apiVersion: networking.istio.io/v1beta1
3kind: Gateway
4metadata:
5 name: dynamic-app-gateway
6 namespace: demo
7spec:
8 selector:
9 istio: ingressgateway
10 servers:
11 - port:
12 number: 80
13 name: http
14 protocol: HTTP
15 hosts:
16 - dynamic-app.local
17
18---
19apiVersion: networking.istio.io/v1beta1
20kind: VirtualService
21metadata:
22 name: dynamic-app-vs
23 namespace: demo
24spec:
25 hosts:
26 - dynamic-app.local
27 gateways:
28 - dynamic-app-gateway
29 http:
30 - match:
31 - uri:
32 prefix: /
33 route:
34 - destination:
35 host: dynamic-app.demo.svc.cluster.local
36 port:
37 number: 8080
Combining Istio Service Mesh with HashiCorp Vault Agent Injector provides a comprehensive security solution for Kubernetes workloads. This approach delivers both secure service-to-service communication and dynamic secret management without requiring changes to your application code.
Key benefits of this integrated approach:
By leveraging these technologies together, you can build a robust, secure, and observable microservices architecture that follows cloud-native best practices while maintaining operational efficiency.
Start enhancing your Kubernetes deployments today by integrating Istio service mesh with Vault Agent Injector, and take advantage of secure, authenticated, and encrypted communication combined with dynamic secret management across your services.
This is an introduction to Kubernetes, an open-source system for automating deployment, scaling, and management of containerized applications.
Introduction to HashiCorp Vault.
Multi-Platform Service Mesh: Connecting Kubernetes, Nomad, Bare Metal, and VMs with Consul Connect Modern infrastructure often spans multiple platforms -
Securing Communication in Kubernetes with Istio Service Mesh and Vault Agent Injector In modern cloud-native Kubernetes environments, security is paramount. One
Securing Communication in Kubernetes with Consul Connect and Vault Agent Injector In modern cloud-native Kubernetes environments, security is paramount. One of
In a cloud-native Kubernetes environment, secrets management is a critical aspect of security. HashiCorp Vault is a popular tool for managing secrets and
Remember last July when we dug into Ollama and its promise for local AI? Well, a lot has happened since then. Ollama has rolled out some pretty significant
You are interested in our courses or you simply have a question that needs answering? You can contact us at anytime! We will do our best to answer all your questions.
Contact us