Everyone can contribute! Learn DevOps and Cloud Native in our cafe ☕
Technology is moving fast in the DevOps and Cloud Native community.
Join the conversation and add your thoughts, tips, experiences, stories.
"Everyone Can Contribute" is inspired by GitLab's mission.
19. cafe: Break into Kubernetes Security
Highlights
We are learning how to deploy and secure Kubernetes into Hetzner cloud in this series:
-
- cafe: Provisioned the server and agent VMs with Terraform and Ansible in the first session with Max.
-
- cafe: Deployed k3s as Kubernetes distribution with Max.
-
- cafe: Learned about pods and the Hetzner load balancer with Max.
-
- cafe: Ingress controller for load balancer cost savings with Max.
In this session, we change the perspective and try to break into a Kubernetes cluster with Philip Welz.
- Scenario: view access to namespace
everyonecancontribute
& edit access to namespacephilips-workspace
- First, we check the permissions in the namespace with
kubectl auth can-i --list
- Inspect all namespaces and pods to fetch as much information as possible
kubectl get pods -o yaml |grep secret -A5 -B5
- Then create a new pod with privileged mode and execute a bash session into it
- Mount the host filesystem, download kubectl from the internet and use kubelet.conf as our
KUBECONFIG
- Gather as much as possible information about the Cluster ( Nodes, IPs, etc. )
- Generate an SSH key, move it to the mounted host filesystem and access the host as root
- Use container runtime CLI
crictl
to retrieve passwords - Authenticate against the Kubernetes API and fetch the remaining tokens/passwords.
- Created again a pod with privileged mode but this time scheduled it with nodeName to one control plane node
- Mount the host filesystem again and use the admin.conf as our
KUBECONFIG
to gain full cluster admin rights - Target etcd to read the secrets in plaintext.
- Defend tactics
- Use Pod Security Policies. They are deprecated in favour of Kyverno or Open-Policy-Agent
Next week, we’ll look into more security topics and more:
- OpenID Connection of the API Server with Dex and GitLab, continued.
- Hetzner storage volumes
- Future ideas touch monitoring with Prometheus, GitLab CI/CD deployments and much more :)
Insights
- Kubernetes group repos
- Repository with all commands from the session
- Twitter thread
- KubeCon 2019 CTF
- Attacking Kubernetes through Kubelet
- BadPods examples & explanations
- deploy a pod that gives us full host access
- kubectl node shell
Recording
Enjoy the session! 🦊
Written by: Michael Friedrich