Containers — single processes virtualized in isolated environments — are becoming important parts of the IT infrastructure at many companies, especially those embracing DevOps or continuous deployment methodologies. And Kubernetes, an open source system for automating container deployment and management, is being embraced by a growing number of companies that use containers. So naturally, testing and improving Kubernetes' security has become an important topic for security professionals.
At the recent ShmooCon in Washington, DC, Mark Manning, technical director of NCC Group, gave a presentation on Kubernetes for penetration testers — a presentation in which he examined the vulnerabilities of the system and the tools available for testing them. Explaining why he thinks the topic is important, he says the reasons come down to numbers.
"Eighty-five percent of customers are definitely deploying containers or they plan to, in the most common thing I see," Manning says. "I think it's pretty safe to say that with the Fortune 500, half of them are currently using containers in production."
In his presentation, Manning began his look at vulnerabilities with those arising through Kubernetes' configuration — something he says is the root of many dangers.
"I think I'd go so far as saying that by default, it is actually an unsafe platform," Manning says. "And if you took just the code from Kubernetes and deployed it into your own environment, the defaults that are provided create a lot of security vulnerabilities in itself." That risk, he says, is why so many companies have abandoned the idea of deploying Kubernetes on their own servers and now use cloud service providers for their Kubernetes platforms.
One of the issues companies face with configurations, Manning says, is that there are so many ways to do it badly. And that wide variety of possible problems informs the way that Manning approaches security testing in Kubernetes deployments.
Divide and Test
"The way that we do assessments is we will look at the application separately from the infrastructure," Manning says. As part of that process, he says that the application testing and evaluation is the same performed on any platform. The source code is evaluated for vulnerabilities and bugs, all of which are reported to the client for remediation. Then comes the Kubernetes evaluation.
In his presentation, Manning described a process of setting up "attack pods" (a pod is a collection of containers) using standard Kubernetes tools such as kuberctl (a control-line interface for Kubernetes) and cURL (an interface for accessing the API-only components of Kubernetes). His team will also use standard security assessment tools like metasploit, aimed at a Kubernetes target. The point of doing all this creation and "curling" the components is that a successful attack could allow someone to gain access to multiple containers within the pod, violating the separation that's supposed to protect every application.
"We start inside a container and we do what we call 'breakout assessments' that will validate whether or not the container technology is effective at isolating an attacker to only be able to affect this one application," Manning explains. And in a Kubernetes deployment, that separation technology might have been intentionally weakened.
Finding Weakness by Design
"Kubernetes is actively making choices designed to make things faster with higher performance, and make things work easier, at the cost of security," Manning says. He contrasts this with Docker, which employs technology similar to that used in the Google Chrome browser's sandbox to maintain separation and security.
In Kubernetes' case, he explains, the security has been intentionally weakened to make it easier for administrators to rapidly create and manage containers and pods. It's part of a pattern that Manning says he's seen in other products, in which they're made feature-rich and easy to use, with the hope that security will somehow be bolted on later.
The main criminal benefactors of all this, Manning says, are cryptominers who love to find unprotected pods. And they don't have to research or use zero-day exploits to do it. "Throughout this whole presentation, I'm not talking about zero-days or CVEs or exploits like that. I'm talking about misconfigurations because that's what's been working in all these assessments," Manning says.