Getting started in security for platform engineering.
One of the key problems I face on an almost daily basis is a strange form of externally imposed imposter syndrome. For a while, I was a DevSecOps engineer, then a Security Architect for DevOps systems. Now our terminology is changing, and the more technical side of DevOps, with which I am most familiar, is rapidly expanding into platform engineering. Before someone ties themselves in a knot trying to declare me as a PlatSecEng expert, maybe the best start is to put a stop to it immediately.
What I truly am is a security nerd. More specifically, a security nerd who, through a mix of chance and circumstance, has spent most of his career working either adjacent to or directly within DevOps and platform engineering.
This has led us here. To distil something approaching wisdom, if you're a DevOps / Platform engineering specialist who wants to migrate towards security, you might find something here of use to you. If you’re a security specialist looking to link yourself to the growing field of platform engineering, stay tuned, as I’ll likely write another blog post about that.
This blog post will be fairly link-heavy, so be prepared for some rabbit holes.
Frameworks, standards, and governance, oh my…
We're going to start with the most boring parts of all (Sorry, Jussi.) Compliance.
Security is driven by two means:
- Security nerds doing technical things.
- Compliance people building compliance.
Of the two, there's a constant back and forth over which one is more important, which I feel I can finally settle.
The answer is **scale**.
In "Sapiens: A Brief History of Humankind," Yuval Noah Harari argues that the cognitive revolution allowed humans to create and believe in shared myths, such as religions. This enabled large-scale cooperation among groups far beyond the natural social limit of around 150 individuals. These shared beliefs provided the foundation for developing complex societies and civilizations, allowing humans to organize and work together on a previously impossible scale.
We see the same in business. At small scales, startups, and SMBs, it's easier to keep the reigns on security without large-scale oversight and standards. 20 coders sitting in a room that smells faintly of energy drinks likely already know a great deal about secure development. The security champions are obvious and active and can generally keep things well in hand. If not, it's just another part of the technical debt that startups accrue.
But at scale, just as early humans required a shared belief to manage behaviour, a company grows to require rules, regulations, and standards. By adopting these standards, a company can set down an operating method that doesn't require constant oversight and allows for large-scale growth.
Conversely, security practitioners who don't familiarize themselves with these standards limit their field of operation to small-scale environments that duck under such legislation.
ISO 27001 and ISAE
ISO 27001 is crucial when transitioning into a security role within DevOps because it provides a comprehensive framework for managing information security risks. This international standard sets out best practices for establishing, implementing, maintaining, and continuously improving an Information Security Management System (ISMS). By adhering to ISO 27001, you can ensure security is treated as a continuous, systematic process, fostering a culture of security within the fast-paced DevOps environment.
ISAE 3402, however, is a different beast. It focuses on the controls and processes within service organizations, ensuring they meet rigorous security, availability, and integrity standards. Understanding and implementing ISAE 3402 provides assurance that the services they develop and manage comply with stringent security requirements, especially when these services are outsourced or rely on third-party providers.
In essence, ISO 27001 checks that your documentation is in order, and ISAE 3402 ensures you operate according to that documentation.
NIS2, CRA, DORA, and the AI ACT
The EU, never one to shy away from ambitious regulations, is currently dropping some fairly comprehensive security frameworks on us. While perhaps a bit heavy on the paperwork, these initiatives are some of the better developments we've seen in security regulation. NIS2 ensures that critical infrastructure operators, among others, are forced to finally take cybersecurity seriously, which, as anyone who has seen my presentations on Industrial IoT will know, is far from the current situation. The important thing to note about NIS2 is the vastly expanded scope from the original standard, which you can read more about in our blog post on NIS2 preparedness.
CRA goes a step further, requiring compliance with “all networked devices.” But with a 36-month delay in applicability after going into force, this is more of a concern to professionals down the pipeline (though the best time to start preparing is now).
DORA is, again, much the same, but for our financial providers.
On the other hand, the AI Act is the world's first attempt to put guardrails around artificial intelligence, language very much in line with where Eficode sees the job of DevOps and platform engineering going soon.
Familiarity with these rulings, even on a basic level, will likely serve you well if you're entertaining ideas of joining the DevPlatSecOpsEng sphere.
CIS Benchmarks
We're finally diving into the nerdy side of things, where the real magic (grunt work) happens. The CIS Benchmarks are a set of guides for system hardening, providing detailed, actionable steps for securing systems, whether servers, network devices, or applications. They offer a solid, evidence-based standard that helps ensure your systems are secure, not just in theory but in practice.
Now, one thing you need to know about the CIS benchmarks is that they're long, with the Windows 11 standalone CIS Benchmark alone measuring in at 1200+ pages. This makes it roughly as long as Gone with the Wind and somehow twice as boring. The advantage, however, lies in that CIS standards closely align with the requirements set out by standards like the ISO, DORA, NIS, etc. So, while it might seem like overkill to some, following the CIS Benchmarks means you're not just ticking boxes but genuinely fortifying your infrastructure and setting a strong foundation for comprehensive security management.
Check out the CIS Benchmarks here.
OWASP Tops 10
The OWASP Top 10 is somewhat famous, but I've pluralized the title in the way I have for a reason. OWASP actually puts out Tops 10 for different aspects of working in the IT world. You may consider checking out their CI/CD-oriented Top 10 list or their depressingly dated (through no fault of their own) IoT Top 10 list.
While these lists don't change greatly, they provide a good set of core concepts to understand.
Security principles and practices
Threat modelling
Threat modelling is an exercise in positive negativity. In essence, it’s a brainstorming session about all the ways things can go horribly wrong. Threat modelling is an essential practice for identifying potential security risks in your systems before they become actual problems. It’s like playing devil’s advocate but with the added benefit of actually improving your security posture. By systematically analyzing potential threats, attack vectors, and vulnerabilities, you can prioritize your defenses and ensure you're not just reacting to security incidents but actively preventing them. So, while it may feel a bit like doomsday prepping, threat modelling is a smart, proactive way to stay ahead of the curve and keep your systems secure. Not to mention, it’s going to be all but mandatory under NIS2.
If you're looking for a resource about the process but, like me, have the attention span of a hummingbird, I've got your back. 5 Minute Threat Modelling from TDOC London 2024 to accompany your next cup of coffee.
Defense in Depth
One of my favourite quotes is “[Security] is like an onion. Tough to crack on the outside, then you peel back the layers to find exactly the same, over and over. Then you start to cry.”
This approach is called defense in depth, a cornerstone of effective security strategy, especially in the complex environments where modern DevOps and platform engineering operate. Instead of relying on a single line of defense, this approach layers multiple security measures across different parts of the system—network, application, data, and endpoints—to create a robust barrier against threats. Each layer is designed to address different aspects of potential attacks, ensuring that if one layer is breached, others remain in place to protect the system. This strategy reduces the risk of a catastrophic failure and provides a comprehensive way to manage security risks. Defense in depth aligns well with the principles of DevOps, promoting a culture of continuous improvement and proactive security management across the entire lifecycle of system development and operations. Defense in depth is essentially the formalized output of your threat modelling scenarios.
It's hard to find a resource about Defense in depth without it degenerating into a sales pitch. That being said, Cloudflare does a decent job of summarizing the principles. Just ignore the last paragraph.
Principle of Least Privilege
This security principle is all about granting users and systems the minimal level of access necessary to perform their tasks, nothing more, which makes sense if you stop to think about it. We've been doing this for years by ensuring server rooms have different keys, but it often gets left behind in the speed of development. While we consider security among the "nice to haves" and not the "Minimum Viable Product," we run into situations where it's easier to grant admin and resolve later.
By restricting permissions to the bare essentials, you reduce the attack surface—the potential attack avenues for hostile actors—and limit the potential damage if an account is compromised. It’s a simple concept but one that’s often overlooked in the rush to get things done. Implementing the Principle of Least Privilege might seem like a hassle at first, but it’s a small price to pay for the peace of mind that comes from knowing that even if something goes wrong, the impact will be contained.
The best summary document for PoLP is, in my opinion, the one put out by Palo Alto Networks.
Automation in security
Whether or not Bill Gates actually said, "I will always choose a lazy person to do a difficult job because a lazy person will find an easy way to do it," it's advice I have taken to heart. I am a lazy security professional. If I have to do a task more than thrice, I'll make a bash script for it. If I have to run something on more than one server, I'll create an Ansible Playbook for it. Because the less actual work I do, the more time I can spend writing meandering guides that no one asked for.
Automation specific to security is actually critical to DevOps. At its core, DevOps is the art of scaling infrastructure, processes, and the pace of development. In this context, we have to ask ourselves: Do we want our systems to outscale our ability to secure them?
Security at any scale other than the same is likely to fall short of actual requirements. Learn quickly to automate everything you can, from scanning for vulnerabilities to checking for indicators of compromise to deploying new security tooling.
As automation is a pretty vast topic, I'm instead going to direct you to this XKCD comic about ROI of automation and tell you to google Bash scripting, Python, and Ansible. Start by building playbooks around extracting logs and scripts to analyze them.
Mindset
One thing that needs to be understood about security: It will not pause. It does not stop. But it is important to know that this doesn't mean it is all incumbent on you. That mentality leads to stress and burnout, which are things we'd like to avoid.
For advice on how to deal with this, we have a podcast episode for you.
Getting your hands dirty
All this information is theoretical. Hands-on practice through labs and practice environments is indispensable for developing practical security skills.
Your first task? Pick a colour: Red or Blue.
If you're interested in the defensive, Blue-team side of things, build a CI/CD system. All the components can be in the form of free, open source software. Spin up Gitea, push some code, and use Jenkins to build and deploy it. Scan the code with Semgrep. Build some Docker containers and scan them with Trivy.
It may be tempting to work with one of the "all-in-one" platforms like GitHub or GitLab. These are great for production but less so for learning. You’ll also find that large parts of their security tooling are only available in premium versions, which is less useful for a learning experience.
If you're more offensively inclined, start running CTFs and set up a home lab for vulnerable image testing. There's even a CI/CD-oriented system called CICD-GOAT. A great way to learn about CI/CD threats would be to go through these, with or without the assistance of the solutions folder, and see how attacks form.
Embedding security in development
The shift-left approach to security emphasizes integrating security measures early in the development lifecycle rather than as an afterthought. By embedding security practices from the beginning, platform engineers can proactively address vulnerabilities and ensure that security is a fundamental part of the development process, reducing the likelihood of issues arising later.
Hear more about this from our Webcast with GitLab.
Learning to say “yes”
Adopting a mindset of collaboration and solution-oriented thinking is crucial in security, where saying "no" can often halt progress and stifle innovation. It's not 2003 anymore. No is unproductive. Learn from improv.
Of course, this doesn't mean saying yes to everything. If someone says they want SSH access open to the general internet with password-only authentication and 67 people sharing one "root" user, feel free to tell them no. Laugh at them while you do it, as well.
But if someone comes to you with a reasonable request which policies prevent, consider leaning in to find a way to make the "yes" as secure as possible.
By leaning in to say "yes" and seeking constructive ways to address security concerns, platform engineers can foster a more collaborative environment, where security is seen as an enabler rather than a roadblock.
That way, you'll be doing your part to dispel the "difficult security engineer" stereotype.
Summa Summarum
Security is hard. Security is vast and only getting larger with every passing day. This is how I intended to end this post until the first proofreader told me it was "too depressing" and "wasn't a proper summary."
So, I'll go on to say that security is by no means insurmountable. A well-placed, well-defined, and strictly followed Defense in depth model will mitigate all but the most resourceful (resourceful as in having resources, not being particularly skilled. Think state actors.)
The key thing to remember is that security always needs more people. One myth we need to bust is the idea that you have to be an expert in everything IT to contribute to security. This idea is rooted in elitism and keeps security constantly on the back foot.
If you're interested in security, regardless of your background, start learning today so you can contribute tomorrow.
Published: Dec 11, 2024
