Programming

IT operators and DevOps engineers have been a nightmare liability for security, governance and business continuity. The best-case scenario for a DevOps/IT operator is that you get a superhero that does everything and knows it all. All responsibilities and security privileges gravitate towards this role, and knowledge sharing becomes impossible. Lastly, it becomes impossible to track the thousands of out-of-band changes initiated by a DevOps team to an auditor or certifier.

Cloud engineering, feature management and IAC tools have made it way better for engineers to build and deploy self-monitoring systems. A modern software ecosystem can be deployed, updated and migrated on an automated schedule. It can be done, safely without any of the responsible engineers having direct access to environment secrets or sensitive data. All of these changes can be set under version control for auditing purposes. If given the option, any smart employer would prefer the option to invest in such a system rather than support a 24-7 response team.

There will always be a need for surgery on a production environment, but there's no reason that can't be a formalised incident. If you are having weekly incidents that require engineers to do operations work, then that's something that needs to be addressed.

We should all be working to eliminate operations work. IT operator needs to be a trusted security role, not a critical glue with all the keys that holds a system together.