I'm a developer, how do I become better at Sysadmin or DevOps roles?

[BretFisher]

A common question from developers:

I've been a full-stack dev for about 4 years, and ... I'm trying to figure out how to actually become a DevOps / SysAdmin guy. Any suggestions on maybe a learning path and/or which courses to take? I feel like the path is more clear for web dev, but it's been hard to find something more clear cut in the DevOps world.

Also a similar question from those just starting out:

I am beginner and just learning the basics. just want to know what is devops? I am a system administrator but dont know about devops. how can I become a devops and what are the requirements what skill do i need?

[BretFisher]

This answer is mostly from the perspective that you're a solid developer looking to put more Ops in your Dev toolset. If you're coming from a sysadmin background, this is still mostly true, but I threw in an extra bit at the bottom just for you.

Well as much as even I use the word DevOps in my title, it's not really a job role, but more of a mindset of how dev's and sysadmin's come together to solve problems... Web Dev, Build Engineer, or Linux/Cloud Sysadmin is a role. DevOps to me is like saying "We Do Agile". It speaks more to how you approach work and the team then your actual job tasking. But for the sake of this post, I'll pretend it's a role.

There are lots of articles and courses claiming to teach you what DevOps is. This post isn't that. I'm not going to repeat what most of those already tell you.

To me like an IT job, I think the most important skills are a strong empathy for your customers and desire to help others. Your customers in a DevOps role will change day to day and could be developers, sysadmins, or users themselves. Gone are the days where most of us can hide in the back of the IT office, work alone, and talk to no one. You must be a self-starter that is good at break/fix, problem analysis, and "systems thinking".

The three core qualities I think that set the best of any IT role so far in front of others are:

• Empathy, the ability to hear what others are saying and put yourself in their shoes to better understand the problem or desired outcome.
• The drive to help others, to even put them first.
• A deep curiosity for understanding how things work. Code, servers, or networks. Always be learning.

Notice none of that said anything about a tool or specific technology. All of that will change. DevOps may not even be a thing in 15 years.

Sadly, I don't have a great learning path for DevOps (but I made a tools list at end of this post that I would expect someone I hire for DevOps to know). I stumbled into it myself after doing "lots of different things" and I had the mindset of DevOps before it was a word, where I was working in Ops/Sysadmin teams that cared about servers and networks, and asking that they start caring more about the Dev's and their job responsibilities of easily deploying apps. I would stand on the mountaintop and tell both groups "It takes a village to raise an app, let's be friends".

A Sysadmin cares about keeping apps and everything underneath them stable, secure, fast, and recoverable. They enjoy creating systems and processes to deploy OS's, monitor those systems for performance and health, keeping security patches current, keeping the networks locked down and error-free, and reducing the number of variables in the infrastructure (keeping OS's the same distro and version, standardizing on certain tools, etc). You'll need some of those skills to bridge the gap between a developer and an operator.

So to start in your sysadmin/ops journey, I would say "start caring about how everything under your code works".

Take Linux courses. Take AWS courses. Learn a ton about networking, the OSI Layers, how TCP packets are made up, how firewalls and NAT really work. Learning those things will make you a better programmer anyway, even if you didn't try to be a sysadmin. Learn common sysadmin CLI tools like SSH, Bash, package managers, and how drivers and system services are configured. Force yourself to use network storage (NFS, iSCSI), load balancers, and do backups and restores of databases. Automate everything you can. Pick a system automation tool like Ansible or SaltStack or Puppet and start using it to control servers rather than manual SSH commands. Pick a monitoring and logging tool and get confident in them. Use them for even your smallest personal projects.

This all sounds like a lot, and it is, but if I were to ask a sysadmin to be a full stack web dev, they would have a similarly long list of libraries and tools that probably took a web dev years to get good at. So the key is to pick one tool in a problem space and learn it to a reasonable level, and then move on.

Most DevOps job descriptions really just mean "We need you to take the dev team's code, and create/manage an automated path of tools that test the code, build the runtime environments (servers and containers) and get the code onto them in a reliable and consistent way."

In reality, like any generic job title, a DevOps job can be lots of things. Some will be more developer focused, some will be "build engineers" that focus only on CI/CD, some will be more operations and sysadmin focused.

So here's a longer job description: "We need someone who understands code and dev tools, but can also get that code setup in CI/CD, knows git, and also knows how to manage and troubleshoot servers. They know AWS/Azure and how to start from zero and build out a whole stack of services, and then back it up, monitor it for health and performance, set up logging and alerting, and maybe knows some security like TLS, SSH, certificates and key generation, IP/Sec, VPN's, Firewall basics."

Someone who knows a few of those things well, we call a Specialist. Someone who knows a little bit about many of those topics, we call a Generalist. To me, anyone great in an IT role, especially DevOps, which bridges two traditional disciplines, is what I call a Versatilist. They are a Specialist in a few things and a Generalist in many others. They can't write Java and Node.js and .NET by memory, but they can get the gist of how it starts up, where to find configuration values, and how it handles dependencies. Maybe they Specialize in AWS, Ansible, and Docker, and the rest they know good enough to get by... Or maybe they are a pro at networking, firewalls, and know Python and Jenkins well, but the rest they have just a passing knowledge in...

Just like no two developers have the same skillset and tool experience, neither will two sysadmins.

Today, DevOps roles tend to define themselves as someone who configures a set of tools for Continuous Delivery, and then just focuses on what can make that feedback loop of "develop, build, test, deploy, monitor, repeat" faster and faster. To do that means you first need to understand the problems of making that happen and the tools to solve the problems it creates. The reason they even came up with the term was to try and bridge the gap, or the "wall" that most organizations had created between Dev (makes apps) and Ops (runs apps). The two groups would often be adversarial and had opposing goals: Dev's wanted to deploy (change) code as fast as possible to meet new business goals, and Ops wanted to standardize and formalize change (keep it stable) to meet business reliability and security standards. DevOps was the dream that each team would take partial ownership of the other teams' success and failures, and start focusing more on the merging of new business goals with existing business standards. This was necessary so the business could move faster with fewer people.

So sure you might read a book or two on DevOps mindset and process, to get a clearer head around why the term even exists, but then the rest is on learning sysadmin and build engineer tools and then putting those into practice in your job today. That's where the real learning and experience matters. Take more interest in shipping the code and how it runs in production. Be curious about everything that comes after you commit your code.

And lastly, if someone said "I need a DevOps job in 12 months and I'm currently a full stack developer" then I'd say here's a short list of things to learn. This isn't a magic list of tools, you could easily replace each item with a competing tool in that problem space. The goal is to make you well rounded as a generalist sysadmin, and then based on your interest/passion, go deep into a few areas you like to be specialized. Learn the basics of these things, then over time, go deeper in the areas you gravitate to.

1. Know the history of DevOps and why it was needed.

1. GitHub and Git Flow.
2. Learn AWS basics. Not just the how, but also the why and when to use each tool. Skip 75% of their products and focus on the core tools everyone uses:
   1. EC2, VPC's, Security Groups, Elastic IP's, ELB's, Route 53
   2. Storage. EBS, EFS, S3
   3. Lambda, CloudWatch, CodeDeploy
   4. CloudFormation ← key to AWS infrastructure automation and "infrastructure as code" but you need to know the services above first and why they exist before automating their creation.
3. Learn TCP/IP networking, NAT firewalls, and the 7 OSI layer basics.
4. Learn Jenkins and the CI/CD workflow.
5. Learn Docker, and then how to use it in Jenkins to build, test, and deploy containers to servers.
6. Learn Docker Swarm for creating a container cluster to deploy containers on many servers as easy as one.
7. Learn Linux. Take an admin course online. Pick one distribution to know better than the others.
8. Learn Ansible for automating sysadmin tasks across many servers as easy as one.
9. Learn Terraform for creating servers in any cloud via "infrastructure as code".
10. On top of Linux, Docker, Swarm, and AWS, learn:
    1. Nginx or HAProxy for HTTP Reverse Proxy.
    2. cAdvisor, Prometheus, and Grafana for Monitoring.
    3. Use something like "swarmprom" to learn how the above tools work together to give you graphs and alerting of your apps and cluster.
    4. REX-Ray, used shared network storage to store your persistent data (databases).
       1. Never stop learning new things!

What if I'm a sysadmin/operator wanting to move into DevOps.

I feel like this path is easier, as it's the one I took. The short answer: learn to code. Be a developer in your spare time to start learning the mindset and problems of a developer. Start with web development which will benefit your whole career and then maybe one other language. If you have no preference, I'd suggest a modern general-use language like Node.js, Go, or Python. You might even know Python as a sysadmin, or Node.js (JavaScript) as a web developer, so win!

The reason to learn a language or two is not because you'll need to write programs much, but so you better understand the mindset of a developer. You can get away with just understanding how to create, build, and run basic programs in those languages.

For a sysadmin stepping into DevOps, assuming you know most of the tools listed above:

1. For each major language, spend a day learning how it uses dependencies, how to install/manage them, and then how it sets up environment variables. Every language is different.
   1. Java, .NET, PHP, Python, Node.js, Go, Ruby, Web Dev (webpack, bower, gulp, etc.)
   2. In a DevOps role it's usually expected you know how to make very basic apps in the languages of the team you're supporting.
   3. Does the ecosystem for that language prefer file-based configs or can it also do envvar based (the best way for containers)? How should it be built for containers? Inside a language, there can be config standards that are specific to a toolkit, so there might be multiple ways depending on what the dev team is doing.
   4. What are the sysadmin-concerns about that language? Does it have memory management settings that usually need to be changed (Java)? Is it single threaded so that you'll need to manage solutions for multi-core servers (Node.js)? Does it have common caching or temp dir settings and permission issues (PHP)? Does it require special ways of installing on a server that are different then developers are used to locally (lots of them, but Ruby can be the worst)?
   5. Often you'll get a code repo from a developer that you need to change the way it deploys for testing and production, which means you need to understand how to install the language build/runtime environment, envvars, and dependencies. Usually, doing this on a server is different than how the developer did it on their computer.

In the end, for any DevOps mid-career role I'd hire, I'd expect them to be able in a single day:

1. Create a very basic hello world app in a language of their choice.
2. Use git to commit that code to a remote git repository. Understand modern CVS workflows like pull-requests, bug tracking systems, code commits, code merges, etc.
3. Create documentation a simple way for other devs to download that repo and get started coding on it as well. (working in a team and having empathy for others is important, remember)
4. Use their choice of CI to test that app in basic ways. CI should run on every code commit.
5. Provision cloud servers, storage, networking, firewalls, and load balancers for the app.
6. Use CD to automate the deployment of code after successful CI to the servers.
7. Do this for a test and prod set of servers. Store the configs of those two environments in a way that's easily manageable and has change tracking.
8. Deploy basic backups, monitoring, alerting, and log collection. Leave the fancy stuff for Ops-focused people.
9. Do all of this "infrastructure as code" style where they use configuration files to store the settings of all these systems.

If they can do that all with the tools of their choice, then it shows they understand the full lifecycle of applications and how all the parts fit together. Understanding both the Dev and Ops roles in these basic ways helps you fully support both teams and bridge the gap.

For a junior DevOps role starting out, I'd expect them to do most of that, with some gaps where they are good at seeking help or quickly learning how to get it done with a few days of internet research.

[itprohelper]

Great explanation!

[BretFisher]

Here's a good beginner course on DevOps topics that I watched some of: https://www.linkedin.com/learning/devops-foundations

[kharels]

Thanks Bret! This definitely helps.

[Aracki]

This is a great text. You should make a blog post out of it. 👍

[BretFisher]

Good idea!

[BretFisher]

Here's an interesting learning path.

https://roadmap.sh/devops

I would change it so that step one is NOT to learn a programming language unless you're a dev (in which you already know one). There are plenty of sysadmins and ops that barely know any scripting languages, much less full-featured programming languages that can do a lot in the DevOps space with CLI commands, YAML, and some JSON. I'd say yes to be a well-rounded DevOps person you'd want to know a little about multiple languages, but I find it's more helpful to know the packaging tools, the dependency and build file languages, and how apps are typically laid out on a file system. When you know that about a language you can usually take an app from the dev, properly package it up, deploy it, monitor it, and understand what it does without ever being able to write a hello world in that language.

[iamfredrickokereke]

I love this and already on this path, but this makes it clearer and relatable. Thank you so much for sharing @BretFisher.

[andrus-code]

It was a great summary of how to pass from sysadmin to devops!
It was a great perspective talk of both worlds of developers and operations.!!!