A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
Echoing the sentiment that you should adjust your perspective in approaching containerization, otherwise you're in for a tough time.
Jerome Petazzoni, one of the more recognizable names in the container community, has a site where he puts all of his workshops, slides, etc. This is his one for getting started with Docker.
Huge thanks for this. I'll look at them tonight.
Is there a PDF
Not that I'm aware of, but the source Markdown files are here, so you could probably make one with e.g. pandoc if you wanted.
Sounds like you, like a lot of others, have come to docker from the perspective of "it's like a mini virtual machine". Maybe you've used VMs before, like virtualbox or VMware or EC2. Maybe you have experience with setting up a cluster of VMs, each with their own OS, own SSH client, own suite of applications, and an overlay network between them all. Maybe someone told you "you should use docker instead, it's like mini lightweight VMs". And you'd be right to assume this is the wrong perspective to approach docker, because it leads to the problems you have faced.
Instead, try to think of docker containers as standalone applications. They don't contain a kernel, they don't have SSH, no Nano or VIM, just simply the Application, in a container, with enough supporting filesystem and OS libraries to make the application work.
That perspective is what helped me to get better at docker, I know it's not exactly answering your question, but it might help.
"like mini lightweight VMs"
That's exactly how I've approached it cause that's exactly how it was explained. But it's not at all like that. Thanks for your explanation.
Same, that's how it was explained to me too. It spent over a year learning docker the wrong way, and trying to use it as a replacement for VMs, after a coworker told me that.
Heh I remember searching for an hour about how to see the GUI of a docker container when I was first getting into it. Didn't help that I was using windows to run docker, either, it's a whole other layer of abstraction.
Yeah, I had the benefit that I was a Linux sysadmin, and have been using Linux for work and at home for the last 15 years. So I was lucky I didn't have to deal with the nightmare that is Docker on Windows, while I was learning Docker.
Since then I have had to provide basic docker training to new developers, most use windows, I have had to deal with docker on windows plenty, it does ramp the learning curve, so I know your pain.
I started learning docker first by how to customize an image. Since Alpine Linux is a popular light weight distro, I started with customizing it and moved from there. I also explored other people's Dockerfiles to see how to use a Dockerfile to customize an image to containerize an app.
I found almost every YouTube howto on the subject matter very frustrating and often making the assumptions that you note. I also read the reference documentation on docker's website. It's not easy and it's time consuming. I'd say I am good enough to be dangerous, not much more than that. Mostly, I've modified other people's images to suite my needs
Thanks I'll look into that. It doesn't help that my introduction to Docker was using portainer so I haven't really had much experience in the terminal outside of docker ps. I really put the cart before the horse there and am regretting it.
Hey, it's okay. I've been there and done that myself. You'll find it useful to look at both Dockerfile and docker-compose.yml files. A mistake I made early on was thinking I could replace a Dockerfile with a docker-compose.yml one. The Dockerfile itself describes how an image should be built. The docker-compose.yml file describes the environment and fetches images. It also may pass variables to the Dockerfile.
A suggestion of mine would be to learn docker-compose. It's basically the docker command line in a file which is easily modifiable, can deploy extremely easy, etc.
Others have pointed out that docker containers aren't idiomatically like VMs and you interact with them differently. Some workflow tips:
The other issue in play here is that the fundamentals necessary to understand how docker containers run aren't actually docker fundamentals. They're:
These things aren't trivial to learn, a thorough understanding of these things is the difference between a junior sysadmin and a senior one, and you WILL get exposed to them when things break. But step one in learning more is definitely to recognize that the hards parts of docker are rarely docker itself. Rather, the hard parts are their own thing. Figure out what Linux system you're struggling with and start learning about that and how docker uses it rather than narrowly focusing your research on docker itself. Anything focusing on the docker piece must necessarily gloss over the real foundations which are often very complex... so this will start you expose you to deeper material that you can assemble in your own mind into an understanding of your own specific docker setup.
@PriorProject @dartanjinn This is why OS containers make more sense to me. #LXC is all I use.
I went through that phase too, but people haven't widely adopted the idioms around immutable infrastructure for no reason. My LXC setup was more work to maintain and left me with much MUCH more upgrade uncertainty than my idiomatic/immutable container setup does. I have a deep understanding of both systems and both approaches and I would never go back to using LXC like VMs.
@PriorProject it's definitely more work but it's not that much. I need to set up some more automation but currently I can keep it updated with a few scripts.
I was in the process of writing basically this same post, but you've already said it very well.
Docker isn't really a technology in and of itself; its an applicable of many of the fundamental components of Linux to create a specific result. To really understand it you have to understand the technologies it's built from.
A recommendation I'll throw in here for OP is How Linux Works by Brian Ward. This book is an excellent first step in really understanding those Linux fundamentals.
This might not be a popular opinion, but I had a very short period of time between graduating school and starting work where I needed to learn docker and I found this course: https://serversforhackers.com/shipping-docker to be very helpful, they had a sale and I got the course and it taught me everything I needed in a few days.
You absolutely don't need a course to learn it, but sometimes these can be easy shortcuts.
It doesn’t make any sense to me from the perspective that I’ve approached it.
I think you've already identified the issue. Docker is pretty "simple", but you've got to approach it from the right angle or else it doesn't make sense. Apart from suggesting the official documentation I can't give you much advice on how to learn it because my path to understanding Docker was pretty much just trying things out and iterating on it. That said here's a tips that could help:
As @[email protected] has already mentioned, building your own images is also pretty helpful since it strips a lot of the "black magic" away and teaches you how to further "mod" existing images.
Lots of great information and suggestions in this discussion. Thank you everyone who are offering your input. I’d like to ask as well, how easy is it to setup docker for use in windows? I ask this because almost every guide I’ve found has docker running in Linux. I realize you are able to setup a docker in windows, but I was struggling to get basic instructions for windows, and even the tutorial in docker required me to switch back to docker types that don’t work in windows.
From my experience the "default" experience, which is Docker Desktop, is a pain, due to what you described. In particular, Linux containers and Windows file permissions just don't mix well.
Other than that, there's three options:
Docker Desktop for windows (DDW) using wsl2 for hosting containers is very easy to get started with. It also integrates directly with visual studio code.
Long term you will likely want any useful container services hosted on a Linux vm rather than wsl, as wsl (and DDW) are tied to and start with your Windows user account.
And WSL 2 is the “Linux within windows” if I recall. And your other recommendation is a Linux vm with dockers running inside it? I’ll give that a try if I decide to switch to dockers again instead of just a vm. My original use case was to have a downloader and vpn bundled inside one docker so the rest of the system doesn’t see the vpn connection, but decided to use a vm to accomplish it instead due to “windows+docker” issues.
Start by making a simple image and make a container of it (I started with a simple python webserver)
Once you grasp basic cli commands and the docker lingo, try pulling portainer and use its web ui.
Next step would be to customize an image.
There is a lot of information on SO, and you can ask specific questions on sites like this one. YT videos are 99% shit. Avoid.
I think docker is so stupidly awesome I nowdays dont want to do anything without it.
Some pointers that might help you:
You can mount the same directory in multiple containers. So you can totally have a container reading files from a directory, and access the same files on that directory from, say, a VSCode server container. Just make sure the containers both use the same UID and GID to prevent permission issues.
I took a look at Dashy, I think I see the confusion. If you are looking at this article, then yes they mention Code Server, but that's purely in the context of using Dashy in a non-docker context. But to be honest, any text editor works.
But I think that's a red herring. That in itself has nothing to do with docker.
What you'll need to do, once you understand the fundamentals of running docker, pull images, start a container based on an imagine, is to expose a docker volume that points to /public/conf.yaml. A docker volume ensures that the file or directory it's mapped to in the container is available and persists outside of the container. This allows you to persist files and directories without losing them once the container stops or restarts.
Once the volume is exposed, then you can use your favorite text editor to update the dashy config file. Code Server is fine, powerful, but overkill.
But first, try getting familiar with pulling, starting stopping docker images using the cli. Gotta start there first before tinkering with docker parameters like volumes.
Yeah Dashy isn't really important to me, it's just another fun project to learn more about Docker. However, what I learned is that I don't know shit about what I'm doing lol. It proved to be a great tool at exposing my absolute ignorance of something I thought I was getting a good grasp on.
Yeah I think I'm gonna shit can Portainer and go through that LinkedIn course someone else posted. Thanks for your insight.
Portainer is definitely useful (I use it on a daily basis), but probably a bad place to start...
I started with the following progression:
Good luck in your journey!
Docker-compose got it done. Once I learned about Volumes and using compose to pass in volumes from other instances I was able to pass in a directory with a custom yaml to the Dashy container then pass the same directory into the code-server container and both are working as I expected they should. Compose and volumes were the missing pieces. I also learned that stacks is how to use compose in Portainer. Not sure why they felt the need to change the naming but it works.
That is AWESOME! Congrats!
Yes that's right, portainer stacks equate to compose.. I might be wrong, but I remember reading somewhere a while back that they (and other container orchestration tools) were not permitted to reference "Docker" or its products (including compose) due to legal and licensing restrictions by Docker.
Not to the level of Reddit, but Docker has its fair share of questionable business decisions.
Containers are hard. This should at least get docker running without sudo: https://www.baeldung.com/linux/docker-run-without-sudo
This is almost worse than using sudo. Because now you essentially have full sudo access without having the protections of sudo, like asking for your password
Podman is supposed to be a sudo-less container manager. Though fair warning I hear it is also quite frustrating to start.
Rootless podman (or docker) is a very different thing that adding yourself to the docker group. Rootless is an advanced topic that makes networking and other fundamental aspects of the container runtime work differently so it's harder to exploit the runtime itself. Adding yourself to the docker group just gives your account permission access the docker daemon running as root... which is a much less fundamental change.
I don't recommend beginners to mess with rootless docker or rootless podman. Rootful podman is much more straightforward (though less well documented than docker).
Yeah I spent a few hours with Podman before I went straight back to Docker.
What was different? I learned podman first because that's what my work wanted to use. I feel like 99% of the time I can just alias docker=podman and everything just works the same way you would expect.
Obviously podman has pods instead of docker compose, but you can get docker compose working for podman if you prefer to do it that way.
The one thing that I feel is extremely wonky about podman is UIDs in rootless containers. But when I want to figure out what the UID a user has outside the container, I just mount an empty volume that has permissions 777, touch a file, then check the UID of that file outside the container.
I've been using my library's membership to LinkedIn Learning to take a Docker Essentials course
I think that might be exactly what I was looking for. Thank you.
Whoops, just remembered that I found the above course to be better suited for IT professionals/people with prior knowledge. Here is his course that I'm currently taking.
I find it to be much easier to follow than his prior-mentioned course
You need to understand the fundamentals of how containers work. What’s inside the container is up to the creator - if you are the creator and want a package manager in there just install it. Normal way for me would be to pick a container for a system I like and add my own stuff. Find one that has what you want and add your stuff there.
Tldr: take your example and ask ChatGPT about the parts you don’t understand