this post was submitted on 05 Jan 2025
34 points (90.5% liked)

Selfhosted

40942 readers
985 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
ruud
Loki
CannaVet
devve
HybridSarcasm
[email protected]
34
Is it possible to run a docker host that has no harddrive? (self.selfhosted)
submitted 3 days ago* (last edited 3 days ago) by Passerby6497 to c/selfhosted
43 comments fedilink hide all child comments
 

So I had a micro PC that was running one of my core services and it only supports NVMe drives. Unfortunately, this little guy cooked itself and I'm not in a position to replace the drive. The system is still good and is fairly powerful, so I want to be able to reuse it.

I'm thinking I want to set up some kind of netboot appliance on another server to be able to allow me to boot the system without ever having a local disk. One thing I want to is run some docker images (specifically Frigate) but i wont be able to write anything to persistent storage locally. NFS shares are common in my setup.

Is it even possible to make a 'gold image' of a docker host and have it netboot? I expect that memory limitations (16GB) will be my main issue, but I'm just trying to think of how to bring this system back into use. I have two NAS appliances that I can use for backend long term storage (where I keep my docker files and non-database files anyway), so it shouldn't be too difficult to have some kind of easily editable storage solution. I don't want to use USB drives as persistent storage due to lifespan concerns from using them in production environments.

you are viewing a single comment's thread
view the rest of the comments
[–] [email protected] 15 points 3 days ago (3 children)

Kind of, but probably not. I started writing this and was like "totally it could be stateless". Docker runs stateless, and I believe when it starts it is still stateless (or at least could be mounted on a ramdrive) - but then I started thinking, and what about the images? Have to be downloaded and ran somewhere, and that's going to eat ram quickly. So I amend to you don't need it to be stateful, you could have an image like you talked about that is loaded every time (that's essentially what kubernetes does), but you will still need space somewhere as scratch drive. A place docker will places images and temporary file systems while it's running.

For state, check out docker's volume backings here: https://docs.docker.com/engine/storage/volumes/. You could use nfs to another server as an example for your volumes. Your volumes would never need to be on your "app server", but instead could be loaded via nfs from your storage server.

This is all nearing into kubernetes territory though. If you're thinking about netboot and automatically starting containers, and handling stateless volumes and storing volumes in a way that are synced with a storage server.... it might be time for kubernetes.

[–] [email protected] 5 points 3 days ago (1 children)

I guess you can also use NFS/iSCSI for images too?

[–] [email protected] 3 points 2 days ago (1 children)

Correct, I run docker on a compute host that has no local storage. The host’s disks are on iSCSI LUNs.

[–] Passerby6497 2 points 2 days ago (1 children)

That's really good to know. Do you ever have issues writing database files on those disks? Database files on nfs mounts have been the bane of my existence.

[–] [email protected] 2 points 1 day ago

FWIW I run only very small databases e.g., sqlite ones shipped with applications, but haven’t had any problems in about a year now, and nothing that wasn’t recoverable from backup.

[–] [email protected] 1 points 2 days ago (1 children)

Yeah, a PXE boot should work, but you'd need a ton of RAM (I'd double to 32GB for Frigate). Drives are cheap, I'd just get one and not deal with network booting at all.

[–] [email protected] 3 points 2 days ago (1 children)

Exactly. Hell 50 bucks you can get a decent SSD. Just grab something, have all of your drives hosted via NFS, but then you aren't hacking docker to run in ram all the time, and wasting your ram hosting stuff it doesn't need to

[–] Passerby6497 1 points 2 days ago (1 children)

Hell 50 bucks you can get a decent SSD.

If only it were that easy, I would have already thrown a spare 2.5" into the system, but it's only got a single nvme slot for local storage.

[–] [email protected] 2 points 2 days ago (1 children)

You can get an NVME drive for <$50, in fact I saw a 128GB one online for ~$15 from a reliable brand (Patriot).

[–] Passerby6497 3 points 2 days ago

That's actually doable. Thanks for that friend.

[–] Passerby6497 1 points 3 days ago (1 children)

So I amend to you don’t need it to be stateful, you could have an image like you talked about that is loaded every time (that’s essentially what kubernetes does), but you will still need space somewhere as scratch drive. A place docker will places images and temporary file systems while it’s running.

Putting the image somewhere is easy. I've got TBs of space available on my NAS drives, especially right now with not acquiring any additional linux ISOs.

For state, check out docker’s volume backings here: https://docs.docker.com/engine/storage/volumes/. You could use nfs to another server as an example for your volumes. Your volumes would never need to be on your “app server”, but instead could be loaded via nfs from your storage server.

I'll check that out. If that allows me to actually write databases to disk on the nfs backing volume, that would be amazing. That's the biggest issue I run into (regularly).

This is all nearing into kubernetes territory though. If you’re thinking about netboot and automatically starting containers, and handling stateless volumes and storing volumes in a way that are synced with a storage server… it might be time for kubernetes.

I don't think I've ever looked into kubernetes. I'll have to look into that at some point... Any good beginner resources?

[–] [email protected] 4 points 2 days ago

Personally I suggest k3s, setting up a test cluster and playing with it. For volume management I use longhorn. It's a HUGE learning curve, but it's officially something companies will shell out big money for too if you're willing to learn it. Soup to nuts from setting up test cluster and playing with it all the way to all of my services running was about 4 months of tinkering for me- but I'll never go back