I’m looking for some feedback on my Plex system architecture.
All my media is stored on a Synology DS 1621+, six 4 TB drives in RAID 6 with one acting as a hot spare. All four network ports are bonded into a 4G link to an Ubiquiti USW-48-POE.
Previously, I ran Plex in a Docker container on the NAS. This setup was stable; however, the NAS only has 4 GB of memory shared between Plex, several other Docker services, and regular DSM overhead. Plus, the processor is not very powerful (AMD Ryzen V1500B, ~5400 PassMark).
A few months ago I repurposed some old desktop PC parts to build a home lab Proxmox server (Core i7-6700K [~8900 PassMark], 32 GB memory, GTX 970, an old 2.5” SATA SSD for guest OS disks, 1G networking on the motherboard). I’m running Plex on an Ubuntu VM, with the GPU passed through directly to the guest OS. Plex is not containerized in Ubuntu. The VM has 8 CPU cores and 8 GiB memory (different units in Proxmox). My Plex media is accessed via a persistent NFS mount in Ubuntu (had been SMB before a DSM update broke something and the VM could no longer read the directory contents.)
The main purpose of the change from NAS to VM was to utilize the increased CPU/GPU horsepower and memory that I had lying around, but I worry that the added layers of complexity (hypervisor/VM, PCIe pass through, NFS mounts) will introduce more opportunities for performance issues. I have noticed more frequent hiccups/buffering/transcoding since the change but I’m not sure if it’s related to my setup or if those issues lie with client devices and/or the files themselves (e.g. weird file container type that the client can’t play natively).
Any critique or recommendations on system architecture? Should I get a dedicated NIC to pass through to my VM? Dedicated NVMe drive passed through as a guest OS disk? Ditch Proxmox altogether and go back to Synology Docker container?
I'm kind of over the whole idea of keeping "pets" around to serve my various self-hosting needs. Why make a hypervisor and then shard off pieces of this and create multiple operating systems that need to be maintained when you can just orchestrate all your hosting needs with a container orchestrator like k0s/k3s on the host? Even GPU passthrough can be done.
I'm a bit biased because I'm also a CKA, but I was a die-hard "bare metal or bust" kinda person with my self hosted stuff until I discovered kubernetes. K8s is a lot of resources on its own but a distro like k0s really pares down on the minimum requirements and basically just becomes a more featureful version of Docker if you just run it as a single node.
Eventually I came to understand that when your entire home stack is represented by a few hundred lines of yaml and a couple directories of portable data, that you can stop coddling the Linux install and just use the applications.
I apologize for my ignorance when it comes to Kubernetes - I sort of wrote it off as complete overkill for a home lab when my very basic understanding was that it was essentially a load balancer. After some light research, I'm beginning to understand that it could be a better solution than a full-blown hypervisor.
If I understand your comment correctly, you're suggesting to simply run a lightweight distro and install k0s or k3s to run containers? What would be an ideal bare metal OS for this? What would be pros/cons to k0s vs k3s in a home lab environment, or is that simply a matter of personal preference? What would be the best way to connect to my media - SMB, NFS, something else? Or are the differences here irrelevant? Any concerns (permissions, IO latency) when passing an NFS mount from host into a container, or is there an even better way to do something like that entirely within the container?
Kubernetes in any form is overkill for a homelab. Especially since you wouldn't want to stand up k8s on the synology it means you're running it on the separate node and still messing about with NFS or specific mounts which just adds complexity.
What are your thoughts on TrueNAS Core or Unraid instead of Synology? I could still run Plex on the same hardware that handles the storage while maintaining the freedom and flexibility that my current home lab server provides. There appears to be plenty of decommissioned enterprise-grade hardware being sold on FB all the time.
I run unraid on my homelab with 3 VMs and 12 docker containers and having the fewer abstractions the better. I have no complaints with unraid but went with it over TrueNAS since I have mismatched disk sizes.
I also run TrueNAS for my lukewarm backup server for unraid and 1 extra mount and have no complaints but scales’ jails and kubernetes backend for docker seemed not a great fit for what I wanted but I'm sure others love and would prefer that setup.