As you all might be aware VMware is hiking prices again. (Surprise to no one)
Right now Hyper-V seems to be the most popular choice and Proxmox appears to be the runner up. Hyper-V is probably the best for Windows shops but my concern is that it will just become Azure tied at some point. I could be wrong but somehow I don't trust Microsoft to not screw everyone over. They already deprecated WSUS which is a pretty popular tool for Windows environments.
Proxmox seems to be a great alternative that many people are jumping on. It is still missing some bigger features but things like the data center manager are in the pipeline. However, I think many people (especially VMware admins) are fundamentally misunderstanding it.
Proxmox is not that unique and is built on Foss. You could probably put together a Proxmox like system without completely being over your head. It is just KVM libvirt/qemu and corosync along with some other stuff like ZFS.
What Proxmox does provide is convenience and reliability. It takes time to make a system and you are responsible when things go wrong. Doing the DIY method is a good exercise but not something you want to run in prod unless you have the proper staff and skillset.
And there is where the problem lies. There are companies are coming from a Windows/point in click background who don't have staff that understand Linux. Proxmox is just Debian under the hood so it is vulnerable to all the same issues. You can install updates with the GUI but if you don't understand how Linux packaging works you may end up with a situation where you blow off your own foot. Same goes for networking and filesystems. To effectively maintain a Proxmox environment you need expertise. Proxmox makes it very easy to switch to cowboy mode and break the system. It is very flexible but you must be very wary of making changes to the hypervisor as that's the foundation for everything else.
I personally wish Proxmox would serious consider a immutable architecture. TrueNAS already does this and it would be nice to have a solid update system. They would do a stand alone OS image or they could use something based on OStree. Maybe even build in a update manager that can update each node and check the health.
Just my thoughts
I think you are looking at this wrong. Proxmox is not prod ready yet, but it is improving and the market is pushing the incumbent services into crappier service for higher prices. Broadcom is making VMware dip below the RoI threshold, and Hyper-v will not survive when it is dragging customers away from the Azure cash cow. The advantage of proxmox is that it will persist after the traditional incumbents are afterthoughts (think xenserver). That's why it is a great option for the homelab or lab environment with previous gen hardware . Proxmox is missing huge features...vms hang unpredictably if you migrate vms across hosts with different CPU architectures (Intel -> AMD), there is no cluster-wide startup order, and things like DRS equivalents are still separate plugins. That being said knowing it now and submitting feedback or patches positions you to have a solution when MS and Broadcom price you out of on-prem.
Pretty sure that's why they made Azure Stack HCI, it's hyper-v, but doesn't work without an up to date azure subscription and charges you monthly fees to run vms on hardware you own.
It's great, the worst of both worlds..... Fucking thing doesn't even report on disk provisioned, only utilization, so get fucked it you want to capacity plan without writing your own report script.
Proxmox has those features. Also I personally wouldn't mix CPU archs but you should be able to as it is all KVM. Maybe there is a different memory layout or something
I'm battling this right now; it SHOULD work but does not work consistently. Again, homelab, not ideal environment. I'm going from 2 R710's with Xeons to a 3-node cluster with the 710's and an EPYC R6525. Sometimes VM's migrate fine, sometimes they hang and have to be full reset. Ultimately this was fine as I didn't migrate much, but then I slapped on a DRS-like thing, and I see it more. I've been collecting logs and submitting diagnostics; even pegging the VM's to a common CPU arch didn't fix it.
To that end, DRS alternatives are still mostly plugins. This was the go-to, but then it was abandoned:
https://github.com/cvk98/Proxmox-load-balancer
And now I'm getting ready to go deeper into this, but I want to resolve the migration hangs first:
https://github.com/gyptazy/ProxLB
Proxmox has load balancing capabilities built in. You can just toggle it on and Proxmox will level everything out. However, if you are having issues with VMs hanging I would get that resolved first.
I've never done a live transfer between AMD and Intel so maybe there is more to the story. Make sure you get on the Proxmox forms as that's where the developers hang out.
Where do you see the load balancing feature? Searching for exactly that was what got me to ProxLB. I have HA groups and fences, but that's less resource allocation than failure resolution in my experience. My cluster is 8.2.7.
I posted to the forums, but I got a "YMMV" kind of answer; the docs say it's technically unsupported: https://pve.proxmox.com/pve-docs/chapter-qm.html#_requirements
I'm setting the CPU Type to x86-64-v2-AES which is the highest my westmere CPU's can do. I have a path to getting all 3 nodes to the 6525 hardware, pending some budget and some decomm's at work.
I've read extensively about that, and this thread was very helpful, and my understanding is that's still not really a DRS equivalent, but more of a recovery mode: https://forum.proxmox.com/threads/ha-cluster-resource-scheduling-filling-in-the-missing-pieces-from-the-docs.139187/
Yeah I just know it goes beepity boop and then the VMs move around. My deployment is also pretty small so it really doesn't matter for me.