Moving to Proxmox

Virtualization is an important tool for a homelab, large or small. For many years I ran some iteration of VMWare, normally ESXi with vCenter. Last year I switched everything from VMWare to Proxmox, a "free" hypervisor platform. In 2022 there are quite a few homelab hypervisor platforms that we could take advantage of.

Hypervisor Options

Homelabbers have plenty of options when it comes to running a hypervisor at home. If you are just getting started you might investigate something free like Oracle's Virtual Box, Microsoft's Hyper-V, or VMWare's Workstation Player. These products are often referred to as Type 2 hypervisors, or hypervisors that virtualize on top of an operating system. These products work well as they allow you to just install the hypervisor directly on your existing operating system, the cost to get started is quite low, if it isn't for you or you feel that you want to move onto more advanced hypervisors, worst case you just uninstall the hypervisor (or in the case of Hyper-V, just turn off the Windows feature).

Hypervisors like VMWare's ESXi and Proxmox Virtual Environment are known as Type 1 hypervisors, or hypervisors that are installed on bare metal. The benefit of Type 1 hypervisors is that they are more performant, designed to run as servers, and include more enterprise-like features.

VMWare At Home?

Yep! While you may associate VMWware with high priced enterprise licenses (and you would not be incorrect about that), there are methods of getting around high priced enterprise licenses....

No, not piracy, the VMUG Advantage! VMUG, or VMWare User Group, offers a number of benefits for it's members, including 365-day "evaluation" licenses of VMWare products, including ESXi and vCenter. While $200 isn't cheap it is remotely in the ballpark of what someone might pay for enterprise software they can run at home. While there are some limitations of the VMUG license, for a home lab running 2-3 hypervisor nodes, the VMUG license works quite well.

Why Not VMWare?

So, why am I not running VMWare at home anymore? Well, VMWare ESXi is enterprise software and while the license you get through VMUG covers a number of vCenter nodes, vCenter is heavy software, not the ESXi hypervisor itself, but vCenter. See, ESXi in a cluster requires a vCenter server to be running. A typical architecture is to run the vCenter server as a VM on the very cluster it is managing. Yes! This isn't normally a problem in actual enterprise, when you have multiple ESXi nodes in your cluster, but for home, where I have two ESXi nodes, vCenter is just too much. The smallest installation size of vCenter that exists (because it only exists as an appliance, not software installed on a server) is 12GB. That is a LOT for a homelab VM that is just managing communication between two ESXi nodes.

Ok, great vCenter is BIG. Why not just run two nodes on ESXi without vCenter? Hmm, well, that is a great question! The answer has to do with the tag line of this site, Probably Security Stuff. I run a number of VLANs at home, and I bind network connections using LACP, annnnd both of those things need VMWare distributed switches, which are only configureable in vCenter.

Why Proxmox

VMWare and Proxmox are not the only hypervisors that we have to choose from, there are quite a few hypervisors available. What I was looking for was an appliance style hypervisor that supports clustering, flexible storage options (local storage, NFS, iSCSI, JBOD), and network interface bonding with VLAN support. Proxmox provides all of these features for an attractive price (free). Proxmox utilizes the license model that a number of open source projects use, for their enterprise customers (or those wishing to support the project or utilize the "enterprise" package repository) there is enterprise support, but for everyone else there is an open source license with the only downside being a "bug" letting you know that you don't have a license when you log into Proxmox, in my opinion, a VERY reasonable tradeoff.

Important Differences

Moving from VMWare to Proxmox was relatively straight forward, if you are generally familiar with virtualization technologies you likely will not run into too many differences. That being said, there are a few things that are different between these products.

Installation Media

A common VMWare ESXI installation method is to utilize a USB drive to be the boot disk for ESXI (at least in a homelab environment, this will differ in enterprise). A big reason for this is that ESXI takes over the full boot disk where you install the ESXI system. Proxmox allows you to utilize the same boot disk for the Proxmox installation and VM/ISO storage so you are more free to use actual disks for installation.

Networking

Proxmox is a linux based OS and uses linux style networking concepts. While VMWare network can take some time to get used to, Proxmox is a bit more similar to what most people may be familiar with. Proxmox networking supports interface bonding and VLAN support.

Updates

In my opinion, VMWare updating is confusing, staging updates, actually applying updates, seems a bit complicated. Proxmox updates work fairly simply, just apply the updates and it runs apt update, reboot to apply kernel updates, done.

Clustering

One of the more attractive features of Proxmox is the ability to have a cluster without a single coordinating system like vCenter. Proxmox systems are able to form a cluster with just the default installation. So, what happens if you only have a 2 node cluster? There still should be some concept of quorum, well Proxmox has the concept of a QDevice that allows you to fix that issue with just a Raspberry Pi.

Backups

VMWare backups have always been a bit difficult, even if you used Veeam free backup you needed a licensed version of vCenter to utilize the backup APIs in VMWare. Proxmox not only has a dedicated enterprise backup solution, but it is built into the Proxmox hypervisor. This allows you to utilize any of the configured storage options to be used as backups for your VMs.

Lessons Learned

Overall my move from VMWare to Proxmox was quite positive. I save $200 a year in VMUG licenses, I better utilize the compute I have (not burning 12GB of ram for a clustering VM), and am supporting software that just feels better in my homelab. Proxmox version 7 has been a breath of fresh air for someone who has been running VMWare for years at home. Removing some of the enterprise complexities while still providing enterprise performance has brought some enjoyment back to home virtualization.