Proxmox on a second-hand workstation

The machine running most of my home lab is a Lenovo ThinkStation P700 that cost less than a mid-range mini PC. It has a Xeon E5-2699 v3 in it — eighteen cores of 2014-era server silicon that nobody wants any more — and it has been more useful than anything else I have bought for the lab.

This is not a build guide. Proxmox installs itself in about ten minutes and the official documentation is good. It is about the decisions that turned out to matter, most of which I got wrong first.

Why old workstation silicon

The instinct is to buy something small and new. A modern mini PC is quiet, sips power, and fits behind a monitor. I have one; it runs my always-on services. But it has four cores and one RAM channel, and the moment you want to run several VMs that each want real memory, you are stuck.

Second-hand dual-socket workstations solve that problem in a way nothing else at the price does. You get:

• Core count that is genuinely spare. Not “enough”, spare. The difference matters, because the entire value of a hypervisor is that you stop thinking about whether there is room for another VM.
• Registered ECC memory at scrap prices. DDR4 ECC RDIMMs are worth almost nothing because almost nothing consumer takes them. 64GB or 128GB costs less than 32GB of desktop DDR5.
• Real PCIe lanes. This is the one people forget, and it is the one that actually decides what the machine can do. A consumer CPU gives you sixteen usable lanes and a chipset arguing over the rest. This class of chip gives you forty per socket, which means a GPU and an HBA and a decent NIC, all at full width, without a lane-sharing spreadsheet.

The cost is electricity and noise. A dual-socket Haswell-EP box idles somewhere around 90–120W depending on what is in it, and the stock cooling is designed for an office rather than a bedroom. If your machine has to live where you sleep, buy the mini PC. If it can live in a garage, loft or cupboard, buy the workstation and spend the difference on RAM.

The three decisions that mattered

1. Storage layout, decided before you install anything

Proxmox will happily install onto a single disk with LVM-thin and you will never think about it again — until you want snapshots, or a disk dies, or you want to replicate to another host.

What I would tell myself: decide up front whether you want ZFS, and if you do, install onto it from the beginning. Converting later means rebuilding the host. ZFS gives you snapshots that are instant and nearly free, checksums that catch silent corruption, and send/receive for backups that are incremental at the block level. The cost is RAM — budget a gigabyte or so per terabyte and do not fight it — which is exactly the resource the second-hand workstation gives you for nothing.

The mistake is to treat the boot disk as unimportant. It holds every VM definition, every firewall rule, and the cluster configuration. Mirror it. Two cheap SSDs in a ZFS mirror cost almost nothing and remove an entire category of bad weekend.

2. Passthrough is a hardware decision, not a software one

I run a GPU passed through to a VM that does image and video generation. Getting there is straightforward if the hardware cooperates and miserable if it does not, and you find out which after you have bought everything.

The thing to check before purchase is IOMMU grouping. Your motherboard decides which devices are isolated from each other, and if your GPU shares a group with, say, the USB controller, you cannot pass one through without the other. Older workstation boards are generally good at this because they were designed to handle multiple discrete cards; consumer boards frequently are not. It costs nothing to check the grouping on a machine you already own before you commit to a plan that depends on it.

The second thing: a GPU being passed through must not be the one the host is using to draw its console. On a workstation with onboard graphics this is free. Without it, you need a second card, or you need the host to run headless and be confident enough to administer it entirely over the network.

3. Backups are not snapshots

A ZFS snapshot protects you from your own mistakes. It does not protect you from the machine catching fire, and it does not protect you from deleting the pool.

The lesson arrived the hard way. I had a box holding the only copy of a project’s configuration — the automation that drove it, and the credentials it needed. The box died. Not corrupted, not recoverable: dead. Everything the machine did had been written down nowhere except on the machine.

The site it published survived, because a static host kept serving the last build for nine months while nothing new appeared. That is the only reason the story has a happy ending, and it is luck rather than design.

What changed afterwards is not really a Proxmox lesson, but it is the most important thing on this page: anything that describes how a system works belongs in a repository, not on the host that runs it. Code, configuration, unit files, the lot. Secrets stay out of the repository and go somewhere they can be regenerated. If the machine disappears overnight, the recovery should be boring.

Proxmox Backup Server is the natural other half of this, and it is worth running even if the only target is a USB disk on the same desk. Deduplicated, incremental, verifiable, and restore-tested — the last of which is the part everyone skips. A backup you have never restored is a hypothesis.

Things I would do differently

Use a separate VLAN for management from day one. Retrofitting network segmentation onto a working lab is tedious in a way that doing it up front is not.

Do not over-allocate vCPU early. It is tempting to give every VM eight cores because there are plenty. It makes scheduling worse, not better. Start small and raise it when something is actually slow.

Name things for what they do, not what they are. Six months later you will not remember which of vm-101 and vm-104 is the one you are not allowed to reboot.

Turn on the no-subscription repository properly, and take the updates. The nag dialog drives people to script it away and then never patch. Set the repository correctly and let it update; a hypervisor is a large piece of attack surface running as root, and it is not the thing to leave sitting on an eighteen-month-old kernel.

Is it worth it

For me, unambiguously. The value is not the hardware, it is that provisioning a machine becomes a five-minute decision instead of a purchase. Wanting to try something no longer has a cost attached, and that changes what you try.

The trade is noise, heat, about ten pounds a month in electricity, and a machine that looks like it was rescued from a skip. Everything else about it has been straightforwardly good.