Using a Dell R720XD for Local Ai Server and Homelab

EVEN MORE INFO About the Dell R720XD I am using as part of my Homelab and Ai infrastructure

I am using this as this R720XD as a storage head-end running Proxmox hosting a VM for TrueNAS and a VM for Ubuntu as well as several LXC and Docker containers. Proxmox is a level one virtualization software and it is really good. It allows you to squeeze many virtual machines in one real physical machine is a good way to think of it. That spells efficiency and cost savings. I anticipate you have questions, que this updated DDR4-5 RAM DISASTER UPDATED FAQ.

• • A Dell R720XD… WHY?
    • It has ample PCIe lanes from the dual socket Intel Xeon processors.
    • It has 24 DIMM lanes. This allows the system to hit 256GB easy and cost effectively with lower density 16GB DIMMS. Impossible with DDR4 and especially DDR5 systems for cheap anymore.
    • It has a lot of PCIe Gen 3 slots in full x16 width mechanically and electrically. You simply must get cheap V2 Intel Xeon E5 2600 chips to have enough lanes as V2 came with 4 extra vs the Sandy Bridge V1 processors. In addition they are faster and have more cores in many instances.
    • It has 2x 8 Pin GPU power slots. Yes, you read that right. GPU’s in an R720XD is a real thing!
    • Cards like the Dell branded 3060 and A4000 fit in this. Cards like a Tesla P4 fit in anything and I am using one in my system. A4000 was a great card I had in mine for a long time and it worked excellent.
    • It comes in a 3.5″ version that is super cool. 12x 3.5″ drive bays up front, 2x 2.5″ drive bays in the back. SAS compatible. Very ZFS friendly Server. Love it.
    • They are cheap. You might consider going with a Dell R720/R730/R730xd and just hunting around for the best price if the use for the system is just planned for Local Ai but the R720XD is likely the larges capacity + cheap DDR3 + cheap base system you will find in large ram config.
    • You can use a tool called IPMI to tune the R720 and any 12th generation servers fans manually. There are great scripts to help with that automatically also.
  • What kind of GPUs are in it?
    • I have a few local Ai GPUs for auxiliary services that are just 8GB however I have tried out several GPUs in it over the years and it is very capable of cooling and providing PCIe Gen3 x16 performance levels. For Inference tasks, that is perfectly fine. I currently am using an NVIDIA Tesla P4, but and have used an A4000 and a 3060 12GB (non-ti), which may also possibly be a great idea as it has 4 extra GB VRAM. That 12GB VRAM and 360GB/s performance with Ampere class handling really does add a lot of extra capabilities over an 8GB GPU in what models you can run. Two of those can make this server possibly a nice “all-in-one” homelab server for doing storage AND local AI at the same time as well as any other homelab tasks you could imagine like Frigate, Home Assistant, Nextcloud, and more. Have you considered subscribing to the newsletter over there in the sidebar? It’s an easy way to stay up to date as I write more content, post results and configs and share deals I find out there.
  • RAM and Processors
    • ZFS uses RAM for buffering writes to disk and reads. It does a great job at that. I do not use L2ARC or a SLOG on this machine. I have 256GB DDR3 1600 in it. 
    • High core freq desirable for writing fast to shared storage and it also provides a nice bump for your decode TPS, about 4+TPS per extra GHz, but you don’t need a ton of cores really. An excellent balanced processor is a pair of E5-2667v2’s. This also has slots for high performance networking but do know if you install it does not support bifurcation nor can you boot from NVMe drives in a carrier card. I did publish a perfect solution to use an m.2 mSATA + NVMe card in this in my R730XD Truenas Setup video, timestamped here. https://youtu.be/dgjBdhS1rCE?t=648 the same will work with an R720XD also. That leave the 2x 2.5″ drives in the back free for a few SSD’s in a RAID 1 config that can run your VM and LXC disks that are mounting your fat file shares that are residing on your 12x 3.5″ drives.
  • Which SAS HBAs in there?
    • LSI 9207-8e for SAS2 JBODs like the NetApp DS4243, DS4246, and DE6600 or the very cool LSI 9207-4i4e if you want a little of both.
  • What is my onboard storage arrangement like?
    • I have a Raid Z2 small array of 4TB drives for storage on there connected to its own onboard flashed PERC HBA and the front 12x 3.5″ slots. I use this for backups, internal hosting of temp files and ai models and my steam game library. I have another HBA that I have dedicated connected to the 2 2.5″ drives in the back. A bit overkill and would be cool to use the 9207-4i4e for it. I have 2x Evo 870 1TB drives in ZFS Raid1 for the boot drive of my Proxmox machine connected. Then I have 2 LSI 9207-8e connected to the JBOD arrays, nice. They are daisy chained in a HA setup currently.
  • Why are you doing all this man, that is a datacenter in your garage. Have you lost your marbles?
    • It allows me to possibly sell off some of my flash stuffs which is part of running an efficient operation. I am a hoarder of tech stuff, so I very much need to do that.