Chia GPU Farming and Plotting in Virtual Machines with a Storage Headend Machine - Part 0

When you click on links to various merchants on this site and make a purchase, this can result in this site earning a commission. Affiliate programs and affiliations include, but are not limited to, the eBay Partner Network. As an Amazon Associate I earn from qualifying purchases. #ad #promotions

Chia Plot 2.0 is here

GPU plotting and GPU farming are now a thing. A quick recap may help some. So much has come about in the past month that I am attempting to use some early observations that I made around a few ideas into action. I am also trying to create a good platform that I think is easily possible for folks that want to grow, test, learn and have fun with their hardware. Part of the planning here is that I expect the unexpected and so I have looked for systems that are flexible in the event of any foreseeable needed change. In addition, we will have a good amount of testing around the Official Chia farming package when it is able to farm end to end as well at various compression levels. As of right now there is a complete end to end system developed by MadMAx (who created the original Chia plotter) called Gigahorse and that is what I am utilizing at the current time. It does have a fee and is closed source, but the tradeoffs for the impressive compression levels do make it a compelling offering for me right now. Do your own research and all that.

Chia GPU Plotting and Farming Hardware in the age of Plot 2.0

Assumptions I am making and the reasons why. As always, I have a bias toward use of hardware I own already and overall cost minimization. This works over VERY well for plot 2.0 which can utilize GPUs for creating plots and incredible speeds and with drastically lower electrical usage. The reason for this is the PCIe bus. There are few instances in daily life of a pc user when you will utilize 100% of the bandwidth on a PCIe bus, but plotting chia with GPUs is actually one of them. This coincides with many desktop systems that do have 1 full 16x PCIe4 slot (and seldom more) having a yet another manufacturer limit which is 128GB DDR4 RAM. While you can utilize a 128GB and even 64GB RAM system to plot with the MadMAx Gigahorse plotter, ideally you will need to have 256 to create a K32 plot. Chia has CHIP 12 in process which plans to have an increased plot filter cutting plan in place for the next 10 years. This should easily outstrip the viability for someone to cost effectively plot grind the blockchain with cheap hardware. It is a planned hard fork but in my opinion from early on and still today, is a must-do proposition. Now you’re up to speed.

What my GPU based Chia farm looks like

Here is my current hardware and storage layout from the video. A lot has changed fast.

Dell R720XD

I am using this as the storage headend running Proxmox hosting a VM for TrueNAS and a VM for Ubuntu. Proxmox is a level one virtualization software, and it is really good. It allows you to squeeze in machines in machines is one way to think of it. That spells efficiency and cost savings. I anticipate you have questions, que the FAQ.

- A Dell R720XD… WHY?
  - It has ample PCIe lanes from the dual socket Intel Xeon processors.
  - It has 24 DIMM lanes. This allows the system to hit 256GB easy and cost effectively with lower density 16GB DIMMS.
  - It has a lot of PCIe 3 slots in full 16 width and full width. You may need V2 Intel Xeon chips to have enough lanes as V2 came with 4 extra vs the Sandy Bridge V1 processors.
  - It has 2x 8 Pin GPU power slots. Yes, you read that right.
  - Cards like the Dell branded 3060 and A4000 fit in this. Cards like a Tesla P4 fit in anything, and I am using one in my system. A4000 to be tested for fit soon.
  - It comes in a 3.5″ version that is super cool. 12 3.5″ drive bays up front, 2 2.5″ drive bays in the back. Love it.
  - They are cheap. You might consider going with a Dell R720/R730/R730xd and just hunting around for the best price if the use for the system is just planned for Chia.
  - You can use a tool called IPMI to tune the R720 and any 12th generation servers fans manually.
- What kind of GPUs are in it?
  - Plotting and Farming GPUs for creating and decompression of plots at C8 levels. I decided to go C8. I currently am using an NVIDIA Tesla P4, but that won’t last long at C8. I have an A4000 arriving today and my friend Jon suggested the 3060 (non-ti), which may also possibly be a great idea. It comes with 12GB VRAM and you need 3GB VRAM to farm C8 and 8GB VRAM to plot so possibly a nice “all in one” card for doing both at decent scale….. Have you considered subscribing to the newsletter over there in the sidebar? It’s an easy way to stay up to date as I write more content and post results.
- RAM and Processors
  - ZFS uses RAM for buffering writes to disk and reads. It does a great job at that. I do not use L2ARC or a SLOG here. I have 368GB in it.
  - High core freq desirable for writing fast, but you don’t need a ton of cores really. I am using E5-2667v2 processors. I plan for 5 GPU rigs to be hitting this at a time. 2 PCIe4 and 3 PCIe3.
- Which SAS HBAs in there?
  - LSI 9207-8e for SAS2 JBODs like the NetApp DS4243, DS4246, and DE6600 or the very cool LSI 9207-4i4e if you want a little of both.
- What is my onboard storage arrangement like?
  - I have a Raid Z2 small array of 4TB drives for storage on there connected to its own onboard flashed PERC HBA and the front 12 3.5″ slots. I use this for backups and non chia stuff. I have another HBA that I have dedicated connected to the 2 2.5″ drives in the back. A bit overkill and would be cool to use the 9207-4i4e for it. I have 2x Evo 870 1TB drives in ZFS Raid1 for the boot drive of my Proxmox machine connected. Then I have 2 LSI 9207-8e connected to the JBOD arrays for Chia HDD storage. They are daisy chained in a HA setup currently.
- Why use RAID0 arrays vs single drives for the JBOD? Have lost my marbles?
  - Marbles intact. I have done some math as to why I am doing this. It will take bit over 25 hours to fill up a raid0 ZFS array at the 1.8GB/s that I can write to each tray at with 153TiB of space.
  - I don’t need landing pools of flash. I have yet to kill any flash with Chia activity, and I don’t plan to start now. It’s just not needed. This is not a recommendation for you, it’s just how I am doing it for this JBOD. Maybe I do it different for the next JBOD.
  - My plots exit RAM, go over 40Gbe to mounted NFSv4 share directly.
  - It’s fun. It’s fewer things doing things. It’s a good test. It seems like it could save cost if you don’t need to buy SSDs.
  - It allows me to possibly sell off some of my flash stuffs which is part of running an efficient operation. I am a hoarder of tech stuff, so I very much need to do that.

Bookmark and refer back to https://digitalspaceport.com/chia. These pages will be updated with new information as it is released to keep you Chia farming. Subscribe to the newsletter in the sidebar to get automatic notification when we release new articles.