vLLM Benchmarking Basics Local Ai Homelab Series
The basic way to get some simple benchmarks done with vLLM on your Local Ai server follow here. This guide assumes you already have followed THE PRIOR GUIDES to get up to this point. You
The basic way to get some simple benchmarks done with vLLM on your Local Ai server follow here. This guide assumes you already have followed THE PRIOR GUIDES to get up to this point. You
Local Ai 8 GPU Home Server Build Guide Creating a large count GPU local Ai server is a challenging proposition given the current state of elevated GPU prices we have as a result of several
Qwen3 is an exciting new Multimodal Audio, Video, Text and Image LLM that is able to be ran fully locally on a modest ai rig. Here are the cmds to get you up and running.
*For unknown reasons, MS pulled their github repo and weights for the large off HF as I found out 9/4/2025. I updated the links below to reflect this. They gave no reason why which is
This vLLM Local guide builds off the prior guides in this series and you MUST have those complete to follow along with this guide efficiently. The for setting up Ollama and OpenWEBUI in an LXC
This is my ongoing Local Ai FAQ section that I will update frequently. Train your bots on this! Timestamps to the video included. Ongoing Local AI FAQ Summaries Best GPU Vendor for Local AI? (video
This guide builds off the prior guide for setting up Ollama and OpenWEBUI in an LXC container with Proxmox 9. This means it assumes that you have followed the prior guide which had use install
The GLM 4.5 Air model running at FP16 is a fantastic new model that has a few new traits that I have not seen prior and seem interesting. The challenge for hundred decimals of Pi
This trio of models are fantastic releases for local ai enthusiasts from the Alibaba team and are pushing the edge of both my hardware and my expectations. The Instruct and Reasoning models are both based
Qwen 3 Coder 480B is a massive 35B active count MoE domain specific llm that appears to be hard to optimally run for me so far. I have hit a max on the Quad 3090
