Lawyer upgrades eight-V100 private AI server to run 80% of DeepSeek V4.1 Flash at 4-bit
Start here
A lawyer running a private AI server on eight used Nvidia V100 GPUs moved about 80% of the DeepSeek V4.1 Flash legal workload from 2-bit to 4-bit NVFP4 precision, adding 1TB of RAM and an RTX 3090. Code output fell from 160–215 to 150–180 tokens per second, but the crashes that affected the new build are gone; the lawyer attributes them to a bad PSU rather than PCIe. The setup is a self-hosted alternative to cloud inference for legal work, trading some throughput for higher precision on hardware that predates native support for the format.
Generated from reports · updated 12 min ago
Report timeline
Follow the reports to see the event from different sides.
- Hacker News · AILawyer upgrades eight-V100 private AI server to run 80% of DeepSeek V4.1 Flash at 4-bit NVFP4
A lawyer running a private AI server on eight used Nvidia V100 GPUs moved about 80% of its DeepSeek V4.1 Flash legal workload from 2-bit to 4-bit NVFP4 precision, adding 1TB of RAM and an RTX 3090. Code output dropped from 160–215 to 150–180 tokens per second, but the crashes that hit the new build are gone, with a bad PSU rather than PCIe the likely culprit.
Heat for this event
Not enough continuous observations yet to draw a trend.