The way I see it is that I make the CAD, send the CAD to the part supplier (Fabric8) who automatically check to make sure it’s printable, they then make the parts and post it to me. I would do the assembly myself if needed but would probably try to design things to be minimal / no assembly. The hope would be that this process would be cost effective at scale so if I need more I could just press a button.
I run a on-prem mini cluster which is water cooled but the customers need edge compute which should stay air cooled. I would probably try to make a blower style vapor chamber for nvidia gpus so I can ram air through without dealing with nvidia driver fuckery. NVidia segments the market based on heat sinks, binning and drivers so the enterprises segment has to pay far more. The 4090 blowers made by OEMs are gimped, rare, and expensive and I think intentionally so.
Not only is consumer grade cheaper but it’s way less hassle to get - the Enterprise sales pipeline is a total pain as the costs keep changing and they keep trying to push older overpriced stuff onto me like I wouldn’t notice. And thats even if they think you’re big enough to talk to. Much easier to pull consumer stuff from the market on an as needed basis.
My software supports graceful degradation so I don’t need ‘enterprise’ reliability. I don’t need high speed interconnects either. I need TFLOPS on dense matmuls, run in parallel batches. Consumer GPUs are fine for this, replace the heatsinks with a blower optimized and put in some powerful fans and take off. I could pack more into a single computer and to have a lower amortized cost and a higher density.
If the vapor chamber blower heatsink is too expensive then I might as well just buy more GPU computers and let them run slower.
I run a on-prem mini cluster which is water cooled but the customers need edge compute which should stay air cooled. I would probably try to make a blower style vapor chamber for nvidia gpus so I can ram air through without dealing with nvidia driver fuckery. NVidia segments the market based on heat sinks, binning and drivers so the enterprises segment has to pay far more. The 4090 blowers made by OEMs are gimped, rare, and expensive and I think intentionally so.
Not only is consumer grade cheaper but it’s way less hassle to get - the Enterprise sales pipeline is a total pain as the costs keep changing and they keep trying to push older overpriced stuff onto me like I wouldn’t notice. And thats even if they think you’re big enough to talk to. Much easier to pull consumer stuff from the market on an as needed basis.
My software supports graceful degradation so I don’t need ‘enterprise’ reliability. I don’t need high speed interconnects either. I need TFLOPS on dense matmuls, run in parallel batches. Consumer GPUs are fine for this, replace the heatsinks with a blower optimized and put in some powerful fans and take off. I could pack more into a single computer and to have a lower amortized cost and a higher density.
If the vapor chamber blower heatsink is too expensive then I might as well just buy more GPU computers and let them run slower.