They build their own servers using some custom components. Economies of scale probably don't really factor into the pricing of their GPU stuff (unless I'm seriously underestimating their GPU customer base), but they do factor into the peripheral stuff like PSUs, fans, hard drives, etc.
There's some videos out there showing off their custom motherboards, it's an odd mixture of standard-ish consumer ATX but stripped down to the absolute bare essentials and with some components rearranged for better airflow in racks chassis they also build themselves: https://youtube.com/watch?v=V2P8mjWRqpk
