Let’s just say hypothetically this was possible and that the laws of silicon were not a thing, and that there was market demand for it for some asinine reason. As well as every OS process scheduler was made to optionally work with this. How would this work?
Personally, I’d say there would be a smol lower clocked CPU on the board that would kick on and take over if you were to pull the big boy CPU out and swap it while the system is running.
With the motherboard I am using (Supermicro X9DRH-7F) I get 6 8x PCIe slots, and 1 16x PCIe slot for a total of 7 slots. All of these are communicating directly with the CPUs, as opposed to some boards where the slots go through the chipset. There are motherboards with even more, but they are more expensive. I got this because I will need the bandwidth for model parallelism using multiple GPUs.
Also, the least power-hungry CPUs you can get for this board are the Xeon E5-2630L, which each consume 60W under full load.
Here’s my slot layout
Specs of my server: https://burggit.moe/comment/59898
Oh nice, you could probably fit a few GPUs in there if your case has room for it.
I’ll probably do that when LLM generating becomes less cost prohibitive. Not a gaymur