You can have video mixing or you can have little CPU load. You can't have them both.
And when I'm talking about CPU load I really mean it. For MCU style conferences, get a machine with the most cores and CPUs you can afford. That's the rule of the game. Ask Industrial Light and Magic about video effects and CPU cycles.
That said, let's see how to be Magicians On the Cheap (TM).
You can achieve very good results without sacrificing much, if you don't need fancy effects:
pcmu or pcma (for example, g711) audio, one channel (mono), 8khz (support is mandatory in WebRTC and ubiquitous in SIP)
video-mode passthrough
(be sure no layout is mentioned in conference profile, if one is mentioned, the conference is automatically started in mux video-mode)
With those settings, you'll have an incredibly low CPU load: you're actually just switching between different video inputs, choosing one, and retransmitting it as-is to all participants. As for audio, you get input streams that are not...