For any of your objects that don't require the most accurate physics calculations, you can tone down their physics interations which is a big performance boost.
Here's the part of the manual in the physics object
Set stepping iterations Set the number of velocity iterations and position iterations used in the physics engine. The default is 8 and 3 respectively. Lower values run faster but are less accurate, and higher values can reduce performance but provide a more realistic simulation.