When planning to destroy large buildings or structures you usually want some parts to stand, however other objects may eventually collide with them, that’s why those parts should be set as static bodies in dynamics, sadly if these static parts have complex shapes,  that will increase significantly the computation time despite they doesn’t move. For example in the image bellow we want to destroy only the main tower on the right but when falling it will surely collide with the rest of the castle so we must set the whole thing in dynamics.

A way to speed up the simulation in cases like this is using simple objects as barriers, so flying fragments cannot pass through them. The idea is those barriers separate the real geometry from the fragments, you can use simple shapes as boxes or cylinders to enclose the area where fragments can move, taking care they approximate the shape behind in case the real geometry is very close to the barrier. In our example we can surround all standing parts with simple boxes and some cylinders,

Modifying  the barriers shape if necessary to approximate correctly the geometry behind, the important thing is keeping always the low poly count. It doesn’t mind if the barriers interpenetrate each other as they are static objects.

At this point you have just to set the barriers as static bodies , you can even hide them from the viewport and PDi will take them in account for collisions anyway. After the barriers are hidden the result can be better than using the real geometry as you are avoiding its complexity and for sure you will save important computation time. See the image at the bottom once the barriers are hidden for our example.