Instead of using families, make a layer called Collision, and make a tiny red box (like, 5x5 pixels), and spread it around where there is collision. Make THIS object solid. Check for collision against this object only. Make sure to make this layer invisible, though (or probably toggle-able, if you want to have a 'debug' area).
This is probably something you should end up doing normally, because it removes ambiguity in collision detection.