This sounds like a good problem for you to troubleshoot.
Here are some tips/questions to help you out.
1. What is the X and Y coordinate of the gem in the secret room? What is the X and Y coordinate of the gem in layout 5? Are they the same?
2. What mechanism is used to determine if a gem should be destroyed? Can you think of a better way? For example, would using the UID of the object work? Would using the Layout of the object help to solve the problem?
Once you answer these then you should be able to fix it.