I don't know if it helps in this situation (I'm not sure if I fully understood the issue), but you can always have audio being unaffected by timescale (Properties/Timescale audio)
Also I made an example of staying time-based (and keeping a few things running, e.g an audio fade) while Timescale = 0 in "Verve!", an example game. You can find it in the examples & tutorials section.
But again, not sure if it helps in this specific case.