I've noticed that this issue has come up several times before, and it seems to be a popular request, so what I will do is I will pass it on to the team so that they can consider it/see if it is possible to implement.
I don't know if such a change would be patchable, or whether it would require an update to the app itself, however I will suggest it.
Back in beta, the idol cooldown was supposed to be longer. The issue was it was too late to implement a cooldown timer for separate items, instead the universal item cooldown was simply increased to 10 seconds. It's really not enough as it makes a lot of bosses trivial. I think OTM need to re-evaluate and start implementing separate timers so that idols and potions can have longer cooldowns (e.g. 1 minute for idols, 30 seconds for potions, etc.).
However, I agree with the OP, idols and sigils shouldn't work in arena at all.
There are two types of people in this world: Those who crave closure
A business is more profitable if they don't gouge and piss off customers.