If you know the ID, yes.
But this sounds like you are someone who's using some macro that uses the key 4.
Instead, consider swapping whatever skill is on 4 to another key with an "useless" skill.
Then, fix your macro to use the new key, instead of 4.