@beldin The skip button I don't think should matter - you aren't drawing glyphs. I don't think I specified that last time so I will now.
Secondly, having other places to put the timer does not mean there would be less accidents. After all, considering that where the timer is now is not one of the locations that spell starts at - the best we could do is move it somewhere where the same is true. As such, drawing larger glyphs should only make it less likely to accidentally hit the timer. I would attribute the accidents - in your case - to the cold and the gloves, something which I believe to be a fairly uncommon situation.
Finally, I would have the very opposite problem should we move it to where you suggested. Often times, for tests and such, I skip through 8-9 seconds of the timer and the turn itself, so it is nice for the button to be accessible. The buttons on the side are fairly unaccessible, the very fact you are using to try and put the timer there. This makes it much more difficult to actually use the timer feature, should it be moved there.