Is it safe to remove the rod with my hand (wearing a glove)?

I'm about to start my first weld. I've watched a lot of how-to videos and I think I have everything figured out to start except one thing. What do you do with the stinger/rod when you're paused between welds to say flip the work piece over? Should I turn off the welder? Should I put the stinger down...