Some of the other answers here are confusing two events I think. In the Fellowship of the Ring, Gandalf says “No, don’t give it to me, put it on the mantelpiece. It will be safe enough there”. As he does that, Bilbo hand jerks back and he drops the envelope with the ring inside. “Before he could pick it up, the wizard stopped and seized it and set it in its place.” The Fellowship of the Ring

So there the wizard obviously wants to have as little to do with the ring as possible. But does touch the envelope it’s inside. The incident where he puts it in the fire is later on, after he returns from his journey to find out more about the ring in Gondor, and his meeting with Aragorn who had captured Gollum, and involves Gandalf with Frodo.

The reason he doesn’t want to touch the ring at all, if he can help it, is because it could be too much temptation for him, he might start to desire it for himself and with his power he could do a lot of good with it - but he knows that would be the path to his eventual downfall.

The only character in the books not affected by the ring at all is Tom Bombadil. For him it is a bauble, which he would soon forget and lose, he not only doesn’t go invisible when he wears it, he actually makes the ring itself vanish for fun. See also Robert Walker's answer to Does Tom Bombadil predate the Istari? Apparently Tom Bombadil is not in the movies and I think that’s a real shame as then you lose that perspective of someone who is not affected by the ring at all.