The tea / coffee room where mathematicians meet in the Maths Dept in Oxford has whiteboard tables - or did when I was there in the late 1980s and early 1990s . And the mathematicians fequently jot equations on them while drinking their tea and coffee. Like this:
Whiteboards on the walls as well, so if a mathematician wants to talk about something more complex in the middle of their tea time chat, and not enough room for it on the table, will often get up and start writing it on the walls as well.
I've done both myself :).
I haven't ever written it on ordinary walls or windows, like the movies. But have woken up from sleep with an idea, or had an idea while out on a walk, and scribbled it on the nearest available scrap of paper, whatever it might be - often used envelopes.
I used to carry paper and a pen around with me in case I thought of something.
The thing is - that in Maths you can come up with rather intricate complex ideas with many parts to them - all in one go. It's a bit like music (where composers can do the same) where a melody might spring into your mind, complete with all the notes and the rhythm. And if you don't write it down right away you may forget that tune.
So you can come up with a maths idea - that is a kind of unity. Not exactly a proof as such, but a sketch, an idea that you may be sure will work, you can see all the parts of the proof in your mind's eye -and just have to fill in the details to make it into an acceptable maths proof.
So yes - it is to do with not losing the thought. Also - that - you can have complex ideas with lots of things inter-related in your mind. But you can't have that along with all the details worked out, crossing all the ts and dotting all the is. Somehow you get an intuition "this is going to work" - not 100% but often reliable. But to go through all the details - most people unless they have a really exceptional memory for maths - they have to get out a sheet of paper and start scribbling.
It's like the joke about a maths professor who in a maths lecture says at one point in the middle of a long complicated proof - "and this step is obvious".
One of the students in the audience interrupts and says "I don't quite see it, how is it obvious?".
The professor clears an area of the board and starts scribbling away furiously for a few minutes, eventually covering nearly the whole whiteboard with equations.
Finally he stops scribbling, wipes out all his workings, turns to the audience again, and says "Yes, it is obvious", and continues with the lecture. :).
So - sorry to explain a joke :). But for non mathematicians:
The thing is - that a very complex idea - somehow the mind can grasp it as a unity. So it becomes a simple obvious thing. But - in maths often you have to do a fair bit of preparatory work before you get to see its simplicity and obviousness - that's what he was doing with his furious scribbling. And when finished, he could see that it was obvious, presumably had forgotten why it was obvious before.
Most mathematicians have had this experience, and that's what makes the joke funny :). If he'd been a better teacher he might then have gone on and explained what he was doing with all his scribblings, and explained why it is obvious to his class, until they also "got it" as he did.
So - then sometimes if you are an experienced mathematician, done lots of proving -then it happens the other way round. You get the insight that it is obvious before you have done the detailed scribbling.
If it happens that way around - you then have to do all that scribbling to confirm your intuition that it is obvious.
It's very like a melody. Including, sometimes you come up with a beautiful, elegant method of proof, like a rather lovely melody, and you don't want to forget it (even if there may be other ways to prove it, mathematicians also like to use the most beautiful, elegant proofs they have if they can).
And - though I'm not at that level myself as an amateur composer, who only writes short simple pieces - I would imagine that a professional composer would not just think of the musical idea, but also have in mind how it could be transformed and used elsewhere in the composition, a bit like seeing the complex implications of a simple proof idea all in a single flash of insight.
As someone who does both, note down maths ideas, and note down musical melodies - I can say that it's rather similar - in what it feels like, and why you do it.
Though, I don't know if music departments at universities have whiteboard tables for composers to write melodies or chords as they chat to each other - anyone know? :).
Oh - and ordinary every day mathematicians do this, don't have to be a genius :).
Another thought also. Is partly because of the visual nature of maths. Need to set out even equations in a visual way with the symbols all in place to check that it works, and often have to draw diagrams - which you can't do in words.
E.g. try explain this to someone else in words:
(simple geometrical proof of Pythagoras theorem).
At some point you'll probably want to take out a pen and paper and write something down as you talk.
Nowadays - if you have a tablet computer you carry around with you - you could get out your stylus and jot things down in situations where, a few years back, you'd get out pen and paper.
That's probably a matter of preference, habit and personality. I can say myself that I've taken out paper and drawn diagrams to explain something to someone even with my Surface Pro sitting on the desk right next to us!