So the new "so" does seem different from the old uses of that handy, shape-shifting conjunction, "So, ..." when used at the beginning of a sentence. So these old usages are nicely analysed and illustrated in the online Cambridge English Dictionary:
Used at the beginning of a sentence to connect it with something that has been said or has happened previously:So ... No! Enough of that!
So, there I was standing at the edge of the road with only my underwear on ...
So, just to finish what I was saying earlier...
Used as a way of making certain that you or someone else understand something correctly, often when you are repeating the important points of a plan:
So we leave on the Thursday and get back the next Tuesday, is that right?
Used to refer to a discovery that you have just made:
So that's what he does when I'm not around!
Used as a short pause, sometimes to emphasize what you are saying:
So, here we are again - just you and me.
Used before you introduce a subject of conversation that is of present interest, especially when you are asking a question:
So, who do you think is going to win the election?
informal Used to show that you agree with something that someone has just said, but you do not think that it is important:
So the car's expensive - well, I can afford it.
This new "so" is interesting to think about in comparison to those examples. For a start, it is clearly not followed by an implicit comma. It is most frequently heard used by academics and experts under mild interrogation on the radio, and it does seem mainly to be a modish throat-clearing let's-kick-off noise that sounds more contemporary and fluent than "Well...", "Um...", or "OK...". But the fact is that "so" has never previously been a conventional response to a question not beginning with "how" or "why" (even if it now emphatically is), and this may mean that something else is going on here.
Now, "so" may not have previously been used to introduce an answer, but it did frequently start off a question. See usage (2) above ("a way of making certain that you or someone else understand something correctly"). Tentatively, might it be that a new question-and-response formula has evolved? That is, from
"So how old are you, professor?"to
" I'm 61"
"So how old are you, professor?"Perhaps the aggressive, nicety-free style of questioning developed by John Humphrys and his ilk has stimulated the "so" response? So it's a theory, but an unlikely one, I think. But I have a feeling that it is not unconnected. You might say that the new "so" implies the use of "so" at the beginning of the question, even when it's absent ("How old are you, professor?" "So I'm 61"). The second exchange above does have a certain symmetry to it, and the second leading "so" seems to grab back some initiative from the questioner, in a semi-sarcastic, passive-aggressive, mirroring kind of way. "So please let's remember that I'm the expert around here, mate", it seems to say.
"So I'm 61"
In fact, "passive-aggressive" behaviour may be a key here, understood as a way of expressing hostility indirectly (for example by repeatedly failing to do the washing-up, or in that curious military offence of "dumb insolence"). Although the new "so" has now undoubtedly escaped into the wild, and really has become just a modish kind of phatic teeing-up noise, I suspect that lurking sulkily behind its origins is a passive-aggressive text something along these lines:
"I know you're probably going to misunderstand and quite possibly belittle what I'm about to say, because you don't have the necessary context or background reading, and I'm feeling rather defensive because I'm a fish out of water in this studio, but I think you should accept that I know what I'm talking about and you most definitely don't, and SO a radically simplified but nonetheless authoritative version of the latest thinking on the subject would go something like this..."Or it might even shade into something a little more arrogant, rather more overtly aggressive, what we might call the new "right?/yeah?" (as in "The latest thinking on string theory -- right? -- is that all string is very long and very thin, yeah?"). Something like this:
"Pretend to pay attention now, you ignorant, preening fool, as I'm about to share some industrial-strength wisdom on a subject you can barely begin to comprehend, and -- even though I know you're inevitably going to latch onto some irrelevant little detail and worry it to death, in the process wasting this entire five minute slot in which I could be educating people on the topic rather than acting as the foil for your ponderous 'wit', Humphrys, you egregious **** -- it goes like SO..."So -- right? -- so "so" is so much shorter, yeah?
So it's a car-park, innit?