The black riders are not really ordinary creatures like us. They thrive off uncertainty and fear and confusion and doubt and worry. They get weak and unsure of themselves when faced with genuine confidence. And they are basically cowardly. In the past they were ordinary humans - but with expertise in sorcery. And - they are enslaved by Sauron, and are in a state of almost alive almost dead, and both in this realm and in another realm.
And as well as being invisible (when not wearing clothes to give themselves shape) - they also can't see as we do - basically they are blind in daylight and rely on other senses, hence the sniffing. They have many followers and can rely on the vision of their followers, also of their horses, but by themselves, they can't see at all by daylight. So think of blind horse riders trying to find Frodo and his friends in daylight. And, at the start of the first book, they are in territory that the Hobbits know well. As darkness falls their senses are more acute - but luckily - that's when they encounter the elves in the woods - and later on have protection of Aragorn.
In book 3 they only get really powerful after Sauron manages to create thick clouds that block out the sun to the extent that even in the middle of the day it remains quite dark. And also as the people of Gondor get more and more terrified - and Denethor himself falls into black despair through the deceits of Sauron controling his glimpses into the seeing stone - those two together, actual darkness in the middle of the day and the despair and fear of the people - and combined with the fell flying creatures they ride on and the ranks of all their minions - those help to make them so powerful in book 3.
At the start of the books, the black riders are on horseback, in the Shire, a long way from Mordor, with no supporters, in an unfamiliar country, and one that has not yet been tainted at all by the reach of Sauron. So they are on their own, and don't have the powers they have in the third book. And they are scattered also as they have split up and gone different ways in their search to try to find Frodo as they don't yet know where he is until just before they flee from the house - and then - that's just one rider that knows so far, at that point. In bright sunshine, in the Shire, about as far removed in spirit as well as geographically from Mordor as you could imagine.
Even when the first black rider to find them does come close, and is right on the track of the ring, he is easily held off by a group of elves, just by the sound of their speech and laughter and singing. It's dark then - but a clear starry night, and the elves are high elves, blessed by Elbereth, who put the stars into the sky. In the shire, so far from Mordor, that's enough to scare the black rider away, too much for it to confront.
You could hardly imagine that making any difference to the black riders at the end of book 3.
Another way of looking at it - they are in no hurry. Predators such as for instance big cats seldom are. Slow and silent and hidden and wait - and then when the right moment comes, pounce. That's how they operate. They can hardly imagine that it is possible that the ring bearer will be able to escape from them. And - predators - they don't like to get hurt. Because a predator has to make many kills, and for instance a lion or a leopard with a broken leg, or damage to its body will probably die because they have to be at the top of their form to survive. So in the wild - "top predators" are often very cowardly, break off a hunt when you would think they could still get their target - but they want to be totally sure that they succeed, undamaged, so they can continue to survive as predators, and hunt again on another day and another day, week after week, month after month, as top predators.
So - that's why for instance after they wound Frodo at the top of Weatherhill - they then are in no hurry to try to do anything else, they think it is a sure thing and they don't want to get hurt themselves, even slightly. I know that they are meant to be pretty much invulnerable - unlike a leopard or lion they wouldn't die but they would have a hard time if damaged.
And though he doesn't know it, Frodo is carrying an object of great power which - if he knew how to use it - would be able to subdue them completely. So they know that and are afraid of the ring at the same time that they are trying to recover it.
Also - in between they lose the track of both Hobbits and the ring completely for several days - have no idea where they have got to - when the Hobbits visit Tom Bombadil. They are foiled partly by Fatty Bolger - and - is it possible that Tom Bombadil also helps to keep them away in some fashion, hides the hobbits from them?
So the Hobbits in book 1 get a respite from the dark riders to get stronger again just when they are at their weakest (in the sense of most despondent and fearful, after getting lost in the forest and the traps of old man Willow) and could easily have been taken by them if they had been able to follow them. Where the singing as naturally as they normally would speak, and merry laughter etc of Tom Bombadill's house is the perfect antidote to the effects of the black riders and old man willow on them just a short while before.
Elements of all of that I think.
That's the "in story" explanation.
Also - the first books were the earliest written. In the earliest version of book 1, then the black rider was instead just Gandalf with a cold. The sniffle of the black riders, so eerie in the final book, starts off as Gandalf's cold, bundled up so that the Hobbits don't recognize him (a bit like Merry later on in the final version).
So the first book was written when the black riders were gradually evolving in the author's ideas. When he wrote the later books, that was much later and by then they became much more terrifying figures. He then backwrote that into the early books as well, nevertheless that kept this progression to more and more terrifying creatures as the story developed, because the idea of the black riders itself also developed as he wrote the books - Tolkein basically started at page 1 of book 1 and wrote the story in a forward direction one chapter after another as if living the story himself. With lots of rewrites and going back and changing things, but - with that basic direction so when he wrote book 1 he didn't know himself how the story was going to end, he may have had some basic ideas. But when he started on the first draft of book 1 he didn't even know that the "one ring" was going to be a central theme of the novel to the extent it was. He wrote many pages of text along same lines as the Hobbit with it no more than an invisibility ring that let Frodo play tricks on farmer Maggot and his family (for instance). Later he rewrote that to make the ring much darker, and also rewrote the Hobbit also, the sections about the ring, to make them consistent with his new ideas for the developing Lord of the Rings story.
But - those two explanations actually go rather well with each other. The way he found the black riders more and more terrifying as he developed the story and wrote the later chapters and books for the first time - and the idea that as they got closer to Mordor they become more terrifying and powerful within the story - those two are kind of synergistic, probably helped him to write the book as he did. And, I think, help make the black riders so much more scary. You thought they were scary already in book 1 but as the story unfolds, it turns out, this is them at their weakest and most vulnerable.
And as well as being invisible (when not wearing clothes to give themselves shape) - they also can't see as we do - basically they are blind in daylight and rely on other senses, hence the sniffing. They have many followers and can rely on the vision of their followers, also of their horses, but by themselves, they can't see at all by daylight. So think of blind horse riders trying to find Frodo and his friends in daylight. And, at the start of the first book, they are in territory that the Hobbits know well. As darkness falls their senses are more acute - but luckily - that's when they encounter the elves in the woods - and later on have protection of Aragorn.
In book 3 they only get really powerful after Sauron manages to create thick clouds that block out the sun to the extent that even in the middle of the day it remains quite dark. And also as the people of Gondor get more and more terrified - and Denethor himself falls into black despair through the deceits of Sauron controling his glimpses into the seeing stone - those two together, actual darkness in the middle of the day and the despair and fear of the people - and combined with the fell flying creatures they ride on and the ranks of all their minions - those help to make them so powerful in book 3.
At the start of the books, the black riders are on horseback, in the Shire, a long way from Mordor, with no supporters, in an unfamiliar country, and one that has not yet been tainted at all by the reach of Sauron. So they are on their own, and don't have the powers they have in the third book. And they are scattered also as they have split up and gone different ways in their search to try to find Frodo as they don't yet know where he is until just before they flee from the house - and then - that's just one rider that knows so far, at that point. In bright sunshine, in the Shire, about as far removed in spirit as well as geographically from Mordor as you could imagine.
Even when the first black rider to find them does come close, and is right on the track of the ring, he is easily held off by a group of elves, just by the sound of their speech and laughter and singing. It's dark then - but a clear starry night, and the elves are high elves, blessed by Elbereth, who put the stars into the sky. In the shire, so far from Mordor, that's enough to scare the black rider away, too much for it to confront.
You could hardly imagine that making any difference to the black riders at the end of book 3.
Another way of looking at it - they are in no hurry. Predators such as for instance big cats seldom are. Slow and silent and hidden and wait - and then when the right moment comes, pounce. That's how they operate. They can hardly imagine that it is possible that the ring bearer will be able to escape from them. And - predators - they don't like to get hurt. Because a predator has to make many kills, and for instance a lion or a leopard with a broken leg, or damage to its body will probably die because they have to be at the top of their form to survive. So in the wild - "top predators" are often very cowardly, break off a hunt when you would think they could still get their target - but they want to be totally sure that they succeed, undamaged, so they can continue to survive as predators, and hunt again on another day and another day, week after week, month after month, as top predators.
So - that's why for instance after they wound Frodo at the top of Weatherhill - they then are in no hurry to try to do anything else, they think it is a sure thing and they don't want to get hurt themselves, even slightly. I know that they are meant to be pretty much invulnerable - unlike a leopard or lion they wouldn't die but they would have a hard time if damaged.
And though he doesn't know it, Frodo is carrying an object of great power which - if he knew how to use it - would be able to subdue them completely. So they know that and are afraid of the ring at the same time that they are trying to recover it.
Also - in between they lose the track of both Hobbits and the ring completely for several days - have no idea where they have got to - when the Hobbits visit Tom Bombadil. They are foiled partly by Fatty Bolger - and - is it possible that Tom Bombadil also helps to keep them away in some fashion, hides the hobbits from them?
(Aside about whether Tom Bombadil could have protected the Hobbits from the black riders more actively as they slept in his cottage:)
(That he could do that is hinted in the flight to the Ford when Frodo tries to tell the black riders to stop and go back to Mordor - but it says, "The Riders halted - but Frodo had not the power of Tom Bombadil". That would seem to suggest some previous episode known to the narrator when Tom stopped them - also Frodo's dream about the black riders and his reaction when he wakes up in Tom Bombadil's house: "There was a noise like a strong wind blowing, and on it was borne the sound of hoofs, galloping, galloping, galloping from the East. ‘Black Riders!’ thought Frodo as he wakened, with the sound of the hoofs still echoing in his mind. He wondered if he would ever again have the courage to leave the safety of these stone walls. He lay motionless, still listening; but all was now silent, and at last he turned and fell asleep again or wandered into some other unremembered dream." - hints at the possibility there with "all was now silent" that his dream might have been triggered by actual hoofs galloping outside of Tom Bombadil's cottage - then after they wake up, reinforcing this hint: ... "He had half expected to see turf right up to the walls, turf all pocked with hoof-prints".
But beyond that - in the book as it is there is no account of Tom stopping them. I'm not sure, but I have a vague memory that perhaps this was made more explicit in an earlier draft, the suggestion that Tom Bombadil somehow held the black riders off while they visited him. Is that true or was it never made any more explicit than this? If anyone knows more about this, do say in the comments).
So the Hobbits in book 1 get a respite from the dark riders to get stronger again just when they are at their weakest (in the sense of most despondent and fearful, after getting lost in the forest and the traps of old man Willow) and could easily have been taken by them if they had been able to follow them. Where the singing as naturally as they normally would speak, and merry laughter etc of Tom Bombadill's house is the perfect antidote to the effects of the black riders and old man willow on them just a short while before.
Elements of all of that I think.
That's the "in story" explanation.
Also - the first books were the earliest written. In the earliest version of book 1, then the black rider was instead just Gandalf with a cold. The sniffle of the black riders, so eerie in the final book, starts off as Gandalf's cold, bundled up so that the Hobbits don't recognize him (a bit like Merry later on in the final version).
So the first book was written when the black riders were gradually evolving in the author's ideas. When he wrote the later books, that was much later and by then they became much more terrifying figures. He then backwrote that into the early books as well, nevertheless that kept this progression to more and more terrifying creatures as the story developed, because the idea of the black riders itself also developed as he wrote the books - Tolkein basically started at page 1 of book 1 and wrote the story in a forward direction one chapter after another as if living the story himself. With lots of rewrites and going back and changing things, but - with that basic direction so when he wrote book 1 he didn't know himself how the story was going to end, he may have had some basic ideas. But when he started on the first draft of book 1 he didn't even know that the "one ring" was going to be a central theme of the novel to the extent it was. He wrote many pages of text along same lines as the Hobbit with it no more than an invisibility ring that let Frodo play tricks on farmer Maggot and his family (for instance). Later he rewrote that to make the ring much darker, and also rewrote the Hobbit also, the sections about the ring, to make them consistent with his new ideas for the developing Lord of the Rings story.
But - those two explanations actually go rather well with each other. The way he found the black riders more and more terrifying as he developed the story and wrote the later chapters and books for the first time - and the idea that as they got closer to Mordor they become more terrifying and powerful within the story - those two are kind of synergistic, probably helped him to write the book as he did. And, I think, help make the black riders so much more scary. You thought they were scary already in book 1 but as the story unfolds, it turns out, this is them at their weakest and most vulnerable.
About the Author
Robert Walker
Writer of articles on Mars and Space issues - Software Developer of Tune Smithy, Bounce Metronome etc.Studied at Wolfson College, Oxford
Lives in Isle of Mull
4.8m answer views110.3k this month
Top Writer2017, 2016, and 2015
Published WriterHuffPost, Slate, and 4 moreHuffPost, Slate, Forbes, Newsweek, Business Insider, and Science 2.0