If it is far enough from the sun, way way beyond Neptune, then it could be heavier than Jupiter but almost certainly not as heavy as a brown dwarf (not heavy enough to have had a phase of deuterium burning when it was born).

If it is closer to the Sun then the limits are much more stringent. It’s much easier to see something if it is closer. If you reduce the distance of an object by a half, it is four times easier to see because the light from the Sun is four times brighter. But it is also four times larger, as seen from the Sun, or from Earth which is also very close to the Sun. The combination of both those effects is that a very distant object which is illuminated by the Sun gets sixteen times brighter if you halve its distance from the Sun. If it is ten times further away, then it is ten thousand times fainter.

So - that’s why it’s impossible that there are extra planets inside of Neptune’s orbit that we have never spotted to date. We’d see them easily. But go to, say ten times the distance to Neptune, and we could miss rather large objects already. If it is over three thousand times the distance to Neptune, we could miss even an object as large as Jupiter out there, easily.

LIMITS FROM THE WISE SURVEY

The original paper for the WISE results implications for planets is here:

A SEARCH FOR A DISTANT COMPANION TO THE SUN WITH THE WIDE-FIELD INFRARED SURVEY EXPLORER

The survey is for gas giants, and stars rather than terrestrial planets shining only by reflected light. It was an automated computer search with any tricky borderline cases investigated by hand.

The scans overlapped at the ecliptic poles. Each spot of the sky was photographed twelve times in the ecliptic, but hundreds of times at the poles. So there is no way it can miss planets that are out of the ecliptic - it is more sensitive to those than planets in the ecliptic.

Artist's impression of the Wide Field Infrared Survey Explorer which has produced the tightest constraints to date on Planet X.

It gives strong constraints on a Jupiter or Saturn sized object. A Jupiter sized object must be at least 82,000 au from the sun, and a Saturn sized object at least 28, 000 au. But a brown dwarf can actually be similar in size to Jupiter and also very cold and as well as that, it can be much darker than Jupiter in appearance (though not invisible in reflected light, - our Moon is as dark as worn asphalt and of course, easy to see).

Anyway, apparently it would be possible for a small, dark and very cold five billion year old brown dwarf to be in our solar system at a distance of 26,000 au, even closer than the limit for Saturn. That's 650 times the distance to Pluto, or 0.41 light years away - so far away it would take light over 21 weeks to get from the brown dwarf to Earth.

The survey could spot the more usual 150 K brown dwarf (that’s a very cold brown dwarf, -123 C) out to ten light years away.

That's why the idea of an unseen star is getting increasingly unlikely. It couldn't have missed a red dwarf star, even the smallest. It couldn’t have missed any kind of a star at all. It just possibly could have missed a very dark very very cold brown dwarf in a rather distant orbit. But brown dwarfs are less common than they used to be thought to be. As a result of the WISE survey again, it's now thought that there are six normal stars for every brown dwarf.

Also it's not too surprising that our solar system turns out to have no companion star, not even a red dwarf - the majority (54%) of sun type stars are single. And a binary system is less likely to have stable planets, unless the companion is very distant, or very close to the sun. And what planets it has are more likely to be in very eccentric far from circular orbits, again unless the companion is very distant. So, given that we live on a planet with a stable near circular orbit - there's a selection bias in favour of non binary solar systems on top of that 54% result.

Constraints on dark matter in our solar system

By dark matter here the authors mean any form of matter not yet known including asteroids as well as the hypothetical "dark matter". So it's also relevant to things like "Nibiru".

See Constraints on Dark Matter in the Solar System

Any matter inside of Saturn would be easy to spot from its gravitational effects on Mars, Saturn and the Earth. The reason they chose those three planets is because we have had Cassini in orbit around Saturn for years, and many spacecraft sent to Mars, and of course with Earth also - it means we have very precise measurements of their position in their orbits going back many years now.

Then, taking account of the effects of all the known matter in the solar system, they concluded that there is at most 1.7 ×10^−10 M⊙ missing from the matter we know about inside of Saturn. That's unfamiliar units, for most of us, expressed in terms of the mass of the sun, which is 1.989 × 10^30 kg So this means we are missing at most 1.17 × 10^20 kg. By comparison, the mass of Ceres is 8.958 × 10^20 kg. So we are missing less than a seventh of the mass of Ceres.

To put it another way, if you had an asteroid with same density as Ceres, with this amount of mass, then its diameter would be 950*cube root(1.17/8.958) km or about 480 km in diameter.

So if all that matter was concentrated into a single object inside of Saturn's orbit, it can't be larger than about 480 km in diameter. Or if it was made of ice, it's diameter can't be larger than 950*cube root(2.161*1.17/8.958) km, or about 623 km in diameter

So there is no way that there are even any unknown large asteroids as large as Ceres inside of Saturn right now. If there were any, then we'd have spotted their effects on the orbits of Saturn, Mars and Earth through the sensitive position measurements we can do nowadays, no matter where they are.

When you get to the region inside of Jupiter, then as a result of the PAN-STARRS all sky survey for asteroids, we have a complete listing of all the asteroids of ten kilometers and larger. The only ones that could be not yet discovered are of order of one kilometer or so downwards. And we are discovering the one kilometer asteroids at one per month and have already found more than 90% of them.

There is another place where some largish objects could be hidden, and that’s close to the Sun, well inside Mercury’s orbit. They could be up to tens of kilometers in diameter, but there have been searches and nothing was found, not yet. These would be objects that permanently orbit the Sun very close to it, and are of no risk to us, not for thousands of years, probably millions of years anyway.

Note, none of the asteroids found by PAN-STARRS etc are headed our way either - though there are thousands of them, space is vast, and for them to hit us is harder than to hit a dust mote by throwing something even smaller in its direction in a large room. Whenever they find a new asteroid, they work out its orbit, and publish the latest calculations - it is all done in an open way and there is no possibility of hiding anything. They usually find out pretty quickly that it is not going to hit us in the next couple of centuries, up to 2200 which is usually as far as they go in the published information.