A water world needs an atmosphere, or some other way to keep the water from evaporating. If the water is exposed to the vacuum of space it will soon evaporate, very quickly, at a rate of over 20 km of depth of water every day. But that’s not really an issue if it has an atmosphere as well - which it could retain if it has a rocky core which can generate a magnetic field to protect it.

Another solution is to have a surface covered with an organic scum which would work even in a vacuum. If you take a comet, which is nearly completely ice and dry ice and similar volatiles like ammonia, somehow get it into a near circular orbit without all the ice evaporating, and let it have an organic scum covering it making it nearly impervious to evaporation, it could be liquid water all the way through.

So that shows you could have a very tiny pure water world. The main thing is, how do you get it close to the sun since it has to form much further out (or it lost all its water long ago) and that indeed is a problem for Earth too.

Another way to get a pure water world is to take any icy moon, such as Europa, Enceladus etc and warm them up. That might happen naturally indeed as our sun gets hotter. Eventually perhaps it will get so hot that Europa becomes a water world. It’s ocean would then be easily deep enough to cover any features in its underlying rocky core. The main question then would be, how long would it last as a stable water world? Ceres, which seems to have lots of ice in its crust, might become a water world first of all. Perhaps in a future with a hotter sun, our own solar system might have many dwarf planet water worlds.

Europa has a very deep ice crust with probably a 100 km deep ocean beneath. As our Sun warms up it may become a dwarf moon planet water world.

In theory you can have quite large planets that consist only of water all the way through.

I worked out with a rough calculation that we can have an ice free planet of pure water with temperatures of -0.16 °C and radius of around 2,127 km or at a temperature of 81.85 °C and radius of about 3,982 km.

That’s for fresh water. A salty ocean would stay liquid at lower temperatures and higher pressures.

Compare the diameter of our Moon of 3,474 km, so it seems you could have a planet that’s a bit larger than our Moon, entirely of water, and still be habitable for at least some microbes. Indeed Hyperthermophiles have optimal temperatures above 80 °C (176 °F).

If it gets larger then it can still consist of water all the way through except that the center of the planet is Ice VI (ice-six).

If it is close enough to its star, and covered in a thick layer of organic scum, then it wouldn’t need a magnetic field and it could stay liquid just through the external heat of its star.

See my Do water planets exist?