No, this doesn’t work. Magnetic fields don’t have a significant effect on motions of planets. The problem is that magnetic fields follow an inverse cube law which means they drop off very very quickly. Magnetic fields are strong close up, and over a very short distance a small bar magnet can easily counteract the gravity of the entire Earth, as happens when you pick up a metal object using a magnet.

Notice here how the crane magnet easily overcomes the gravitational attraction of the entire Earth on a large heavy piece of metal. But it only does so when it is very very close to the piece of metal.

Notice that the lumps of metal don’t even begin to move until the magnet is very close to them. Then they wobble a bit and jump up to the magnet, showing that the magnetic field is very strong but only over small distances

Earth does have a strong magnetic field at its core, but though it is strong compared to most planets, by the time you get to the surface, it has hardly any noticeable effect. It does affect compass needles but only because they are so delicately balanced that a tiny field can make a difference.

We don’t get flybys of huge comets and asteroids anyway. The largest ones are of order 10 km or so or a bit larger. In the early solar system we got larger ones but they have all been swept out of Earth’s orbit long ago and we are now protected by Jupiter which does a pretty good job of sweeping out the larger objects from further afield though it doesn’t do such a good job of the smaller asteroids.

To affect us it would need to be of similar size to Earth, otherwise Earth would attract the object rather than vice versa. But even a huge magnetic field nearby would have very little effect on the Earth’s motion because of this inverse cube drop off.

Now it’s perhaps just as well. Because with an inverse cube force law, orbits are unstable.

This shows motion of a planet under an inverse cube force law, the so called Cotes's spiral If planets attracted each other under magnetism instead of gravity then there would be no stable orbits in the universe.

Magnetic fields do have an effect on charged particles. Atomic nuclei and electrons can be propelled at huge speeds by the solar magnetic fields. That’s how solar storms work. The Aurora also is the result of particles that are accelerated in our own magnetic field and then hit the atmosphere. And the Earth’s magnetic field helps protect the ISS and satellites from harmful charged particles from the Sun especially during the occasional solar storm.

But for that to happen the particle has to be small (because otherwise effects of gravity and just momentum would overwhelm this tiny effect) and to have a positive or negative electric charge. These are particles that are no heavier than atoms - typically atomic nuclei, or the very lightweight electrons.

Now the reason a magnet has only an inverse cube force law is because it has both a North and South magnetic field. The North and South poles generate inverse square law fields but when you find the difference, taking account of their separation, the result is an inverse cube law. For details see the answers here to: Why does the magnetic field obey an inverse cube law?

The theoretical magnetic monopoles, if they exist - an isolated North or South pole on its own - would have an inverse square force law. But nobody has ever found one of those yet. We don’t know if they are just very rare and we have to keep looking, or if for some reason they can’t form at all. And you’d need a lot of magnetic monopoles inside a planet to affect another planet filled with the opposite sense of monopoles to attract it, or the same sense of monopoles to repel it. Also they would have the awkward property that gravity doesn’t have that poles of the same polarity repel. So a pile of monopoles of the same polarity would push each other apart.

So planets following inverse square law orbits using magnetism based on magnetic monopoles seem unlikely too.