Why Planet Nine might not exist

[or: what keeps me up at night]

As you will see in the next post, I think Planet Nine is really out there. But that doesn’t mean you should think it is out there. You might be skeptical. In fact, I would prefer that you were skeptical. I would prefer that you read the scientific paper, looking for potential flaws, caveats, and places where we might have been led astray. But, OK, I understand that the actual scientific paper is on the weighty side, so, rather than make you wade through it finding the potential piutfalls, instead, I will give you my top list of things that might be wrong.

First, though: what is not wrong.

If there is an approximately 10 Earth mass planet on an extremely elliptical orbit in the outer solar system, it would definitely line up the orbits of Kuiper belt objects with similarly elongated orbits, it would create Kuiper belt objects with orbits twisted by 90 degrees to the planets of the solar system, and it would make objects, like Sedna, which have elongated orbits which don’t ever come close to the rest of the Kuiper belt. These effects we now know from a general mathematical analysis and from detailed computer simulations to double-check the mathematical analysis. This analysis, I am confident to say, is iron-clad. Astronomers will try to reproduce it (I hope), and they will get the same results (I know). There truly is no wiggle room here. A 10 Earth mass planet does exactly all of the things that we are trying to explain.

If I am so confident of this, how could Planet Nine possibly not exist? Just because Planet Nine can explain all of these effects, it doesn’t not mean that there is no other possible explanation. We tried to think of everything that we could, and systematically ruled out alternatives, but that doesn’t mean that someone else won’t come up with an idea that works. Again, I hope that there are skeptical astronomers working right now to come up with alternatives. I am confident that they will not come up with them (because I do actually think we considered everything that could possibly be out there), but, unlike my statement above, I will definitely not say that this one is iron-clad. Aluminum-clad, maybe. Stainless steel, perhaps. I’d be willing to bet a lot of money against the idea that someone will find an alternative explanation for all of the effects that we are seeing. But it is possible I could lose.

There is one insidious way in which we may have been fooled into thinking that Planet Nine exists, however, and it is a problem that permeates all of experimental science. My single biggest worry is that perhaps – just perhaps -- we have been fooled into seeing a pattern where none exists. Humans excel at recognizing patterns, even when they are not there (see: everything single face-on-Mars claim ever, for example). Could we have been similarly fooled? Absolutely (again: I don’t think we have been, for reasons detailed in that next post, but is it possible? Of course). Here’s how:

In our analysis, we show that the six most distant objects that have orbits extending outward from the Kuiper belt all line up within a 100 degree quadrant and all have orbital planes which are tilted away from the plane of the planets by about 20 degrees (and within 6 degrees of each other). From some very simple calculations we can show that the probability of these alignments happening due to chance is only about 0.007%. You could also say that there is a 99.993% chance that the alignments we are seeing in the outer solar system are real, and that we are not simply being fooled into seeing a pattern where none exists.

But, really, if you said that, you’d be wrong. Real statistics don’t work that way. You can’t, for example, flip 100 coins, realize that 10 of them in the far upper right corner all turned up heads and then say “wow; the chances of 10 heads in the far upper right corner is only one in 1024 so something must be happening up there.” And if you flipped all of the coins again, chances are you wouldn’t get 10 heads in the far upper right corner (in fact, chances are 1 in 1024). The real statistical question that you should be asking at the first coin flip is more like “what is the probability that something that seems anomalous will appear just due to chance?” That question is essentially impossible to answer, because it relies on knowing what a human who is looking for anomalous patterns would call anomalous.

There are two good ways to combat these sorts of flawed statistics. The first I just mentioned above: replicate the experiment and look for the same result. Your eye and brain might pick out a random pattern from the noise one time, but the same pattern will not occur again. You might see different patterns, but that just shows you how easy it is to find patterns in data.

How do we replicate the finding that the most distant objects in the outer solar system are unusually aligned? We find more of them. If the alignment was just random pattern finding by easily fooled humans, it will quickly go away when the next half dozen objects are discovered. And while it took 12 years to discover the first 6 aligned objects, the next few should be much faster, as telescopes and surveys continue to get bigger and more powerful. One caveat: our computer simulations do not predict that 100% of the most distant objects will be clustered. Just the vast majority. So finding one or two outside of the cluster is not the end of Planet Nine. But finding the next six objects randomly distributed around the sky would be a pretty clear indication that we fell into the pattern-matching trap and that Planet Nine is a fantasy. The Planet Nine hypothesis makes strong predictions, and these can be used to show that the hypothesis is wrong, if it is.

The second way to combat the flawed statistics of pattern matching is to use your explanation for the pattern that you see to predict something entirely unrelated to the pattern. In our example of the coins above, you could hypothesize that the explanation for all of the coins coming up heads in one spot on the table is that there is a powerful magnetic field in that one location of the table (ok, I’m not sure how that could make things come up all heads, but work with me on this one). You could then make predictions. Perhaps you would predict that a set of ball bearings placed on the table would systematically roll towards that location. Or something. The key is that the prediction is something that you don’t know the answer to ahead of time, is not directly related to your original observation, and has a low probability of occurring on its own. Here you are not replicating the experiment but are instead performing a different experiment and predicting a very specific answer.

Our hypothesis passed this test with flying colors – in my opinion – with the prediction and subsequent realization of the existence of what we call the distant twisted orbits (maybe we need a better term for these; definitely we need a better term for these). But maybe it’s all just a bigger case of pattern matching? Such an explanation begins to get unlikely, but now we have a second set of objects that can be replicated, and we’ll all be watching the results come in.

There is one other more mundane in which we can be “wrong.” Given the small number of objects and our, currently, limited number of computer simulations (“limited” here still means ~6 months of super computer time, but we haven’t had time to precisely explore all of the possible parameters of Planet Nine), it is possible that our estimates are not precisely correct. Maybe Planet Nine js 20 times the mass of the Earth instead of 10. Maybe it is actually a giant terrestrial planet instead of a small gas giant. Maybe it is slightly further away or tilted into a slightly different plane. These tweaks are OK.  We would still say that the Planet Nine hypothesis is correct.

If, however, a planet is found beyond Neptune and it is totally different from what we have described, if it exists, but it fails to cause the basic gravitational interactions that we have discussed, then, quite simply, we are wrong. We are not predicting that there is some planet beyond Neptune, we are predicting that there is this planet beyond Neptune and it is causing these effects on the outer solar sytem.

Now, finally, is what you can tell your friends and family to impress them by your informed skepticism of the Planet Nine hypothesis:

“I worry that they have underestimated the likelihood of finding an intriguing pattern in the orbital data and that they have just been fooled into finding a pattern where there is none. I am waiting for the next few discoveries of distant object to see if they, too, have aligned and twisted orbits like the theory demands. And, for now, I am also running some computer simulations to check some ideas I have about other ways that the patterns can develop. Ask me again in six months.”