There are examples of 2 distinct species (with different chromosome count) creating (sometimes) fertile offspring: https://revistapesquisa.fapesp.br/en/when-hybrids-are-fertile-3/
But genetically the neanderthalers were far less different from us than those examples. Apparently all modern humans share 99.9% of DNA and neanderthalers shared 99.7% of that. https://www.livescience.com/archaeology/are-neanderthals-and-homo-sapiens-the-same-species
So the no viable offspring rule might not be that good for differentiating species, but that also doesn't mean that neanderthalers and us were not the same species. The more I read on it, the more I think that we were. Apparently we interbred quite a lot over the millennia.
I think that might be possible with mitochondrial dna (it always comes from the mother), but I only found 1 speculative source that draws a conclusion: "Nobody today has mitochondrial DNA like that in Neanderthals and, since it’s passed only maternally, this implies that interbreeding was more often between their men and our women." https://aeon.co/essays/what-do-we-know-about-the-lives-of-neanderthal-women
It's an essay, not a research paper, I wouldn't bet any money on this conclusion being correct.