As to "what's falling faster" my point is still that everything's falling at the same speed, because the only non-arbitrary reference point to measure things from is the centre of gravity of the whole system, earth, feather, ball, all of them together.

Well it may still be arbitrary, but at least it's not geocentric or feathercentric or ballcentric. All three can be unhappy with the choice which means it's fair.

Flip that reference point to the earth though and yes the ball is approaching ever so slightly faster than the feather (side note: is our earth spherical or are we at least making it an oblong?). Flip it to the ball and the feather is falling a lot slower towards it than the earth is. Which is probably how I should have started explaining this: The mass difference between feather and earth with respect to the ball is so massive that it actually makes quite a difference while between feather and ball wrt. earth it's negligible.