Greentext

I think you have some terms mixed up. Orbit would refer to a body moving around another body is some type of elliptical revolution. High atmosphere balloons do require propulsion to get to altitude. The atmosphere coupled with the ground makes no sense. (think of a constant spinning marble in a gold fish bowl, the surrounding water doesn't stay perfectly in line with the marble)

The cartoon above does answer the question to some degree though. Essentially in order for the balloon to be stationary while the earth rotates below you would need a propulsion system to maintain its height (fight gravity) as well as combat high winds (that would prevent it from staying "stationary" relative to the earth below). I would imagine the benefit of getting outside of the earth atmosphere completely (which is what I believe you meant by orbit) is there would be no wind to fight against. The problem then is that the balloon would've popped by then.

I think that's right but someone could correct me.

EDIT: Realizing we're talking about high altitude balloons which wouldn't need a propulsion system to achieve altitude, but would not be able to maintain altitude before popping. I think the rest of my statement stands true.