Please note: this feedback will mostly be focused from my own perspective as a forever GM. I am less concerned with how powerful X is against Y, as I am for the health of the game, and the ease of running encounters.
During my playtests I was given the opportunity to see both a Rogue and a Ranger, and they both feel good and bad in opposite and complementary ways. In short, the Rogue is fun but a breakdown of damage spreadsheets reveal it’s in a bad spot. In contrast, the Ranger has a lot of moments that detract from its fun, but joyless math shows that it’s super-effective.
Rogues are affable & attractive idiots
For the rogue’s part, being able to tactically choose what cunning strike to use at each given moment allowed for ranged rogues to opt into a controller role in combat, making one enemy’s life hell for a turn. When this works, it feels amazing as you are able to severely hamstring or simply exclude opponents from contributing at all on their turns. Unexpectedly, the Trip Attack with the Light Crossbow Weapon mastery allowed the rogue to make enemy melee bruisers get nailed to the ground, with a net -20 to their next movement due to being slowed, and having to use half of their reduced movement to stand up. This forced enemies to default to the party frontliner when able, or to use their action to dash instead of doing anything useful in the turn.
However, their damage is quite low. The player didn’t really notice this at the time, but a look at their output showed that they were not really big contributors to damage, and there were circumstances where their Cunning Strikes really didn’t have any impact on enemies.
The Ranger is a high performance hot mess
As for Rangers, they have so many problems that just amount to their spells all requiring concentration. There are so many toys that a Ranger simply can’t access because it would mess with their attempts at optimal damage output, and the mechanisms of bonus action to cast, concentrate, and then hit to apply the riders feels quite outdated, especially compared to the treatment that the Smite Spells received in this playtest.
However they were the number 1 damage dealer at the table. Honestly I think it’s too much damage for a ranged class, particularly because my Hunter-Ranger optimized around getting 60 movement speed, and getting the ‘Hunter’s Leap’ level 7 subclass feature. I simply couldn’t get them in melee because of how elusive they are, using their reaction the moment an enemy closed to engage them, prior to me being able to put attack actions onto them.
But this puts the dev team in a strange spot. If they fix how mutually exclusive all their damage increasing spells are, their damage output will skyrocket to preposterous levels. The solution is one that will be unpopular: Lower ranger damage quite a bit.
Additionally, a raise to rogue's damage should be performed, particularly in Tier 2-4. Maybe increase the number of die they get starting at level 5, 11, etc, or increase the size of their sneak attack dies at those places.
Finally, people who think that the new Hunter’s Mark is a nerf really need to actually give it a shot, because this feels much better.
As per usual, I have crossposted this to /r/onednd, feel free to upvote it for higher visibility. Cheers!
Rogues are resourceless utility/skirmishers. They don't need to be amazing at damage. I could see maybe an extra d6 at levels 5 and 11 to help them catch up and lessen the impact of Cunning Strikes on their damage slightly, but them not being a strictly great damage dealer is a good thing.
If the Ranger was optimized for being fast, and the Rogue was constantly tripping the enemies, why are you surprised it was hard to hit the Ranger? Also, Hunter's Mark makes them focus on single target damage, so of course they should be great at that. Hunter's Mark feeling better is a very iffy statement. The damage is more reliable when upcast to 3rd-level, but it's debatable if it's even worth upcasting over using a different spell unless you're fighting a BBEG and your whole plan is "Hit it really hard".
I'm honestly not as sold on the idea that resources are a balancing consideration, all told. Or at the very least they aren't as big of a consideration as many people presume. But perhaps you're right and rogues are fine, at the very least I don't think rogue players will generally notice that they are doing less damage due to how clutch they can be with their cunning strikes.
As to Hunter's Mark damage, it's higher damage at level 5, when you miss once. Which means that the damage floor is higher in general starting at level 2 spellslots (character level 5), and the damage ceiling is lower only if you are able to make 3 attacks. At level 3 spellslots (character level 9) and onward, it's simply the same or higher damage, excluding gimmicky action surge builds.
As to when it is useful to upcast, it's useful in any fight that you think is serious. The biggest problem is that the rest of the spellcasting toolkit is sitting there, doing very little for the most part (standout exception is Conjure Barrage which is fantastically powerful and fun).
edit: Oh and to answer this...
These were two different playtests, with different classes. The ranger and the rogue weren't in the same playtest group, so this wasn't the result of complementary kits. It's mostly a consequence of the the enemies having only average movement speeds, and the counters being somewhat melee heavy.
But even in the encounter where I had 60 flight speed dragon wyrmlings, the situation constantly played out that this ranger was never grouped up with the rest of the party, so I couldn't hit them in the breath attacks. This amount of safety is a bit problematic with their higher damage output, IMO.
Hunter's Mark damage scales every other spell level, so it goes to 2d6 at level 9 if you use your highest spell slot at the time. No point in casting at 2nd level at all.
Ah, my mistake. The way you compared them made it seem more like they were in the same playtest.
I promise you, "Don't group up when you fight things with breath weapons" is not a strategy unique to Rangers. That's just something kind of busted with ranged builds in general. I've been in a ton of games where the party saw a monster use a breath weapon and all immediately agreed "Split up, and if we have to be in melee at least flank so it needs to take an opportunity attack to get us both." I also just recently ran an encounter with a dragon where the Bard got out and took no damage at all, but that's on me for letting them hide around the corner and not targeting them earlier.
Ah whoops I goofed on the scaling of hunter's mark! In my defense I was juggling a lot of new mechanics and the ranger was a last minute addition to one of my playtests.
But looking at the general numbers, one fewer d6 per turn would have been a drop in the bucket to the damage they were outputting. It was sincerely surprising on how impressive they were, I expected them to perform worse based on my first impressions of reading the class. Actually playtesting revealed that they are in no way nerfed.
That's kind of my point, that Ranged builds having guaranteed high damage is an issue because there are so many perks, like ease of safe positioning. I think Rangers are doing great damage, and that might be a problem.