When I stepped away from my own (mildly successful) Free software project, I had the same concerns: it's about the reputation.

The project had earned a decent amount of trust when I was running it, and presumably people were installing new updates without going over the changes. If I handed off the project to someone new, I wasn't just handing over the work, but that trust as well.

So rather than handing over the project to someone new, I archived it and someone else (thankfully someone not-evil) forked it. Anyone installing the fork immediately understood that the relationship was new. They'd have to decide whether to trust this new maintainer or not.

For my money, this is the way. If you're burning out, remember that your reputation is tied to your project name, and that it has considerable value. If you don't want to continue, the disruption of a fork is better/safer than the smooth-but-risky hand-off.