I recently found that the OsmAnd app lets you adjust the safety of your bike routes, so you can prefer safety over distance.
When navigating, click the "Ride Style" button and choose "prefer unpaved roads." That name made me think it would find gravel/off-road trails, but it actually selects safer roads. In my experience this setting chooses the optimal routes--it's finding the same general path that I would pick based on local knowledge, and it found improvements where I could take a slightly different street for a few blocks to avoid cars!
Also, OsmAnd~ is available via Fdroid with all the paywalls removed.
I know I'm late to the discussion, but I was experimenting with this recently so wanted to add my experience in the UK.
I'm on a gravel bike and I'm a beginner cyclist although I've done lots of walking/hiking in the past so I'm used to being out and about. It's currently autumn so relatively wet.
With the default option (balanced), I get sent down a mix of minor country roads (good) and some paths that are in theory bridleways but in practice they are extremely rutted footpaths 1m wide with 35cm of brambles on either side. When I say "rutted" I mean 0.5m deep (and steep) holes that are full of mud/water. Absolutely impossible to ride down on a gravel bike as the holes are the full width of the path, and I'd be amazed if anyone managed it even on a mountain bike.
With "prefer byways" I get minor roads, which in my area means paved (asphalt) minor roads and country lanes, plus firm(ish) cycle routes. The routing engine doesn't refuse to use a non-paved roads, but it avoids it where possible.
Basically, "prefer byways" is a good option generally for me. "Balanced" would probably be OK in summer except where the paths are overgrown (although a machete would be required as mentioned by someone else!).