If it's just the fallback that's failing for the occasional video, under settings > general settings > Fallback to Non-Preferred Backend on Failure: - Check that the invidious instance there actually works.
If it's happening regularly I would say there's something else because that's not normal, it used to be less reliable than it is now but it's been very good for the last year or so. Only very occasionally I have to fall back to using yt-dlp these days (unless of course I'm keeping the video).