iFixIt has some pretty comprehensive sets... I probably use 5-10% of the bits... But thats also kind of the point, right? You go for them when you run into a "what the fuck is this shit" kind of fastener, and then you probably won't use it again for another few years.
I would recommend getting some longer torx bits that can go in a drill / impact gun, since those are the more common "odd" fasteners you run into, and the short ones for screwdrivers dont give you a lot to work with if you chuck them in a drill