I thought similarly until I tried it out. There's community control schemes you can copy for games, plus there's 4 extra buttons in the back you can quick remap them to either keyboard/mouse buttons or controller inputs, along with two click touchpads that can do that and more. Plus you can quick call a touch/navigatable keyboard to the screen. It takes a bit to get used to but once you do, it's hard going back. I actually ended up preferring using it handheld over using a dock to hook it to a TV with a controller paired to it.
The only games I'd say are hard to use would be like some mmorpg games that are best for keyboard & mouse.
I like the way you think. Really great way to take advantage of how illusions misdirect the senses so it doesn't matter that it isn't "real", it's real enough to manipulate your targets perception, even for just a vital second in combat which is the difference between victory and defeat. Best part is that the way you are talking using it isn't even really broken, it's as effective as what you can literally imagine, and no more.