You are correct that the actions you listed are not attacks, but the Invisibility spell says ...
The spell ends early immediately after the target makes an attack roll, deals damage, or casts a spell.
This whole sentence is a way to check your ability to avoid interactions with the world by being invisible while still interacting with it.
Some interactions bend this rule. Not many break it with a fair DM. The closest I have come was an Arcane Trickster character who can cast Mage Hand (which had Invisibility due to a class ability) then potion of Invisibility. I could use the Mage Hand for the duration of the spell ... but then I couldn't recast it without dropping Invisibility.
Self reply to share that I did this in a one shot and it was great! Level 3 character with expertise in Perception and Insight and 14 WIS, 16 DEX for decent AC and rogue skill checks, 18 INT for Investigation and True Strike. All 16 passives. Big fun.