I'd still call it a legacy game. While legacy games usually involve physically, permanently, altering the pieces, it's not required; just that previous sessions carry some impact over to the next. At least that's what I've seen.
I would say simply adding and removing cards from the game counts as a legacy mechanic, since at any time my copy of Oath is going to play different than your copy based on our play history.
Maybe "legacy-lite"?
Edit: seems the term might be "Fable"? https://boardgamegeek.com/boardgamefamily/44081/series-fable-game-system