Thanks for the input!
So the thing I was trying to print was this: https://www.thingiverse.com/thing:6365845
Ideally you'd just print it in PLA but I only had one colour PLA so I thought, might as well gather some PETG printing experience :D
The part in question is red_panda_low_poly_V1_-_head_A_red.STL and I tried to print it on an odd angle to make sure that most of the outwards facing walls were on the top, because anything that needed support turned out so hideous.
Also it seems to appear on any side of the model, where you have a slight overhanging angle, in no particular axis :)
I'll print it in pla once my current print is done :D
It's kinda not at the start or end of the layer. The layer starts with the inner wall, which prints fine and then it does the outer wall which doesn't do fine.
Which corresponds to the wall here
Bambu Studio 1.8.4.51