this post was submitted on 21 Aug 2024
320 points (99.4% liked)
196
16279 readers
1898 users here now
Be sure to follow the rule before you head out.
Rule: You must post before you leave.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
really? ddg will show me reddit links, did they have to make a webscraper or something
There's a cutoff date, anything indexed before the robots.txt was changed stays in the index