Fbook does it but I don't know about other sites.
Fbook does it but I don't know about other sites.
I'm pretty sure that's what robots.txt is for, but there's no way to enforce such a thing.
You could always place Phuks on an .onion (tor) but that has its own issues.
Hey thanks! I understand about half of it but get the gist. Is that how scrapers work? I have zero coding experience but wanted to try and create a scraper (I looked into trying to learn how to write a python or a java based scraper script) but it seems too complicated for a beginner. My interest is scraping high quality images of old hollywood film stars.
The idea of being obscured and hard-to-find intrigues me, but wouldn't something like that hamper site growth? Unless we want to always stay relatively small.