4

Fbook does it but I don't know about other sites.

Fbook does it but I don't know about other sites.

7 comments

The idea of being obscured and hard-to-find intrigues me, but wouldn't something like that hamper site growth? Unless we want to always stay relatively small.

[–] E-werd 4 points (+4|-0)

I'm pretty sure that's what robots.txt is for, but there's no way to enforce such a thing.

You could always place Phuks on an .onion (tor) but that has its own issues.

Could you explain the robots.txt thing?

[–] E-werd 0 points (+0|-0)

Hey thanks! I understand about half of it but get the gist. Is that how scrapers work? I have zero coding experience but wanted to try and create a scraper (I looked into trying to learn how to write a python or a java based scraper script) but it seems too complicated for a beginner. My interest is scraping high quality images of old hollywood film stars.