Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The point about govdeals missing is real, but in practice most of these sites block scraping aggressively for good reason. The operational cost of maintaining scrapers against hostile targets is non-trivial. Curious what the OP's plan is when they start rate-limiting or serving honeypot data.


I'm not scraping sites like govdeals, so no need to worry about that thankfully.


That's the right call then. How are you pulling govdeals data without scraping?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: