Distributed web/API scraping implementation, migrated from goscrape.
New features:
- Pub-sub output: ZeroMQ is the primary form of output for scrape results, decoupling all handlers from the scraping daemon.
- Priority-rated companies: Higher priority companies are scrapped more frequently, while lower priority are scrapped less frequently. This is implemented to reduce outbound request rate.
