Scraper Development workflowΒΆ
- Create a scraper git repository
- Create the scraper on DataHen
- Write a seeder and parser scripts
- Run a seeder script locally against DataHen to see if it works
- Run the parser scripts locally against the global pages
- Deploy the scraper
- Run the scraper on DataHen
- Check the job outputs on DataHen