Scraper Maintenance workflowΒΆ
- Look at the job outputs and/or job logs
- If there is an error, pause or cancel the affected job on the DataHen scraper
- Go to the scraper directory locally
- Modify the seeder script or parser scripts locally
- Run the seeder script or parser script locally against some fetched job pages
- Deploy the scraper
- Resume the scrape job, or run another scrape by creating another scrape job.
- Check the job outputs on DataHen