WebScraper

WebScraper uses the Integrity v8 engine to quickly scan a website, and can output extracted data as CSV or JSON. Plus download images to a folder.

• Easy to scan a site - just enter the starting URL and press "Go"
• Easy to export - choose the columns you want
• Plenty of extraction options, including HTML elements with certain classes or IDs, regular expressions, or entire content in a number of formats (html, plain text, markdown)
• 'helper' utilities within the app make it easy to find a suitable class / id or produce a regular expression (regex) to extract the data you want
• Since v4.1 can download to a folder all images discovered
• Configuration of various limits on the crawl and the output file size

• Can use the ProxyCrawl service to use different proxy servers and user-agent string etc for each request. Simply set up an account with ProxyCrawl (free up to 1000 successful requests per month), enter your token in Preferences, switch on "Use ProxyCrawl" in your site's advanced settings.
• File menu now has a 'Save Project' option as well as a 'Save Project As...' option which work as you'd expect.
• Fixes issue causing black / whitelist rules from a previously open project to appear in a project after a certain sequence of events.
• Fixes main tab view switching to empty results tab after a saved project is opened.