I want to scrape all the pages and subpages of a web site (by scrape, I mean save all the online content to local HTML files).
What is the best utility to crawl all the pages? Ideally, I would like to specify how many layers deep to scrape.
You have two options:
You can use the
Or, you can use SiteSucker. A GUI application that recursively downloads websites. You can also specify how far down to recurse with SiteSucker.