Is there a way I can run a script with a certain web page as the argument that will download all the images from there?
-
2Have you looked at Automator? There is an Get Image URL from webpage plug in already written. Be careful with overly broad questions (as well as flooding the site with too many questions and no answers - the site works best when you give back as well as just ask - even if the asks are all good).– bmike ♦Jan 16, 2013 at 5:36
5 Answers
You can use an Automator workflow to download images embedded in a web page, or images linked from a web page. A good starting point for a workflow is:
- Get Current Webpage from Safari
- Get Image URLs from Webpage
- Download URLs
You can change the workflow to use a list of web pages to fetch from.
Automator is included with Mac OS X in the Applications > Utilities
folder.
-
Copy all image urls?
Lynx -dump
could work in downloading but a lot of parsing -- no easier method? -1 unless the second point clarified -- it can be a lot of work...– hhhFeb 6, 2013 at 23:59 -
@hhh the second section, get image URLs, is performed by the Automator action displayed in the screen shot. There is no significant work for the user. Feb 7, 2013 at 9:02
-
+1 hey this is cool, why cannot I change my downvote to upvote? I did not know this is so easy! Thanks for sharing :)– hhhFeb 7, 2013 at 18:54
wget -nd -r -l1 -p -np -A jpg,jpeg,png,svg,gif -e robots=off http://www.apple.com/itunes/
- -nd (no directories) downloads all files to the current directory
- -r -l1 (recursive level 1) downloads linked pages and resources on the first page
- -p (page requisites) also includes resources on linked pages
- -np (no parent) doesn't follow links to parent directories
- -A (accept) only downloads or keeps files with the specified extensions
- -e robots=off ignores robots.txt and doesn't download a robots.txt to the current directory
If the images are on a different host or subdomain, you have to add -H to span hosts:
wget -nd -H -p -A jpg,jpeg,png,gif -e robots=off http://example.tumblr.com/page/{1..2}
You can also use curl:
cd ~/Desktop/; IFS=$'\n'; for u in $(curl -Ls http://example.tumblr.com/page/{1..2} | sed -En 's/.*src="([^"]+\.(jpe?g|png))".*/\1/p' | sort -u); do curl -s "$u" -O; done
-L follows location headers (redirects). -O outputs files to the current directory with the same names.
-
where is it downloading to? only the main page shows up in the working directory ... are the linked files being saved somewhere else?– inaFeb 25, 2020 at 6:54
Here is a hacky solution (but it works). Hope someone can find a better one.
In Terminal, use
wget --page-requisites http://example.com/
. This will download the webpage at example.com and all of the resources linked from it (such as images, stylesheets, and scripts). More info on --page-requisites. Note: You can add many URLs separated by spaces to download lots of them at once. If many are from the same server, you should use something likewget --wait=2
to avoid slurping down files too fast.Open the folder you downloaded those files to and use Spotlight to separate the images from the other files. I'm going to assume you have Mountain Lion. Type "Image" into the search field and select Kinds > Image.
-
It only downloaded the HTML code of it, not any of the images. It appears to be the same as if I had done "Save As..." in my web browser.– JShoeJan 17, 2013 at 2:13
-
Either the site is using JavaScript to load in the content, or they block
wget
by user agent. In the second case, you can try usingwget -U "enter your web browser's user-agent here"
to pretend you are an actual web browser. Jan 17, 2013 at 2:21 -
@JShoe I just tested the -U flag with Safari's user agent, and it works. Jan 17, 2013 at 2:40
-
I was using Chrome, and trying to download from imgur. Also, what is a user-agent?– JShoeJan 17, 2013 at 2:46
-
1@hhh I compiled it from source, but you can also use Homebrew or MacPorts. (I think it used to come with OS X?) Feb 10, 2013 at 6:07
If you know the pattern in the url, you could use the *ix solution with Curl: Use curl to download images from website using wildcard?
Check out the Automator Space on MyAppleSpace http://www.myapplespace.com/pages/view/14664/automator-script-library
-
1
-
1...and link-only answers are discouraged and the accepted answer already shows how to do it in Automator.– beroeAug 30, 2014 at 0:25