Is there a way to use curl to download non-sequential images with a wildcard? I want to download all the panoramas I upload to my account at 360 Panorama to a local folder so I don't have to do this manually each time.

The images files follow the pattern of: http://occipital.com/images/viewer/XYZ_flat.jpg with the XYZ being random. It also seems from that URL that I need to be logged in or seen as logged in to the site.

  • I'm not too familiar with curl, as I only just discovered it when looking for a way to accomplish this task and curl (or wget) seemed the solution.
    – George C
    Commented Aug 11, 2012 at 3:34
  • See CURL to download a directory on Super User.
    – Arjan
    Commented Aug 11, 2012 at 7:42
  • Can you add some specific (real) URLs to your question? Maybe we can find a pattern there which would be possible to guess when using curl.
    – nohillside
    Commented Aug 11, 2012 at 8:18
  • @patrix here's an actual URL: occipital.com/images/viewer/qSJGuD_flat.jpg — seems they don't offer a direct path that's tied to the username. Here's the link to my user account page when I am not logged in: occipital.com/user/01ca-255/george-coghill
    – George C
    Commented Aug 12, 2012 at 13:23
  • @Arjan I am trying to just download the newer images as they are posted, not the entire directory, and not just one time. Trying to avoid having to go to my account and manually download each image every time I upload new panoramas.
    – George C
    Commented Aug 12, 2012 at 13:28

4 Answers 4


You can use this bash code for the actual URL you provided in your comment.

for i in $(for j in {1..8}; do
    curl http://occipital.com/user/01ca-255/george-coghill/$j/;
  done \
  | sed -n 's,.*/viewer/\(......_flat\)_small\.jpg.*,http://occipital.com/images/viewer/\1.jpg,p'
); do
  curl -L -o "${i##*/}" "$i";

Feel free to write this command as one line; all line breaks were only added to increase legibility. You may copy or remove them, whatever you like.

What this code does:

  1. Iterate over the 8 pages of your account gallery
  2. Extract the image names from the preview images
  3. Fetch all full-sized images using this list of names

If you want to only download files which don't exist, and don't know the number of gallery pages up front, you can adjust the code to this:

for i in $(
  while curl --max-redirs 0 -L -f \
      http://occipital.com/user/01ca-255/george-coghill/$((j++))/; do
  done \
  | sed -n 's,.*/viewer/\(......_flat\)_small\.jpg.*,http://occipital.com/images/viewer/\1.jpg,p'
); do
  [[ -f "${i##*/}" ]] || curl -L -o "${i##*/}" "$i";

The first code now increments the page number until a redirect or error occurs. The server will redirect you to the last existing page if you try to visit a page after the existing ones. The [[ -f … ]] || part will execute curl only if the corresponding file does not exist yet.

  • This did the trick, I have to say it's way over my head but I really appreciate the help! How would I tweak the code to specify the download directory?
    – George C
    Commented Nov 3, 2012 at 21:11
  • @GeorgeC: "${i##*/}" is the name of the file on your local machine; you may prepend a directory name to that, e.g. "some/dir/${i##*/}". Or you simply cd into the desired directory first.
    – MvG
    Commented Nov 4, 2012 at 20:28

So you want to download pictures from http://occipital.com/images/viewer/*_flat.jpg?

This is not possible. You can't treat viewer as a folder. The web is a service that provides you with things when you ask for it. It doesn't have a list of all of the files. Unless you try every file from AAA to ZZZ, this is not possible without a list.

  • That's what I was thinking was going to be the problem. But let's say there was a URL that this could work on—how would one use curl with non-sequential URLs to automatically check and download new files? Or is it just not possible without a sequential URL pattern?
    – George C
    Commented Aug 12, 2012 at 13:25

While you can't use a true wildcard you can specify parts within braces ie. curl mysite.{alpha,beta,gamma}.com or number ranges mysite.com/files[0001-0100].txt

See the curl man page for more info


So for your files that are random you might do mages/viewer/[000-999]_flat.jpg You'd end up with lots of file not found but it should get all the files that are there.


Like ClarkGoble said, use something like this "brute-force" solution:

    curl http://occipital.com/images/viewer/[1-999:1]_flat.jpg -f -o -image_#1_flat.jpg

The "-f" option will prevent curl from writing output files if it gets a 404 error. So you will get output only for the files found on the server.

-o stands for "output"

#1 matches the replacement pattern, i.e. in the case, the numbers starting from 1, up to 999, with a step of 1.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .