5

My main goal is to be able use Terminal to turn

http://t.umblr.com/redirect?z=https%3A%2F%2Farchiveofourown.org%2Fworks%2F1885509&t=N2M3YmNhNjQzODU5MjI2NmE5ZjljY2Y4NWVhOTVhM2IwYzgyYzYyYixLOHhQakVRbA%3D%3D

into

https://archiveofourown.org/works/1885509

I thought a good place to start would be with finding a way to remove the http://t.umblr.com/redirect?z=.

I triedtr -d; however, there does not seem to be a way to remove only the first instance of a group of characters.

The following (using cut) will only work consistently if there length of the string never changes, which means there is no guarantee for future use.

 echo 'http://t.umblr.com/redirect?z=https%3A%2F%2Farchiveofourown.org%2Fworks%2F1885509&t=N2M3YmNhNjQzODU5MjI2NmE5ZjljY2Y4NWVhOTVhM2IwYzgyYzYyYixLOHhQakVRbA%3D%3D' | cut -c31-83

This looked promising:

 var="http://t.umblr.com/redirect?z=https%3A%2F%2Farchiveofourown.org%2Fworks%2F1885509&t=N2M3YmNhNjQzODU5MjI2NmE5ZjljY2Y4NWVhOTVhM2IwYzgyYzYyYixLOHhQakVRbA%3D%3D" | var=${var:30} 

But because I can not get | pbcopy to work, not could I get outer to print, I have idea what it actually does.

I don't know if there is shell script that will allow me to keep what appears between the equal signs. From the tutorials and questions I have found, sed looks like it would be my best option (if I can figure it out) but it seems to work for files, not strings.

Where is a better place for me to start?

5

This seems to work:

$ echo 'http://t.umblr.com/redirect?z=https%3A%2F%2Farchiveofourown.org%2Fworks%2F1885509&t=N2M3YmNhNjQzODU5MjI2NmE5ZjljY2Y4NWVhOTVhM2IwYzgyYzYyYixLOHhQakVRbA%3D%3D' | awk 'BEGIN{FS="=|&"}{gsub(/%3A/, ":")}{gsub(/%2F/, "/")}{print $2}'
https://archiveofourown.org/works/1885509
$

What happening here is the URL string is placed in single quotes and using echo is piped | into an awk program.

Example Syntax:

echo 'URL' | awk 'BEGIN{FS="=|&"}{gsub(/%3A/, ":")}{gsub(/%2F/, "/")}{print $2}'
  • One of the reasons I decided to start with removing the characters at the beginning was because replacing %3A into : and %2F into / was proving too much--So thank you. I've tried this with a number of different links, and it works. I don't remember awk showing up in my research, I'll be spending some time on it's man page. I'm wondering about BEGIN{FS="=|&"}. Is that start after the first '=' and go until the next '=', but no really, stop at the '&'? – 345422 Jul 29 '16 at 9:55
  • @boredrandom, I wish I knew awk well enough that I could explain how the field separator works when using it the way I did. I have to admit that it could have been written as {FS="[=&]"} and it too would have worked in this instance. I originally wrote the awk program using {FS="="} and when I saw the &t at the end of the output I modified the field separator hoping it would remove it, and it did. Because I couldn't explain that part is why I said "This seems to work:". Although that also assumes other URL's to be processed follows the same pattern, which is a typical pattern. – user3439894 Jul 29 '16 at 11:41
  • @boredrandom, BTW Not sure how you ultimately want to use this, however I thought I'd mention because the awk program is returning the output before the prompt returns you can redirect the output, e.g. adding ` | pbcopy` at the end would place it on the clipboard before the prompt returns and you'd not see the output until you pasted it elsewhere. There are other ways of incorporating the code depending on how you want to use it, so if you need help with that just ask. – user3439894 Jul 29 '16 at 12:05
  • Thanks. Actually, the first thing I did was add pbcopy. I want to create a Automator Service to Copy Link, apply this script and add the clean URL to my clipboard (because Tumblr has started using redirects). I wanted to get this part done first. – 345422 Jul 29 '16 at 13:11
2

Using Perl, create this text file and name it something like "redo.pl":

my $url = $ARGV[0];
$url =~ s/%3A/:/g;
$url =~ s/%2F/\//g;
$url =~ /=\K(.*)(?=&)/;
print $1;

It can be run from a terminal as:

 perl redo.pl http://t.umblr.com...

The output will be:

https://archiveofourown.org/works/1885509
  • I have recently downloaded perl via Homebrew, but I'm not quite sure how to install and use it. Do you know of any useful guilds? (Or should I set this up another question?) – 345422 Aug 4 '16 at 23:44
  • 1
    The main issue you need to focus on is that Homebrew will install Perl, but it will not make its version the default because Mac OS comes with its own Perl (System Perl) and Homebrew does not want to disable it. The best resource I know to learn how to use your own Perl while leave the System Perl active is to look at the helper program called Perlbrew. Its homepage is perlbrew.pl. I also highly recommend the site LearnPerl at learn.perl.org and the associated book "Learning Perl" by Randal L. Schwartz, brian d foy, Tom Phoenix. There is no better or easier way to learn Perl. – Pinecone Aug 5 '16 at 18:03
1

If you want to use sed:

echo 'URL' | cut -d = -f 2 | sed -e 's|\%3A|:|g' -e 's|\%2F|/|g'

or even without cut and just one call to sed

echo 'URL' | sed -e 's|.*z=\(.*\)\&.*|\1|' -e 's|\%3A|:|g' -e 's|\%2F|/|g'
  • One of the reasons I decided to start with removing the characters at the beginning was because replacing %3A into : and %2F into / was proving too much--So thank you. Just out of curiosity, what part of this removes the t.umblr.com bit? Is it the -f 2 part? – 345422 Jul 29 '16 at 9:49
  • @boredrandom -d = sets the delimeter, -f 2 selects the second field. – nohillside Jul 29 '16 at 11:02
  • And, on second thought, my solution doesn't remove the part after the & yet. Can be done with cut or sed, but let me get back in front of a computer running bash first :-) – nohillside Jul 29 '16 at 11:04
  • One of the reasons I started using awk more often is, too often I found myself writing code that relied on piping through multiple commands that could easily be replaced with a single awk program. For me this seems like a better way of doing things, and from what I read, is more efficient then running multiple pipes. Either way though, both methods have there uses. – user3439894 Jul 29 '16 at 13:43
1

Here's an example using php and it's urldecode() function

echo 'URL' | php -r '$str = urldecode(fgets(STDIN));
$str = explode("=", $str);
$str = explode("&", $str[1]);
printf("%s\n", $str[0]);'
  • the function urldecode() decodes any %## encoding
  • the function explode() turns the var str into an array split on the equal sign
  • The second explode() function turns the value of the array str[1] into an array split on the & sign overwriting the array str
  • then print the first value of the new array str

Another solution using pure shell syntax (no external programs): ksh, zsh, bash- OSX 10.6 and up versions

url='http://t.umblr.com/redirect?z=https%3A%2F%2Farchiveofourown.org%2Fworks%2F1885509&t=N2M3YmNhNjQzODU5MjI2NmE5ZjljY2Y4NWVhOTVhM2IwYzgyYzYyYixLOHhQakVRbA%3D%3D'
nurl=${url//\%2F//}; nurl=${nurl//\%3A/:}
nurl=${nurl%=*}; nurl=${nurl#*=}
echo ${nurl%\&*}
  • Hi. And thanks for your addition. You said "pure shell syntax (no external programs)," do you mean programs like .Replace ? – 345422 Jul 29 '16 at 17:46
  • Like sed, awk, php....etc, other than the shell. – fd0 Jul 29 '16 at 17:48
  • Oh. Okay. I didn't know those were external programs. Is there something in the man that points out if commands are for internal or external? (So I have an idea what to look for.) – 345422 Jul 29 '16 at 20:08
  • Each shell has its own programing language. These languages allow you to manipulate strings without the need to call other programs (external programs) such as awk or sed ... In the example, we are using Parameter Expansion to manipulate the string contained in the variable url. Assuming that your shell is bash use the following command and read away on the man page- man -P "less -p 'Parameter Expansion'" bash – fd0 Jul 29 '16 at 20:39

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .