Removing characters from a string via Terminal

Question

My main goal is to be able use Terminal to turn

http://t.umblr.com/redirect?z=https%3A%2F%2Farchiveofourown.org%2Fworks%2F1885509&t=N2M3YmNhNjQzODU5MjI2NmE5ZjljY2Y4NWVhOTVhM2IwYzgyYzYyYixLOHhQakVRbA%3D%3D

into

https://archiveofourown.org/works/1885509

I thought a good place to start would be with finding a way to remove the http://t.umblr.com/redirect?z=.

I triedtr -d; however, there does not seem to be a way to remove only the first instance of a group of characters.

The following (using cut) will only work consistently if there length of the string never changes, which means there is no guarantee for future use.

 echo 'http://t.umblr.com/redirect?z=https%3A%2F%2Farchiveofourown.org%2Fworks%2F1885509&t=N2M3YmNhNjQzODU5MjI2NmE5ZjljY2Y4NWVhOTVhM2IwYzgyYzYyYixLOHhQakVRbA%3D%3D' | cut -c31-83

This looked promising:

 var="http://t.umblr.com/redirect?z=https%3A%2F%2Farchiveofourown.org%2Fworks%2F1885509&t=N2M3YmNhNjQzODU5MjI2NmE5ZjljY2Y4NWVhOTVhM2IwYzgyYzYyYixLOHhQakVRbA%3D%3D" | var=${var:30} 

But because I can not get | pbcopy to work, not could I get outer to print, I have idea what it actually does.

I don't know if there is shell script that will allow me to keep what appears between the equal signs. From the tutorials and questions I have found, sed looks like it would be my best option (if I can figure it out) but it seems to work for files, not strings.

Where is a better place for me to start?

Answer 1

This seems to work:

$ echo 'http://t.umblr.com/redirect?z=https%3A%2F%2Farchiveofourown.org%2Fworks%2F1885509&t=N2M3YmNhNjQzODU5MjI2NmE5ZjljY2Y4NWVhOTVhM2IwYzgyYzYyYixLOHhQakVRbA%3D%3D' | awk 'BEGIN{FS="=|&"}{gsub(/%3A/, ":")}{gsub(/%2F/, "/")}{print $2}'
https://archiveofourown.org/works/1885509
$

What happening here is the URL string is placed in single quotes and using echo is piped | into an awk program.

Example Syntax:

echo 'URL' | awk 'BEGIN{FS="=|&"}{gsub(/%3A/, ":")}{gsub(/%2F/, "/")}{print $2}'

Answer 2

Using Perl, create this text file and name it something like "redo.pl":

my $url = $ARGV[0];
$url =~ s/%3A/:/g;
$url =~ s/%2F/\//g;
$url =~ /=\K(.*)(?=&)/;
print $1;

It can be run from a terminal as:

 perl redo.pl http://t.umblr.com...

The output will be:

https://archiveofourown.org/works/1885509

Answer 3

If you want to use sed:

echo 'URL' | cut -d = -f 2 | sed -e 's|\%3A|:|g' -e 's|\%2F|/|g'

or even without cut and just one call to sed

echo 'URL' | sed -e 's|.*z=\(.*\)\&.*|\1|' -e 's|\%3A|:|g' -e 's|\%2F|/|g'

Answer 4

Here's an example using php and it's urldecode() function

echo 'URL' | php -r '$str = urldecode(fgets(STDIN));
$str = explode("=", $str);
$str = explode("&", $str[1]);
printf("%s\n", $str[0]);'

• the function urldecode() decodes any %## encoding
• the function explode() turns the var str into an array split on the equal sign
• The second explode() function turns the value of the array str[1] into an array split on the & sign overwriting the array str
• then print the first value of the new array str

Another solution using pure shell syntax (no external programs): ksh, zsh, bash- OSX 10.6 and up versions

url='http://t.umblr.com/redirect?z=https%3A%2F%2Farchiveofourown.org%2Fworks%2F1885509&t=N2M3YmNhNjQzODU5MjI2NmE5ZjljY2Y4NWVhOTVhM2IwYzgyYzYyYixLOHhQakVRbA%3D%3D'
nurl=${url//\%2F//}; nurl=${nurl//\%3A/:}
nurl=${nurl%=*}; nurl=${nurl#*=}
echo ${nurl%\&*}