3

I am looking for a command line equivalent to the Get text from webpage option in /Applications/Automator.app. The Get text from webpageoption is pretty self explanatory, it gets the text only, without HTML tags, CSS, JavaScript, etc. from a webpage. I know I can use wget or curl but that will give me all the HTML tags, CSS, JavaScript, etc. not the only text version of the webpage.

  • I don't know of any command to do this. You could try writing a PHP script that downloads the content then strips out all of the tags. That could still leave you with javascript or css depending on how the page is built. You'd need to strip that stuff with extra regexp first, then use strip_tags() to dump the rest. – l008com Apr 11 '18 at 23:37
  • Hmm. I'm looking for speed here do you think it would be quicker than just using the automator thing. – JBis Apr 11 '18 at 23:39
  • Web pages are pretty small, I think any solution is going to run very quickly. I don't know anything about the automator solution though. – l008com Apr 11 '18 at 23:46
  • Ok. Thanks for your help will look into the PHP script idea, and possibly run from command line with the php command. – JBis Apr 11 '18 at 23:48
  • 2
    Off the bat you could do something like curl https://apple.stackexchange.com | sed 's/<[^>]*>//g' and it will strip out the tags. Taking it further, you could use this PHP code to do just what you asked: nadeausoftware.com/articles/2007/09/…. Though, with a page like this, it will give you irrelevant info. Your best bet is to just find the text your're looking for. – Allan Apr 12 '18 at 2:05
1

Your best option is the textutil command. Read the man page, but something like:

textutil -convert txt webpage.html

should work.

  • 1
    Can you please explain why textutil is better than other options? – fd0 Dec 16 '18 at 18:57
  • It's simpler than a PHP script. It's more readable than sed. – benwiggy Dec 16 '18 at 19:02
  • I'll try it out soon. See if it works. – JBis Dec 16 '18 at 19:37
  • User user3439894 give an example in a comment above. Ran an my mac just fine. – historystamp Dec 17 '18 at 0:27
  • @historystamp if you want to address someone use the @ followed by the name – Ruskes Dec 19 '18 at 22:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .