2

The following command will dump the dom for a web page using Google Chrome:

/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome  --headless --disable-gpu --dump-dom http://example.com

I'm wondering if something similar is possible for Safari.

I can do this with AppleScript but I'd prefer the solution for Safari to be "headless" (see --headless option in command above), so I think an AppleScript will not achieve what I'm looking for.

** UPDATE **

Selenium driver has a headless mode for other browsers. But it doesn't look like there is such a mode for safari: https://stackoverflow.com/a/58314435

  • Why not just use e.g.: curl -s http://example.com – user3439894 Jul 30 at 14:37
  • because curl doesn't have a javascript engine. – StevieD Jul 30 at 14:38
  • I don’t have a direct answer and suspect it’s - this function isn’t implemented in Safari. However, I’m curious - what do you do with the dump once you have it? Apple and Xcode do unit testing in other ways if that’s your end goal. – bmike Jul 30 at 17:43
  • Scrape the contents. – StevieD Jul 30 at 17:44
1

I’m fairly confident Safari isn’t amenable to scraping, so use Chrome or Firefox for that on macOS if you need a browser to be scriptable.

However, swift and python are exceptionally good at this by calling code directly via NSURL or equivalent as well as using the NSURLSessiond daemon and working with the OS directly and not even needing Safari.

Of course you have to learn some new language and API/SDK for this to pay off. The pay off can be huge if you have a command line tool that does all the heavy lifting far more efficiently than Chrome or Safari can.

| improve this answer | |
  • Safari is scriptable and it's definitely possible to scrape a Safari page with Selenium (I've done it); it just can't be done in headless mode, AFAICT. Do you know if NSURL will run the html through a js engine? – StevieD Jul 30 at 17:56
  • 1
    I’ve never wanted more js, so no - not something I’ve done. I block it pretty widely in browser as well, tbh. However, I can see how that’s needed in some cases - try this -pypi.org/project/cloudscraper @StevieD – bmike Jul 30 at 17:58
  • 1
    Awesome, my dude. Thanks so much! I needed a script just like that wasn't heavyweight. It loads the contents in less than a second. – StevieD Jul 30 at 18:09
  • @StevieD I linked cloud scraper in my answer to the python words. Please put up your own answer if you end up with a script. I have enough rep here, I’d rather you answer if you can do better. Feel free to edit into my answer if you want to accept it at some point after testing. – bmike Jul 30 at 20:04
  • 1
    Ok but the only problem is that it doesn't answer the exact question I posted (I asked about Safair). I don't care about rep. And just before you posted your comment, I posted on stackoverflow and I put a link to to cloudscraper there: stackoverflow.com/questions/63178669/… – StevieD Jul 30 at 20:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .