1

Does a wget showing full request headers, exist on mac? The GNU wget on mac doesn't but the GNU wget on windows (Wget 1.19)

The example output that I want, is what I get on GNU Wget in Windows (This is from the windows binaries listed https://www.gnu.org/software/wget/faq.html#download )

C:\blah>wget -d www.google.com
DEBUG output created by Wget 1.19.4 on mingw32.

Reading HSTS entries from http://www.google.com/.wget-hsts
URI encoding = 'CP1252'
converted 'http://www.google.com' (CP1252) -> 'http://www.google.com' (UTF-8)
Converted file name 'index.html' (UTF-8) -> 'index.html' (CP1252)
--2019-04-10 16:25:48--  http://www.google.com/
Resolving www.google.com (www.google.com)... seconds 0.00, 216.58.204.4
Caching www.google.com => 216.58.204.4
Connecting to www.google.com (www.google.com)|216.58.204.4|:80... seconds 0.00, connected.
Created socket 3.
Releasing 0x00000000002fb820 (new refcount 1).

---request begin---
GET / HTTP/1.1
User-Agent: Wget/1.19.4 (mingw32)
Accept: */*
Accept-Encoding: identity
Host: www.google.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 200 OK
Date: Wed, 10 Apr 2019 15:25:54 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Set-Cookie: 1P_JAR=2019-04-10-15; expires=Fri, 10-May-2019 15:25:54 GMT; path=/; domain=.google.com
Set-Cookie: NID=181=eSn6z-R-x0B1ewMV1qhIqoLLQbZFwjf_dq3GeNk8nCpMmG5UIdhoirK8fMGswItGholH9cs-7XSJwAmyJCat0Z24Uff_o9IxGDruiq0ds35p_idqiyY8awPhNQqhrMLe9JODOppcGIMaXSEQjAKlnvhEYa6zuE-e1BKunVlrze
s; expires=Thu, 10-Oct-2019 15:25:54 GMT; path=/; domain=.google.com; HttpOnly
Accept-Ranges: none
Vary: Accept-Encoding
Transfer-Encoding: chunked

---response end---
200 OK
cdm: 2
cdm: 3
cdm: 4
cdm: 5
cdm: 6
cdm: 7
cdm: 8

Stored cookie google.com -1 (ANY) / <permanent> <insecure> [expiry 2019-05-10 16:25:54] 1P_JAR 2019-04-10-15
cdm: 2
cdm: 3
cdm: 4
cdm: 5
cdm: 6
cdm: 7
cdm: 8

Stored cookie google.com -1 (ANY) / <permanent> <insecure> [expiry 2019-10-10 16:25:54] NID 181=eSn6z-R-x0B1ewMV1qhIqoLLQbZFwjf_dq3GeNk8nCpMmG5UIdhoirK8fMGswItGholH9cs-7XSJwAmyJCat0Z34Uff_o9
IxGDruiq0ds35p_idqiyY8awPhNQqhrMLe9JODOppcGIMaBSEZjAKlnvhEYa6zuE-e1BKunVlrzes
Registered socket 3 for persistent reuse.
URI content encoding = 'ISO-8859-1'
Length: unspecified [text/html]
Saving to: 'index.html.5'

index.html.5                                        [ <=>                                                                                                  ]  11.69K  --.-KB/s    in 0s

2019-04-10 16:25:48 (35.6 MB/s) - 'index.html.5' saved [11968]


C:\blah>

Meanwhile on OSX,

~/bleh$ which wget
/usr/local/bin/wget
~/crp$ ls -l /usr/local/bin/wget
lrwxr-xr-x  1 apple  admin  32  9 Feb  2018 /usr/local/bin/wget -> ../Cellar/wget/1.19.4_1/bin/wget
~/crp$ wget -d -S www.google.com
Debugging support not compiled in. Ignoring --debug flag.
--2019-04-10 18:17:18--  http://www.google.com/
Resolving www.google.com (www.google.com)... 216.58.201.36, 2a00:1450:4009:814::2004
Connecting to www.google.com (www.google.com)|216.58.201.36|:80... connected.
HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Date: Wed, 10 Apr 2019 17:17:10 GMT
  Expires: -1
  Cache-Control: private, max-age=0
  Content-Type: text/html; charset=ISO-8859-1
  P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
  Server: gws
  X-XSS-Protection: 0
  X-Frame-Options: SAMEORIGIN
  Set-Cookie: 1P_JAR=2019-04-10-17; expires=Fri, 10-May-2019 17:17:10 GMT; path=/; domain=.google.com
  Set-Cookie: NID=181=yamm_hb_BvDNDrOORdbgAQgagjGPGAGAMiTYVGhM4BDpDfsZGbwumBM0c75lmUe5-weAvOBX-2Vk6RjJ-fzR5Q4a9xXzO7IYe7_oD7opH8FKZtJ1pTIk_P8eoOVxHmHz-np5sSJPqAuBy4kjejoGLLsALJuri18vvpsvdUc0g4o; expires=Thu, 10-Oct-2019 17:17:10 GMT; path=/; domain=.google.com; HttpOnly
  Accept-Ranges: none
  Vary: Accept-Encoding
  Transfer-Encoding: chunked
Length: unspecified [text/html]
Saving to: ‘index.html.1’

index.html.1            [ <=>                ]  11.67K  --.-KB/s    in 0.001s  

2019-04-10 18:17:18 (11.1 MB/s) - ‘index.html.1’ saved [11945]

~/bleh$

Added note

I notice on OSX it says

~/bleh$ wget -d www.google.com
Debugging support not compiled in. Ignoring --debug flag.

This link https://discourse.brew.sh/t/how-to-install-wget-with-debug-support/1672

says brew reinstall wget --with-debug . But that doesn't work

~/crp$ brew reinstall wget --with-debug
Usage: brew reinstall [options] formula

Uninstall and then install formula (with existing and any appended install
options).

Unless HOMEBREW_NO_INSTALL_CLEANUP is set, brew cleanup will be run for the
reinstalled formulae or, every 30 days, for all formulae.

    -d, --debug                      If brewing fails, open an interactive
                                     debugging session with access to IRB or a
                                     shell inside the temporary build directory
    -s, --build-from-source          Compile formula from source even if a
                                     bottle is available.
        --force-bottle               Install from a bottle if it exists for the
                                     current or newest version of macOS, even if
                                     it would not normally be used for
                                     installation.
        --keep-tmp                   Don't delete the temporary files created
                                     during installation.
    -f, --force                      Install without checking for previously
                                     installed keg-only or non-migrated
                                     versions.
    -v, --verbose                    Print the verification and postinstall
                                     steps.
        --display-times              Print install times for each formula at the
                                     end of the run.
    -h, --help                       Show this message.
Error: invalid option: --with-debug

And brew info wget doesn't show that option

~/crp$ brew info wget
wget: stable 1.20.3 (bottled), HEAD
Internet file retriever
https://www.gnu.org/software/wget/
/usr/local/Cellar/wget/1.19.4_1 (50 files, 3.7MB) *
  Poured from bottle on 2018-02-09 at 16:46:38
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/wget.rb
==> Dependencies
Build: pkg-config ✘
Required: libidn2 ✘, openssl ✘
==> Options
--HEAD
    Install HEAD version
==> Analytics
install: 166,419 (30 days), 475,010 (90 days), 1,489,305 (365 days)
install_on_request: 146,838 (30 days), 417,361 (90 days), 1,304,327 (365 days)
build_error: 0 (30 days)
~/crp$ 

(I'm guessing that maybe HEAD version is not what i'm looking for.. I noticed in man wget mention of making a HEAD request rather than a GET request and maybe that's what that would be)

2

This link mentions to use macports for this instead of homebrew

https://stackoverflow.com/questions/48734977/how-to-install-wget-with-debug-support-on-homebrew

this is the line sudo port install wget

And you can get macports from https://guide.macports.org/chunked/installing.macports.html

running the relevant pkg file to install macports.

Then it installs wget with debugging support, so -d works

~/carp$ wget -d www.google.com
DEBUG output created by Wget 1.20.3 on darwin16.7.0.

Reading HSTS entries from /Users/apple/.wget-hsts
URI encoding = ‘UTF-8’
Converted file name 'index.html' (UTF-8) -> 'index.html' (UTF-8)
--2019-04-10 19:24:38--  http://www.google.com/
Resolving www.google.com (www.google.com)... 172.217.23.36, 2a00:1450:4009:80d::2004
Caching www.google.com => 172.217.23.36 2a00:1450:4009:80d::2004
Connecting to www.google.com (www.google.com)|172.217.23.36|:80... connected.
Created socket 5.
Releasing 0x00007fb166501bf0 (new refcount 1).

---request begin---
GET / HTTP/1.1
User-Agent: Wget/1.20.3 (darwin16.7.0)
Accept: */*
Accept-Encoding: identity
Host: www.google.com
Connection: Keep-Alive

---request end---
HTTP request sent, awaiting response... 
---response begin---
HTTP/1.1 200 OK
Date: Wed, 10 Apr 2019 18:24:29 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Set-Cookie: 1P_JAR=2019-04-10-18; expires=Fri, 10-May-2019 18:24:30 GMT; path=/; domain=.google.com
Set-Cookie: NID=181=Ggz9g1lZdhiLr62-quiY9H9HbAyfJejHBiCpENjuXrz9fnPVn0x1rwOp7jsklZHT8gZSjCtzPQmxlUZQ1WrCJkN0G1qH0CkNVO5mz1ECzkMdivp_d-GPuz8VWERjQ2d0NJNwlAejY88ci9_Gv0ANg2JeNsdFC2xHoJrhEDk2xfQ; expires=Thu, 10-Oct-2019 18:24:30 GMT; path=/; domain=.google.com; HttpOnly
Accept-Ranges: none
Vary: Accept-Encoding
Transfer-Encoding: chunked

---response end---
200 OK
cdm: 1

Stored cookie google.com -1 (ANY) / <permanent> <insecure> [expiry 2019-05-10 19:24:30] 1P_JAR 2019-04-10-18
cdm: 1

Stored cookie google.com -1 (ANY) / <permanent> <insecure> [expiry 2019-10-10 19:24:30] NID 181=Ggz9g1lZdhiLr62-quiY9H9HbAyfJejHBiCpENjuXrz9fnPVn0x1rwOp7jsklZHT8gZSjCtzPQmxlUZQ1WrCJkN0G1qH0CkNVO5mz1ECzkMdivp_d-GPuz8VWERjQ2d0NJNwlAejY88ci9_Gv0ANg2JeNsdFC2xHoJftEDk2xfQ
Registered socket 5 for persistent reuse.
URI content encoding = ‘ISO-8859-1’
Length: unspecified [text/html]
Saving to: ‘index.html.2’

index.html.2            [ <=>                ]  11.70K  --.-KB/s    in 0s      

2019-04-10 19:24:38 (24.0 MB/s) - ‘index.html.2’ saved [11976]

~/carp$ sudo port install wget

and of course it also installs the latest/latest stable, version GNU Wget 1.20.3 built on darwin16.7.0., and

~/carp$ which wget
/opt/local/bin/wget
~/carp$ 

as opposed to what it was before, before it was /usr/local/bin/wget.

Path was updated from /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin to /opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin

| improve this answer | |

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .