How to recover lost NTFS Partition?

Question

I have a mid-2012 Macbook pro 13" Model No. A1278 (EMC 2554*). I'm running MacOS Sierra 10.12 and Windows 10 Build 10240 on 750GB disk.

Without doing proper research, I stupidly used Disk Utility to add a new 139GB partition to the same disk that had MacOS and Bootcamp and made the filesystem as ExFat because I wanted both my MacOS and Windows to be able to read and write to it.

Lo and behold, Boot Camp was no longer functioning. Whilst MacOS still boots, the Boot Camp partition now no longer showed up in the Startup boot option menu, though it did in the Disk Utlility and on the MacOS desktop. Attempting to boot straight off of bootcamp showed the following message: No bootable device --- insert boot disk and press any key

So I followed this tutorial: http://nerdr.com/bootcamp-partition-lost-repairing-mac-partitions/

I already know I may have made a mistake because I was using that tutorial and an answer from other forum posts (https://discussions.apple.com/message/19071422#19071422 & I can no longer boot into macOS after resizing my bootcamp partition). The nerdr tutorial in the link above says to use gptfdisk and I was using the terminal app gdisk based on the answer in the forum.

After following the Nerdr procedure exactly using gdisk, however, it had all the same menu options so I didn't know if there is a difference. After I was done, Bootcamp was completely erased. Now here's what gdisk shows as my new partition table:

Command (? for help): p
Disk /dev/disk0: 1465149168 sectors, 698.6 GiB
Sector size (logical): 512 bytes
Disk identifier (GUID): DC08A243-0BDF-4D84-870F-044744CBFD1A
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 1465149134
Partitions will be aligned on 8-sector boundaries
Total free space is 70891845 sectors (33.8 GiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1              40          409639   200.0 MiB   EF00  EFI System Partition
   2          409640      1100019007   524.3 GiB   AF00  
   3      1100019008      1101288543   619.9 MiB   AB00  
   4      1101288544      1392987759   139.1 GiB   0700  
   5      1392987760      1394257295   619.9 MiB   AB00  Recovery HD

These terminal outputs might also help:

diskutil list:

   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *750.2 GB   disk0
   1:                        EFI EFI                     209.7 MB   disk0s1
   2:                  Apple_HFS 750GB HD                563.0 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
   4:       Microsoft Basic Data TD                      149.4 GB   disk0s4
   5:                 Apple_Boot                         650.0 MB   disk0s5

diskutil cs list:
No CoreStorage logical volume groups found

I think the data is still probably there, it is now just missing on the partition table. As you can see, the Recovery HD disk ends on sector 1394257295 and the last available sector is 1465149134. It says there is 33.8gb of free space, and the Windows bootcamp partition was around 39gb if I remember correctly.

I don't have anything backed up or have time machine running.

I thought restoring the partition I made back to the original partition may fix. I was not able to restore the ExFat partition back to the 750GB hard drive. I get message "Restore Process Has Failed" and the following list of commands: Validating target... Validating source... Repartitioning target device... Source volume is read-write and cannot be unmounted, so it can't be block copied. Operation failed…

I get the above message over and over, even after unmounting it, reformatting the file system as MacOS journaled, and so on. Lastly I don't know if bootcamp was booting using MBR or GPT, but the disk did have a windows folder on it if that helps.

How do I recover my lost NTFS partition?

Edit: Output for David Anderson

There is no output for "dd if=/dev/disk0 bs=512 skip=1394257296 count=50000000 | fgrep -o -a -b "BOOTMGR is compressed":

Answer 1

If you need a better explanation of any step given below, please post a comment.

The answer given below assumes a sector size of 512 bytes. For larger sector sizes, you will need to make the appropriate adjustments. Below are the basic steps needed to find and restore a lost Windows Boot Camp partition. None of the steps require the use of third party software.

1. If applicable, disable System Integrity Protection (SIP). You need to do this to get access to /dev/disk0. In other words, you need to be able to read the data stored on the disk you are booted from.

   The command to disable SIP is given below. You will need to boot to one of several macOS recovery modes or an external Mac startup disk, before executing the command from the Terminal application.

   csrutil  disable
   

   Actually, SIP does not need to be totally disabled. The following undocumented version of the csrutil command can be used to at least disable the filesystem protections of SIP.

   csrutil  enable  --without  fs
   

   Note: I have only tested the above command while running High Sierra (macOS 10.13.1).

   After partially or completely disabling SIP, boot back to macOS (OS X).

   Note: rEFInd can be also be used to selectively disable/enable SIP.

2. Launch the Terminal application and enter the command given below.

   sudo  chmod  a+w  /dev/disk0
   

   This allows you to read (but not write) from /dev/disk0 without preceding your commands with sudo.

   Note: This read access will be removed automatically when you restart your computer.

3. Next, you need to determine which sectors may contain your missing NTFS partition. This can be accomplished by entering the command given below.

   gpt  -r  show  /dev/disk0
   

   Below is the output from my computer. Since my computer has Fedora (linux) and rEFInd installed, the output below will differ from what would be typical. However, the Windows NTFS volume still resides in partition 4, as expected for a 2011 iMac.

   Marlin:~ davidanderson$ gpt -r show /dev/disk0
   gpt show: /dev/disk0: Suspicious MBR at sector 0
         start       size  index  contents
             0          1         MBR
             1          1         Pri GPT header
             2         32         Pri GPT table
            34          6         
            40     409600      1  GPT part - C12A7328-F81F-11D2-BA4B-00A0C93EC93B
        409640  585937496      2  GPT part - 48465300-0000-11AA-AA11-00306543ECAC
     586347136    1269536      3  GPT part - 426F6F74-0000-11AA-AA11-00306543ECAC
     587616672  193097312      4  GPT part - EBD0A0A2-B9E5-4433-87C0-68B6B72699C7
     780713984     409600      5  GPT part - 48465300-0000-11AA-AA11-00306543ECAC
     781123584    1024000      6  GPT part - 0FC63DAF-8483-4772-8E79-3D69D8477DE4
     782147584  192528384      7  GPT part - E6D6D379-F507-44C2-A23C-238F2A3DF928
     974675968          8         
     974675976    1835008      8  GPT part - 48465300-0000-11AA-AA11-00306543ECAC
     976510984     262144      9  GPT part - C12A7328-F81F-11D2-BA4B-00A0C93EC93B
     976773128          7         
     976773135         32         Sec GPT table
     976773167          1         Sec GPT header
   

   In your case, you can substitute your output from gdisk. What you need to look for is any large amounts free space. Your output shows free space starting at sector 1394257296 and ending a sector 1465149134. This is 70891839 sectors which is about 36 GB of free space.

4. You are now ready to start searching for the missing NTFS partition. To hex dump the first sector of my Windows NTFS Boot Camp partition, I entered the command given below.

   hexdump  -C  -v  -n  512  -s  587616672b  /dev/disk0
   

   Note: In the above command, 587616672b is 587616672 followed by the letter b. It is NOT 5876166726.

   Below is the output from the above command.

   Marlin:~ davidanderson$ hexdump -C -v -n 512 -s 587616672b /dev/disk0
   460ca34000  eb 52 90 4e 54 46 53 20  20 20 20 00 02 08 00 00  |.R.NTFS    .....|
   460ca34010  00 00 00 00 00 f8 00 00  3f 00 33 00 a0 51 06 23  |........?.3..Q.#|
   460ca34020  00 00 00 00 80 00 80 00  5f 6e 82 0b 00 00 00 00  |........_n......|
   460ca34030  00 00 0c 00 00 00 00 00  02 00 00 00 00 00 00 00  |................|
   460ca34040  f6 00 00 00 01 00 00 00  f8 1a 1d 9e f0 ed 53 24  |..............S$|
   460ca34050  00 00 00 00 fa 33 c0 8e  d0 bc 00 7c fb 68 c0 07  |.....3.....|.h..|
   460ca34060  1f 1e 68 66 00 cb 88 16  0e 00 66 81 3e 03 00 4e  |..hf......f.>..N|
   460ca34070  54 46 53 75 15 b4 41 bb  aa 55 cd 13 72 0c 81 fb  |TFSu..A..U..r...|
   460ca34080  55 aa 75 06 f7 c1 01 00  75 03 e9 dd 00 1e 83 ec  |U.u.....u.......|
   460ca34090  18 68 1a 00 b4 48 8a 16  0e 00 8b f4 16 1f cd 13  |.h...H..........|
   460ca340a0  9f 83 c4 18 9e 58 1f 72  e1 3b 06 0b 00 75 db a3  |.....X.r.;...u..|
   460ca340b0  0f 00 c1 2e 0f 00 04 1e  5a 33 db b9 00 20 2b c8  |........Z3... +.|
   460ca340c0  66 ff 06 11 00 03 16 0f  00 8e c2 ff 06 16 00 e8  |f...............|
   460ca340d0  4b 00 2b c8 77 ef b8 00  bb cd 1a 66 23 c0 75 2d  |K.+.w......f#.u-|
   460ca340e0  66 81 fb 54 43 50 41 75  24 81 f9 02 01 72 1e 16  |f..TCPAu$....r..|
   460ca340f0  68 07 bb 16 68 52 11 16  68 09 00 66 53 66 53 66  |h...hR..h..fSfSf|
   460ca34100  55 16 16 16 68 b8 01 66  61 0e 07 cd 1a 33 c0 bf  |U...h..fa....3..|
   460ca34110  0a 13 b9 f6 0c fc f3 aa  e9 fe 01 90 90 66 60 1e  |.............f`.|
   460ca34120  06 66 a1 11 00 66 03 06  1c 00 1e 66 68 00 00 00  |.f...f.....fh...|
   460ca34130  00 66 50 06 53 68 01 00  68 10 00 b4 42 8a 16 0e  |.fP.Sh..h...B...|
   460ca34140  00 16 1f 8b f4 cd 13 66  59 5b 5a 66 59 66 59 1f  |.......fY[ZfYfY.|
   460ca34150  0f 82 16 00 66 ff 06 11  00 03 16 0f 00 8e c2 ff  |....f...........|
   460ca34160  0e 16 00 75 bc 07 1f 66  61 c3 a1 f6 01 e8 09 00  |...u...fa.......|
   460ca34170  a1 fa 01 e8 03 00 f4 eb  fd 8b f0 ac 3c 00 74 09  |............<.t.|
   460ca34180  b4 0e bb 07 00 cd 10 eb  f2 c3 0d 0a 41 20 64 69  |............A di|
   460ca34190  73 6b 20 72 65 61 64 20  65 72 72 6f 72 20 6f 63  |sk read error oc|
   460ca341a0  63 75 72 72 65 64 00 0d  0a 42 4f 4f 54 4d 47 52  |curred...BOOTMGR|
   460ca341b0  20 69 73 20 63 6f 6d 70  72 65 73 73 65 64 00 0d  | is compressed..|
   460ca341c0  0a 50 72 65 73 73 20 43  74 72 6c 2b 41 6c 74 2b  |.Press Ctrl+Alt+|
   460ca341d0  44 65 6c 20 74 6f 20 72  65 73 74 61 72 74 0d 0a  |Del to restart..|
   460ca341e0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
   460ca341f0  00 00 00 00 00 00 8a 01  a7 01 bf 01 00 00 55 aa  |..............U.|
   460ca34200
   

   A description of the above output can be found at either Microsoft: NTFS Physical Structure or Wikipedia: NTFS Internals.

   Imbedded in this sector are several strings. To find your NTFS partition, you will need to search for one of these strings. I will randomly choose the string BOOTMGR is compressed. Since I know where my NTFS partition is, I can enter the command below to search for this string.

   dd  if=/dev/disk0  bs=512  skip=587616672  count=50000000  |  fgrep  -o  -a  -b  "BOOTMGR is compressed"
   

   The largest count value my computer will allow is about 50000000. This value may be different for your computer. So, if I wanted to search my entire NTFS partition, I would have to enter this command 4 times. Each time with a skip value increased by 50000000. Of course, one could write a shell script with a loop to repeat these commands, but I feel adding such a script here would be beyond the scope of this answer.

   The output, from entering the above command on my computer, is given below.

   Marlin:~ davidanderson$ dd if=/dev/disk0 bs=512 skip=587616672 count=50000000 | fgrep -o -a -b "BOOTMGR is compressed"
   425:BOOTMGR is compressed
   338433776:BOOTMGR is compressed
   790493856:BOOTMGR is compressed
   1095625536:BOOTMGR is compressed
   1405559232:BOOTMGR is compressed
   3533975792:BOOTMGR is compressed
   3534741744:BOOTMGR is compressed
   4601366880:BOOTMGR is compressed
   6145609496:BOOTMGR is compressed
   6146292528:BOOTMGR is compressed
   6146951928:BOOTMGR is compressed
   6951738816:BOOTMGR is compressed
   8739643840:BOOTMGR is compressed
   8846594496:BOOTMGR is compressed
   9138315712:BOOTMGR is compressed
   9241170368:BOOTMGR is compressed
   21212440040:BOOTMGR is compressed
   50000000+0 records in
   50000000+0 records out
   25600000000 bytes transferred in 874.399247 secs (29277244 bytes/sec)
   

   As expected, the the string was found inside the first sector searched. The rest of the matches can be ignored.

   If your case, you would enter the command given below, where I assume a count value of 50000000 would be valid.

   dd  if=/dev/disk0  bs=512  skip=1394257296  count=50000000  |  fgrep  -o  -a  -b  "BOOTMGR is compressed"
   

   Note: You can enter a "control T" while the dd command is executing. This will cause the dd command to display its progress.

   If this command does not return a match, you can assume your NTFS partition is not completely intact.

5. Verify a match resides in a valid Partition Boot Sector. Any given match value can be converted to a Partition Boot Sector starting value by first dividing the match value by the bs value. Any remainder can be discarded. Next, add the skip value to the result. This is computed below for my first match value of 425.

   425 / 512 + 587616672 = 587616672
   

   Next, enter a command to hex dump the computed sector. For my computer, the command and output is shown in step 4. For the -s option, you will need to substitute your starting sector value. Do not forget to append the letter b to your value.

   Minimally, the sector should start with the hex sequence eb 52 90 4e 54 46 53 20 20 20 20 00 and end with the hex sequence 55 aa. Other values, to check for, can be found in the references given in step 4.

6. Determine the NTFS partition start and size values. The partition start value is the same as Partition Boot Sector starting value. To determine the size, you will need to acquire the 8 hexadecimal byte values, stored in the Partition Boot Sector, starting at the 41st value. For my Partition Boot Sector shown in step 4, these values are shown below.

   5f 6e 82 0b 00 00 00 00
   

   Next, you will need to convert from little endian to big endian by reversing the order of the bytes, as shown below.

   00 00 00 00 0b 82 6e 5f
   

   Convert these values to single hexadecimal value by removing the spaces and preceding zeros, then adding 0x the beginning of the result. Do so, for the above values, results in the single value shown below.

   0xb826e5f
   

   Determine the decimal equivalent by using the command shown below.

   Marlin:~ davidanderson$ echo $((0xb826e5f))
   193097311
   

   Finally, round up this value to the nearest integer divisible by 8. In this case, 193097311 is not divisible by 8. Therefore, this value needs to be rounded up to a value of 193097312.

   For those who did not know, a value is divisible by 8, if its last 3 digits are divisible by 8. In this case, 311/8=38.875 and 312/8=39, so 193097311 is not divisible 8, but 193097312 is.

7. Add the found partition to the GUID partition table (GPT). If this partition did not already exist in my computer, I could add it by entering the command shown below. In your case, you would use index option -i 6 in place of -i 4. Also, your beginning sector -b and size -s option values would be different.

   sudo  gpt  -f  add  -i  4  -b  587616672  -s  193097312  -t  windows  /dev/disk0
   

   My computer is running macOS 10.13.1. This command may not execute, if running OS X or an earlier version of macOS. If so, you will need to first start up from macOS Recovery over the Internet or an external Mac startup disk, then execute the alternative command given below from the Terminal application. Again, your -i, -b and -s option values would be different from what is shown below.

   gpt  add  -i  4  -b  587616672  -s  193097312  -t  windows  /dev/disk0
   

   Another alternative would be to use gdisk to add the partition to the GPT.

   Note: After editing the GPT, a restart of the computer may be required.

8. If necessary, enable SIP. This requires the same procedure as step 1, except the command to enable SIP is shown below.

   csrutil  enable