13

I've got a lot of highlighting and notes in iBooks that I've read, and would like to be able to collect them in one easy to use and manipulate format (e.g for writing papers and citing quotes).

For example, I'd like a highlight like this

to produce something (e.g., in CSV) like

Quod me nutrit me destruit – that which sustains me also destroys me,14,Tamburlane Parts One and Two,Christopher Marlowe,Anthony B. Dawson ed.,Bloomsbury,https://itun.es/us/qSrZ0.l

I can see how to do this (sort of) laboriously, one note at a time, using iBook's "sharing" feature (or copy and paste, of course) but I don't see any way to do it in bulk, for all of my notes from a book, or even all my books.

Is there a way to accomplish this, with an Apple Script or using Automator for example? Or perhaps there is a text or XML file containing my notes and highlighting that I could write a script (in Python, preferably) to parse.

11

iBooks doesn't have AppleScript support. The annotations are stored in a SQLite file: ~/Library/Containers/com.apple.iBooksX/Data/Documents/AEAnnotation/.

You could try to parse that. This answer gives a link to Digested, which reads that database and then allows you to export your annotations to Evernote, but I don't know what formatting they'll have or if you want to mess with Evernote.

One (possibly) simple solution would be to open the book in iBooks for iOS. You could then email the annotations in bulk to yourself.

  1. Open the book
  2. Press the "list button" to bring up the Table of Contents
  3. Switch to the Notes tab
  4. Press the Share button
  5. Select Edit Notes
  6. Select All
  7. Share via email.

Edit:

Actually, after reading a comment on reddit, there seems to be a way to export them all from iBooks on OS X as well:

You can export your notes in an email from Notes -> Select All -> Share (you need to hold ctrl while you right click to retain your selection). Your highlighted portions will be copied into the email with your notes and formatted nicely. Strangely, on the Mac the application doesn't care about whether the book is copy protected—it will always copy the highlighted portion. The iOS application indeed discriminates though. If your book is copy protected, then only the chapter name will be shared. That seems to be the only way to do it unfortunately. :/

Using my laptop's trackpad, I had to hold down ctrl + shift while tapping on the trackpad in order to bring up the contextual menu while retaining the selection.

  • This is very helpful. I'm still a ways away though from getting the annotations into a CSV or other convenient form. I can't make anything out of the SQL database, and the mail, while nice looking, isn't programmatically accessible. – orome Dec 19 '14 at 19:27
  • Is it just me, or is this no longer possible in the new version of iBooks? I don't see an Edit Notes button anymore. In which case, how do I export all my notes? – incandescentman Jul 2 '15 at 22:42
  • @incandescentman It's there for me in iOS 8.4. – fred Jul 15 '15 at 17:15
  • 1
    @incandescentman The numbered steps in my answer above were for iOS. The last part of the answer—the section after "Edit"—is for OS X. That still works for me on Yosemite. – fred Jul 16 '15 at 19:14
  • 2
    So, I'm on el capitan now, but the directions from that reddit comment still work for me. I guess the process does differ slightly depending on whether you're using a mouse or a trackpad. Using the laptop without an external keyboard/mouse, after I have the comments selected using select all, I then press control + shift + tap on trackpad. That brings up this:screenshot. Comments across chapters are selected. – fred Oct 6 '15 at 2:33
3

I've written a script for this purpose that extracts the notes from your Mac and outputs Evernote export files, ready for double clicking. Perhaps you could modify my script if it doesn't suit your purposes precisely.

In short, it reads the SQLite databases in: ./Library/Containers/com.apple.iBooksX/Data/Documents/BKLibrary ./Library/Containers/com.apple.iBooksX/Data/Documents/AEAnnotations

... and in this case, exports them to Evernote's .enex format.

https://github.com/jorisw/ibooks2evernote/

    <?php
    /*
     *  iBooks notes to Evernote converter
     *  by Joris Witteman <joris@jor.is>
     *  
     *  Reads the iBooks Annotations library on your Mac and exports
     *  them, tagged with their respective book title and imported in
     *  separate notebooks.
     *
     *  Usage:
     *  
     *  Move this script to the top of your personal home directory on your Mac.
     *  This is the folder that has your name, which the Finder opens if you
     *  click on the Finder icon in the Dock.
     *
     *  To export your notes to Evernote:
     *  
     *  1. Run the following command in the Terminal:
     *
     *     php ./ibooks2evernote.php
     *    
     *  2. Open the newly created "iBooks exports for Evernote" folder from your
     *     home folder, open each file in there, Evernote will open and start 
     *     importing your notes.
     *
     */




















    // Default file locations for required iBooks data 
    define('RESULT_DIRECTORY_NAME',"iBooks exports for Evernote");
    define('BOOKS_DATABASE_DIRECTORY','./Library/Containers/com.apple.iBooksX/Data/Documents/BKLibrary');
    define('NOTES_DATABASE_DIRECTORY','./Library/Containers/com.apple.iBooksX/Data/Documents/AEAnnotation');


    if(file_exists(RESULT_DIRECTORY_NAME)){
        die("The destination folder for the exports already exists on your Mac.\nPlease move that one out of the way before proceeding.\n");
    }

    // Verify presence of iBooks database

    if(!file_exists(BOOKS_DATABASE_DIRECTORY)){
        die("Sorry, couldn't find an iBooks Library on your Mac. Have you put any books in there?\n");
    }else{
        if(!$path = exec('ls '.BOOKS_DATABASE_DIRECTORY."/*.sqlite")){
            die("Could not find the iBooks library database. Have you put any books in there?\n");
        }else{
            define('BOOKS_DATABASE_FILE',$path);
        }
    }


    // Verify presence of iBooks notes database

    if(!file_exists(NOTES_DATABASE_DIRECTORY)){
        die("Sorry, couldn't find any iBooks notes on your Mac. Have you actually taken any notes in iBooks?\n");
    }else{
        if(!$path = exec('ls '.NOTES_DATABASE_DIRECTORY."/*.sqlite")){
            die("Could not find the iBooks notes database. Have you actually taken any notes in iBooks?\n");
        }else{
            define('NOTES_DATABASE_FILE',$path);
        }
    }


    // Fire up a SQLite parser

    class MyDB extends SQLite3
    {
      function __construct($FileName)
      {
         $this->open($FileName);
      }
    }


    // Retrieve any books.

    $books = array();

    $booksdb = new MyDB(BOOKS_DATABASE_FILE);

    if(!$booksdb){
      echo $booksdb->lastErrorMsg();
    } 

    $res = $booksdb->query("
                SELECT
                    ZASSETID,
                    ZTITLE AS Title,
                    ZAUTHOR AS Author
                FROM ZBKLIBRARYASSET
                WHERE ZTITLE IS NOT NULL");

    while($row = $res->fetchArray(SQLITE3_ASSOC) ){
        $books[$row['ZASSETID']] = $row;
    }

    $booksdb->close();

    if(count($books)==0) die("No books found in your library. Have you added any to iBooks?\n");

    // Retrieve the notes.

    $notesdb = new MyDB(NOTES_DATABASE_FILE);

    if(!$notesdb){
      echo $notesdb->lastErrorMsg();
    } 

    $notes = array();

    $res = $notesdb->query("
                SELECT
                    ZANNOTATIONREPRESENTATIVETEXT as BroaderText,
                    ZANNOTATIONSELECTEDTEXT as SelectedText,
                    ZANNOTATIONNOTE as Note,
                    ZFUTUREPROOFING5 as Chapter,
                    ZANNOTATIONCREATIONDATE as Created,
                    ZANNOTATIONMODIFICATIONDATE as Modified,
                    ZANNOTATIONASSETID
                FROM ZAEANNOTATION
                WHERE ZANNOTATIONSELECTEDTEXT IS NOT NULL
                ORDER BY ZANNOTATIONASSETID ASC,Created ASC");

    while($row = $res->fetchArray(SQLITE3_ASSOC) ){
        $notes[$row['ZANNOTATIONASSETID']][] = $row;
    }

    $notesdb->close();


    if(count($notes)==0) die("No notes found in your library. Have you added any to iBooks?\n\nIf you did on other devices than this Mac, make sure to enable iBooks notes/bookmarks syncing on all devices.");


    // Create a new directory and cd into it

    mkdir(RESULT_DIRECTORY_NAME);
    chdir(RESULT_DIRECTORY_NAME);

    $i=0;
    $j=0;
    $b=0;

    foreach($notes as $AssetID => $booknotes){

        $Body = '<?xml version="1.0" encoding="UTF-8"?>
        <!DOCTYPE en-export SYSTEM "http://xml.evernote.com/pub/evernote-export3.dtd">
        <en-export export-date="'.@strftime('%Y%m%dT%H%M%S',time()).'" application="iBooks2Evernote" version="iBooks2Evernote Mac 0.0.1">';

        $BookTitle  = $books[$AssetID]['Title'];

        $j = 0;

        foreach($booknotes as $note){

            $CappedText = null;
            $TextWithContext = null;

            // Skip empty notes
            if(strlen($note['BroaderText']?$note['BroaderText']:$note['SelectedText'])==0) continue;

            $HighlightedText = $note['SelectedText'];

            // Cap the titles to 255 characters or Evernote will blank them.

            if(strlen($HighlightedText)>255) $CappedText = substr($note['SelectedText'],0,254)."…";

            // If iBooks stored the surrounding paragraph of a highlighted text, show it and make the highlighted text show as highlighted.
            if(!empty($note['BroaderText']) && $note['BroaderText'] != $note['SelectedText']){
                $TextWithContext = str_replace($note['SelectedText'],"<span style=\"background: yellow;\">".$note['SelectedText']."</span>",$note['BroaderText']);
            }

            // Keep some counters for commandline feedback
            if($j==0)$b++;
            $i++;
            $j++;

            // Put it in Evernote's ENEX format.
            $Body .='
    <note><title>'.($CappedText?$CappedText:$HighlightedText).'</title><content><![CDATA[<?xml version="1.0" encoding="UTF-8" standalone="no"?>
    <!DOCTYPE en-note SYSTEM "http://xml.evernote.com/pub/enml2.dtd">
    <en-note>
    <div>
    <p>'.($TextWithContext?$TextWithContext:$HighlightedText).'</p>
    <p><span style="color: rgb(169, 169, 169);font-size: 12px;">From chapter: '.$note['Chapter'].'</span></p>
    </div>
    <div>'.$note['Note'].'</div>
    </en-note>
    ]]></content><created>'.@strftime('%Y%m%dT%H%M%S',@strtotime("2001-01-01 +". ((int)$note['Created'])." seconds")).'</created><updated>'.@strftime('%Y%m%dT%H%M%S',@strtotime("2001-01-01 +". ((int)$note['Modified'])." seconds")).'</updated><tag>'.$BookTitle.'.</tag><note-attributes><author>joris@jor.is</author><source>desktop.mac</source><reminder-order>0</reminder-order></note-attributes></note>';

        }

        $Body .='
        </en-export>
        ';

        file_put_contents($BookTitle.".enex", $Body);
    }

    echo "Done! Exported $i notes into $b separate export files in the '".RESULT_DIRECTORY_NAME."' folder.\n\n";
3
  1. Install the free DB Browser for SQLite.
  2. Go to the iBooks annotations folder: ~/Library/Containers/com.apple.iBooksX/Data/Documents/AEAnnotation/
  3. Copy the .sqlite file somewhere (like Desktop) to keep the original safe.
  4. Open the file with DB Browser.
  5. Find some notes in your target book by browsing the data.
  6. Filter by the the ZANNOTATIONASSETID to show only notes in the target book.
  7. Copy and paste the annotations you want into Numbers or whatever application you prefer.
  • Apple's famed ease of use! – orome Jul 11 '16 at 21:10
  • @raxacoricofallapatorius: Seriously. That's some rigmarole just to get a list of the vocabulary words I highlighted. – Gavin Jul 11 '16 at 21:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .