In Safari on Mavericks, I sometimes get a dialog from web sites indicating that some site functions will be blocked or degraded unless I turn off Private Browsing mode. Most users probably expect their choice of whether to use Private Browsing to be private. My impression is that other major browsers do not leak Private Browsing state to web sites.
I have noticed that while in private Browsing mode, Safari will not request favicons, so in theory a site could block caching of the favicon and make a pretty good guess that Private Browsing is on.
In contrast, on iOS, Safari doesn't request favicons at all. But it will, even in Private Browsing mode, request any "apple-touch-icon" if you access the Sharing panel (the apple-touch-icon is used as the Home Screen icon if you subsequently "Add to Home Screen" for that URL).
Other than favicon, are there other known ways that a web site could determine (or guess with reasonable accuracy) that you are using Private Browsing mode in desktop Safari?