Remotely distribute an iOS app compiled in Xcode 7 without App Store, developer program sub, or jailbreak?

Question

Following these instructions (see also Apple's developer documentation and BoltClock's answer to Test iOS app on device without apple developer program or jailbreak), I was able to compile an iOS app (in this case provenance, a video game console emulator app which doesn't meet Apple's guidelines and will never be allowed in Apple's App Store) in Xcode 7, sign it with my personal iCloud account, sideload it onto my iPhone (by plugging it into my Mac by USB, and selecting my iPhone in xcode menu -> Product -> Destination), and allow my phone to execute it by "trusting" apps with this signature in Settings -> General -> Profile, which installs the certificate associated to my icloud account onto the device.

I would like to replicate these steps with an over the air download. Can I get Xcode to produce an executable that I can share with my brother, who has an iPhone, but does not have access to a Mac with Xcode?

The StackExchange Network accomplished something like this apparently for its iOS app beta program, by using Apple's enterprise distribution via Mobile Device Management, as mentioned in this question. There is also the BuildStore (http://builds.io/), which for $10/year also enables this functionality, apparently by adding users to a "developer team" associated to a paid Apple Developer Program account (see Apple dev docs). With Xcode 7, is it possible to do such a thing without an enterprise or even regular paid developer account?

update: Thanks to Stacksza's help, I have made some progress, which I will summarize here.

1. XCode will create an archive (Product -> Archive), but it does not allow you to export an archive to to an .ipa file for ad hoc distribution without subscribing to the paid ADP account tier.
2. No problem though, because the command line xcode tool does create an .ipa executable, even with no paid ADP account. Use a command like xcodebuild -exportArchive -archivePath Provenance-Release\ 11-20-15\,\ 22.24.xcarchive/ -exportOptionsPlist ~/manifest.plist -exportPath ~/Provenance (see Khawar's answer at How to create ipa in xcode 6 without Apple Developer account?)
3. But this command requires you to specify a manifest .plist file. The answer by Razvan at Over The Air (OTA) iOS IPA File Distribution For Public? contains a sample manifest.plist, as well as instructions for the hyperlink format necessary for providing a download link, which must be hosted on an SSL webpage. Might be that the SSL URL must math the app's bundle id, I'm not sure.
4. But the above only works if the app is provisioned for the device which downloads it, and the device has already installed the developer certificate. So for instance a device which has previously been connected via USB to my Xcode can now download the app over the air, which is great. This is progress. But for clean devices, I get the error Could not install embedded profile: 0xe8008012 (This provisioning profile cannot be installed on this device.) (as viewed in the device's console, which can be viewed in Xcode -> Window -> Devices, or else in the Apple Configurator app).
5. For an app to be provisioned for a device, it needs to contain the device's UDID in the embedded.mobileprovision file, see How Do You Embed a Provisioning Profile in an iOS App, How to automate checking a field in an ipa's embedded.mobileprovision?, How to find out what profile was used for building *.ipa file?, and Is there a way to see what UDID are included in a build?. You cannot just edit this file to add the UDID, however, because then the CODESIGNATURE of the app doesn't match, and the device will report the error Failed to verify code signature of /private/var/mobile/Library/Caches/com.apple.mobile.installd.staging/temp.7jBjpk‌​/extracted/Payload/Provenance.app : 0xe8008017 (A signed resource has been added, modified, or deleted.)}
6. Ok, so we need to resign the app. Xcode provides a commandline tool called codesign which can do this. Instructions are at How to re-sign the ipa file? and Re-sign IPA (iPhone) and Resign an iPhone App, insert new Bundle ID and send to Xcode Organizer for Upload and How to manually re-sign an IPA with a new provisioning profile. I had to specify my developer certificate, the precise name of which can be found in Keychain Access, under the Certificates tab, and looks like "iPhone Developer: [email protected] (ABCDEFGHIJ)". Then the command is like codesign -f -s "iPhone Developer: [email protected] (ABCDEFGHIJ)" Payload/Provenance.app. The example commands in the instructions above included a flag called --resource-rules, but I found no such file in my compiled app, so I removed that flag. I wonder what it does, and whether it is necessary. Edit: The excellent blog post Inside Code Signing by Thomas Kollbach explains that the --resource-rules flag used to be used to state which parts of an app must be signed, but since iOS8 is deprecated, and the entire app must be signed.
7. Now that the app has the UDID added and is signed, downloading it on the fresh device gives the errors Could not install embedded profile: 0xe8008003 (This provisioning profile is malformed and A valid provisioning profile for this executable was not found. My guess it the problem now is that the device still has not installed the developer certificate. When sideloading via Xcode, before the app will execute, you have to find the installed certificate in Settings -> General -> Profile, select your certificate and tap "trust". However this option does not appear after the over-the-air download of the ad hoc app, so I am unsure how to proceed. I think I need some way to install the developer certificate on the device. According to Installing provisioning profile on iOS 8.0.2, it used to be possible by simply emailing the certificate to the device, but that no longer works as of iOS 8. I have not tried to confirm. How can I install my developer certificate onto an iOS device remotely?
8. I tried exporting my iPhone developer certificate from Keychain Access, and emailing it to the device. The device was able to install the certificate, but it displayed with a red warning that it was not signed, whereas when the cert was installed via Xcode/USB had no such warning. According to Rhythmic Fistman at is possible to re-encrypt an embedded.mobileprovision file?, the profile must be signed by Apple to work. This may be the sticking point. Apple will only sign your cert if you have a paid ADP account.
9. This blog post on provisioning by Sean Heber was very illuminating about the general theory of developer certificates and provisioning profiles (along with Kollbach's they should both should be required reading). It contains this paragraph:

   The reason all of this works and is secure is that Apple generates the provisioning profiles in the portal and then signs them with their own private keys before delivering them to you. The signing of the provisioning profiles is something only Apple can do. The file you download can therefore not be tampered with without rendering it invalid. An invalid provisioning profile will not be accepted by iOS and thus Apple can control exactly what can and cannot be provisioned by a developer by simply restricting access to the signing of the provisioning profile to things the the developer portal gives you permission to configure in the first place - even though provisioning profiles can support any number of other awesome options you can’t use without jailbreaking. This is why you have to register testing devices in the portal, add your certificates to the portal, etc - only things in the portal (and thus the numbers of which can be controlled and limited arbitrary by Apple) can be included in a generated and properly signed provisioning profile. The portal is where Apple’s provisioning policies and limitations are actually enforced.

This seems to definitively answer my question (can I get the app to run without a paid ADP account) with a "no". Although this article predates Xcode 7. Apparently Xcode 7 will send a device's UDID to the Apple portal, and receive in response a provisioning profile signed by Apple, containing that UDID. Xcode 7 only does this when the device is plugged in via USB and recognized by Xcode, but one wonders whether a manual Xcode request to Apple could be crafted? That gets deeper into reverse engineering Apple's stuff than I'm comfortable with, and probably violates ToS or whatever, so I think I'm satisfied with the "no" answer.

According to the about page for BuildStore, they manage something similiar via email, so it may be possible.

Xcode “Build and Archive” from command line is quite old (circa xcode 3), but suggests using xcrun instead of xcodebuild which I've been using.

Answer 1

The answer is almost YES!

I managed to install new builds of the same app on the devices via Fabric.io without apple dev subscription. (I guess you can also distribute them with iTunes. Send ipa to tester, and it should be able to install it via iTunes.)

The only disadvantage is that you have to connect the device to Xcode at least once. Then you can remotely install new versions. (I didn't find the way to do it manually knowing the udid)

When you connect the "unknown" device to xcode 7, there is a fix issue button in Target general settings. Fix the issue and Xcode will generate a new Provisioning profile with this device included. (you can find it in ~/Library/MobileDevice/Provisioning Profiles)

Than you can use this provisioning profile to distribute apps via Fabric.io

1. Signup on Fabric.io
2. Setup Fabric for your project (there is a step by step guide)
3. Make sure that xcode uses right prov. profile to sign it. Automatic worked for me.
4. Xcode>project>archive
5. fabric helper will detect the build and you will be able to distribute it.

Sorry guys who are not proficient enough with Xcode, I don't have much time to add less error prone guide, but I hope this answer will help somebody with ideas.

TIP: use this quicklook plugin to see inside ipas and prov. profiles!

Answer 2

As of some version of iOS 9.x released since my answer was originally posted, this is no longer even a possibility. Apple now restricts OTA downloads to non-free accounts, as shown by errors in the device logs upon the install failing.

I have tested this extensively, as this would be amazing for my apps too.

Can I distribute my apps OTA without a Developer Program Membership (costs money)?

You can create and sign an IPA and create the Manifest file for it, but it will fail to install on the device. The reason for this is Xcode needs to configure the IPA for that device (supposedly signing it properly with the UDID). You can even see this happen if you try to plug in a new device to the Mac and run your app via Xcode on it. The short answer is no.

What can I do to make this work?

You can sign up for the Developer Program with Apple (costs money) and add the UDID of his device to your profile, then create an IPA and manifest file, upload them to a web server and send him the URL.

It really is for security, as the price barrier is usually what stops malicious apps from being distributed over the web. Not to mention keeping the majority of app downloads in the App Store.

I've signed up for the developer program? How do I distribute it now?

You must have a HTTPS web server (and a SSL certificate which you can make your own root for). You also must have added their UDID to your Developer profile (I'm not exactly sure how to do this). You must use Xcode to create the IPA (Product > Archive > Export > Save for Ad-hoc Deployment > Sign in with Apple ID > Follow instructions) and you will get a Plist and an IPA file. Put those both on your server and change the values in the Plist file to match the HTTPS URLs on your server. Then you use a special link to link to the Plist manifest, and it will pop up asking the user to install your app. If everything goes right, the user will have your app. You also have to code your own update method if you want that.

If this is an app you want to distribute at this point, it might make more sense to actually put it on the App Store, or if this is for beta testing use TestFlight. If you have a developer account you can use TestFlight and skip all of this stuff for app testing. All you need is their email and to get them to install the TestFlight app from the App Store. TestFlight can be managed from iTunes Connect.

Answer 3

Basically, someone has to sign the app. You can get the IPA file, but it won't run on arbitrary devices. Focus on that and you'll quickly decide if you want to endure the hassle of each person using Xcode or if paying for Apple to sign your apps makes sense.

If you are relying on iOS 9 non-developer signing with any old Apple ID, your testers all need to sign in with your Apple ID to run the app.

You can pay for Apple to handle the accounts and signing by paying for a year of developer accounts. Then you can email, pop the app on any server, use TestFlight or all over easier for you methods other than handing out your Apple ID to your testers / users of the app.

You can always send the source code to your brother if he has Xcode, but you mentioned that was the point of not having someone have a Mac to install and sign the app.

Jail breaking is easier than forging Apple's certificate chain, but I don't recommend that.