13

I'd love to understand more about how Rosetta 2 works. The Apple Developer article is brief. Has anyone done a deep analysis on how Rosetta 2 works, how it is invoked and whether it's possible to use it via an API?

Some questions:

  • How are x86_64 applications launched under Rosetta?
  • Is it possible to dynamically invoke translation for a portion of x86 instructions?
  • Might it be possible to bridge Rosetta to QEMU or similar to allow fast virtualization of Intel Docker images?
4
  • We strongly encourage one question per question. Even when related, it works far better to ask each clearly and then follow up with one question that combines one or two items to synthesize an answer that depends on a couple items.
    – bmike
    Dec 1, 2020 at 12:26
  • Question 2 needs to stand alone “ Is it possible to dynamically invoke translation for a portion of x86 instructions?” it’s an excellent question and should be asked and answered here in some detail past a “yes/no” summary answer.
    – bmike
    Dec 1, 2020 at 12:45
  • Same with docker and qemu questions so those two questions would be idea, to ask now as both are running on M1 now. I absolutely encourage these questions in proper sized chunks.
    – bmike
    Dec 1, 2020 at 12:46
  • 1
    I suppose at the moment there’s so little information on this subject on Google or elsewhere that any links to relevant information are useful. Agree separate questions would be better once anyone has managed to delve into the black box which is Rosetta-2. Dec 1, 2020 at 22:30

2 Answers 2

17

Rosetta 2 works by doing an ahead-of-time (AOT) translation of the Intel code to corresponding ARM code. It is able to do this efficiently and easily mainly because the M1 CPU contains a special instruction that switches the memory-ordering model observed by the CPU for that thread into a model equivalent to the Intel x86 model (TSO - total store order). This has to do with how programs can expect memory consistency to work when having multiple processors (i.e. cores in this case).

User's can observe the translation the first time they launch an Intel app on the M1 as the first launch is slow. The translated code is cached and used on subsequent, much faster launches.

If you have a binary that is valid for several different architectures, you can specifically invoke Rosetta 2 by specifying that you want to launch the Intel code. You can do that from the terminal like this:

arch -x86_64 ./mycommand

Note that this setting also applies to any program that the "mycommand" process should choose to run.

Rosetta 2 as delivered by Apple in macOS Big Sur is not setup to dynamically invoke translation for a portion of x86 instructions. It is entirely focused on doing an AOT translation of the whole binary in advance. There's no user interface for translating a few small set of instructions on the fly. Rosetta 2 does include a JIT engine that allows translating instructions on the fly (for example if you run an Intel-based browser with a JIT JavaScript engine) - it is however not a general purpose JIT-engine that you could use for other purposes through an API or similar.

If you want to do that for research purposes or just out of "pure interest", then you could just take the instructions you want to translate and add them to a simple application shell (essentially adding them to a simple main()-only C program for example) and run it. The cached, translated version of the program then includes the translated instructions for inspection.

The cache is available in these folders:

/var/db/oah/
/System/Library/dyld/aot_shared_cache

There's no immediate way of "bridging" Rosetta 2 to QEMU to allow fast virtualization of Intel Docker images. QEMU contains its own Intel x86 emulation, so you could get it to run Intel Docker images on the M1 without involving Rosetta 2 at all. In this case, "fast" is a very subjective measure.

11
  • 2
    aot_shared_cache is a (big) file, are there any tools available to extract specific content?
    – nohillside
    Dec 1, 2020 at 12:12
  • I don't think Apple supplies any tools specifically for working with the aot_shared_cache file, including ones that would allow you to search for and extract specific content. For now you'll have to make do with a hexeditor or similar tools.
    – jksoegaard
    Dec 1, 2020 at 12:35
  • This probably makes it unusable for practical purposes, at least until somebody reverse-engineers the file format. The file is over one GB, at least here.
    – nohillside
    Dec 1, 2020 at 12:57
  • 1
    Thanks for the detailed reply. FWIW the 'AOT' claim is at least disputed - instead Hector Martin calls it a JIT + a code-cache: twitter.com/marcan42/status/1328956667548082177?s=20 Dec 1, 2020 at 13:22
  • 1
    @JamesBlackburn It's not really disputed - it's just people splitting hairs. These terms do not have the strict definitions which that tweet assumes. If you translate from one set of machine code (be it a virtual machine or actual physical machine) to another set of machine code, and you do it before you run the program - then that's ahead-of-time. This is what Apple essentially do with Rosetta 2. If you want to be able to run the vast majority of Intel apps, you also need to do JIT (just-in-time) compilation as you'll find lots of programs doing code-manipulation on the fly.
    – jksoegaard
    Dec 1, 2020 at 14:54
7

This is an answer for a deep analysis on how Rosetta 2 works.

I have reverse-engineered Rosetta 2 a little bit. For more details, see the GitHub pages of this project.

Cache files of Rosetta 2 are located at both /System/Library/dyld/aot_shared_cache and /var/db/oah/.

/System/Library/dyld/aot_shared_cache contains only translated results for System dylibs (e.g., /usr/lib/system/libsystem_blocks.dylib, /usr/lib/system/libxpc.dylib You can see the full list here for macOS Big Sur version 11.1). This file contains multiple cache files for System dylibs, so its file size is huge.

For non-system binaries such as third-party x86_64 binaries, the files with aot extension under /var/db/oah contain the translated results.

aot_shared_cache is a (big) file, are there any tools available to extract specific content?

@nohillside I have created a simple python script to show the contents of aot_shared_cache. You can use this script to extract specific content.

3
  • Great work! Can't wait for the next "chapters"!
    – jksoegaard
    Feb 26, 2021 at 9:02
  • sudo ls -l /var/db/oah -> ls: oah: Operation not permitted :-\
    – arya
    Jun 9, 2021 at 21:10
  • You need to disable System Integrity Protection (SIP) to access /var/db/oah because files under /var/db/oah is protected by SIP by default. Jun 11, 2021 at 7:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .