How is HEIC image decoding very fast in iOS?

Question

I used libheic c++ library to decode heic image files, but it is extremely slow. It takes about 1 - 2 seconds to decode an image. Same is the case in windows photo viewer too. But how can I swipe hundreds of heic photos instantaneously on the iphone photo viewer?

Answer 1

I'm going to assume you mean "libheif" where you wrote "libheic" (the former is a popular library for decoding HEIC files, whereas the latter is not something that can be found anywhere).

There are several reasons why the built-in iOS photo viewer is fast whereas your own libheif-based decoder is slow:

1. iOS devices with an A10 or newer CPU (i.e. iPhone 7 and newer) has hardware decoding built-in. This means that software taking advantage of that feature can do hardware accelerated image decoding - which is generally much faster than purely CPU-based image decoding.

2. Apple engineers spent a lot of time and effort optimising the software for decoding HEIC-files as they are widely used throughout the iOS system. This includes making it very easy for app developers (first or third party) to take advantage of hardware accelerated decoding.

3. You might not have compiled libheif and its dependencies in the most efficient matter, or have used the most efficient parameters during runtime of your program. I.e. which options you use when compiling the program can make a huge difference in performance.

   libheif uses libde265 for actually decoding the HEIC image data. libde265 does not support the Apple A or M series hardware accelerated decoding features. It does support the more general SSE acceleration (on x86 CPUs, when enabled) however.

4. libheif is not known for being "super fast". I don't think it is the intention that libheif is "the speediest option" - rather it seems to be focused on supporting a large range of features, ease of use and making it work with a large range of programming languages.

   For example I saw that performance regressions have been introduced in the past in exchange for more features. At one time, I saw a benchmark noting image decoding performance dropping to almost a third of its former level when upgrading libheif to a recent version compared to the older 1.6 version.

5. It is possible to use various "tricks of the trade" to optimize for example scrolling through a long list of photos in a photo viewer. In particular, HEIC files can actually contain embedded thumbnails encoded with JPEG. So you can do very fast decoding of 320x240 thumbnails embedded inside the HEIC file, in cases where you only need a thumbnail and can delay actual image decoding to later.