Steve Jobs Core Animation demo from WWDC 2007

33 points by jeanthomas


david_chisnall

Core Animation included a load of stuff that was arguably more important than the animation part (but was necessary for it).

In classic OpenStep, your window was a buffer (you could choose whether it actually got its own buffer or whether it rendered into the frame buffer directly, but OS X added the buffering automatically). When your view had some updates pending, you would set its needsDisplay property. Each run loop iteration, the window would kick the top-level view to recursively visit its children, query that property and call drawRect: on any view that needed redisplay.

In old unbuffered windows, it would also send a drawRect: message to any newly exposed region (e.g. if you dragged a window that overlapped the edge of the screen back on, the regions that used to be off the edge would now be drawn). With OS X, this didn't happen: the window server pretended that the window was always exposed, so dragging windows around was nice and fast: no redrawing needed, just compositing the windows, and by 2007 this was all done on the GPU. With buffered windows, you'd still get things like scroll views exposing parts of child views and asking them to redraw the newly visible regions.

With CoreAnimation, views could set their needsLayer property and they'd now render to a CALayer object, rather than into the same buffer as the parent. These would then be composited on the GPU. This had two big advantages. First, it meant that views could cache their contents easily. NeXT's NSScrollView did some tricks to copy pixels around (and have child views draw into a slightly larger rectangle than was visible) but now any child view with a layer could just have the coordinates of that layer moved by the scroll view without ever needing to be redrawn for scrolls (almost no CPU work, and the GPU work was trivial because GPUs are really fast at compositing). But, in addition to the caching, it also enabled parallelism. Two different views could advertise that they were thread safe (i.e. their controllers would acquire locks, or do read-only operations on the model) and then they could be rendered in parallel using libdispatch's work queues.

This combination of things meant better parallelism for rendering and lower total CPU rendering work, at the expense of more GPU memory. A complex view hierarchy might need 2-5x as much total memory as the window buffer would need without CoreAnimation. And a display might need 20x as much frame buffer with the window server providing a texture buffer for each window. The 1440x900 display in a MacBook Pro around then needed around 3.7 MiB for the frame buffer and the compositing window manager plus core animation would turn this into maybe 370 MiB of cached textures. The window server could ask windows to discard cached CALayers (and recompute them on next display) and used texture compression for windows, so it wasn't often that much, but it was easily 100+ MiB of texture memory. That was clearly infeasible on the NeXT machines, which had 8-32 MiB of total memory (and a single core and no GPU, so parallelising the CPU work and offloading to the GPU made no sense) but was a big win on the kind of machines Apple was selling by 2007 (the cheapest MacBook Pro then had two cores).

The animation bits on top were almost free: they were just some affine transforms added in the compositing step, and a little bit of processing to do interpolation around the values in the affine transforms. Once you'd built the core parts, the animation bits were trivial in comparison.

The iPhone's UIKit built on top of CoreAnimation from the start. Not having overlapping windows meant that its display server could discard (or aggressively compress) the contents of background apps if RAM was tight.

Note that the combination of the damage, fixes, render, composite, and GLX extensions on X11 meant that X11 could do everything that CoreAnimation did. It just needed assembling and cooperation from the compositing manager. The thing Apple did was rip out the core of their toolkits to rebuild around this model.

bhoot

The overall usability has degraded too. Often, after I type in an app name in full in spotlight, the first suggestion is a folder instead of the app itself. The app does show up momentarily in the suggestion list while I am typing, but vanishes by the time I'm done.

xeubie

One really clear turning point for Apple on a technical level was the switch from Objective-C to Swift. At the time it seemed like an exciting transition, bringing a stricter compiler and a syntax that younger web developers saw as familiar. But after going through years of churn to switch to this supposedly modern language, what did they actually gain? Objective-C was really elegant, and (unlike C++) a strict superset of C. Swift became bloated with features, including excessive type inference that made it look good on slides but absolutely tanked compile times.

Francisco Tolmasky (part of the original iPhone team) talked about the many mistakes they made when introducing Swift, including not consulting the army of application developers within Apple itself before announcing it. It's a language made by LLVM compiler geeks that had little application-level experience, and had dubious features like declarative syntax bolted on afterwards. I'd argue this was actually their biggest mistake on the software side, because it affected their entire ecosystem across all of their products.