Steve Jobs Core Animation demo from WWDC 2007
33 points by jeanthomas
33 points by jeanthomas
Core Animation included a load of stuff that was arguably more important than the animation part (but was necessary for it).
In classic OpenStep, your window was a buffer (you could choose whether it actually got its own buffer or whether it rendered into the frame buffer directly, but OS X added the buffering automatically). When your view had some updates pending, you would set its needsDisplay property. Each run loop iteration, the window would kick the top-level view to recursively visit its children, query that property and call drawRect: on any view that needed redisplay.
In old unbuffered windows, it would also send a drawRect: message to any newly exposed region (e.g. if you dragged a window that overlapped the edge of the screen back on, the regions that used to be off the edge would now be drawn). With OS X, this didn't happen: the window server pretended that the window was always exposed, so dragging windows around was nice and fast: no redrawing needed, just compositing the windows, and by 2007 this was all done on the GPU. With buffered windows, you'd still get things like scroll views exposing parts of child views and asking them to redraw the newly visible regions.
With CoreAnimation, views could set their needsLayer property and they'd now render to a CALayer object, rather than into the same buffer as the parent. These would then be composited on the GPU. This had two big advantages. First, it meant that views could cache their contents easily. NeXT's NSScrollView did some tricks to copy pixels around (and have child views draw into a slightly larger rectangle than was visible) but now any child view with a layer could just have the coordinates of that layer moved by the scroll view without ever needing to be redrawn for scrolls (almost no CPU work, and the GPU work was trivial because GPUs are really fast at compositing). But, in addition to the caching, it also enabled parallelism. Two different views could advertise that they were thread safe (i.e. their controllers would acquire locks, or do read-only operations on the model) and then they could be rendered in parallel using libdispatch's work queues.
This combination of things meant better parallelism for rendering and lower total CPU rendering work, at the expense of more GPU memory. A complex view hierarchy might need 2-5x as much total memory as the window buffer would need without CoreAnimation. And a display might need 20x as much frame buffer with the window server providing a texture buffer for each window. The 1440x900 display in a MacBook Pro around then needed around 3.7 MiB for the frame buffer and the compositing window manager plus core animation would turn this into maybe 370 MiB of cached textures. The window server could ask windows to discard cached CALayers (and recompute them on next display) and used texture compression for windows, so it wasn't often that much, but it was easily 100+ MiB of texture memory. That was clearly infeasible on the NeXT machines, which had 8-32 MiB of total memory (and a single core and no GPU, so parallelising the CPU work and offloading to the GPU made no sense) but was a big win on the kind of machines Apple was selling by 2007 (the cheapest MacBook Pro then had two cores).
The animation bits on top were almost free: they were just some affine transforms added in the compositing step, and a little bit of processing to do interpolation around the values in the affine transforms. Once you'd built the core parts, the animation bits were trivial in comparison.
The iPhone's UIKit built on top of CoreAnimation from the start. Not having overlapping windows meant that its display server could discard (or aggressively compress) the contents of background apps if RAM was tight.
Note that the combination of the damage, fixes, render, composite, and GLX extensions on X11 meant that X11 could do everything that CoreAnimation did. It just needed assembling and cooperation from the compositing manager. The thing Apple did was rip out the core of their toolkits to rebuild around this model.
The overall usability has degraded too. Often, after I type in an app name in full in spotlight, the first suggestion is a folder instead of the app itself. The app does show up momentarily in the suggestion list while I am typing, but vanishes by the time I'm done.
This drives me insane. It's basic search. I don't understand it.
And yes, I've disabled a bunch of types from Spotlight indexing.
One really clear turning point for Apple on a technical level was the switch from Objective-C to Swift. At the time it seemed like an exciting transition, bringing a stricter compiler and a syntax that younger web developers saw as familiar. But after going through years of churn to switch to this supposedly modern language, what did they actually gain? Objective-C was really elegant, and (unlike C++) a strict superset of C. Swift became bloated with features, including excessive type inference that made it look good on slides but absolutely tanked compile times.
Francisco Tolmasky (part of the original iPhone team) talked about the many mistakes they made when introducing Swift, including not consulting the army of application developers within Apple itself before announcing it. It's a language made by LLVM compiler geeks that had little application-level experience, and had dubious features like declarative syntax bolted on afterwards. I'd argue this was actually their biggest mistake on the software side, because it affected their entire ecosystem across all of their products.
But after going through years of churn to switch to this supposedly modern language, what did they actually gain?
I think a sizable gain was that many more things have both spatial and temporal memory safety throughout, which aiui was a large reason that Swift is now in the kernel and Secure Enclave.
(I do also know of a sizable number of people who still vastly prefer the ergonomics to Objective C, but of course that's a more subjective thing.)
Improved memory safety is nice, but its use case remains primarily for high-level app development, not kernel or secure enclave development. App developers cared more about async support than null safety, yet it took 7+ years for Swift to get the former. The entire ecosystem was upended for the sake of features that weren't very relevant to the primary use case of the language. A gradual evolution to Objective-C 3.0 could've improved safety and performance, while other low level languages could've been adapted to kernel development rather than attempting to build a jack-of-all-trades language, as the original Swift presentation laughably portrayed it as.
I beg to differ. When we first got our hands on Swift and deemed it stable enough to ship with, the first thing the iOS dev community did was make illegal states unrepresentable in our apps' data models and state. We wiped out classes of bugs and crashes because we had better tools for correctness.
Concurrency did take a while, and there was a lot of churn there. Combine was a distraction. But it was only because of progress in the then-private Swift project that we got features we loved in Objective-C 2.0 like ARC.
Improved memory safety is nice, but its use case remains primarily for high-level app development
I'd argue that moving away from a memory unsafe language is good here too! Userland apps can intake all sorts of untrusted data and connect to all sorts of places. Having a more "modern" language, specifically one not built around raw pointers, is still a net good there. Even if the underlying frameworks still rely heavily on Obj C as stated (aiui they're trying to get away from this, but yeah it's still a very long journey), that doesn't mean other parts can't be hardened ahead.
Certainly, I wouldn't say that all their decisions here were optimal. But I think as for "after going through years of churn [...] what did they actually gain" (which was the original question) this is still a sizable win, and probably one that wouldn't have been possible without some churn anyway.
You sound like you haven't used Objective-C much. I used it extensively starting around 2006; I was active on CocoaDev and other sites in those days. It was absolutely not "built around raw pointers". In fact pointers were opaque references that you rarely had to explicitly dereference or do arithmetic on except for C interop. Also, once ARC was introduced in 2011, pointers essentially were just managed references. Objective-C was already quite a safe and high-level language.
Regarding the "sizable win", you're forgetting opportunity cost. What could they have achieved if they evolved Objective-C instead? It's impossible to say, but we know it took 7+ years to get async, a feature that app developers actually were clamoring for. Nobody in those days was clamoring for reducing of devirtualization or getting null safety; both are good, but to high level app developers this was not even on the list of most important things.
But after going through years of churn to switch to this supposedly modern language, what did they actually gain?
A lot of performance. Objective-C imposes a lot of dynamic language overhead on top of C, which has some benefits, but makes it hard to devirtualize. I'd rather eat compile time than run time. And as the sibling comment points out, it's a lot more secure and appropriate for systems work.
As David Chisnall will point out, Objective-C is a good language for the problems of the 90s, but we have a different set of problems now.
Making an Objective-C successor that is optimized for kernel and secure enclave work makes no sense -- its primary use case is for making userland apps. This is a classic example of focusing on the wrong problem. How much Swift code is being written in the kernel or secure enclave vs in apps?
Moreover, most of Apple's frameworks that Swift apps are built upon are still written in Objective-C. A typical Swift app is spending most of its execution time in the underlying frameworks. Tolmasky already wrote about this but people still accept Apple's marketing about Swift uncritically.
Objective-C mac and iOS apps were not suffering from massive performance problems. What app developers needed was things like better async support, which ended up taking 7+ years to finally arrive in Swift. If instead they did a steady evolution, say an Objective-C 3.0, that focused on async support and performance improvements, the ecosystem would've suffered less churn and they would've been less distracted by pointless rewrites.
I think SwiftUI was the bigger mistake. The simplification of UI structure to the lowest common denominator made Mac UIs bland.
Apple's design switched from bespoke UIs that were inventing widgets and interactions tailored to each problem, to cramming everything into lists and grids of buttons.
I just clicked around the system settings on my work Macbook (M4 Max) and yeah, it's surprisingly slow.
I feel like Linux is actually the place where serious graphics work is happening now. Hyprland is kind of an extreme case, I don't think most "regular users" want that in their desktop experience, but it sure is fast and smooth.
Even vanilla Gnome you get with Fedora is a really smooth and lag free experience. I am always surprised by that when I switch between osx and linux.
Similar experience here. My work computer is a three year old M3 MacBook Pro. My personal computer is an unremarkable six year old Dell with an integrated Intel GPU running Debian KDE plasma.
Working on the Mac is like working through a pane of syrup. There are pointless animations everywhere, wasted space in the name of "design", and every action (even just typing and clicking) has some noticable latency.
On KDE, everything is just instant with zero delay or fuss. I even put it on an ancient Chromebook with 4gb of RAM recently. You have to give everything a few moments to load but once it's in memory, it's every bit of usable.
Core Animation when it came on the scene was amazing. Apple used to do these traveling events called Tech Talks where you could signup, for free, and they would hold developer-oriented sessions at a hotel. It was like a mini WWDC. They really focused on Core Animation for a while and I learned a ton about how to implement things efficiently.
Of course I want Apple to bring software quality back. But I don't think Apple's quality is the sole problem. A lot of us prefer their computers even with this deficiency, because the alternatives still look worse. I want competition to make them improve. Why don't we expect great things from anybody else?
Interesting how obvious it is that "water", "fish", "amazon" and "beach" are literally the only tags that exist in the demo, as the results appear from the first letter already.
I made the grave mistake of updating to Golden Gate. It's a severely degraded experience, to put it kindly. I can't full-screen apps anymore, because the multiple desktops manager (Mission Control) becomes unusably laggy. Without full-screen windows it's just very laggy. I really cannot understand how they could allow such a buggy piece of software to ship. For the first time in 15 years I'm actually considering switching back to linux for my laptop.
I’m not sure that such a self-contained demo would be slow today. The problem is that software is not made like that anymore.