The Case for Message Passing in Swift

Update: Chris Lattner has asked developers to not file duplicate radars for feature requests. Please do not duplicate any radars listed in this post.

Apple introduced Swift as “Objective-C without the C”, but in many ways, Swift is closer to “C++ without the C” (or Objective-C without the Smalltalk). Swift’s generics, for example, are closer to C++'s templates than to any feature of Objective-C. One of the more subtle ways that Swift resembles C++ is in how it calls methods. Unlike Objective-C, where methods are called using message passing, Swift uses a vtable for method dispatch, and when the compiler has enough information, inlines the function call directly. While this improves performance over message passing, it trades power and flexibility for performance gains that aren’t necessary for most apps.

In this article, I will present use cases which are currently impossible or difficult to implement without message passing. For each of these cases, I have filed radars asking for support in Swift. That way, even if message passing is not added to Swift, support for these other use cases may be added. There is also a radar requesting message passing. If you have any use ceases for message passing not covered in this article, please include them in your radar, and let me know and I’ll update this article with links to your use cases, but please do not file duplicates of the radars in this article.

A Small Note on Binary Compatibility

At WWDC, Apple promised that although Swift’s syntax may change, the code you write now will be “binary compatible” with Swift in the future. Binary compatibility might make radical changes, like switching to message-passing, impossible. However, I don’t think “binary compatibility” in this case means that Swift frameworks written today will work with future Swift code. The Swift runtime and standard library is linked into all Swift apps: the OS does not include either. Some of the Swift developers have already indicated that they plan to make major changes to the language, like adding access modifiers. By “binary compatibility”, I believe Apple means that apps compiled today will continue to run in the future. As such, I don’t think binary compatibility precludes adding message passing.

Background

Objective-C began as a fusion of C and Smalltalk. It was an attempt to balance high-level, object-oriented features with the performance of a low-level language like C. C calls functions directly, as the compiler knows the exact address of the function to call. Objective-C added Smalltalk-style message passing, which determines the function to call at runtime. Each class contains a dictionary mapping selectors to function addresses. When an instance of a class is sent a message, it looks up the address that corresponds to the message’s selector, and calls the function at this address.

C++ classes, on the other hand, use vtables to handle dynamic dispatch. By the name, you might think a vtable is similar to the dictionary used in message passing, but vtables are usually just arrays of function pointers. The vtable is constructed at compile time, and functions are inserted into the vtable in the order they are declared. The compiler then translates message calls into vtable lookups. For example, someInstance->foo() might be translated to someInstance->vtable[3]().

Because the compiler must know how to translate a method call into the proper vtable index, the vtable must be known at compile time, and cannot change at runtime. In contrast, the message passing dictionary can be modified at runtime, and its possible to send messages to a class which are not in its message passing dictionary, though you may trigger a runtime exception by doing so. This allows you to send messages that aren’t known at compile-time, and enables some powerful metaprogramming features.

Swift utilizes all three approaches. Swift classes that inherit from Objective-C classes will use message passing. Other classes will use vtables, except in cases where the compiler has enough information to call the method directly. Since Cocoa is still written in Objective-C, apps written in Swift will use message passing a large percentage of the time.

Safety First

In all of the Swift presentations and documentation released at WWDC, Apple stressed that Swift is “safe by default”, unlike Objective-C. Since Objective-C was based on C, Objective-C objects were mostly just raw C pointers. This allowed programmers to write data corruption, crashing, and security bugs by not performing correct boundary checks. The most famous example of such a bug is the Heartbleed bug which compromised the security of most servers on the Web. When the swift compiler can detect an out-of-bounds access, it will throw an error. At runtime, an out-of-bounds access will trigger a crash. Swift references are even non-nullable by default, helping developers to avoid the billion-dollar mistake.

One of the ways that Swift practices this “safety-first” mentality is through type safety. Whereas Objective-C was a dynamically-typed language with optional static typing, Swift, like C++, is a statically-typed language with the ability to ignore type information when necessary. This is partially due to the history of Objective-C. In its infancy, the only object type was id. Partially due to this, selectors have never encoded type information, making it possible to send a message to an object with arguments of the incorrect type. This can open security holes, corrupt data, or cause a crash.

If Swift were to adopt message passing, it could do so in a way that encoded static type information into the message. This would be slower than Objective-C message passing, but it would probably be fast enough. Message passing is much slower than a C++ virtual method call, but message passing is rarely the bottleneck in Objective-C code, and when it is, there are optimizations like IMP caching that can be used to speed up critical code. Objective-C has powered rich, fast, and smooth interactions on iOS for alost 7 years, while Android still struggles with garbage-collection and JIT-induced slowdowns. There are certain areas (e.g. high-end games, signal processing) where message passing is truly too slow, but this is not the case for most apps.

Objective-What?

It has been argued that, since Swift and Objective-C have distinct advantages but are interoperable, it’s beneficial to have both languages. Microsoft’s strategy to have multiple languages which all support the Common Language Runtime means that developers can choose the right language for the job. But Apple is not Microsoft, and maintaining two mainstream languages is a lot of work. Apple is a company that thrives when it is focused on a few things, and it’s Apple’s culture to provide customers with one best product.

Objective-C isn’t going away any time soon, but if Swift succeeds in becoming the dominant language for iOS and Mac development, Objective-C will go away. Eventually the cost of maintaining two languages and runtimes will cease to be worth it. Like Carbon before it, Objective-C will be deprecated, leaving Swift as the only supported language for app development on Apple platforms. In general, I am loathe to argue that my way is the one true way to do things, but since Swift will eventually be the one way to do things, and since now is the time when large changes will be easiest to introduce, I feel compelled to argue that Swift should forswear all but message passing.

Mocks and Other Uses of Message Forwarding

Mock objects are extremely useful for test-driven development. In Objective-C, OCMock implements mock objects through a proxy object. OCMock’s partial mocks in particular are a great example of the power of message passing. A partial mock object stores the original object as a property. It then intercepts any messages sent to it, and either forwards the message to the original object or, if the method has been stubbed in the test, returns the stub value instead.

To achieve the same effect in Swift, the mock object would have to implement all of the methods of the original object. Each of these methods would have to test if they have been stubbed, and call the method on the original object. Because classes change over time, it’s important to always remember to update the mock object when the class is updated, which is tedious and time-consuming. This can be made better if the class designer designs a protocol that the original class and the mock class both implement. However, since extensions can add methods to classes, this effort is ultimately futile, as a class designer cannot preemptively add extension methods to the mock object.

Another approach would be to subclass the original object and override the stubbed methods. This is the approach that the Google C++ Mocking Framework takes. There are a few problems with this approach. First, subclassing has additional side effects. For example, in Swift, convenience initializers are only inherited if the subclass doesn’t have its own designated initializer. If we needed to stub the designated initializer, the convenience initializers would disappear from the mock object. This method also has the disadvantage that a separate subclass must be created for every combination of stubbed methods. In practice, this leads to a large number of mock subclasses, which are mostly just boilerplate code.

The bottom line is that languages with message forwarding implement partial mocks much more completely and cleanly than languages without. I’ve filed a radar asking for partial mock objects in Swift.

There are, however, many other uses of message forwarding. It’s been used to implement something akin to multiple inheritance. It’s been used to automatically implement a facade pattern for database access, that updates as the database schema does. It’s been used to implement a cancelable editing interface. Each of these things could be implemented without message forwarding, but they would require a lot of glue code that would need to be constantly updated. Message forwarding is a powerful tool that allows us to better decouple our classes and do more with less code.

Method Swizzling or: No Mac is an Island

Method Swizzling is an occasionally-used technique where two method implementations are swapped in the message passing dictionary. If you swizzled two methods, -method1 and -method2, sending a message for method1 would execute -method2, and vice-versa. This technique is not possible with vtables.

Method swizzling is most often used to patch bugs in frameworks, or for plug-ins, to inject code into the host application, without resorting to more dangerous techniques such as mach_override. Before the introduction of base localization, method swizzling was used by many Mac apps to automatically localize nibs. Of course, Apple engineers strongly discourage swizzling methods in Apple’s frameworks. Occasionally, it is required to work around a bug or add important functionality, but since Apple is continually fixing bugs and adding features, any solution is temporary. And since Apple strives to not break compatibility with older apps, swizzling their methods makes their job harder. But for frameworks that will never be updated, and for tight deadlines, sometimes swizzling can save a project.

On the Mac, swizzling is used to implement plug-in functionality, even for apps that don’t have a plug-in API. This adds features that are very important to power users. Many popular Photoshop and Xcode plug-ins use method swizzling. GPGMail, a plug-in for Mail.app which adds OpenPGP encryption to Mail.app, makes extensive use of method swizzling. This functionality is incredibly important to security-minded individuals, especially in the wake of the Snowden leaks. If Mail.app were rewritten in Swift, this would not be possible.

On iOS, sandboxing prevents this kind of thing, and on OS X, sandboxed apps are more difficult to inject code into. Maybe app extensions will eventually solve the problems that current plug-ins are addressing. Until then, however, the malleability of OS X apps allows plug-ins to be developed for them even when the apps’ authors never considered adding plug-in support. For this reason, message passing should be the default for all Swift apps on OS X. Even if an app’s author doesn’t think they need message passing, it automatically makes their app part of a vibrant third-party ecosystem, which power users consider an irreplaceable part of OS X.

Configuration

Message passing makes it easier to configure your app at runtime. This can help technically-minded designers work faster. One famous story from the development of the original Mac is the Calculator Construction Set. Steve Jobs was unhappy with the design of the Mac’s calculator, and kept rejecting changes made by the developer, Chris Espinosa. Chris eventually wrote an application that allowed Steve to configure the calculator at runtime, and Steve was quickly able to come up with a design he liked.

The ability to convert a string containing a method’s name to a method call is useful for anyone creating a similar tool, or a DSL, to customize the interface of an app. (Converting strings to classes is another issue beyond the scope of message passing.) Interface Builder can, at compile time, compile action strings to calls to Swift methods, but third party tools can’t easily do this. In addition, this kind of customization is useful in NSUserDefaults for changing how an app behaves.

I have filed a radar asking for the ability to convert a string to a method call.

Remote Methods

Objective-C’s Distributed Objects have problems, but the basic concept is useful in many situations. Sending a method name from a server to a client app to execute a method on client can make the server and client easy to code and maintain. There are obvious security implications, so the client must be careful of which methods it allows to be executed, but imagine writing an IRC client that converts IRC client commands to method names, and then calls those methods. Then, rather than keeping some kind of mapping from IRC commands to methods, you simply need to add a method for each IRC command you wish to process. This is not only less code, but can also run more quickly than a large if/else or a dictionary lookup.

Runtime Metaprogramming

Most importantly, message passing enables runtime metaprogramming, which while currently not used to solve most problems, is increasingly important to the future of programming. Over the past few decades, developers have programmed using increasingly higher-level abstractions. Since computers excel at automation, the ultimate goal is to tell a computer, in natural language, what a program should do. Metaprogramming is an important step in that direction, as it allows you to write declarative systems that respond to changes at runtime.

Metaprogramming allows you to do a lot of very cool things. In one noteworthy example, Steven G. Harms used metaprogramming to automatically conjugate all Latin verbs for all tenses, voices and moods. Typically, this would require writing thousands of methods, but his implementation only required 24 methods. For many problems, Swift requires fewer lines of code than Objective-C, but for a large number of very interesting problems, runtime metaprogramming allows Objective-C to be much more concise than Swift.

Swift is an opportunity to do metaprogramming right. Objective-C’s metaprogramming capabilities are a tad limited compared to languages like Ruby. By making a break from the Objective-C runtime, we have a chance to implement a more complete metaprogramming API. One of Swift’s major advantages over Objective-C is its REPL, which allows Swift playgrounds to compile and execute code while the application is running. If that functionality could be exposed to developers with an eval() function, along with functions to modify existing classes, Swift could be a premier language for metaprogramming.

I have filed a radar asking for an eval() function in Swift. There is also a radar asking for runtime introspection and reflection functions, which help enable runtime metaprogramming.

The Big Picture

In the NeXT days, Objective-C was marketed to developers as a way to develop faster in fewer lines of code. Swift achieves this better than Objective-C for some problems, but the lack of message passing makes Objective-C much better suited for other problems. Although direct and vtable dispatch allow many compiler optimizations that message passing does not, message passing is rarely a bottleneck in Mac and iOS apps, and when it is, there are optimizations that can be used to alleviate the problem. Furthermore, since most Swift code uses message passing to communicate with Apple’s Objective-C APIs, we already know that message passing is fast enough in Swift. As a language that will be primarily used to write apps for Apple’s ecosystem, Swift should strive for as much developer productivity as possible, while keeping performance acceptable, and implementing message passing across the board is a great way to achieve these goals.