Ask a physicist and you’ll get the answer to a well-known riddle. “What’s Nu?,” it’s “c over lambda”. Physicists use the Greek letter nu to represent the frequency of light, which can be expressed as its speed c divided by its wavelength lambda.
Nu also is the name I chose for a project that relates a different C to a different lambda. Nu is a new programming language that binds the expressive power of Lisp to the pervasiveness and machine-level efficiency of C by building on the power and flexibility of Objective-C.
A major goal of Nu was to make it easier to build and reuse software components. Consistently with that, Nu was built from many preexisting components and many well-developed ideas. There’s not much that’s new about Nu except for its particular combination of recycled parts.
Nu is object-oriented and functional. It is written in Lisp-like S-expressions but conforms to no preexisting Lisp standard. Instead, Nu was designed to follow Yukihiro Matsumoto’s “Principle of Least Surprise” while staying tightly integrated with C via the Objective-C runtime and object model.
Objective-C is an object-oriented extension of C that provides powerful messaging primitives, deep runtime introspection, and dynamic binding, all important tools for bridging to dynamic languages. From that, Objective-C has been bridged to many dynamic languages: Python, Ruby, Perl, Scheme, and many others. It has also been used to build at least one completely new scripting language, Philippe Mougin’s F-Script.
Nu is written in Objective-C, and Nu is designed to allow deep interaction with Objective-C programs. As a result, Nu has deep access to its own implementation. That gives its users an unusual ability to explore and understand Nu. Where there is understanding, there are few surprises.
Inspired by Alexander Burger’s Pico Lisp, I decided to write Nu in early 2007. Previously I had been involved with two projects that bridged Ruby and Objective-C: first RubyCocoa, an open-source bridge created by Hisa Fujimoto and now actively supported by Apple, and then RubyObjC, a completely new bridge that I wrote to address reliability issues with RubyCocoa and to go deeper into the Objective-C runtime. In both projects, I was attempting to link the expressiveness of Ruby with the efficiency of C by building on Objective-C.
Why not Ruby?
Of all the existing dynamic languages, Ruby seems like the best choice to bridge to Objective-C. Both are children of C and Smalltalk. From Smalltalk they inherited very similar object models, and both languages are layered on C. That suggests a natural combination: Ruby for high-level programming and scripting and Objective-C for higher-performance C subsystems.
But because they developed independently, Ruby and Objective-C each has its own set of essential and supporting elements, and like cantankerous newlyweds, each prefers its own way of doing things. Here are some examples.
- Ruby and Objective-C have completely different and inconsistent syntaxes for message sending. Along with many other language bridges, the Ruby to Objective-C bridges force an awkward solution on Ruby programmers for the sake of predictability. This is a minor but annoying problem that conveniently disappears when we switch to a syntax of S-expressions.
- Ruby and Objective-C have completely different representations of standard data elements such as strings, numbers, arrays, hashes, and exceptions. Bridges must either automatically convert values when they cross the bridge (which is expensive) or try to teach the types of one language to masquerade as the types of the other. Deceptively, this sometimes works, but it is nearly impossible to fully hide this problem.
- Ruby and Objective-C have large and overlapping libraries. It’s tempting to believe that combining them provides the best of both, but in practice a developer usually has to favor one set of libraries over another, and like the standard elements, library objects such as files, dates, and ranges must be converted whenever they cross the bridge.
- Ruby and Objective-C store objects differently. As a result, objects with data in both languages (such as instances of Ruby classes derived from Objective-C classes) must be stored in two places and both parts must be correctly maintained.
- Ruby and Objective-C have completely different memory management schemes. Ruby is garbage collected and Objective-C is typically reference counted. An important challenge for a Ruby bridge is to make sure that the Ruby garbage collector never deletes objects that are needed by Objective-C instances. Mistakes in memory management usually lead to sudden and hard-to-debug crashes.
- The Ruby and Objective-C threading models are different and completely incompatible. In a multithreaded application, it is impossible to use Ruby in any thread but the main application thread, and even this requires a patch to the Ruby language implementation. With multicore systems all around us, this is simply unacceptable.
- Ruby and Objective-C libraries occasionally use the same names to represent different methods. To resolve these conflicts, one side (usually the Ruby side) must convert any conflicted names into something different, typically by adding a prefix. This is workable but can lead to surprising errors.
A good bridge must correctly and reliably address all of these issues, but no bridge can hide everything. For mixed language programming, these inconsistencies are a fact of life, and this is for a well-matched pair of languages. The problems of binding Objective-C to any other scripting language are worse.
Nu is my solution to these problems. Instead of grafting two mature and overlapping language implementations together, I wrote Nu on, with, and for Objective-C. Instead of being problems to be bridged, the rich set of Objective-C classes became the building blocks of Nu.
The Objective-C object model is the Nu object model. Nu directly uses the Objective-C runtime to store its class descriptions. Nu code can be used to create new classes and to add class methods, instance methods, and instance variables to existing classes. Like Objective-C, Nu uses Smalltalk-style message sending, and like in Smalltalk, everything in Nu is an object. Standard Objective-C types such as NSString, NSNumber, NSArray, NSDictionary, and NSException are used throughout, and Nu provides some new classes and extensions to existing classes to make it easier to program in both Nu and Objective-C.
Nu is interpreted; the interpreter is built as a framework that can be imported into any Objective-C application. The Nu framework includes a built-in console that can be easily activated to allow developers to directly interact with their Cocoa applications using Nu.
Nu also borrows some tricks from Ruby that make its code more readable and concise. Here documents can be used to declare multiline string constants. Strings can contain interpolated Nu expressions. Names that begin with “$” have global scope; names beginning with “@” denote instance variables. For experienced Lisp programmers, the “__” prefix is used to declare generated symbols, which are important tools for safe programming with macros.
I’m still working on Nu, but I’ve already used it to build a few things, including the program that serves and manages this blog. In future posts I’ll describe that in more detail and share more about Nu.
So that’s Nu. A new language that’s not so new. It’s just an evolutionary mix of recycled ideas, but what else are revolutions made of?
This sounds very interesting. Is there any sample code or a release yet?
Ezra Zygmuntowicz — August 12, 2007
Second that…even if you’re not ready for a release, it’d be cool to see what it looks like!
joe — August 12, 2007
Sounds interesting. Wish I had more time to play with stuff like this!
Brendan Rankin — August 12, 2007
I have a couple of procedural things to work out first, but it’s coming. It’s been kind of crazy getting this ready in time for C4, Jonathan Rentzsch’s Mac developer conference where I announced this yesterday. It won’t be long, though.
Tim — August 12, 2007
Examples ! pleaaaase…
Lionel Barret — August 12, 2007
I am also looking forward trying it :-) Congratz for your presentation, it was apparently a success!
Laurent — August 12, 2007
You may have a winning concept, here. I suppose a pretty accurate way to characterize Nu would be to say it is pretty similar to F-Script, but Lispy instead of Smalltalk-like, right?
Are you making any attempt, btw, at reconciliating NS* class and method identifier names with the hyphen-separated lowercase name convention prevalent in Lisp? (I would hope so, as merely wholesale adopting the NS- prefix and camelCaseWording would be pretty sure to turn off Lispers.)
Arto Bendiken — August 13, 2007
s-expressions are an important element of Nu. So if that’s what it means to be “Lispy”, then yes.
I don’t think that Nu will be interesting to experienced Lispers. But Lispers already have lots of Lisps, and if people want to say that Nu isn’t Lisp, I’m fine with that. Others have described Nu as “Ruby with parentheses”, but I think it’s best to just call it Nu. As I described it at C4, I wrote it to be a glue language for Objective-C, so I think Nu should mix well with Objective-C code and use similar naming conventions. Anyone who prefers existing Lisp conventions and idioms should probably stay with whatever he or she is currently using.
For my purposes, I wanted a structured way to range from a very high level language (with s-expressions) down to C, and for me, that’s Nu.
Tim — August 13, 2007
I’ve wanted a higher-level language integrated with the Objective-C runtime for yonks. F-Script is nice, but has the significant shortcoming that you can’t create classes in it. One thing I’m wondering about is combining Nu and Objective-C in application implementations. My ideal would be to be able to start a new project in Xcode (with an appropriate template) and mix Objective-C and Nu implementation files. Is this likely to happen? If not, would it be a simple matter of including the Nu files as application resources and loading them at launch time, or are there likely to be additional complications? While I’m asking dumb questions, you mention ref counting vs. garbage collection. Is it safe to assume you’re writing this with an eye to working with GC in Leopard? I’ve been yearning for a compiled Smalltalkish language with full ObjC runtime integration for a while now, but an interpreted Lispish one might scratch my itch too. :-)
Ahruman — August 14, 2007
Share? As a lisper, I’m interested to see how you combined the idioms.
Ahruman, have you seen Ambrai Smalltalk?
Thom — August 14, 2007
That list of liabilities in bridging Ruby is hard-won and valuable (and your documentation efforts on RubyCocoa have been much appreciated as I’ve been picking it up the last couple of weeks). The thread limitation comes as a big surprise.
Would one be correct in assuming that RubyObjC is not slated for a great deal of future attention?
Nat — August 15, 2007
I’m not a fan of Xcode, preferring to edit in TextMate and organize my builds with Rake (and a Nu equivalent that I called “nuke”). I suppose if you are willing to limit yourself to desktop Cocoa apps, Xcode is fine, but to me it has too much of a “we’re Apple, trust us” way of overspecifying tasks and constraining developers. On the other hand, a general-purpose build tool like Rake can be used to do everything from building your desktop app to deploying the web site that you use to sell it – and in a way that is easy to reuse from project to project. But all that said, an Xcode project for a Nu app would have the same structure as one for a RubyCocoa app. Your .nu files would go in your bundle’s resource directory (that’s what my nuke task does now) and you’d need a custom main.m to load those .nu files when your application starts. All that can be easily managed with Xcode, and although I don’t expect to use Xcode for builds, I’d be willing to merge contributed Xcode project templates into the Nu project.
Right now all memory management in Nu uses reference counting, but it’s nice that at least theoretically, Apple’s got a garbage collector that we can use when Leopard is released. I haven’t tried a GC’d Nu, but understand that a compiler switch is all that it takes to convert. But I think the catch is that every framework you use has to use GC. I’m going to wait and see if Apple can make it work for their own products first.
Tim — August 15, 2007
This sounds wonderful … except for the S-expressions. I’ve used a whole lot of languages, including LISP as well as others as syntactically diverse as Smalltalk-80, FORTH and Pascal; and LISP is my second least-favorite syntax.
The rigid prefix (Polish) notation, and the authoritarian nesting of parentheses, are both very unpleasant for me to write and to read. The appeal of S-expressions appears to be that the language syntax has a trivial mapping to the data structures the program works on, which means code and data are interchangable, which in turn makes tricks like LISP macros and generating code on the fly easy. But I don’t want to directly work with a syntax that’s so mechanical.
I realize some people love LISP, and that of course you’re not going to change your mind in midstream … I just wanted to vent.
(Regarding GC: In Leopard, all the system frameworks support GC … so compatibility is only an issue if you’re using 3rd party frameworks. And in many cases refcounted code works fine in a GC environment; retain and release just turn into no-ops.)
Jens Alfke — August 15, 2007
Thanks for your comments. I agree that s-expressions are a matter of preference.
For me, the upside of macros and easy code manipulation outweighs the downside of syntactic awkwardness. Actually, I think most of the time, the syntax isn’t awkward, but in certain situations (like complex algebraic expressions) it can be a pain. But I’d be writing those expressions in Objective-C for performance reasons.
If you decide to write a glue language that satisfies your own syntactic preferences, you can still get a lot of leverage from the Objective-C libraries, and if you build it on the Objective-C runtime, then our languages will interoperate. :-)
Tim — August 15, 2007
I have been migrating my projects from RubyObjC to Nu and haven’t run into any serious problems yet. Based on the list that you mentioned, I’ve decided that Nu is a better tool for the things I want to do, which generally include writing significant amounts of Objective-C. RubyCocoa or RubyObjC might be good choices for you if your primary language is Ruby, but you’ll still have to deal with the issues I listed.
You’re right, I don’t have any plans to add to RubyObjC. But I’d be glad to add new contributors and maintainers.
Tim — August 15, 2007
Simple, “nu” is “now” in Swedish ;-)
Magnus — August 15, 2007
On the natural language meaning of “nu”, it means “now” in Dutch too, but “naked” in French. Nice twist.
luc dubois — August 15, 2007
I wonder if you shouldn’t have called it “nil” instead. As in “Nil Is not Lisp” :-)
alastair — August 15, 2007
Tim, I asked about Xcode integration partly because it’s what I’d personally like, but also because I think it’ll be much more widely adopted if people can use it easily with existing tools and workflows. Learning a new language is obviously less work than learning a new language and new tools. Of course, how much this matters depends on how much you want to take over the world. ;-)
Ahruman — August 15, 2007
Re. syntax: it’s a question of the technical benefits of S-expressions vs. the marketing advantages of C/Pascal-like syntaxes.
However, I don’t think it has to be an either-or choice: you could have the best of both worlds by 1. standardising on S-expressions as the default Nu syntax, and 2. encouraging other developers to develop cross-compilers that translate scripts from [their syntax of choice] to standard Nu S-exps.
(It always surprises me that more languages don’t take this approach, considering all the benefits of modularity in other areas of software design.)
has — August 16, 2007
Awesome! I was going back and forth until you said the build tool is called ‘nuke.’ Can’t wait to try this out.
Chris — August 16, 2007
I think this is exciting. And the approach seems sane–build new from old that works. I like this approach because I won’t have to go through the contortions that I must using two languages together. Count me in when you release Nu. Na-Nu, Na-Nu!
Excellent — August 16, 2007
What kind of license are you thinking about this, is this a commercial product? Best regards.
Steven — August 17, 2007
It’s not final yet, but will probably be similar to the Ruby license. But I’m tempted to make it viral, at least for BigCos; I’m getting tired of tripping over bugs in (closed-source) Cocoa.
Tim — August 17, 2007
For the sake of the children, I hope you don’t make the license viral and instead stick with the tried-and-true Ruby license. There are several reasons why: First, we have enough OSS licenses for lawyers to review as it is - proliferation has to stop somewhere. Second, you’re hardly going to pry open Cocoa or any other closed source software by introducing a viral license into the ecosystem. The presence of such things merely trigger quarantine measures, not increased openness (in other words, this generally leads to the exact opposite effect of the one you’re trying to achieve).
jkh — August 21, 2007
jkh: Noted. Thanks.
Tim — August 21, 2007
What’s “authoritarian” about s-expressions? You might as well speak of C’s “authoritarian” operator precedence levels.
Sunnan — August 21, 2007
I need a scripting language for the iphone.
Please remember to include security concepts in the software.
email@example.com — May 25, 2009