Source Code of the Day

From shutdown.c, part of Darwin's system_cmds project:

#ifdef __APPLE__

void log_and_exec_reboot_or_halt(void);

#else

void die_you_gravy_sucking_pig_dog(void);

#endif

Probably not the cause of our "WARNING: couldn't lock kext manager for reboot: (ipc/send) invalid destination port" problems, but still good to know.

More evidence that C++ is the work of the Devil

libstdc++ is the project providing Darwin's standard C++ library. (You can't criticise the naming, anyway.) The binary root (.tar.gz) is 36.7Mb. Decompressed, it's 196Mb. Thinned down to only its i386 version this falls to 176Mb. 5.6Mb of this is the libraries. Yes, that's almost 170Mb of headers and docs.

I've produced a special libstdcxx_libs root for use in the code Darwin installation I'm working on. You can thank me later.

Darwin Server

You probably don't want to read about today's debugging exploits — how I managed to track down a couple of un-implemented methods using the power of the printf(), how I finally twigged that I'd only need the two OS X frameworks to get gdb working in PureDarwinXmas, and how I eventually discovered that most of dscl's problems were caused by the memory PureFoundation was allocating for objects not being zeroed — so I though I'd just quickly tap out a few lines about where I see Darwin going.

(Of course, these won't include any of my super-secret plans. For those you'll have to wait a little while longer.)

I think you can guess from this post's title the direction I think that Darwin should be headed. It's not that I've got anything against it becoming a GUIed desktop system, it's just that I don't think there will be much demand for it. Remember that we already have a class-leading desktop OS built on top of Darwin. This leaves us in an awkward position: do we go down the Unix/Linux route, slap on the Gnome desktop and get lost in the crowd? Or do we try and emulate OS X — possibly by using code from Cocotron — with predictable sub-par results? Personally, I'm happy to let others decide. My interest lies elsewhere.

I think the idea of a version of Darwin which can be installed on (the way driver support is going, a narrow sub-set of) generic server PCs is a very compelling prospect, and may offer a feasible alternative to Linux in the web server space. A good number of web site designers and web app developers already use Macs as their primary machines. And while they run all the same software as your typical Linux machine (Apache, PHP, Ruby, Python, [insert your favourite web stack here]), I know from bitter experience that there are enough subtle gotchas to make deployment from OS X, at least on the first try — shall we say — interesting.

One question which is always hanging in the air in situations like this is "How does this mesh with Apple's plans?" — accompanied by the unspoken "And how quickly will they shut us down if they don't approve?" I won't go so far as to argue that a commoditised Darwin server would be good for Apple and therefore they should embrace it with open funding, but I will say that they shouldn't feel threatened by it. An open server built on Darwin is likely to appeal to sectors such as discount web hosting — the typical $9.99 per month virtual private servers on shared boxes type of arrangement — where XServes or even OS X Server under virtualisation were never going to be an option.

I think the PureDarwin boys are getting ready to put out another build, which will mark the next step on the road towards this bright future of Darwin on everything.

PureFoundation: Exceptions

(I've decided that this blog is going to become the unofficial PureFoundation development blog, in the hope that (a) it might attract a little interest — and who knows, maybe a contributor or two — to the project, and (b) I can get some of why I'm coding what and how I'm coding down before I forget about it.)

The last couple of days have been spent chasing down bugs in PureFoundation. The next couple of days will probably be spent in the same way. The number and stupidity of them is getting depressing, and the fact that I don't have access to a working version of gdb, or even a version of DTrace which will trace Objective-C method calls, doesn't help. Yes, I know that real men debug using printf()s, but this is getting ridiculous.

My current target is dscl. We've just got to the point where the DirectoryService daemon will run, so now we wouldn't mind using it to add a few users. (Yes, I know real men always run as root...) dscl is exactly the kind of project which spurred the creation of PureFoundation: it's both completely vital and written in Obj-C. Given the general paucity of admin tools available to us (and I'm really going to have to kick-start a Darwin Admin Tools project one of these days with the aim of cloning Apple's systemsetup and networksetup), I can't see PureDarwin getting anywhere without it.

My single aide was that I had access to the dscl source. Now, I honestly don't know if it compiles. I didn't try. darwinbuild has been enough of a frustration recently that I didn't even want to try. The moments of "why the hell am I wasting my time on this?" introspection are getting further apart again, but I'm still easily discouraged. If it had built I could have filled it full of more printf()s, which may have helped. Instead, all I could do was tag Foundation and see which library calls it made, following along as best I could with the source code. (My groovy new two monitor setup — using an old 15" CRT I found lying around — really helped here. If I'd had to do all this plus constantly shuffle windows about...)

The deep pessimism I've developed over the last 30+ years led me down a few blind alleys. Chiefly, the fact that dscl uses exceptions to propagate error conditions in exactly the way you aren't meant to, coupled with the fact I was sure logging into the local directory domain would fail, led me to believe that PureFoundation's exception handling was broken. Cue today's diversion.

Exception handling is one of those things which seems magical until you start digging into it. It's part language feature, part OS trickery. But it turns out that, at least in the default objc4 runtime version, it's fairly simple. Those @try and @catch blocks are converted into simple function calls by the compiler. These functions are defined in the file objc-exception.m, and basically just manipulates a linked-list of exception handlers and compiler-produced contexts. The comments on the code plainly state that this is place-holder code until Foundation starts up and provides its own, which is enlightening but ultimately unhelpful.

I've bored you enough without going into the details of the quick solution I hacked together. You can see for yourself by examining NSException.m. It uses thread-local storage to manage the chains of exception handlers, which I think is a neater and more elegant way of doing it, which also has the benefit of being thread-safe. Also added are both an ultimate default uncaught exception handler, and implementations of NSGetUncaughtExceptionHandler() and NSSetUncaughtExceptionHandler().

The punch line is that it turns out that no exceptions were being thrown by dscl, the local domain was being successfully accessed, and the first of many bus errors was in fact caused by my retarded double CFRelease()ing of a CFTimeZone type in my shiny new NSLog(). Which is why I really need an extra pair of eyeballs on the project. Any takers?

Extreme Open Source

I've been hacking (in the neckbeard sense of the word) Darwin for a couple of months solidly now, and have decided to christen the experience "Extreme Open Source". What's the difference between this and normal open source? It's the difference between "Sports" and "Extreme Sports". Basically, the difference can be summed-up as "why the hell would you want to do that when there are plenty of equally exciting but exponentially safer alternatives?" I think insanity plays a part.

It would be easy for me to type some stuff here about the bloody shocking state of the Darwin source code, but I've slowly come to the realisation that that isn't the problem. The source is, in fact, of quite excellent quality. The problem is it simply isn't designed to compile and run separately from the proprietary parts of OS X. And it's not just the Foundation-and-Objective-C using components, either. I've spent the past fortnight writing implementations of two function groups — CFNotificationCenter and CFFileDescriptor — which are absent from CFLite but relied upon by (the really rather vital) DirectoryService daemon.

darwinbuild — the chroot environment designed for compiling the separate projects — can be an exercise in frustration. When it works it's an absolute dream: packages are downloaded, dependencies met and code neatly assembled and arranged. But when it doesn't you're left pretty much helpless. I'm currently bashing my head (and not for the first time) against the fact that one particular tool (copystrings) is written in Ruby and yet there's no working Ruby install in the chroot and all my attempts to install one have failed. And without copystrings — which does exactly what its name suggests and so shouldn't be the most complicated application ever written — certain builds fail before they even really begin. In this case I'm trying to build a version of the DHCP-marshling IPConfiguration which doesn't rely on Apple's closed-source 80211 code. Working, pure-source networking is one broken script away.

So there you have Extreme Open Source: it's stressful, time consuming, and while you're working on it there's always this nagging suspicion at the back of your mind that as soon as you reach the finish line Apple's going to turn around and say, "Actually... I think we're going to close source all this stuff."