Scott James Remnant

Now that edgy has been frozen for the beta release in a week’s time, and the next developer summit, where the plans for the next release will be discussed, has been announced, I think it’s the perfect time to start thinking about the kinds of features we want to see in that release.

At our recent sprint in Wiesbaden, Mark reminded us that when Ubuntu first released we were leading the field in many of the components we installed by default. Many users came to us because we provided Linux 2.6, hotplug/udev and Project Utopia by default; along with the latest GNOME release and development framework. Now everybody does that, butwe led the way.

With the dapper release, we produced something that could be supported for a long time. With edgy we’ve scratched our itches and fixed things under the hood that have been causing us problems. edgy+1 seems a good time to integrate the hottest new technologies and lead the way again.

So what do I think is exciting, and what do I want in edgy+1?

telepathy, farsight and galago

Anybody who has used the 2006 software release for the Nokia 770 Internet Tablet will have seen these in action already.

They’re, in my opinion, some of the most exciting projects currently being worked on; and ripe for integration into Ubuntu.

Telepathy is a communication framework built on DBus that allows any application to establish communication with users all around the world, using any protocol from IRC to MSN to VoIP.

It uses Farsight to provide the codecs and protocol support on top of the GStreamer media framework.

And Galago glues it all together; providing presence information about contacts and allowing users to specify their own status.

This means that you’ll be able to contact your contacts. Your calendar alarm tells you that it’s a colleague’s birthday, you’d be able to click their name and see a selection of different ways that you can contact that person right now. From e-mail, opening an MSN or Jabber window or even establishing a VoIP or SIP call to them.

The most exciting thing is that because every application on the desktop has easy access to this, people are coming up with fantastic ideas for using it!

zeroconf

By this, I mean zero configuration networking. Something the OLPC project appears to be going to do very well indeed.

Where no existing network infrastructure exists, your computer should still be able to communicate with other nearby computers. It should use ad-hoc wireless networks, link-local IP addresses and server-free DNS resolution with libnss-mdns.

Play games with your friends on the train without touching a button. Chat with other people in a lecture hall or at a conference where the existing network is unavailable. Hassle-free home and office networking without any complicated set up.

Networking that “just works”.

avahi

Whether your computer is on an infrastructure or ad-hoc network, it should be able to locate resources on the network and share its own to other people.

Avahi allows applications to, via DBus, do this by using a mulicast form of DNS; compatible with Apple’s Bonjour protocol.

The immediately obvious benefits of this are that one can locate such things as file servers without any complicated directory; more fun things can be done too, such as sharing your music from RhythmBox with other users on the network.

Combine it with telepathy, etc. and you can establish communication with anyone else, with a wide range of different protocols.

The immediate problem here is that we currently have a “no open ports” policy, which if we’re going to give the users the opportunity to establish ad-hoc networks is almost certainly a very good thing. We already have exceptions for that policy for DNS and DHCP, it may be that Avahi (mDNS) joins that list. Another option is a “mobile-phone like” interface for enabling and disabling discovery and sharing.

Bluetooth

I don’t think we should stop discovery and sharing at 802.* networking either. Linux actually has a pretty decent bluetooth stack, it’s something that we should start taking advantage of.

Your computer should be able to discover paired or known bluetooth devices when in range without intervention and communicate with them.

If I attempt to establish a network connection, and my phone is nearby, that should be offered as a method for getting on the Internet. If possible, it should also be offered through telepathy as a method for contacting people in my address book!

Synchronisation

These technologies together give us a really big possibility, automatic synchronisation of user’s information.

When I bring my laptop home, it should automatically update my desktop with any changes I’ve made to my address book or calendar, and any shared files, etc.

Likewise it should be automatically be updated with any changes made on the network, to a company address book on a server, for example.

And why stop there? Don’t use share your music, synchronise it so that your laptop, desktop and iPod have the same selection available at all times.

Hassle-free backups is an obvious win here.

But we’re stopping again. Why is my mobile phone, and PDA excluded from this? Via bluetooth, my address book and calendar should be automatically updated from these and these should be automatically updated themselves.

If I add a new number to my phone’s memory, it should appear in my computer address book; and my computer should offer me the ability to contact that person (using VoIP, for example).

It’s been over a year now since my last proper upload to Debian, and nine months since I announced my intent to put aside working on Debian for a while.

With Matthew Garrett’s resignation from Debian, several people have compared it to my own “resignation” from Debian. It has got my thinking about whether I currently intend to ever end my “Sabbatical” and return as an active Debian Developer.

I think that the end of my love-affair with Debian started at Debconf last year where several developers treated those of us who also worked on Ubuntu quite rudely. Someone was attacked for wearing an Ubuntu t-shirt at the conference, while someone else was applauded for wearing a “Fuck Ubuntu” t-shirt. That’s where I realised that maybe I didn’t have as much in common with these people as I thought I did.

I still don’t understand why Debian singles Ubuntu out for this kind of treatment, we’re still the only derviative distribution that makes all of our patches to Debian available, yet Ubuntu is claimed to not do anything at all.

Another example is that Ubuntu is being asked to change the Maintainer field of every package, something no other derivative is being asked to do or has ever been asked to do.

Martin Krafft has had some interesting things to say about this strange relationship in the past.

If that was the start of my falling out with Debian, I think that Debian considering removing documentation and firmware from the distribution, especially the documentation, was another point I started wondering whether I shared anything in common with the project anymore.

Call me strange, but I think that one of the fundamental purposes of a Linux distribution is to be useful to its users. If nobody can use the distribution because it doesn’t support their hardware, and even if it did, all the documentation has been stripped out; I started to wonder what it’s aims are. It became increasingly apparent that the only users Debian was considering a priority were its own developer.

And the third thing is simply a matter of Fun.

Fun for me, at the moment at least, has been to build a system that fits together extremely nicely with each component doing the right thing. For Ubuntu this has meant being a driving force for it being Linux 2.6 only, with reliance on udev for hardware, etc. Upstart is just a continuation of these goals, getting a system which all “just works”, even if it means throwing out a few things people were previously fond of.

All of these things would have been impossible to do in Debian itself. Getting upstart installable required changes to twelve different packages, including sysvinit itself; at a worst case, this would have required the agreement of twelve different maintainers in Debian. It’s often exhausting just persuading one of the reason for the change, persuading a dozen would have been a herculian effort.

Perhaps Matthew is right, what Debian lacks is a single leadership. There’s nobody in Debian I could have gone to for approval over the changes I wanted to make, whose decision developers cannot overrule. Ubuntu has such people (the Technical Board and sabdfl), which gives the project an obvious direction instead of a couple of hundred people all pulling in different directions.

In a way, I don’t feel that I’ve left Debian. I feel like I was happily going along, only to realise the mob had gone in a different direction and with no easy way of rejoining the group.

Today I reached another milestone in the development of upstart, the packages in universe can now replace the existing sysvinit package.

Before trying this, make sure your installation is up to date as we’ve had to split out some parts of sysvinit into a new sysvutils package. If you’re up to date, and want to try it out, install the upstart and upstart-compat-sysv packages from universe.

Note that the first reboot after you’ve installed the packages (from sysvinit to upstart) will be a little tricky … use reboot -f.

If your system boots and shuts down normally, everything’s working just fine. Note that both will be somewhat more quiet than you’re used to, unless you have usplash running.

Throughout the rest of this entry, I’ll try to answer some of the questions and comments that I’ve received since the last post.

Events

As I talked about previously, upstart is an event-based init daemon. Events are the primary force for having your services and tasks started and stopped at the appropriate time and in the appropriate order.

So what are events and where do they come from? (Note that this part is under development, so may change in later releases).

Events are just simple strings that may be sent by any process when something it is tracking the state of changes. They have no state or longevity, and if, when queued, they do not cause any job state changes, then they have no effect unless they are sent again.

Jobs can list which events cause them to be started if they are not already running and which events cause them to be stopped if they are running. Multiple start and stop events may be listed, in which case the first to occur changes the job until the next one occurs.

upstart itself generates the following system events:

• “startup”, on system boot.
• “shutdown”, when the system is about to be shut down.
• “stalled”, when there are no jobs running and no events in the queue.

The shutdown tool included in the package also causes one of the following events to be sent once the “shutdown” event has been handled:

• “reboot”,
• “halt”,
• “poweroff”,
• “maintenance” (aka. going into “single user” mode),
• any user-defined event with shutdown -e event.

Jobs also generate events whenever they change state, this is the primary source of events for ordering:

• “jobname/start”, when the job is first started.
• “jobname/started”, once the job is running.
• “jobname/stop”, when the job is first stopped.
• “jobname/stopped”, once the job has stopped.
• “jobname”, for services this is generated once it is running; for tasks this is generated once it has finished.

And as mentioned, any other process on the system may send events through the control socket or just by using initctl trigger EVENT. For now this is just the event string, however it’s intended that the event may include other details including environment variables and even file descriptors.

Typical example

To clarify how it all hangs together, here’s an example (using fictional names) of how the tasks and events can be arranged to provide race-free mounting of filesystems.

• “udev” service started on the “startup” event.
• udev daemon is configured to send a “new-block-device” event whenever a new block device is added to the system.
• “checkfs” task is started on the “new-block-device” event to check the filesystem.
• “mountfs” task is started when the “checkfs” task has finished and mounts the filesystem if listed in /etc/fstab
• “filesystem-mounted” event is generated whenever a filesystem is mounted.
• “fstab” task is started on the “filesystem-mounted” event, it checks the list of mounted filesystems against /etc/fstab and if all are mounted, generates the “writable-filesystem” event.
• other services and tasks would be started on the “writable-filesystem” event.

By breaking this job into these small tasks, we can see how the pieces fit together. Because everything is now done on events, there are no race conditions; we know that any filesystem listed in /etc/fstab will be checked and mounted.

The only reason they wouldn’t be is if there’s an error of some kind, and that means you have larger problems anyway and the system administrator would have a shell to fix it. Of course, the moment they finish checking the filesystem and mount it, the boot process would carry on.

There’s no reason that any of these events need to be generated by the upstart daemon itself, it can receive them from any other daemon on the system such as udev, acpid, etc. This keeps the focus of the init daemon narrow.

A large part of the future development will be working out exactly what kinds of events we want init itself to generate, what kinds we want to come from elsewhere, and what the contents of an event can be.

Getting Involved

If you want to get involved with trying to nudge the direction of upstart development, you can join the upstart-devel mailing list at http://lists.netsplit.com/.

Or if you just want to grab the source code, tarballs are published at http://people.ubuntu.com/\~scott/software/upstart/ and the bzr archive is at http://bazaar.launchpad.net/\~keybuk/upstart/main