The One With D News and Opinions of the Digital Mars D Programming Language

Q & A: Gregor Richards

Compared to my past Q&A subjects, Gregor Richards is a relative newcomer to D. But in the time he's been part of the D community, he has been quite active. He's the creator of DSSS, which has become one of the most popular D projects out there. He's a frequent contributer to the newsgroups, spends a great deal of time in #D, and recently gave a presentation at the D conference. And he has a thing for hats.

I first contacted Gregor about the Q&A a few days before the conference. He graciously accepted. I had answers to most of the questions almost as soon as I sent them to him. But one of the questions, the one about the future of DSSS, was left unanswered. Since this particular subject was going to be part of his presentation, he asked me to hold off on posting until after the conference was over. Now that it's done, he has sent me a revised set of answers, modified to reflect events since the first draft.

As usual, my questions are in bold and his answers in normal type.

What do you do to pay the bills?

I work at a company which I don't think I need to name in this Q&A. Suffice to say that my job has nothing to do with my life short of paying the bills. It's not that it's such a terrible job, it's just not something I've ever considered a particularly big part of my life.

Before you found D, what was your programming background? With which languages were
you most familiar and which was/were your preference?

Well, the answer to this question could be a bit loaded ... I'm quite interested in programming language design. As such, I know and use a fairly wide variety of languages. I was using C and C++ as static, compiled languages until D, and those projects that I originally wrote in C or C++ are probably not going to change very quickly, simply because changing a project's language isn't exactly an easy process.

Because I'm interested in programming language design, languages like Java did interest me, but I found myself balking at the fairly extreme limitations they imposed. My opinion of Java in particular is that somebody said "C++ is hard to use. Let's remove all the features so it's easy." Removing all the features is not a good way to design a language.

I know a lot of dynamic languages well enough that I could get into a project in one in a couple of days. That is to say, I don't know most of them well enough to just sit down and start hacking, but it wouldn't take me long to get there. It would be hard to list them, but suffice to say that if it's a popular, Free and Open Source dynamic language, I've probably written at least one program in it. I quite like Smalltalk, and part of why I wrote DSSS is to try to get some of the ease with which people share software in Smalltalk into D.

How and when did you first learn of D? What was it about the language that hooked you?

I think it was around September of 2006. I probably started really using it later.

It's hard to say what about the language hooked me. Coming from C++, probably the top two "hook" features were its much leaner object orientation and awesome dynamic array support.

Which of D's current features do you like the most? Are there any you dislike?

Actually, the dynamic array support has always been my favorite feature. It seems like a bizarre thing to be the real "killer feature" of a programming language, but the simple ease with which arrays can be used never ceases to put a smile on my face. When the videos of the conference are out, watch Kris' presentation and you'll understand.

Aside from that, there are a lot of features I love, but I'd say what I most love is the features D does not have. Walter gets tons of points from me for rejecting often-requested features which in reality would be terrible. Good languages aren't about quantity of features, but quality of features.

As GNU/Linux is my primary operating system, I'd say the worst feature of D right now is DMD. OK, that's not a feature per se, but I find myself spending ridiculous amounts of time in DSSS working around bugs in DMD. I don't think its ELF backend is particularly well-implemented. I get silly emails complaining that DSSS doesn't support .so files, and have to explain that DMD doesn't do ELF right, so the .so files it makes aren't happy with using symbols in the host. And that's only one of the silly problems I've had to work around. Probably the worst part of all of this is that there's a superior option, GDC, but many people don't use it because they'd rather bleed on the cutting edge than use a compiler better-suited to the platform. Furthermore, from the front-end source code of DMD that's available, it seems that Walter's theory of portability is not portability at all. It's "write code that works on only one platform, then to port it to another platform, write code that only works on that platform." As a POSIX guy, that disgusts me a bit.

Actually, that leads into the one feature of the D language itself I most dislike: No negative versions. Portability, to me and most POSIX people, is about writing code that works in most places and then putting exceptions in for where it doesn't. I remember Walter replying to a post about this on the newsgroup by saying that you shouldn't use version (!Windows), you should use version (linux), because positive versions are better. But you'll notice that version (!Windows) and version (linux) are not the same thing. They are to Walter, because his theory of portability is not portability.

Let me make it clear that I don't mean any disrespect against Walter with my comments about DMD. On Windows, DMD is a great compiler. The fact is that he clearly isn't used to GNU/Linux or ELF, and it's no easy task getting that working at all. But it doesn't work well.

Also, I'll put forth that there's a small irony in the fact that I wrote those statements about DMD before GDC 0.24 was released, and the very first thing I did with GDC 0.24 was find a hang bug. Whoops

Are there any features you feel are missing from the language, or that you think
would improve it if they were implemented?

When I first read and considered my response to this question, I had not yet seen Walter's presentation at the conference. The one feature I was really hoping for, seamless class extensions, were implemented in a way that's both properly static and sensible. So, I'm quite happy with D where it is, but haven't really had time to find my next nit yet either.

What was it that led you to create DSSS?

There are two reasons. First, I was learning D and noticed that every build system was incredibly limited and horrible. The primary method of installing libraries was (and still is to many people) to copy them into your import directory. Does that work? Sort of. Is it scalable? Not at all. Furthermore, if you have ten dependencies, it's just very annoying for somebody else building your software to go groping through the Internet looking for them. `dsss net deps`, now that's easy.

The second reason is more from my past in other languages. I've built a lot of software. A LOT of software. In fact, I worked as an intern at Intel automating builds of F/OSS software. What I've noticed is that a lot of build systems try to be extremely flexible, and by doing so, often make the actual process of building inconsistent. Flexibility is nice, but in my opinion consistency is far more important. For 95% of projects, this level of flexibility is totally unnecessary, and this flexibility often made the build systems difficult to use - this is true particularly of the autotools (autoconf, automake, libtool, pkg-config, the list goes on). To use autoconf, you need to learn a whole new language! They're so difficult, in fact, that some people just don't use them. In my opinion, the process of building your software should barely be a minor footnote in your life. It needs to be that easy.

Luckily, as D was (and is) fairly young, I felt that I had the unique opportunity of trying to get a good build system in before it became standardized on something horrible. So, I aimed to make a build system which, while less flexible, was so easy to use and so automatic that it's difficult to do wrong. I'm slowly increasing the level of flexibility at users' behest, but I often reject new features that would make things inconsistent. I am absolutely committed to maintaining DSSS builds as 100% consistent, and very easy.

What are your ultimate goals for the project? Are you hoping for DSSS to become the
preferred means of distributing D libraries and building D software, or do you see it
more as an alternative available to those who want it?

There's a chicken-and-egg problem with build systems, especially with integrated network features like DSSS. DSSS is most useful when all D software is available through it, but new D software often isn't made available through it because not all D software is available through it. Ow. Suffice to say that my ultimate goal is for DSSS to become the de facto standard - if it wasn't, its usefulness would be significantly degraded.

Luckily, thanks to the net feature's patching ability, it is actually possible to `dsss net install` a lot of D software that isn't maintained with DSSS. Derelict is probably the top example of this. However, I don't have the energy to maintain all of those builds, so it's more like a temporary shim until things are standardized.

I realize that it doesn't do everything for everyone, but I think if people gave it a chance, most would notice that it does everything they need with no fuss. And, although I'm reluctant to add features (like Walter), I will add them if I'm convinced that it's impossible or needlessly difficult to do so otherwise.

DSSS was originally built on top of Derek Parnell's Bud. You then developed your own
version, Rebuild, because of, as you say on the project wiki, "general unhappiness"
with Bud. What are the problems with Bud that Rebuild addresses?

Well, this is a fairly extensive list by now. At first, I wanted to use an unmodified bud (or 'build' at the time). But, the version that was out then didn't work on GDC. So, I made a branch and kept it in DSSS. It seemed like every release of bud was broken on at least one platform, and for whatever reason Derek was really reluctant to add patches for supporting shared object files (.so files, shared libraries), amongst other things. I don't believe any version of bud has ever supported building libraries properly on OS X out of the box. I fixed all those bugs, and tried to get things upstream where possible, but got very little response. My branch was turning into a fork. Finally, I decided it would be easier for me to write my own tool than to continue to hack bud into a working system - but I still didn't want to do it, as I didn't want to rock the boat. I talked a bit on #d, and it seemed like a lot of people would be very happy with a replacement for bud. So, I made one.

As per actual enhancements of rebuild with respect to bud, here's a list off the top of my head, which is most certainly not complete:

* Better platform support.
* Support for building libraries properly on all platforms.
* Support for shared libraries where possible.
* More broad support for configuration of version() flags and other configuration. To my recollection, bud has only a limited list of version() flags which are supported, and I don't recall whether these
can be added properly to the config files.
* -oq. I've found -op to be very obnoxious, since it spews files all over the place, and -od is of course problematic with multiple modules of the same name. -oq puts all the files in one directory, and names them fully.qualified.name.o (or fully-qualified-name.obj on Windows+DMD, since OPTLINK doesn't like dots in file names). This makes them easy to clean up and has the added bonus of making .a files (static libraries on POSIX) extractable.
* This isn't really fair, but: Generally fast email response and ticket turnaround. I'm fairly good about those kinds of things. That being said, if you have a real bug, post a ticket; I get so many emails about DSSS now that I occasionally miss a few.

Are there any new features to look forward to in the DSSS/Rebuild pipeline?

Tighter integration with DSource is one. I'd like for the net feature to be served by DSource and for a lot of DSSS' auxiliary tools (such as the networked source manager and testing framework) to be connected to DSource.

I'm also aiming for scripting support, and it's looking like the scripts are actually going to be in D. I have a fairly good plan for how that could work now.

Good library versioning support is one of those big issues that I've been struggling with since the beginning. It will definitely come into being, and I'd like to say I'll have it really ideal and seamless. But, it's a difficult problem.

Aside from your own projects, which other D projects do you make regular use of?

I use Tango, and in fact maintain the GDC port of it. I've made fair use of wxD, which is a really nice binding despite its small mindshare. Because DSSS by its very nature shouldn't have many dependencies, I haven't used many of the libraries available all that much. I have used a lot of them a bit in the process of making or helping make them DSSS compliant, and occasionally I make "look at this when I have time" notes to myself about them.

Are there any libraries or tools missing from the D community that you would like to
see? Any you or are working on, or thinking of working on, yourself?

From a purely self-serving point of view, I'd like to see an IDE with integrated DSSS support. I think that such a combination could work very well, and make life easier on end-users who don't use the same IDE. To this end, I've made a small graphical project manager for DSSS I intend to complete and release some time that bridges this gap a bit.

Of course, it's hard to guess what libraries or tools may be useful. Hopefully, somebody will think of one and make us all say, "Wow! What an excellent idea! Why didn't I think of that?" And hopefully, I'll be able to install it by typing `dsss net install coollib`.

A big thanks to Gregor for taking the time to answer these questions.

I mentioned in a previous post that I would be putting up two Q&As this month to make up for the lack of one in July. It doesn't look like that's going to happen, unfortunately. However, I've got plenty of time to turn September into a twofer, so look for the next Q&A in a couple of weeks (fingers crossed).

Technorati Tags: D Programming Language, DSSS, build tools