The One With D

Q & A: Gregor Richards

Compared to my past Q&A subjects, Gregor Richards is a relative newcomer to D. But in the time he’s been part of the D community, he has been quite active. He’s the creator of DSSS, which has become one of the most popular D projects out there. He’s a frequent contributer to the newsgroups, spends a great deal of time in #D, and recently gave a presentation at the D conference. And he has a thing for hats.

I first contacted Gregor about the Q&A a few days before the conference. He graciously accepted. I had answers to most of the questions almost as soon as I sent them to him. But one of the questions, the one about the future of DSSS, was left unanswered. Since this particular subject was going to be part of his presentation, he asked me to hold off on posting until after the conference was over. Now that it’s done, he has sent me a revised set of answers, modified to reflect events since the first draft.

As usual, my questions are in bold and his answers in normal type.

What do you do to pay the bills?

I work at a company which I don’t think I need to name in this Q&A. Suffice to say that my job has nothing to do with my life short of paying the bills. It’s not that it’s such a terrible job, it’s just not something I’ve ever considered a particularly big part of my life.

Before you found D, what was your programming background? With which languages were
you most familiar and which was/were your preference?

Well, the answer to this question could be a bit loaded … I’m quite interested in programming language design. As such, I know and use a fairly wide variety of languages. I was using C and C++ as static, compiled languages until D, and those projects that I originally wrote in C or C++ are probably not going to change very quickly, simply because changing a project’s language isn’t exactly an easy process.

Because I’m interested in programming language design, languages like Java did interest me, but I found myself balking at the fairly extreme limitations they imposed. My opinion of Java in particular is that somebody said “C++ is hard to use. Let’s remove all the features so it’s easy.” Removing all the features is not a good way to design a language.

I know a lot of dynamic languages well enough that I could get into a project in one in a couple of days. That is to say, I don’t know most of them well enough to just sit down and start hacking, but it wouldn’t take me long to get there. It would be hard to list them, but suffice to say that if it’s a popular, Free and Open Source dynamic language, I’ve probably written at least one program in it. I quite like Smalltalk, and part of why I wrote DSSS is to try to get some of the ease with which people share software in Smalltalk into D.

How and when did you first learn of D? What was it about the language that hooked you?

I think it was around September of 2006. I probably started really using it later.

It’s hard to say what about the language hooked me. Coming from C++, probably the top two “hook” features were its much leaner object orientation and awesome dynamic array support.

Which of D’s current features do you like the most? Are there any you dislike?

Actually, the dynamic array support has always been my favorite feature. It seems like a bizarre thing to be the real “killer feature” of a programming language, but the simple ease with which arrays can be used never ceases to put a smile on my face. When the videos of the conference are out, watch Kris’ presentation and you’ll understand.

Aside from that, there are a lot of features I love, but I’d say what I most love is the features D does not have. Walter gets tons of points from me for rejecting often-requested features which in reality would be terrible. Good languages aren’t about quantity of features, but quality of features.

As GNU/Linux is my primary operating system, I’d say the worst feature of D right now is DMD. OK, that’s not a feature per se, but I find myself spending ridiculous amounts of time in DSSS working around bugs in DMD. I don’t think its ELF backend is particularly well-implemented. I get silly emails complaining that DSSS doesn’t support .so files, and have to explain that DMD doesn’t do ELF right, so the .so files it makes aren’t happy with using symbols in the host. And that’s only one of the silly problems I’ve had to work around. Probably the worst part of all of this is that there’s a superior option, GDC, but many people don’t use it because they’d rather bleed on the cutting edge than use a compiler better-suited to the platform. Furthermore, from the front-end source code of DMD that’s available, it seems that Walter’s theory of portability is not portability at all. It’s “write code that works on only one platform, then to port it to another platform, write code that only works on that platform.” As a POSIX guy, that disgusts me a bit.

Actually, that leads into the one feature of the D language itself I most dislike: No negative versions. Portability, to me and most POSIX people, is about writing code that works in most places and then putting exceptions in for where it doesn’t. I remember Walter replying to a post about this on the newsgroup by saying that you shouldn’t use version (!Windows), you should use version (linux), because positive versions are better. But you’ll notice that version (!Windows) and version (linux) are not the same thing. They are to Walter, because his theory of portability is not portability.

Let me make it clear that I don’t mean any disrespect against Walter with my comments about DMD. On Windows, DMD is a great compiler. The fact is that he clearly isn’t used to GNU/Linux or ELF, and it’s no easy task getting that working at all. But it doesn’t work well.

Also, I’ll put forth that there’s a small irony in the fact that I wrote those statements about DMD before GDC 0.24 was released, and the very first thing I did with GDC 0.24 was find a hang bug. Whoops

Are there any features you feel are missing from the language, or that you think
would improve it if they were implemented?

When I first read and considered my response to this question, I had not yet seen Walter’s presentation at the conference. The one feature I was really hoping for, seamless class extensions, were implemented in a way that’s both properly static and sensible. So, I’m quite happy with D where it is, but haven’t really had time to find my next nit yet either.

What was it that led you to create DSSS?

There are two reasons. First, I was learning D and noticed that every build system was incredibly limited and horrible. The primary method of installing libraries was (and still is to many people) to copy them into your import directory. Does that work? Sort of. Is it scalable? Not at all. Furthermore, if you have ten dependencies, it’s just very annoying for somebody else building your software to go groping through the Internet looking for them. `dsss net deps`, now that’s easy.

The second reason is more from my past in other languages. I’ve built a lot of software. A LOT of software. In fact, I worked as an intern at Intel automating builds of F/OSS software. What I’ve noticed is that a lot of build systems try to be extremely flexible, and by doing so, often make the actual process of building inconsistent. Flexibility is nice, but in my opinion consistency is far more important. For 95% of projects, this level of flexibility is totally unnecessary, and this flexibility often made the build systems difficult to use - this is true particularly of the autotools (autoconf, automake, libtool, pkg-config, the list goes on). To use autoconf, you need to learn a whole new language! They’re so difficult, in fact, that some people just don’t use them. In my opinion, the process of building your software should barely be a minor footnote in your life. It needs to be that easy.

Luckily, as D was (and is) fairly young, I felt that I had the unique opportunity of trying to get a good build system in before it became standardized on something horrible. So, I aimed to make a build system which, while less flexible, was so easy to use and so automatic that it’s difficult to do wrong. I’m slowly increasing the level of flexibility at users’ behest, but I often reject new features that would make things inconsistent. I am absolutely committed to maintaining DSSS builds as 100% consistent, and very easy.

What are your ultimate goals for the project? Are you hoping for DSSS to become the
preferred means of distributing D libraries and building D software, or do you see it
more as an alternative available to those who want it?

There’s a chicken-and-egg problem with build systems, especially with integrated network features like DSSS. DSSS is most useful when all D software is available through it, but new D software often isn’t made available through it because not all D software is available through it. Ow. Suffice to say that my ultimate goal is for DSSS to become the de facto standard - if it wasn’t, its usefulness would be significantly degraded.

Luckily, thanks to the net feature’s patching ability, it is actually possible to `dsss net install` a lot of D software that isn’t maintained with DSSS. Derelict is probably the top example of this. However, I don’t have the energy to maintain all of those builds, so it’s more like a temporary shim until things are standardized.

I realize that it doesn’t do everything for everyone, but I think if people gave it a chance, most would notice that it does everything they need with no fuss. And, although I’m reluctant to add features (like Walter), I will add them if I’m convinced that it’s impossible or needlessly difficult to do so otherwise.

DSSS was originally built on top of Derek Parnell’s Bud. You then developed your own
version, Rebuild, because of, as you say on the project wiki, “general unhappiness”
with Bud. What are the problems with Bud that Rebuild addresses?

Well, this is a fairly extensive list by now. At first, I wanted to use an unmodified bud (or ‘build’ at the time). But, the version that was out then didn’t work on GDC. So, I made a branch and kept it in DSSS. It seemed like every release of bud was broken on at least one platform, and for whatever reason Derek was really reluctant to add patches for supporting shared object files (.so files, shared libraries), amongst other things. I don’t believe any version of bud has ever supported building libraries properly on OS X out of the box. I fixed all those bugs, and tried to get things upstream where possible, but got very little response. My branch was turning into a fork. Finally, I decided it would be easier for me to write my own tool than to continue to hack bud into a working system - but I still didn’t want to do it, as I didn’t want to rock the boat. I talked a bit on #d, and it seemed like a lot of people would be very happy with a replacement for bud. So, I made one.

As per actual enhancements of rebuild with respect to bud, here’s a list off the top of my head, which is most certainly not complete:

* Better platform support.
* Support for building libraries properly on all platforms.
* Support for shared libraries where possible.
* More broad support for configuration of version() flags and other configuration. To my recollection, bud has only a limited list of version() flags which are supported, and I don’t recall whether these
can be added properly to the config files.
* -oq. I’ve found -op to be very obnoxious, since it spews files all over the place, and -od is of course problematic with multiple modules of the same name. -oq puts all the files in one directory, and names them fully.qualified.name.o (or fully-qualified-name.obj on Windows+DMD, since OPTLINK doesn’t like dots in file names). This makes them easy to clean up and has the added bonus of making .a files (static libraries on POSIX) extractable.
* This isn’t really fair, but: Generally fast email response and ticket turnaround. I’m fairly good about those kinds of things. That being said, if you have a real bug, post a ticket; I get so many emails about DSSS now that I occasionally miss a few.

Are there any new features to look forward to in the DSSS/Rebuild pipeline?

Tighter integration with DSource is one. I’d like for the net feature to be served by DSource and for a lot of DSSS’ auxiliary tools (such as the networked source manager and testing framework) to be connected to DSource.

I’m also aiming for scripting support, and it’s looking like the scripts are actually going to be in D. I have a fairly good plan for how that could work now.

Good library versioning support is one of those big issues that I’ve been struggling with since the beginning. It will definitely come into being, and I’d like to say I’ll have it really ideal and seamless. But, it’s a difficult problem.

Aside from your own projects, which other D projects do you make regular use of?

I use Tango, and in fact maintain the GDC port of it. I’ve made fair use of wxD, which is a really nice binding despite its small mindshare. Because DSSS by its very nature shouldn’t have many dependencies, I haven’t used many of the libraries available all that much. I have used a lot of them a bit in the process of making or helping make them DSSS compliant, and occasionally I make “look at this when I have time” notes to myself about them.

Are there any libraries or tools missing from the D community that you would like to
see? Any you or are working on, or thinking of working on, yourself?

From a purely self-serving point of view, I’d like to see an IDE with integrated DSSS support. I think that such a combination could work very well, and make life easier on end-users who don’t use the same IDE. To this end, I’ve made a small graphical project manager for DSSS I intend to complete and release some time that bridges this gap a bit.

Of course, it’s hard to guess what libraries or tools may be useful. Hopefully, somebody will think of one and make us all say, “Wow! What an excellent idea! Why didn’t I think of that?” And hopefully, I’ll be able to install it by typing `dsss net install coollib`.

A big thanks to Gregor for taking the time to answer these questions.

I mentioned in a previous post that I would be putting up two Q&As this month to make up for the lack of one in July. It doesn’t look like that’s going to happen, unfortunately. However, I’ve got plenty of time to turn September into a twofer, so look for the next Q&A in a couple of weeks (fingers crossed).

Technorati Tags: D Programming Language, DSSS, build tools

Posted in Q & A | 2 Comments »

Q & A: Jarrett Billingsley

To date, my Q&A victims have been battle-scarred veterans of different software industries. Their opinions on D are tinted by years of experience with different languages and projects. This month, I wanted to get at someone who has a different perspective.

Jarrett Billingsley is a prominent member of the D community who hasn’t worked professionally in the software industry. Currently attending college as a computer science major, Jarrett has contributed two projects to the D community: nonagon, a DirectX-based 3D game engine, and MiniD, a D scripting language that currently sits in a unique position in D-Land.

While I was in #D the other day, I asked Jarrett if he would be willing to answer a few questions about his background, D, and his projects. He accepted. I emailed him some questions a day or so later. Following is the result, with my questions in bold and his answers in plain text.

Note: ‘Python’ is escaped in single quotes in this text in order for me to be able to post this to the blog. See my post titled Blog Upgrade for details.

Your DSource profile says you are a CS student. Which year are you in now? What is the most interesting course you’ve taken so far?

I’m currently a CS student at the University of Pittsburgh.  I’ve just completed my second year, so if all goes well I should have two years left, unless I decide to go to graduate school.

So far, most of the classes I’ve taken in CS have been introductory classes — programming, some computational theory, logic.  They reserve most of the fun classes for the upper-level students ;)  If I had to choose from the ones that I’ve taken, though, it would have to be my “Computer Organization and Assembly Language” course (CS 0447 at Pitt).  The course was about the basic architecture of computer processors, getting down into the register file and the ALU and microprogramming and such, which I found interesting in and of itself.  The main reason I liked the course so much, though, was the professor, Dr. Mark Kahrs.  He doesn’t teach the class very often, but I’m very glad I had him.  He was one of the few professors I’ve had who seemed genuinely excited about the material he was teaching.  He always had a plan, he came up with challenging assignments (write an emulator for an old vector graphics device in MIPS assembly!), his tests actually tested your ability to think and reason rather than memorize, and he knew what he was talking about.  I really respect the man and hope I have the opportunity to take another class with him.

Once you graduate, is there any particular industry you’d like to work in? Have you done any internships or such?

To be honest, I haven’t thought much about what I’d like to do after college!  I’ve always kind of expected that I’d end up doing something with programming, though.  Whether I’ll end up in a typical business setting is an entirely different story.

I originally wanted to go to Full Sail in Florida after graduating from high school.  It’s a college that specializes in entertainment technology, and I was very interested in their video game design and development degree.  The price tag, the lack of on-campus housing, and the proposition of moving 1,000 miles from home only three months after graduating from high school changed my mind. After two years at Pitt, though, my desire to work in the video game industry has started to fade, and I’ve been thinking of a variety of other possibilities: going into an academic career, working at some kind of business startup, or maybe even doing something completely off the wall like starting a bakery and keeping programming a hobby!  As of now, I’m leaning towards an academic career, but we’ll have to see what comes of that.

As for internships, I’m in one right now.  It’s my first job outside of a grocery store and it feels great.  I’m working at the University of Pittsburgh Medical Center as part of an in-house software development team.  We do everything with .Net and C# (which is not a bad language, I’ll admit).  I’m not working on the most exciting programs, and I still don’t know whether or not I like the office environment, but it’s certainly a lot better than getting $6.80 an hour putting apples on a shelf.

When did you first start programming? Which languages have you used? Were you self-taught?

I started when I was about 13 or so.  Being addicted to the Sega Genesis and the N64, my initial desire was to make video games.  To that end, I started using a 3D game language called Dark Basic.  Looking back at it, it’s a terrible language with an even worse 3D engine, but at the time I was blown away :)  It really took me a year or so before programming ‘clicked’ and I was able to begin effectively translating my ideas into code.

I used Dark Basic and its successor Dark Basic Pro for a few years.  I never really made anything of consequence with it, but it certainly got me thinking about things.  After Dark Basic I moved on to its competitor, Blitz3D.  After participating in numerous flamewars between the two communities defending Dark Basic, I tried out B3D and found out it had a much more capable language and better engine.  To this day I still use it for simple graphical programs, testing out 3D concepts, and for making other little utilities.  It’s an incredibly capable language, probably a little better than C in terms of its data and function capabilities (though not as low-level), and it even has a form of RTTI which can be used to simulate polymorphism.  I used B3D to write a tech demo for my high school senior project, and I was just blown away at not only the capability of the language, but of the engine as well. You’d be amazed at what you can squeeze out of the DirectX 7 fixed-function pipeline.

By this time I had taken an interest in C++, because I was getting tired of some of the limitations of languages like Dark Basic and B3D.  They were proprietary, so I could only work with other programmers who owned the language, they were relatively slow, they were only good for games, their data structuring mechanisms were lacking, and they were hard to extend.  I thought I could do better by learning C++ and writing my own game engine.  I started with DirectX because both Dark Basic and B3D were based on it.  What started as a program that basically set up the device and drew some teapots eventually grew (VERY slowly) into a Blitz3D-style engine.  It only did the simplest things — basically you could load 3D objects, texture them, and move them around.  Once I started learning more about classes, I started refactoring the design into a more OOP style, updated to DirectX 9, and renamed it nonagon!

I never had any formal training in programming.  I went to a rural school district so we didn’t have any kind of computer classes beyond “using Word”.  Everything I learned, I taught myself.  It probably took me longer than the people who are just learning programming in college now, but I’ve got quite a head start, and programming is second nature to me by now.

I also know some Java because of school, but think it’s a terrible language.  It’s amazingly inexpressive and inefficient for the sake of “compile once, run anywhere.”  I’ve never actually used a Java program which perhaps speaks for how well that philosophy has panned out.  I also know C# from work, and respect it much more than Java.  If I had to program in a mainstream language other than D for a career, it’d probably be C#.

How and when did you first learn of D? What was it about the language that hooked you?

In a language discussion in a Dark Basic community forum, I posted about my exploits with C++, saying that I liked its power but couldn’t stand how complex it was.  In response, someone (wish I could remember who) replied with a link to the D site, saying that it seemed like a neat language.  I went, started reading the spec, and it just went from there ;)  This was in the fall of 2003. The first version of D that I downloaded was 0.74.  I didn’t really start really using D, though, until almost a year later, around 0.95.

I think when I got hooked was when I was reading the Overview page, in the “features to keep/drop” and the “who D is/is not for” sections.  I remember reading “Those who decide the promise of C++ object oriented programming is not fulfilled due to the complexity of it” and I knew that this was the language for me.

After having worked on nonagon for about a year and a half in C++, I was really starting to hate it.  I became aware of the concept of “memory management” and that I was responsible for cleaning up all the stuff I allocated.  I started reading articles about smart pointers and reference counting and stuff, and I basically said “that’s it, I’m done.”  I gave up on nonagon because C++ was making it so ugly.

What was weird about D is that everything seemed to work the way I expected C++ to.  Arrays knew their length; objects cleaned up after themselves; downcasting a class reference actually did something worthwhile (I didn’t know about dynamic_cast in C++ at the time).  It was like I had been trying to program in D the entire time I had used C++.  That’s probably what appealed to me the most — the language just worked the way I expected it to.

Most D programmers come to the language after having used other languages professionally, often for several years. In your case, without all of that experience behind you, how was the learning curve for D? Was it easy to pick up?

I found it very easy to pick up.  When I switched to D, I still wasn’t that clear on most OOP concepts (most likely because the C++ OOP model is just a mess).  It only took me a few months for that stuff to really click, and it mostly came about during the time when I was translating nonagon from C++ to D.  That was a really fun experience — being able to copy and paste code, remove the quarter that was devoted to memory management, combine the declarations and definitions, remove all the -> operators, and see the clean, simple logic and structure that was underneath that ugly C++ exterior.

What’s funny is how far my abilities have come even in the past couple of years.  I look back at some of the old nonagon code now and think “man, this is messy.  I could do it so much better this other way.”  Probably the best thing about learning D, though, is that it scales nicely.  Any beginner Java programmer would be able to write useful programs in D using only simple OOP concepts.  I did that with nonagon at first as well.  But when you get better at programming and want more power, D doesn’t fail to impress — operator overloading, custom memory management, templates that would make C++ programmers drool, compile-time function evaluation and so on.  Don’t believe the D overview page that says D is a good second language — I think it’s just as good a first language as Java is, but it provides features to satisfy you as you improve.

What are some things you really like about D? What don’t you like about it?

They’re kind of underrated, but dynamic arrays and associative arrays as core data types of the language are probably some of the best features.  Most of my classes are just wrappers around these two incredibly powerful constructs.  They mean I don’t have to deal with clumsy ArrayList/Vector, Hashtable, Stack, or Set classes.  It’s to the point where any language I use that doesn’t have them as core types feels crippled.

Some other things I like:  No function or class prototypes.  Fast compile times (I can compile MiniD, which is just over 20,000 lines I believe, as well as a large chunk of Tango in about two or three seconds on my 1.6GHz Athlon).  Templates that make sense, compile fast, and look nice. Separation of identity versus equality.  The ability to interface with C code.  The abandonment of the ugly preprocessor (even C# still has #if — ugh), replacing it with actual language features.  Foreach, the best loop ever.  Delegates and inline delegate/function literals.  Nested functions in general.  Scope class references.  Scope statements.  Finally.  IFTI.  Automatic virtualization of class methods and destructors.  String mixins + CTFE + templates = wow.

I don’t have that many complaints about D, but they’re there.  The “calling a function as a property” syntax — I think explicit properties are a better way to go.  The adoption of some odd properties of C++’s OOP model for no reason besides “it’s the way C++ does it.”  One, a derived class defined in a different module from its parent class may access its own protected members, but not the protected members of other instances of the base class.  Two, base class methods do not overload with derived class methods unless you explicitly put in an alias to do so (seen lots of threads on the newsgroups about that one).  Some other things: no requirement to put ‘ref’ or ‘out’ on parameters at the call site as in C#.  No requirement to put ‘override’ on overriding methods as in C#.  Lack of static closures, though that can be simulated somewhat satisfactorily.  Lack of scope for anything other than class references (scope arrays, anyone?).  Uhh, no way to use $ in slices/indexing for a user-defined type.  I’m really scraping here

Of course there are some “non-language features” that I’m not particularly fond of.  I feel like the language which is supposedly “community-driven” is very rarely so.  I feel like more emphasis is placed on “cool new features to lure over C++ programmers” than on fixing bugs.  I feel like more credence is given to leading academic voices in the C++ world than to practical experience from programmers who have used D extensively in their own projects.  And I feel like the development of D is a haphazard, mysterious process to which only a select few are privy.  I’d like to see more updates like the few updates before 1.0 (50 bugfixes?  I’ll take that over a single new feature), a more open development process, and for the actual users of D to have a say in what they think is best for the language.

The first publicly announced D project you worked on (that I know of) was nonagon. There aren’t many D projects that I’ve seen using DirectX. What prompted you to create nonagon and to choose DirectX?

As was mentioned before, I started nonagon in C++ in an attempt to emulate some of the game languages I had used, both of which used DirectX.  I’ve also never really used any OS besides Windows extensively until very recently, so cross-platform compatibility was not high on my list when I started it.  It was actually kind of a project getting all of DirectX to work with D in the first place, since the core library is easy to link against, but DirectX had some static libraries that were impossible to use from D.  I think I was the first to actually get D3DX to work with D, which is kind of nice

I’ve been thinking about revamping nonagon, as I haven’t really touched it in the past year and a half.  I’ve been thinking of making it API-agnostic, or just based on OpenGL, SDL, and such.  But that’s quite an undertaking, and with a job and MiniD on my plate this summer, I don’t know if it’ll get done any time soon.  That’s mostly why I made the project open-source recently, so in case I never get back to it, it still might be useful for someone!

MiniD is, to me, a tremendously interesting project. The project page has an informative section regarding why you created it. How have the design and implementation been for you? Has it been a difficult process or an easy one? Did you have experience with creating scripting languages or have you been learning it all as you’ve gone along?

I’ve always been interested in scripting languages for some reason — I made one or two really simple ones in Dark Basic, as well as a relatively complex assembly-like language in Blitz3D.  MiniD actually started as a culmination of two small “messing around” projects that I worked on in the spring of 2006.  One of them was a compiler for a simple C-style language that never had a name.  I wanted to see how far I could get without any kind of help, and it got pretty far.  I wrote the lexical, syntactic, and part of the semantic passes before I realized that I didn’t really know what I was doing anymore.  Of course, I look at the code now and it looks like an absolute joke :)  The other project was an attempt to make a D implementation of Lua.  I found out about Lua and thought that it was really cool, but didn’t like the C API at all.  I actually translated the entire Lua compiler into D, and it generates code that is almost bit-for-bit identical to the code that the official Lua compiler generates.  After I did that, though, I looked at how much more of the language I had yet to port, and thought “why am I doing this?  Why don’t I just make a language of my own?”

The initial design of MiniD was a statically-typed language, kind of like D but with many of the features stripped out.  The language was never implemented; it was all specifications in the DSource wiki.  I wanted to make the compiler right this time, so I followed the lead of the DMD frontend, using very similar patterns of lexing and parsing.  When I started with the compiler, I started to realize the complexity of the project — having only implemented the lexical and syntactic passes of the compiler, it was already approaching 7,000 lines of code, and the semantic pass was going to be by far the largest part.  Then would have to come code generation, the virtual machine and the standard libraries.  I realized that the language I had created was about as complex as C# and that perhaps I was in too deep!

I retargeted the language as a Lua-like embeddable extensible scripting language instead.  By switching to dynamic typing and dropping a large proportion of the features, the project suddenly became much simpler.  I had a (kind of) working compiler in a matter of weeks, and the VM followed soon after.  I was and still am particularly proud of the compiler — it’s based on the DMD compiler for lexing and parsing, and uses my own “expression stack” to generate the code.  It’s basically a stack machine in the compiler that converts “popping” operations into RISC-style instructions.  That was one hell of a learning experience :D  But the compiler is very modular and easy to modify; it made it absolutely trivial to remove the static typing when I refactored the language, and it’s been incredibly easy to modify and extend the language with it ever since.  I’ve also used the compiler as a basis for a few other small personal projects that involved parsing of some sort, which speaks for how flexible it is.  It really taught me a lot about compiler design and I feel a lot better having implemented it myself instead of using something like flex/bison or yacc.

MiniD has, of course, evolved quite a bit even after I made it a dynamic language.  The very first useable release had no standard library, but was basically an implementation of Lua with C-style syntax.  I realized that I set out to create my own language, but had just ended up copying Lua instead!  So I started taking features that I liked from other languages — classes from Squirrel, array slicing and modules from D, more robust list manipulation functions from Lisp (which I was playing around with at the time).  Slowly, the syntax changed, the compiler and VM matured, and the current incarnation of MiniD emerged.  It’s now become closer to Squirrel than to Lua, though it’s got its own features, like syntactic support for coroutines and the module system.  I really liked how Squirrel solved the method call problem (if you have a.b(), should it index b and just call it, or pass a as the first parameter?  passing an implicit ‘this’ to all functions, method or not, solves this), and that has made the language more traditional-looking than before as well.

I wouldn’t say that the process was particularly easy or difficult — I found it challenging, of course, and most of all fun.  All the “scripting languages” I wrote before it were extremely simplistic, usually barely Turing-complete and usually without a compiler (typing in bytecodes into a hex editor == fun).  MiniD was a huge departure from what I had done before and has been an amazing learning experience.  I also can’t downplay the influence of other languages on its development — most of what I learned about implementing the language I learned from reading the Lua source, as well as Squirrel, Io, and the DMD frontend.  They have been wonderful references.

Where are you planning to go with MiniD? What sort of features can we expect to see in the future? What sort of applications do you think the language is ideal for?

I’m not real sure!  Right now it’s targeted mostly at the same kinds of tasks that Lua is: adding scriptability to a host application, while being extensible itself so that scripts can access whatever the host app wants them to.  Even so, the language is really starting to mature and take on a personality of its own, and can probably be used for many independent programs as well.  One of my friends wrote a small game in ‘Python’ using pygame for a recent coding competition (our team used D, surprise surprise), and I did some translation of the ‘Python’ source into MiniD to see how well it fared.  I was pleased with how easily the translation went, and only ran into a few things that ‘Python’ could do that MiniD couldn’t.  Because of this, I wouldn’t be surprised if MiniD would be able to do as well as ‘Python’ for small apps.  Of course, it doesn’t have the standard library that ‘Python’ does, but that can change!

As for future features: working on that :)  I’ve frozen the current featureset until 1.0 comes out, so virtually all the changes until then will just be API changes.  After that, though, the 2.0 beta branch can begin!  I’ve had some ideas such as separating indexing from field access and coming up with a new ’symbol’ type for faster indexing and to make it possible to implement properties without stepping on the indexing operator’s toes.  List comprehensions are really sweet and would be awesome to have in the language.  Some kind of non-collaborative threading (as opposed to coroutines) would also be nice to have.  Some kind of parameter type constraints for functions is also possible.  Beyond that, I’m not sure — MiniD’s flexibility in terms of where and how you can declare functions and classes places some restrictions on the language.  I’d like to keep MiniD as a scripting and extension language; making it a whole language in its own right is a tempting idea but would sacrifice a lot of the flexibility in order to gain some other features.  Maybe it could branch into two languages — a lightweight scripting language and a heavyweight apps language!  Man that’s a lot of work

I’ve also had non-language ideas, such as a competing library to pygame (MDGame perhaps?) and a simple cross-platform GUI framework based on OpenGL and SDL and using MiniD for the logic.  These libraries got me interested in loading dynamic libraries into MiniD, which will hopefully be possible once DDL gets updated and converted to Tango.

What would I think that MiniD is ideal for?  Well, Lua has been used extensively in many games and applications.  Moho (now Anime Studio), a 2D animation program, exposes a Lua interface for writing plugins, and there has been a great proliferation of them.  Lua plugins can then be integrated into the UI as if they were native tools.  That’s a great example of the power of scripting languages.  Some games, like FarCry, use Lua to write a large majority of the game code.  They write the engine and the performance-intensive code in the host language, and then use Lua to write virtually the entire game.  Lua is also used in the user interface of World of Warcraft, and for scripting in many other games.  MiniD could be used for much the same purposes.  MiniD is expressive enough to allow for useful code to be written with very little effort, and the API does and will make it extremely easy to interface between native code and MiniD code.

With some help from external libraries (like MDGame or the GUI framework), MiniD could also be a very capable rapid prototyping language.  It’s already got a module system and classes, as well as semantics very similar to D’s, meaning that you could come up with a quick idea in MiniD, see the results right away, and then convert it to D code once the idea has solidified.

Lastly I think MiniD could be a good learner’s programming language.  It gets you used to the C-style syntax that has become virtually ubiquitous in mainstream programming languages without exposing you to a lot of low-level detail.  It can provide instant feedback through a command-line interpreter, making it possible to have interactive labs where students can try things out and see what they do right there.  It can also be used to teach about code modularity and encapsulation with the module system, and about OOP as well.

Aside from your own projects, which D libraries and/or tools do you regularly make use of?

Well I recently converted MiniD over to using Tango as the standard library.  I really like the idea of a community-driven standard library, as it feels like I can make a difference.  I can’t complain about the turnaround time for bugfixes either — I put in a ticket and it’s fixed in a day, an hour, or even a couple minutes!  I’m not entirely fond of all of Tango, but that’s probably just because I’m used to Phobos.  Tango also provides some really useful features that Phobos doesn’t at all — Fibers, proper handling of all Unicode encodings, more flexible IO and formatting, and extensive array utilities to name a few.  I could really see Tango becoming the library that finally gets D the attention it deserves, especially if other “pluggable” libraries are built on it, such as GUI toolkits.  A lot of Java and C# programmers have become spoiled by the extensive standard libraries that those languages provide, and they’re beginning to expect the same level of capabilities from systems programming languages like D.

The other tool I use is Derek Parnell’s Build/Bud.  I have virtually replaced my personal batch file compilation system with Build and response files.  It’s so convenient to be able to have compilation that’s almost as automatic as you get with an IDE from a command-line tool.  And it’s cross-platform to boot!

I’ve looked at DSSS too, but I haven’t had much experience with it or use for it.  I could use DSSS/Rebuild to build my apps, but so far Build has suited me well.  Nonetheless MiniD is available through DSSS’s net feature and compiles (usually) with Rebuild as well.  I haven’t needed DSSS much though, because I don’t make much use of other peoples’ libraries — I’m usually writing my own!

I also used Derelict in a small “game” that I made for an informal game coding competition this past spring.  Long story short: it’s great.  No trouble using it, easy to set up, completely unintrusive.  I didn’t get to test if it worked on anything but Windows but there’s no reason it shouldn’t.

Lastly, after a short but tumultuous stint with DLLs in D, I became interested in DDL.  The idea of having a sort of Java-y .Net-y run-time code binding system in a natively compiled language is incredibly tempting and could open up a lot of possibilities.  I’m hoping it’s everything it seems to be so that loadable native code libraries can be implemented in MiniD.

Are there any specific, and currently unimplemented, libraries or tools you would like to see developed in the future? Any you’re thinking of implementing?

I’d like to see a proper Unicode library, maybe based on IBM’s ICU library.  D has already proven itself as capable in the area of numeric programming with support for extended-precision reals and complex/imaginary datatypes.  Given its ability to handle all three major encodings of Unicode, it could very easily become a great text-processing and internationalization language as well.  That being said, I probably wouldn’t even think of implementing something like this :D  There are programmers out there who are really interested in this area, but I’m not one of them.

Another area where I think D is really promising is for games.  D could really use a “killer” game engine or 3D framework.  (No, nonagon’s not that engine :D)  C++ has long been considered the de facto standard for writing games because it allows so much control of memory management.  Now that games are getting more complex and computers are getting more cores, wasting development time on memory management that could be spent writing more complex AI or supporting multiple cores is starting to seem stupid. D already provides custom allocators and deallocators for classes and structs, just as C++ does, as well as allocating classes on the stack.  An advanced GC implementation (such as an incremental GC that doesn’t “stop the world” running on a separate core which can be highly tuned for different kinds of memory allocation strategies within the same program) coupled with custom allocation schemes for some classes and structs could make games written in D considerably simpler than their C++ counterparts while offering almost the same performance.

And it’s been said once, it’s been said a million times — D needs a proper IDE.  I think it’s been discussed enough, but many large companies/programming teams make heavy use of IDE features, and without it, few business people will take D seriously.

A big thanks to Jarrett for taking the time to do this. He even went above and beyond and formatted the answers for me in a text file!

Technorati Tags: D Programming Language, nonagon, MiniD, scripting languages

Posted in Q & A | 9 Comments »

Q & A: Eric Anderton

This month’s Q&A victim is Eric Anderton. Around the D community he is most commonly known as ‘Pragma’. His most well-known project is DDL (D Dynamic Libraries), but he is also the maintainer of Enki, a frontend parser generator. Another of his projects, DSP (D Server Pages), is awaiting the completion of DDL. Something many probably don’t know is that he also works on the TracForums project for DSource. And with all of this going on, he still finds the time to post frequently in the D newsgroups.

When I recently asked Eric on very short notice to take some time out for a Q&A, he graciously accepted. As usual, my questions are in bold and his answers in normal type.

Your DSource profile says you are a web developer. How long have you been doing that? Which technologies do you usually work with? What are your favorites?

I’ve been doing web development and design for almost eight years now. It all began by working for a few start-ups during the dot-com boom, and then gradually moving on to consulting and then contracting; all in that arena. I’ve done other things besides web development (intern work, C++, tech support), but those avenues gave way to what became the most lucrative; a man’s gotta eat.

Technology-wise, I’m all over the map. I’ve done some personal stuff with PHP, some consulting work using ASP, and have used .NET, Java and ColdFusion professionally. At one point, even VB and Access were on the menu. Right now, ColdFusion combined with AJAX (dojo and YUI mostly) pays the bills. There have been plenty of other hobby languages, but all but D have failed to hold my interest.

My favorite web technology has to be ColdFusion, despite how much I’ve bashed it in the past. Ever since MX7 came out, it’s demonstrated itself to be very mature, well-documented and flexible. For all its quirks, it’s very even-handed at both RAD and highly-scalable designs. This translates into less time spent in “hacker land” trying to get trivial stuff done. I’ve also seen it work wonders for helping teams, with a broad range of proficiencies, work together as a team. I’m also very excited about what Adobe has done for CF8 and Flex.

Browsing through the DDL project pages, it seems obvious that you have a strong interest in things like linkers, grammars and such. What is your background with all of that? Is it just a hobby or have you done anything professionally?

Mostly, it’s just a hobby. I’ve had minimal classroom instruction in language theory and compiler mechanics, so I’m not formally trained in any of this. I’ve never composed a compiler, wrote my own linker, or even composed a code generator professionally - although I’d love to be paid for that kind of work. Thankfully, I find this kind of hacking on my own time very rewarding.

I just read what I can, and experiment even more. I’m one of these guys that can’t stand not knowing how something works, so I was always taking stuff apart as a kid; I always thought dinosaurs looked much more impressive as skeletons since you could see how they “worked” on the inside. So you could say that when undertaking DDL, I had a strong internal motivation to take all this “meta education” on. I certainly didn’t know as much about all this three years ago.

How and when did you first learn of D? Wat was it about the language that hooked you?

At the time I was working on something resembling a MUD/MMOG framework in .NET, largely due to the platform’s strengths: garbage collection, reflection and remoting. I found the performance to be a real drawback and the locked-in feeling of being tied to the .NET platform itself even worse. Java really didn’t feel much better. So I went back to investigating how C++ could be enhanced with macros and such to give it some of these capabilities without these problems.

Then I learned about D on slashdot.org, gave it a glance and promptly forgot about it. D just didn’t seem ready yet. I came back to it about six months later and decided to give it a chance after I realized it had everything I was looking for: no lock-in agenda, native compilation, and garbage collection. I also saw a golden opportunity to jump into a fledgling technology and help make it something really great. In then end, I think that’s really what has kept me interested: the D community and the raw potential of D.

If you could add one new feature to D right now, what would it be? What other features do you think would be useful to have?

I think the language, at the 1.0 mark, is adequate for most of what anyone could ask for. Someone is always going to come along and say “well ‘Python’ has this”, “Forth has that”, or “someone else broke LISP again.” That’s not going to change. Where D is weak, and has been in need of improvement for some time, is in how it’s conveyed to the world online.

People are clamoring for something that does what D does, and gives project managers warm-fuzzy-feelings by being a complete product. Honestly, having better documentation and a completely fleshed-out ABI is a must. And it doesn’t need a miracle to happen either: if the DigitalMars.com D Programming Language web were opened up as a wiki, I think we’d see an overnight improvement in this area. After that, the top support projects, a good IDE and GUI Debugger need to find their way into a DMD/GDC installer; a DigitalMars branded installer would be even better.

It seems to me that DDL must have been quite challenging to put together. What led you to undertake such a challenge?

The short version: I’ve learned to turn moments of naivete into an asset. Sometimes, the conventional approach exists for reasons that may not apply to what you are trying to accomplish.

Back in high-school, a friend of mine was showing me how to use C for the first time. When it was explained to me what an .obj file was (symbolic information, unlinked code and data), my first reaction was: “Can we load that into memory from our program and run it?” My friend then explained that it wasn’t possible, and that’s what .dll files were for. I took that for the plain truth and moved on to getting a for() loop to work correctly.

Fast-forward 10 or so years later and I find myself trying to turn D into a web-programming language (the DSP project). I was dismayed at how huge a .dsp page was when compiled into a .dll file, not to mention the technical drawbacks due to how partitioned a .dll is from its host. I saw how tiny my .obj files were and thought: “If only I could load those instead.” I then realized that I had the correct insight to solving this very problem a decade ago, but merely lacked the skill to do it then. Scratching that old itch was all the motivation I needed.

What has been the biggest difficulty during the development of DDL? Was there ever a time when a feature you wanted to implement had you stumped?

OMF parsing, OMF parsing and OMF parsing. This file format was the bane of my very existence for well over a year of working on DDL, and caused me to be stumped on more than one occasion.

The documentation for this .obj file format is almost more of a reference than a specification to code against, for reasons that completely escape me. There were so many times that little gotchas in the docs allowed me to code down a blind alley for weeks before a bug would manifest itself. Then, finding where I strayed from the specification took even more effort. This was all made worse by the fact that runtime linking has proven to be very hard to debug in and of itself. Thankfully, DMD’s handling of OMF seems to cut extremely close to the specification - so at least I knew that I could trust the data coming into my test harness as valid.

Once I had my head around most of the idiosyncrasies of this arcane format, it all started to come together. Things improved even more when I got some help from the community at large, which was great since it helped both my morale and the project itself by leaps and bounds. Now my only major problem is having the time to work on it!

What is the future direction for DDL? What sort of additional features do you have on the drawing board?

At the moment, I’m porting DDL to Tango, and removing the old Mango-parser scheme. So this is a win-win for most folks as back-porting the new DDL to Phobos should prove pretty easy. However, I have no explicit plans to support both libraries in the near future.

The other planned features for DDL really haven’t changed and are all on the project roadmap: ELF support, COFF support, better reflection, module unloading and better performance via lazy loading. I’ve also heard that there are some performance issues with large libraries (big symbol tables and such), that should be addressed.

How did Enki come about? What are your plans for it?

In retrospect, I guess Enki came about because D doesn’t have AOP.

When composing parsers for various tasks, I found myself repeating the same idioms, over and over again, for function after function. What was worse was that I learned that I wasn’t really very good at converting a grammar decomposition (typically BNF) to code, since the result was typically rife with assumptions and shortcuts that led to bugs and bad behavior. So I also wound up fixing the same bugs, over and over again.

So I couldn’t find that magic balance point between compactness, readability and performance that I wanted. Either I had too much code to maintain, too much overhead to be efficient, or not enough code and/or overhead to be correct. AOP could, in theory at least, allow me to decorate my code with the needed checks, preambles and epilogs without sacrificing clarity or performance. But since this wasn’t an option, I had to look elsewhere.

Next I started looking at The Language Machine as a possible alternative to hand-coding a parser. While it is a brilliant piece of software, I found the syntax to be a little too hard to get my head around. I also found it to be too generalized. All I wanted was something like LEX/YACC for D, without imposing so much on the developer. Just “a front-end generator,” I thought, “with a minimal runtime so you can maintain the rest in D.” When I realized that such a tool could effectively generate it’s own frontend, I couldn’t resist the urge to code one up.

Enki’s future looks very bright. I’m in the process of re-vamping the project to use Tango. I’ve also refactored the engine so it uses a visitor pattern for code generation, which will make composing alternate backends for Enki much easier. As Enki loves to eat it’s own dog food, this technique is now used to generate both the BNF output and D code output. The consequence of this has been a vastly improved code generator, and a redesigned parser backend that will now accept a distinct lexer/tokenizer, improved error handling, and something I call “repeated predicate optimization.” There’s even some new EBNF constructs I concocted to support repetition ranges and delimiters, plus a new attribute syntax that will allow coders to target multiple backends from a single .bnf file.

I’ve also toyed around with making a compile-time parser generator with some limited success. I’ve held off on developing it further until Walter comes up with some better meta-programming support.

Aside from your own projects, which D libraries and/or tools do you regularly make use of?

I would have to say that Tango, Mango and DBBI have had the most impact on my D projects. The first two are obvious as they feed directly into DDL and Enki. I made great use of DBBI when composing a PHPBB to TracForums translator for the TracForums project. We haven’t used it on dsource.org yet, but I anticipate it coming in very handy once TracForums has matured.

Are there any specific, and currently unimplemented, libraries or tools you would like to see developed in the future? Any you’re thinking of implementing?

On the unfinished project front, my pet R&D project DSP comes to mind. I’m also still working on TracForums, which currently enjoys a very nice life as a functional beta on the Tango site.

I’ve also talked with the Tango team about drafting an “updater tool” (think: “apt-get update”), possibly based on DSSS. I’ve held off on that only because the D front-end is still maintained in C++. Of course, this would be a grand opportunity for Enki, or even for anyone willing to just translate the DMD front-end to D since D 1.0 is now frozen. As every translation effort in the past has been crushed by DMD’s Beta schedule, there’s not as much (aside from bugfixes) from keeping such a project from flourishing now.

DBBI needs some serious TLC, since it seems to languish for huge stretches at a time. I’d love for someone to put up a “call for patches” on the forum, merge everybody’s improvements in, and then fold in a solid Variant type (like the one that popped up on the Tango forums recently) for good measure.

I’d also like to play a bit more with Rodin, since I bugged Gregor to try that concept out in the first place. I think D needs better reflection and supporting that via a pre-processor is probably the most elegant way to go short of using DDL. Not to mention, that those two approaches are highly compatible, and are just bristling with possibility.

Thanks again to Eric for taking the time to do this.

Technorati Tags: D Programming Language, DDL, Enki, DSP, Trac

Posted in Q & A | 2 Comments »

Q & A: Sean Kelly

Before Sean Kelly teamed up with Kris Bell to create Tango, he was the founder and maintainer of the Ares project. He has been a prolific poster in the D newsgroups, not only to help people with their D questions but also as an active participant in feature discussions (the ones that I tend to stay out of since they usually go way over my head). If you were looking for people who know D as well as Walter, Sean would be one of the few who come to mind.

You’ll notice that several of the questions are identical (or nearly) to some I asked of Kris Bell last month. I hadn’t considered it until I sat down to make a list of Sean’s questions, but that’s the format I’ve settled on for these little Q & A sessions. At least my next target will know what to expect.

My questions are in bold, his answers in normal type.

When you aren’t hacking D, what do you do to pay the bills?

I am currently at a financial services firm developing enterprise-level investment management and accounting software.

Before you found D, what was your programming background? Which languages were you most familiar with and which was/were your preference?

The job I remember most fondly was one in R&D for a switched long-distance carrier, which mostly involved developing custom applications for a high-volume telephony system. The remainder of my personal and professional background is fairly broad, but the common thread seems to be distributed problem solving. I most enjoy projects which require some form of communication, be it in-process concurrency, distributed server platforms, or social AI.

Before D my programming language of choice was C++, and C++ remains the language I use most often professionally. C++ is an extremely powerful language, but its support of a broad array of programming styles and tendency towards non-obvious syntax tends to complicate its use in large projects. Some of the other programming languages I’ve used professionally are Pascal, C, and Java.

How and when did you first learn of D? What was it about the language that hooked you?

I first learned of D from a Slashdot article a few years ago. At the time, my issues with C++ had really begun to crystallize and D seemed to address them all in a truly elegant manner. I was an instant convert.

What are some things you really like about D? What don’t you like about it?

I most like that D is a systems programming language but doesn’t feel like one — it is this breadth of application that makes it most appealing to me. Some of the language features I find most appealing are slicing, nested functions, delegates, contract programming, and optional garbage collection. Much of the rest is difficult to quantify. For example, D seems to naturally lend itself to producing elegant code. This is very important to me, but it is impossible to point at one or two features and say “this is why.”

Many of the things about D that I didn’t like have been changed or fixed in the time since I first discovered the language. However, mixins still feel like an unfinished feature and also raise concerns about code readability.

If you could add one new feature to D right now, what would it be? What other features do you think would be useful to have?

It’s not a language feature, but I would like a production debugger that works with all of D’s features. Classes, templates, mixins, the lot of it. I’ve gotten quite used to printf debugging since discovering D and I really don’t mind it too much, but there are times when printf alone won’t do.

Aside from that, I’d like to eliminate the differences between constructing different types of data. It’s not a big thing, but I find it occasionally annoying that concrete types don’t support ctor syntax, for example. Another feature would be some way to determine the size of a class at compile-time. I’ve also considered requesting built-in “placement new” support, though I haven’t given this last issue enough thought to be certain it is truly necessary.

Long before Tango, you created Ares as an alternative to Phobos. What was it that first motivated you to do so? What were your goals for the project?

Ares began as an attempt to provide the community with a proving ground for standard library features before submission to Phobos. The original idea was that much of the evaluation and review of code could be handled by the community in an attempt to offload some of the work it appeared Walter might otherwise have to do himself.

Once the project got off the ground however, I began inspecting the innards of Phobos, hoping to formalize the requirements for a third party to implement their own standard library for D. I finally concluded that the “all in one” approach of Phobos made such third party development impossible, so I decided to change that. Ares ultimately became a project to clearly separate and define the responsibilities of three parties: the compiler writer, the systems programmer, and the standard library developer.

As I understand it, your role in Tango is fairly low-level, whereas Kris works at a higher level. Could you clarify that? How are design issues and implementation details worked out?

Our roles were largely defined by what each of us contributed to Tango when the project was conceived: Kris had a robust user-level library and I had experience with the innards of the D runtime. This has given each of us a somewhat different perspective on library design, and I think it has been helpful in our design discussions, which are essentially continuous. As a result, I think it would be fair to say that if we didn’t work together well, Tango would never have been realized.

How has Ares influenced the development of Tango? Is it fair to say that the core of Tango is Ares++, or has it gone in a different direction?

The philosophy behind the Ares design was to produce a modular and configurable runtime on top of which to build a user-level D library. The original intent was simply to provide more structured support for D library developers, but the project ended up attracting developers who needed low-level control of the D runtime for one reason or another. One of my primary goals with Tango was to improve support for this type of development, and I am quite satisfied with the result.

Tango has come quite a long way in a rather short time and is still evolving. Are there any features currently missing from Tango that you would like to add? What can we expect to see in future versions?

As D is a new language, the D programming community contains a large number of library developers and other “pioneers” compared to some of the more entrenched programming communities. Also, recent exposure to Ruby Gems and similar projects have provided useful insight into some ways a completely modular library might be realized. With Tango, we are attempting to unify these ideas by providing a framework for cohesive development as much as we are a robust, user-level library. It is my hope that this will encourage the D community to continue amazing us with their work rather than stifling development because Tango “already does that.” Once Tango reaches 1.0, I suspect much of our library development work will shift into separate projects designed to integrate with or to extend Tango, but they will not be bundled with Tango releases. This places our work on non-essential features on the same footing as that of the community as a whole, and will encourage us to make the distribution and maintenance of external packages as easy as possible.

Aside from Tango, which D libraries and/or tools do you regularly make use of?

Almost none, to be honest. There is a lot I would like to do with D, but currently my free time is dominated by my work on the Tango core. I hope that once Tango reaches 1.0 that I will be able to spend more time on projects that are more immediately visible to users.

Are there any specific, and currently unimplemented, libraries or tools that you would like to see developed in the future? Any you’re thinking of implementing?

Most of my work involves network and database programming, so either of those are likely candidates for future development. I am also very interested in concurrent programming, artificial intelligence, and compiler design, so any of those are possibilities as well.

I’d like to thank Sean for taking the time to participate.

Technorati Tags: D Programming Language, Tango, Ares

Posted in Q & A, Tango | 3 Comments »

Q & A: Kris Bell

There are a number of names and handles you will be familiar with from the different D forums (newsgroups, DSource, #D) and projects. Most of them were early adopters of the language and have devoted many man-hours to develop tools, libraries, wiki entries and tutorials. I’ve been wanting to do some interviews for this blog, so I thought it would be interesting to get to know these people a bit better. We know the names, now let’s learn a bit about the people behind them.

So this is the first in a series of short Question & Answer sessions with some of the earliest adopters and most active members of the D community. The keyword here being short. I’m not a professional journalist and I certainly don’t have the time to write up lengthy interviews. My goal is to shed some light on who is at the keyboard behind the handles and why they are doing the great things they do.

Once I settled on the format of the interviews, I immediately decided to approach Kris Bell for the first Q & A. You will know him on the different forums by the handle kris. Not only has he been around D for a long while, he has also been at the heart of two major D libraries: Mango andTango. He’s an active participant in discussions about D features on d.D and can be found regularly in #D.tango and #D. He also makes me extremely jealous — I can never seem to format my D modules as nicely as his.

So when Kris agreed to answer my queries, I sent him some questions about his background, D and Mango & Tango. My questions are in bold, his answers in normal type.

When you aren’t hacking D, what do you do to pay the bills?

Last paying job was at PARC. August 2006 to February 2007 was spent traveling, since I always wanted to go on a long trip and finally made the time to do it. Recently I’ve been focused on Tango instead of working. Seems like a good deal to me

Before you found D, what was your programming background? Which languages were you most familiar with and which was/were your preference?

I’ve been lucky to become involved in a wide range of software applications. Started off doing firmware for factory automation and so on; then did some really interesting immersive environment-simulation; then some real-time 3D graphics before the hardware started doing it all.

After that, I spent a number of years building RAD platforms for various companies, and constructing high-performance clustering systems. More recently I went back to the firmware again, creating interactive-clothing and related products.

It’s much more fun building interactive systems (simulation, and interactive clothing) since it visibly affects people at the emotional level ~ far more rewarding to see people get excited or have a good laugh over something you’ve built than, for example, to design or build the “most amazing” RAD platform for deploying HR and Financial applications. I imagine that’s why gaming is so popular these days ~ it’s just fun & rewarding to create stuff that others get a buzz from.

Languages? For commercial systems I’ve used Basic, Pascal, Modula, C, and Java; plus many assemblers. Simula was the first real language I learned and, from memory, D is probably closer to it than any other language I’m familiar with (Simula turns 40 this year). Don’t really have a preference for one in particular, though it’s true that Java makes it easier to get a big project (with lots of people involved) actually working correctly; that scores big points for Java in my book.

How and when did you first learn of D? What was it about the language that hooked you?

Went looking to see what Walter Bright had been doing; had used Zortech C back in the day, and the codegen was excellent. It was simply a matter of “hrm, wonder what happened to that dude”.

The ‘hook’ was when I realized D could be a way to avoid ever having to design or construct anything large in C++ again. The latter is, in my view, the worst thing that could possibly have happened to software in the nineties. Some people love C++; I loathe what it does to project costs.

Now that you’ve had quite a bit of experience with it, what are some things you really like about D? What don’t you like about it?

The three things that stand out are array-slicing, foreach and closures. Most other features that affect the writing of daily code are pretty common across (other) Algol-based languages.

I don’t much care for the pedantic manner in which interfaces are supported, or how alias is a central tenet of the language. Having said that, I suspect mixin is the most glaring design fault in D — mixins seem to encourage creating opaque, obtuse, and unmaintainable code, with too little in the way of tangible benefit. As an example: there was one very capable (nameless) package evaluated for Tango, and it failed the first hurdle due to problems related to mixin. Where D really shines is in concise, clear, readable code; the detrimental aspects of mixin tarnish that polish in one way or another.

(n.b. the one mixin statement within the Tango codebase is being replaced)

There’s a number of so-called “quality of implementation” issues with the DM compiler: D binaries are bloated with typeinfo that is only rarely used. For example, moving from v1.0 to v1.005 resulted in a huge binary-size increase for Tango applications ~ all of which is additional typeinfo, 99% of which will never be used in daily programming. That’s a growing problem with D, and it really ought to be addressed in a manner that makes the meta-data optional. Stack-based classes (via ’scope’) ought to be checked for the existence of a dtor, before the compiler injects a try/catch across the class creation and destruction. Then there’s the lack of full debug support. It would also be good to get some kind of roadmap for a 64bit compiler for Windows. These are not language issues per se, but are important in one way or another.

If you could add one new feature to D right now, what would it be? What other features do you think would be useful to have?

It’s hardly a feature in this day & age, but full support for a debugger would be nice.

The syntax for instantiating classes could be simplified somewhat (for those who are more comfortable with script-style code), and structs need a good ctor syntax also. It would be great if structs could be inspected dynamically for what they offer, in a manner similar to what interfaces can expose; that would blur the line between class and struct notably, and would likely result in one or two Tango classes becoming structs instead. D should probably enforce things like “override”, since they make for more robust code and thus help to reduce long-term costs.

Better runtime reflection (in general) would be a nice option to have, but purely as an option.

Before Tango came along, Mango was the gold standard for D libraries. There was quite a bit of praise for it. What was it that motivated you to create the project in the first place? Did you think it would be as popular as it has been?

Mango has a number of different facets, for scratching different itches. The initial motivation was to construct a scalable and efficient layer for developing the kind of platforms I’m used to. Was a personal thing; I wanted to build a really efficient web-server as a way to learn D. After that, it just grew to include a few handy tools, like the servlet engine and the basic clustering package, along with some i18n and XML tools that just weren’t available elsewhere.

I truly have no clue how many Mango users there are or how popular it is, but if it made someone happy then that’s cool.

I know that Tango arose out of discussions between Sean Kelly and yourself. Can you give a little insight into how those talks came to be? How exactly was Tango born?

Sean was doing a number of progressive things with Ares, way beyond what was downloaded with the compiler. Mango gained support for Ares early on, and it seemed that a merger might help to focus direction for both libraries. That happened in a big way, I think.

Tango, as a notion, was conceived in early April 2006 via a series of emails between Sean and I. After that, it was surprising how many were happy to get involved; Lars Ivar Igesund (larsivi) was the first. We’ve got some really capable people on board, and that’s where the Tango richness originates.

Tango, as it stands, is a fairly comprehensive library. Where do you see it heading in the future? What is the ultimate goal?

Tango will remain a medium-weight library; it’s not intended to be a Java-style runtime, with everything bundled. The core goals are to offer the kind of services we find useful, and to enable those in an efficient manner.

We decided early on that Tango would take the route of “library extensions”, instead of tossing the proverbial kitchen-sink into the core. The intent behind that is to provide “enough” useful facilities in the core to spur development elsewhere, but not to the point where it stifles innovation. For example, there are a few network-oriented clients in Tango, but no servers ~ there are many ways to skin the server cat, so we omit those to provide space for both innovation and alternatives. Clients are much duller, so we include them as a means to jump-start development.

What will Mango’s role be now that most of it has been integrated into Tango? Do you expect to expand it at all?

Just the lower level aspects of Mango made the migration into Tango, and were heavily revised during the transition. The ‘platform’ layers of Mango (servers, XML, etc) remained behind, since that’s not the kind of thing that belongs in Tango. Given that, I guess the role of Mango will remain a “network services” extension to Tango? It also stands as a reasonable example of how to apply Tango.

Aside from Mango and Tango, which D libraries and/or tools do you regularly make use of?

I’m still focused on the library at this time, so have limited use for anything but Build and Rebuild. When we turn more attention to building higher-level toolsets instead, a variety of other D projects will become much more useful. Right now the hands-down best tool for D, which I use daily, is MSVC6

Are there any specific, and currently unimplemented, libraries or tools that you would like to see developed in the near future? Any you’re thinking of undertaking yourself?

RAD tools, such as RoR, are a huge boon for a large number of people. Tools/platforms like that are of interest to some Tango contributors, so that’s likely where I’ll be focusing next. Personally, I’d love to do some fun things with D for cell-phones and other commodity devices. I think that’s where a large part of software development will be directed for the next 10 years.

A big thanks to Kris for taking the time to answer these questions. Can you believe he’s free to work on D stuff full time? Some guys have all the luck.

I already know who my targets are for the next few Q & A posts. One of them will be hearing from me in a week or so

Technorati Tags: D Programming Language, Tango, Mango

Posted in Q & A, Tango | 3 Comments »