To date, my Q&A victims have been battle-scarred veterans of different software industries. Their opinions on D are tinted by years of experience with different languages and projects. This month, I wanted to get at someone who has a different perspective.

Jarrett Billingsley is a prominent member of the D community who hasn’t worked professionally in the software industry. Currently attending college as a computer science major, Jarrett has contributed two projects to the D community: nonagon, a DirectX-based 3D game engine, and MiniD, a D scripting language that currently sits in a unique position in D-Land.

While I was in #D the other day, I asked Jarrett if he would be willing to answer a few questions about his background, D, and his projects. He accepted. I emailed him some questions a day or so later. Following is the result, with my questions in bold and his answers in plain text.

Note: ‘Python’ is escaped in single quotes in this text in order for me to be able to post this to the blog. See my post titled Blog Upgrade for details.

Your DSource profile says you are a CS student. Which year are you in now? What is the most interesting course you’ve taken so far?

I’m currently a CS student at the University of Pittsburgh.  I’ve just completed my second year, so if all goes well I should have two years left, unless I decide to go to graduate school.

So far, most of the classes I’ve taken in CS have been introductory classes — programming, some computational theory, logic.  They reserve most of the fun classes for the upper-level students   If I had to choose from the ones that I’ve taken, though, it would have to be my “Computer Organization and Assembly Language” course (CS 0447 at Pitt).  The course was about the basic architecture of computer processors, getting down into the register file and the ALU and microprogramming and such, which I found interesting in and of itself.  The main reason I liked the course so much, though, was the professor, Dr. Mark Kahrs.  He doesn’t teach the class very often, but I’m very glad I had him.  He was one of the few professors I’ve had who seemed genuinely excited about the material he was teaching.  He always had a plan, he came up with challenging assignments (write an emulator for an old vector graphics device in MIPS assembly!), his tests actually tested your ability to think and reason rather than memorize, and he knew what he was talking about.  I really respect the man and hope I have the opportunity to take another class with him.

Once you graduate, is there any particular industry you’d like to work in? Have you done any internships or such?

To be honest, I haven’t thought much about what I’d like to do after college!  I’ve always kind of expected that I’d end up doing something with programming, though.  Whether I’ll end up in a typical business setting is an entirely different story.

I originally wanted to go to Full Sail in Florida after graduating from high school.  It’s a college that specializes in entertainment technology, and I was very interested in their video game design and development degree.  The price tag, the lack of on-campus housing, and the proposition of moving 1,000 miles from home only three months after graduating from high school changed my mind. After two years at Pitt, though, my desire to work in the video game industry has started to fade, and I’ve been thinking of a variety of other possibilities: going into an academic career, working at some kind of business startup, or maybe even doing something completely off the wall like starting a bakery and keeping programming a hobby!  As of now, I’m leaning towards an academic career, but we’ll have to see what comes of that.

As for internships, I’m in one right now.  It’s my first job outside of a grocery store and it feels great.  I’m working at the University of Pittsburgh Medical Center as part of an in-house software development team.  We do everything with .Net and C# (which is not a bad language, I’ll admit).  I’m not working on the most exciting programs, and I still don’t know whether or not I like the office environment, but it’s certainly a lot better than getting $6.80 an hour putting apples on a shelf.

When did you first start programming? Which languages have you used? Were you self-taught?

I started when I was about 13 or so.  Being addicted to the Sega Genesis and the N64, my initial desire was to make video games.  To that end, I started using a 3D game language called Dark Basic.  Looking back at it, it’s a terrible language with an even worse 3D engine, but at the time I was blown away   It really took me a year or so before programming ‘clicked’ and I was able to begin effectively translating my ideas into code.

I used Dark Basic and its successor Dark Basic Pro for a few years.  I never really made anything of consequence with it, but it certainly got me thinking about things.  After Dark Basic I moved on to its competitor, Blitz3D.  After participating in numerous flamewars between the two communities defending Dark Basic, I tried out B3D and found out it had a much more capable language and better engine.  To this day I still use it for simple graphical programs, testing out 3D concepts, and for making other little utilities.  It’s an incredibly capable language, probably a little better than C in terms of its data and function capabilities (though not as low-level), and it even has a form of RTTI which can be used to simulate polymorphism.  I used B3D to write a tech demo for my high school senior project, and I was just blown away at not only the capability of the language, but of the engine as well. You’d be amazed at what you can squeeze out of the DirectX 7 fixed-function pipeline.

By this time I had taken an interest in C++, because I was getting tired of some of the limitations of languages like Dark Basic and B3D.  They were proprietary, so I could only work with other programmers who owned the language, they were relatively slow, they were only good for games, their data structuring mechanisms were lacking, and they were hard to extend.  I thought I could do better by learning C++ and writing my own game engine.  I started with DirectX because both Dark Basic and B3D were based on it.  What started as a program that basically set up the device and drew some teapots eventually grew (VERY slowly) into a Blitz3D-style engine.  It only did the simplest things — basically you could load 3D objects, texture them, and move them around.  Once I started learning more about classes, I started refactoring the design into a more OOP style, updated to DirectX 9, and renamed it nonagon!

I never had any formal training in programming.  I went to a rural school district so we didn’t have any kind of computer classes beyond “using Word”.  Everything I learned, I taught myself.  It probably took me longer than the people who are just learning programming in college now, but I’ve got quite a head start, and programming is second nature to me by now.

I also know some Java because of school, but think it’s a terrible language.  It’s amazingly inexpressive and inefficient for the sake of “compile once, run anywhere.”  I’ve never actually used a Java program which perhaps speaks for how well that philosophy has panned out.  I also know C# from work, and respect it much more than Java.  If I had to program in a mainstream language other than D for a career, it’d probably be C#.

How and when did you first learn of D? What was it about the language that hooked you?

In a language discussion in a Dark Basic community forum, I posted about my exploits with C++, saying that I liked its power but couldn’t stand how complex it was.  In response, someone (wish I could remember who) replied with a link to the D site, saying that it seemed like a neat language.  I went, started reading the spec, and it just went from there   This was in the fall of 2003. The first version of D that I downloaded was 0.74.  I didn’t really start really using D, though, until almost a year later, around 0.95.

I think when I got hooked was when I was reading the Overview page, in the “features to keep/drop” and the “who D is/is not for” sections.  I remember reading “Those who decide the promise of C++ object oriented programming is not fulfilled due to the complexity of it” and I knew that this was the language for me.

After having worked on nonagon for about a year and a half in C++, I was really starting to hate it.  I became aware of the concept of “memory management” and that I was responsible for cleaning up all the stuff I allocated.  I started reading articles about smart pointers and reference counting and stuff, and I basically said “that’s it, I’m done.”  I gave up on nonagon because C++ was making it so ugly.

What was weird about D is that everything seemed to work the way I expected C++ to.  Arrays knew their length; objects cleaned up after themselves; downcasting a class reference actually did something worthwhile (I didn’t know about dynamic_cast in C++ at the time).  It was like I had been trying to program in D the entire time I had used C++.  That’s probably what appealed to me the most — the language just worked the way I expected it to.

Most D programmers come to the language after having used other languages professionally, often for several years. In your case, without all of that experience behind you, how was the learning curve for D? Was it easy to pick up?

I found it very easy to pick up.  When I switched to D, I still wasn’t that clear on most OOP concepts (most likely because the C++ OOP model is just a mess).  It only took me a few months for that stuff to really click, and it mostly came about during the time when I was translating nonagon from C++ to D.  That was a really fun experience — being able to copy and paste code, remove the quarter that was devoted to memory management, combine the declarations and definitions, remove all the -> operators, and see the clean, simple logic and structure that was underneath that ugly C++ exterior.

What’s funny is how far my abilities have come even in the past couple of years.  I look back at some of the old nonagon code now and think “man, this is messy.  I could do it so much better this other way.”  Probably the best thing about learning D, though, is that it scales nicely.  Any beginner Java programmer would be able to write useful programs in D using only simple OOP concepts.  I did that with nonagon at first as well.  But when you get better at programming and want more power, D doesn’t fail to impress — operator overloading, custom memory management, templates that would make C++ programmers drool, compile-time function evaluation and so on.  Don’t believe the D overview page that says D is a good second language — I think it’s just as good a first language as Java is, but it provides features to satisfy you as you improve.

What are some things you really like about D? What don’t you like about it?

They’re kind of underrated, but dynamic arrays and associative arrays as core data types of the language are probably some of the best features.  Most of my classes are just wrappers around these two incredibly powerful constructs.  They mean I don’t have to deal with clumsy ArrayList/Vector, Hashtable, Stack, or Set classes.  It’s to the point where any language I use that doesn’t have them as core types feels crippled.

Some other things I like:  No function or class prototypes.  Fast compile times (I can compile MiniD, which is just over 20,000 lines I believe, as well as a large chunk of Tango in about two or three seconds on my 1.6GHz Athlon).  Templates that make sense, compile fast, and look nice. Separation of identity versus equality.  The ability to interface with C code.  The abandonment of the ugly preprocessor (even C# still has #if — ugh), replacing it with actual language features.  Foreach, the best loop ever.  Delegates and inline delegate/function literals.  Nested functions in general.  Scope class references.  Scope statements.  Finally.  IFTI.  Automatic virtualization of class methods and destructors.  String mixins + CTFE + templates = wow.

I don’t have that many complaints about D, but they’re there.  The “calling a function as a property” syntax — I think explicit properties are a better way to go.  The adoption of some odd properties of C++’s OOP model for no reason besides “it’s the way C++ does it.”  One, a derived class defined in a different module from its parent class may access its own protected members, but not the protected members of other instances of the base class.  Two, base class methods do not overload with derived class methods unless you explicitly put in an alias to do so (seen lots of threads on the newsgroups about that one).  Some other things: no requirement to put ‘ref’ or ‘out’ on parameters at the call site as in C#.  No requirement to put ‘override’ on overriding methods as in C#.  Lack of static closures, though that can be simulated somewhat satisfactorily.  Lack of scope for anything other than class references (scope arrays, anyone?).  Uhh, no way to use $ in slices/indexing for a user-defined type.  I’m really scraping here

Of course there are some “non-language features” that I’m not particularly fond of.  I feel like the language which is supposedly “community-driven” is very rarely so.  I feel like more emphasis is placed on “cool new features to lure over C++ programmers” than on fixing bugs.  I feel like more credence is given to leading academic voices in the C++ world than to practical experience from programmers who have used D extensively in their own projects.  And I feel like the development of D is a haphazard, mysterious process to which only a select few are privy.  I’d like to see more updates like the few updates before 1.0 (50 bugfixes?  I’ll take that over a single new feature), a more open development process, and for the actual users of D to have a say in what they think is best for the language.

The first publicly announced D project you worked on (that I know of) was nonagon. There aren’t many D projects that I’ve seen using DirectX. What prompted you to create nonagon and to choose DirectX?

As was mentioned before, I started nonagon in C++ in an attempt to emulate some of the game languages I had used, both of which used DirectX.  I’ve also never really used any OS besides Windows extensively until very recently, so cross-platform compatibility was not high on my list when I started it.  It was actually kind of a project getting all of DirectX to work with D in the first place, since the core library is easy to link against, but DirectX had some static libraries that were impossible to use from D.  I think I was the first to actually get D3DX to work with D, which is kind of nice

I’ve been thinking about revamping nonagon, as I haven’t really touched it in the past year and a half.  I’ve been thinking of making it API-agnostic, or just based on OpenGL, SDL, and such.  But that’s quite an undertaking, and with a job and MiniD on my plate this summer, I don’t know if it’ll get done any time soon.  That’s mostly why I made the project open-source recently, so in case I never get back to it, it still might be useful for someone!

MiniD is, to me, a tremendously interesting project. The project page has an informative section regarding why you created it. How have the design and implementation been for you? Has it been a difficult process or an easy one? Did you have experience with creating scripting languages or have you been learning it all as you’ve gone along?

I’ve always been interested in scripting languages for some reason — I made one or two really simple ones in Dark Basic, as well as a relatively complex assembly-like language in Blitz3D.  MiniD actually started as a culmination of two small “messing around” projects that I worked on in the spring of 2006.  One of them was a compiler for a simple C-style language that never had a name.  I wanted to see how far I could get without any kind of help, and it got pretty far.  I wrote the lexical, syntactic, and part of the semantic passes before I realized that I didn’t really know what I was doing anymore.  Of course, I look at the code now and it looks like an absolute joke   The other project was an attempt to make a D implementation of Lua.  I found out about Lua and thought that it was really cool, but didn’t like the C API at all.  I actually translated the entire Lua compiler into D, and it generates code that is almost bit-for-bit identical to the code that the official Lua compiler generates.  After I did that, though, I looked at how much more of the language I had yet to port, and thought “why am I doing this?  Why don’t I just make a language of my own?”

The initial design of MiniD was a statically-typed language, kind of like D but with many of the features stripped out.  The language was never implemented; it was all specifications in the DSource wiki.  I wanted to make the compiler right this time, so I followed the lead of the DMD frontend, using very similar patterns of lexing and parsing.  When I started with the compiler, I started to realize the complexity of the project — having only implemented the lexical and syntactic passes of the compiler, it was already approaching 7,000 lines of code, and the semantic pass was going to be by far the largest part.  Then would have to come code generation, the virtual machine and the standard libraries.  I realized that the language I had created was about as complex as C# and that perhaps I was in too deep!

I retargeted the language as a Lua-like embeddable extensible scripting language instead.  By switching to dynamic typing and dropping a large proportion of the features, the project suddenly became much simpler.  I had a (kind of) working compiler in a matter of weeks, and the VM followed soon after.  I was and still am particularly proud of the compiler — it’s based on the DMD compiler for lexing and parsing, and uses my own “expression stack” to generate the code.  It’s basically a stack machine in the compiler that converts “popping” operations into RISC-style instructions.  That was one hell of a learning experience   But the compiler is very modular and easy to modify; it made it absolutely trivial to remove the static typing when I refactored the language, and it’s been incredibly easy to modify and extend the language with it ever since.  I’ve also used the compiler as a basis for a few other small personal projects that involved parsing of some sort, which speaks for how flexible it is.  It really taught me a lot about compiler design and I feel a lot better having implemented it myself instead of using something like flex/bison or yacc.

MiniD has, of course, evolved quite a bit even after I made it a dynamic language.  The very first useable release had no standard library, but was basically an implementation of Lua with C-style syntax.  I realized that I set out to create my own language, but had just ended up copying Lua instead!  So I started taking features that I liked from other languages — classes from Squirrel, array slicing and modules from D, more robust list manipulation functions from Lisp (which I was playing around with at the time).  Slowly, the syntax changed, the compiler and VM matured, and the current incarnation of MiniD emerged.  It’s now become closer to Squirrel than to Lua, though it’s got its own features, like syntactic support for coroutines and the module system.  I really liked how Squirrel solved the method call problem (if you have a.b(), should it index b and just call it, or pass a as the first parameter?  passing an implicit ‘this’ to all functions, method or not, solves this), and that has made the language more traditional-looking than before as well.

I wouldn’t say that the process was particularly easy or difficult — I found it challenging, of course, and most of all fun.  All the “scripting languages” I wrote before it were extremely simplistic, usually barely Turing-complete and usually without a compiler (typing in bytecodes into a hex editor == fun).  MiniD was a huge departure from what I had done before and has been an amazing learning experience.  I also can’t downplay the influence of other languages on its development — most of what I learned about implementing the language I learned from reading the Lua source, as well as Squirrel, Io, and the DMD frontend.  They have been wonderful references.

Where are you planning to go with MiniD? What sort of features can we expect to see in the future? What sort of applications do you think the language is ideal for?

I’m not real sure!  Right now it’s targeted mostly at the same kinds of tasks that Lua is: adding scriptability to a host application, while being extensible itself so that scripts can access whatever the host app wants them to.  Even so, the language is really starting to mature and take on a personality of its own, and can probably be used for many independent programs as well.  One of my friends wrote a small game in ‘Python’ using pygame for a recent coding competition (our team used D, surprise surprise), and I did some translation of the ‘Python’ source into MiniD to see how well it fared.  I was pleased with how easily the translation went, and only ran into a few things that ‘Python’ could do that MiniD couldn’t.  Because of this, I wouldn’t be surprised if MiniD would be able to do as well as ‘Python’ for small apps.  Of course, it doesn’t have the standard library that ‘Python’ does, but that can change!

As for future features: working on that   I’ve frozen the current featureset until 1.0 comes out, so virtually all the changes until then will just be API changes.  After that, though, the 2.0 beta branch can begin!  I’ve had some ideas such as separating indexing from field access and coming up with a new ‘symbol’ type for faster indexing and to make it possible to implement properties without stepping on the indexing operator’s toes.  List comprehensions are really sweet and would be awesome to have in the language.  Some kind of non-collaborative threading (as opposed to coroutines) would also be nice to have.  Some kind of parameter type constraints for functions is also possible.  Beyond that, I’m not sure — MiniD’s flexibility in terms of where and how you can declare functions and classes places some restrictions on the language.  I’d like to keep MiniD as a scripting and extension language; making it a whole language in its own right is a tempting idea but would sacrifice a lot of the flexibility in order to gain some other features.  Maybe it could branch into two languages — a lightweight scripting language and a heavyweight apps language!  Man that’s a lot of work

I’ve also had non-language ideas, such as a competing library to pygame (MDGame perhaps?) and a simple cross-platform GUI framework based on OpenGL and SDL and using MiniD for the logic.  These libraries got me interested in loading dynamic libraries into MiniD, which will hopefully be possible once DDL gets updated and converted to Tango.

What would I think that MiniD is ideal for?  Well, Lua has been used extensively in many games and applications.  Moho (now Anime Studio), a 2D animation program, exposes a Lua interface for writing plugins, and there has been a great proliferation of them.  Lua plugins can then be integrated into the UI as if they were native tools.  That’s a great example of the power of scripting languages.  Some games, like FarCry, use Lua to write a large majority of the game code.  They write the engine and the performance-intensive code in the host language, and then use Lua to write virtually the entire game.  Lua is also used in the user interface of World of Warcraft, and for scripting in many other games.  MiniD could be used for much the same purposes.  MiniD is expressive enough to allow for useful code to be written with very little effort, and the API does and will make it extremely easy to interface between native code and MiniD code.

With some help from external libraries (like MDGame or the GUI framework), MiniD could also be a very capable rapid prototyping language.  It’s already got a module system and classes, as well as semantics very similar to D’s, meaning that you could come up with a quick idea in MiniD, see the results right away, and then convert it to D code once the idea has solidified.

Lastly I think MiniD could be a good learner’s programming language.  It gets you used to the C-style syntax that has become virtually ubiquitous in mainstream programming languages without exposing you to a lot of low-level detail.  It can provide instant feedback through a command-line interpreter, making it possible to have interactive labs where students can try things out and see what they do right there.  It can also be used to teach about code modularity and encapsulation with the module system, and about OOP as well.

Aside from your own projects, which D libraries and/or tools do you regularly make use of?

Well I recently converted MiniD over to using Tango as the standard library.  I really like the idea of a community-driven standard library, as it feels like I can make a difference.  I can’t complain about the turnaround time for bugfixes either — I put in a ticket and it’s fixed in a day, an hour, or even a couple minutes!  I’m not entirely fond of all of Tango, but that’s probably just because I’m used to Phobos.  Tango also provides some really useful features that Phobos doesn’t at all — Fibers, proper handling of all Unicode encodings, more flexible IO and formatting, and extensive array utilities to name a few.  I could really see Tango becoming the library that finally gets D the attention it deserves, especially if other “pluggable” libraries are built on it, such as GUI toolkits.  A lot of Java and C# programmers have become spoiled by the extensive standard libraries that those languages provide, and they’re beginning to expect the same level of capabilities from systems programming languages like D.

The other tool I use is Derek Parnell’s Build/Bud.  I have virtually replaced my personal batch file compilation system with Build and response files.  It’s so convenient to be able to have compilation that’s almost as automatic as you get with an IDE from a command-line tool.  And it’s cross-platform to boot!

I’ve looked at DSSS too, but I haven’t had much experience with it or use for it.  I could use DSSS/Rebuild to build my apps, but so far Build has suited me well.  Nonetheless MiniD is available through DSSS’s net feature and compiles (usually) with Rebuild as well.  I haven’t needed DSSS much though, because I don’t make much use of other peoples’ libraries — I’m usually writing my own!

I also used Derelict in a small “game” that I made for an informal game coding competition this past spring.  Long story short: it’s great.  No trouble using it, easy to set up, completely unintrusive.  I didn’t get to test if it worked on anything but Windows but there’s no reason it shouldn’t.

Lastly, after a short but tumultuous stint with DLLs in D, I became interested in DDL.  The idea of having a sort of Java-y .Net-y run-time code binding system in a natively compiled language is incredibly tempting and could open up a lot of possibilities.  I’m hoping it’s everything it seems to be so that loadable native code libraries can be implemented in MiniD.

Are there any specific, and currently unimplemented, libraries or tools you would like to see developed in the future? Any you’re thinking of implementing?

I’d like to see a proper Unicode library, maybe based on IBM’s ICU library.  D has already proven itself as capable in the area of numeric programming with support for extended-precision reals and complex/imaginary datatypes.  Given its ability to handle all three major encodings of Unicode, it could very easily become a great text-processing and internationalization language as well.  That being said, I probably wouldn’t even think of implementing something like this   There are programmers out there who are really interested in this area, but I’m not one of them.

Another area where I think D is really promising is for games.  D could really use a “killer” game engine or 3D framework.  (No, nonagon’s not that engine )  C++ has long been considered the de facto standard for writing games because it allows so much control of memory management.  Now that games are getting more complex and computers are getting more cores, wasting development time on memory management that could be spent writing more complex AI or supporting multiple cores is starting to seem stupid. D already provides custom allocators and deallocators for classes and structs, just as C++ does, as well as allocating classes on the stack.  An advanced GC implementation (such as an incremental GC that doesn’t “stop the world” running on a separate core which can be highly tuned for different kinds of memory allocation strategies within the same program) coupled with custom allocation schemes for some classes and structs could make games written in D considerably simpler than their C++ counterparts while offering almost the same performance.

And it’s been said once, it’s been said a million times — D needs a proper IDE.  I think it’s been discussed enough, but many large companies/programming teams make heavy use of IDE features, and without it, few business people will take D seriously.

A big thanks to Jarrett for taking the time to do this. He even went above and beyond and formatted the answers for me in a text file!

Technorati Tags: D Programming Language, nonagon, MiniD, scripting languages