D1/D2 Compatibility Annoyances
An annoying aspect of D for me these days, as a library maintainer, is keeping compatibility between D2 and D1. Derelict is very simple in terms of code. Aside from the shared library loading routines and a bit of logic in the DerelictGL package to help with OpenGL context management and extension loading, the majority of the project is just a bunch of enums and function pointer declarations. When I got started on the Derelict2 branch, the goal of which was to add support for D2 (in addition to the D1-Phobos/Tango support already there) I didn't anticipate any issues. I was wrong.
The first issue I came across was with const/immutable. Some of the Phobos routines I needed to call (particularly from std.string and std.conv) use simple char arrays in D1, but use immutable(char)[] in D2. This was easily fixed with aliases. This is also where the first annoyance came. Because of the nature of version statements, everything inside a version(D_Version2) block is still parsed in D1. I had seen this in the newsgroups before, so I knew the way around it was string mixins. That felt like a hack then, and it still does now. I mean, what's the point of a D_Version2 if we have to use mixins to do anything with it?
Next came the declarations of the C function pointers. At first, I was going to ignore const in the C declarations entirely, just as I do in Derelict 1. Besides, I thought, function parameters in this case are dealt with on the C side, so the declaration on the D side shouldn't matter. But, in the end I was convinced that this is impractical. So, I started making these declarations const correct. For function parameters, decorating with in rather than const does the trick and doesn't require separate declarations for D1/2. The return values are a different story. Fortunately, there are very few functions that declare const return values in the C libraries with which Derelict binds. So for those few cases, I aliased to a common type based on D version, again, however reluctantly, using string mixins.
I was sure that my D2 compatibility issues were behind me at this point and began to port more of the Derelict trunk over to the new branch. Then not so long ago, someone brought up a new issue in the forums. When using multiple threads and D2's shared paradigm, the function pointers in Derelict were being put in Thread Local Storage and were not usable outside of the thread in which they were loaded. The solution is to declare all of the function pointers __gshared. My reflexive, naive solution was to add a mixin("__gshared:") in front of the function pointer declarations in D2, and do nothing in D1. This, of course, doesn't work. As I should have remembered from the extern(C)/extern(Windows) problem a while back.
So now, in order to make sure that all the function pointers and any other global variables actually are global in D2, we need to declare them as __gshared, while declaring them normally in D1. The only solution I see that still fits with Derelict's philosophy of sticking as close to the original, bound APIs as possible, is to wrap all of the global declarations in string mixins. This, to me, is just absolutely ridiculous. It's not difficult to do and doesn't really add much to maintenance, but on principle it's just silly. A large portion of code in Derelict, somewhere around 50% or more I would guess, is about to become string mixins. Yuck. What a hack.
In C it would be a simple matter of a couple of preprocessor defines. We absolutely need something that simple in D, rather than resorting to mixing in all of the code in multiple modules. I think the majority of people who are concerned about D1/2 compatibility are library maintainers. I wonder, with such a hackish solution for maintaining compatibility, how many won't just shrug their shoulders and drop D1 support altogether? I'm tempted to, but I know that's quite impractical given that a number of Derelict users are also Tango users, which is D1-only.
So, am I the only one that would like a nice solution that can allow D1/2 compatibility without resorting to mixins? A D_Version2 block that is ignored in D1 would be a good start. Allowing statements like __gshared: to affect everything following the version block in which it's declared would be another. And if that's not possible, then something that achieves the same result is all I'd want. I'd love to hear opinions on this situation.
Since it's likely just wishful thinking on my part (this has been discussed in the newsgroups more than once and nothing has ever come of it), I'm already looking at ways to get the most bang for my buck. If I'm going to be forced to use string mixins, there's got to be something I can do to take advantage of them. One thing I'm considering is DerelictObjects. Wouldn't it be interesting if you could configure Derelict to use free functions, __gshared free functions, or to wrap the functions in classes based on your own preferences or project needs? Hmmm.
How Neat It Is
This may sound silly, but when working with D I sometimes catch myself just sitting here admiring the beauty of a source file. It never ceases to amaze me how neat and concise D code can be. I don't think I've just sat and stared at a source file that way since I first started learning how to program. There's something quite gratifying about it at times. And now that I'm using D more than ever, I often find myself in quite the good mood when I finally close my text editor at the end of a coding session. I hope I'm not the only one 
Mayhem Intergalactic on Steam
Some of you may recall that the indie game Mayhem Intergalactic was created with D. I read a while ago at the Indie Gamer forums that it was going to be published on Steam, though I don't recall if Chris announced it in the D newsgroups. As I was checking Steam's latest releases, I saw that the day has arrived. As of a week ago, Mayhem Intergalactic is available to however many thousands of Steam users there are. Congratulations to Chris. I imagine he'll see a nice jump in sales from this. The majority of the users won't know or care that the game was made with D, but it's good news for D all the same.
DSSS and I
After Gregor stopped posting Rebuild binaries separately from DSSS, I downloaded the dsss-light version for Windows, which is DSSS without all of the net stuff. All I really wanted was access to the Rebuild executable without building it myself. I don't care how they do it in the Linux world, I want my binaries without having to compile them first, thank you.
So I don't really need, or want, DSSS to manage my libraries for me. I've got my own system that works just fine. That, actually, is one of the reasons I never bothered to use DSSS before nor wanted to use it when I downloaded it. I also didn't like that it generated .di files for everything. Nor did I like the idea that I had to learn how to create dsss.conf files in order to make it do what I wanted with my own projects. I was quite happy with using the simple response files I use for Bud and Rebuild.
Recently, I came to realize that it would be a good idea to add a dsss.conf file to the Derelict trunk. Since more and more people are using Tango with Derelict, and since the provided build script cannot be run by DMD when using Tango (at least on Windows, it has to be compiled into an executable first), DSSS seemed like it could potentially be a solution to the problem. So it was time to buckle down and dig into dsss.conf files.
As it turns out, creating a dsss.conf file is not nearly as complicated as the docs make it seem (Gregor did say doc writing isn't one of his strengths). It wasn't long before I had DSSS doing my bidding and building all of the Derelict libraries. There are still a few nits to work out, like I'm fairly certain that the DerelictSDL stuff is being compiled into every single DerelictSDL* library. I'd prefer that not to happen, but I haven't yet seen any obvious way to exclude entire packages (nor have I really dug around for one or asked Gregor). There's also a bug with library names that Gregor has already fixed for the next release. So when Gregor does release the next version, Derelict will be getting a dsss.conf file.
My intent was to provide dsss.conf alongside buildme.d, so that people who prefer can still run the build script. Some time ago I had refactored it to allow support for multiple build tools. That idea has now gone the way of the dodo, so I streamlined it and cleaned it up a bit just to support bud only. But now, I'm thinking, do I really need the build script? Providing two different methods of building the same project could potentially lead to confusion. I get enough confusion from just the single build script as it is. The majority of Derelict users don't bother reading the docs, judging by the majority of support requests I get in the forum. I may very well drop the build script and just require dsss if you want to build the libraries.
Today, I set some time aside to get started on the new, reinvented Smoothie. The first thing I would normally do is set up a couple of response files for Rebuild (or Bud as the case may be) for debug/release configurations. This time, I decided to go with a dsss.conf file instead. I still don't like that it generates .di files (is there a way to turn that off?) or creates extra files that I don't need, but it's not a big deal. Besides, a 'dsss distclean' solves that problem.
Still, if that were all, I'd see no real reason to give up Rebuild for DSSS. The big things for me are the ability to set up multiple builds in a single config file and the variety of hooks and commands available. I tend to configure my projects in such a way that I often have multiple test apps set up in their own packages. Using Rebuild or Bud, I'd need to configure a separate response file for each app. Also, I can already see how some of the hooks are going to be very useful for setting up a complex build process that involves manipulating more than just D source files.
I've said before in a NG post that DSSS solves problems I don't have. That's true, to an extent, but I see now that there's so much more to it than just library management and easy project compilation. To say I've warmed up to it is an understatement. I'll be replacing the response files for my game, and any other projects I have lying around, with DSSS configuration files. Color me converted.
Technorati Tags: D Programming Language, DSSS
On Closures in D
Slow news day for D, but there's a blog post over at hans-eric.com titled, D doesn't have real closures. Hans-Eric talks about D's delegates, how they are similar to closures, and how they aren't quite real closures because they don't keep surrounding variables alive after the scope has changed. His conclusion is that it's not a big deal, but at least one commenter so far disagrees with him.
Personally, I don't see it as a big deal either. For my purposes, I don't really need true closures in D. I do like to see them implemented in scripting languages, though. They are very handy for game AI. Since Lua added closures, that has become one of its major selling points for game developers. If I do come across a case where a true closure would come in handy, I have no problem making use of an inner class instead. D just has so many other useful features, such as anonymous delegates and lazy evaluation (two among many), that not having true closures doesn't bother me a bit.
Technorati Tags: D Programming Language, delegates, closures