The One With D News and Opinions of the Digital Mars D Programming Language

D1/D2 Compatibility Annoyances

An annoying aspect of D for me these days, as a library maintainer, is keeping compatibility between D2 and D1. Derelict is very simple in terms of code. Aside from the shared library loading routines and a bit of logic in the DerelictGL package to help with OpenGL context management and extension loading,  the majority of the project is just a bunch of enums and function pointer declarations. When I got started on the Derelict2 branch, the goal of which was to add support for D2 (in addition to the D1-Phobos/Tango support already there) I didn't anticipate any issues. I was wrong.

The first issue I came across was with const/immutable. Some of the Phobos routines I needed to call (particularly from std.string and std.conv) use simple char arrays in D1, but use immutable(char)[] in D2. This was easily fixed with aliases. This is also where the first annoyance came. Because of the nature of version statements, everything inside a version(D_Version2) block is still parsed in D1. I had seen this in the newsgroups before, so I knew the way around it was string mixins. That felt like a hack then, and it still does now. I mean, what's the point of a D_Version2 if we have to use mixins to do anything with it?

Next came the declarations of the C function pointers. At first, I was going to ignore const in the C declarations entirely, just as I do in Derelict 1. Besides, I thought, function parameters in this case are dealt with on the C side, so the declaration on the D side shouldn't matter. But, in the end I was convinced that this is impractical. So, I started making these declarations const correct. For function parameters, decorating with in rather than const does the trick and doesn't require separate declarations for D1/2. The return values are a different story. Fortunately, there are very few functions that declare const return values in the C libraries with which Derelict binds. So for those few cases, I aliased to a common type based on D version, again, however reluctantly, using string mixins.

I was sure that my D2 compatibility issues were behind me at this point and began to port more of the Derelict trunk over to the new branch. Then not so long ago, someone brought up a new issue in the forums. When using multiple threads and D2's shared paradigm, the function pointers in Derelict were being put in Thread Local Storage and were not usable outside of the thread in which they were loaded. The solution is to declare all of the function pointers __gshared. My reflexive, naive solution was to add a mixin("__gshared:") in front of the function pointer declarations in D2, and do nothing in D1. This, of course, doesn't work. As I should have remembered from the extern(C)/extern(Windows) problem a while back.

So now, in order to make sure that all the function pointers and any other global variables actually are global in D2, we need to declare them as __gshared, while declaring them normally in D1. The only solution I see that still fits with Derelict's philosophy of sticking as close to the original, bound APIs as possible, is to wrap all of the global declarations in string mixins. This, to me, is just absolutely ridiculous. It's not difficult to do and doesn't really add much to maintenance, but on principle it's just silly. A large portion of code in Derelict, somewhere around 50% or more I would guess, is about to become string mixins. Yuck. What a hack.

In C it would be a simple matter of a couple of preprocessor defines. We absolutely need something that simple in D, rather than resorting to mixing in all of the code in multiple modules. I think the majority of people who are concerned about D1/2 compatibility are library maintainers. I wonder, with such a hackish solution for maintaining compatibility, how many won't just shrug their shoulders and drop D1 support altogether? I'm tempted to, but I know that's quite impractical given that a number of Derelict users are also Tango users, which is D1-only.

So, am I the only one that would like a nice solution that can allow D1/2 compatibility without resorting to mixins? A D_Version2 block that is ignored in D1 would be a good start. Allowing statements like __gshared: to affect everything following the version block in which it's declared would be another. And if that's not possible, then something that achieves the same result is all I'd want. I'd love to hear opinions on this situation.

Since it's likely just wishful thinking on my part (this has been discussed in the newsgroups more than once and nothing has ever come of it), I'm already looking at ways to get the most bang for my buck. If I'm going to be forced to use string mixins, there's got to be something I can do to take advantage of them. One thing I'm considering is DerelictObjects. Wouldn't it be interesting if you could configure Derelict to use free functions, __gshared free functions, or to wrap the functions in classes based on your own preferences or project needs? Hmmm.