Yet Another News Roundup
Once again, I've fallen way behind on D news. For a while, it was a bit dry. Then in the past few weeks a few announcements came and I kept putting off reposting them here. So, here's some of the stuff I've not blogged recently.
LDC has been accepted into the Fedora 14 official repository, so if that's your Linux flavor of choice you'll be able to use it quite easily. The package maintainer, Jonathan Mercier, also has packages for Tango and Derelict. Both of those are under review for inclusion in the official repo. I'm sure he'll let us know the result.
Speaking of Derelict, I've just given the build system another overhaul. You can now use the makefiles to compile with either DMD or LDC by specifying DC=dmd/ldc when executing make (e.g. 'make -flinux.mak DC=ldc').
David Simcha has evolved a neat little plotting package, now called Plot2Kill (formerly dflplot). The project, currently in alpha, is described as "capable of drawing basic scientific and statistical plots, including bar graphs, line graphs, histograms, scatter plots, heat maps, quantile-quantile plots, and ROC curves."
awishformore has released a binding/wrapper for SQLlite 3.6. Unfortunately, he doesn't seem to have provided a page about the project, only a direct download link. Normally, I don't like linking directly from here, but once again I'll make an exception. Here's a link to sqlite4d.rar. Currently supports D2 only.
FreepingCreature has announced FCC, a "simple compiled C-like language with a compiler written in D." Again, there's no project page, but there is a git repository that you can browse. Looks like it'd be fun to play with, but I don't see any information about the license.
Nick Sabalausky has announced Goldie 0.3, an engine for the Gold Parser Builder that supports D1/Tango. He says D2 support is planned for the future.
Jacob Carlborg has announced the initial release of Orange, a Boost-licensed serialization library that supports D1/D2, Phobos/Tango (I know what a headache that can be). The architecture is split into a front-end (the serializer) and a back-end (the archiver) so that you can implement custom archive formats.
On a side note, gone are the days when you could be content with just having Subversion installed for your D development. There are three different source control tools used in the projects above (svn, git and mercurial). So if you plan to use any of them in earnest, make sure you're up to speed on the ones you aren't familiar with. Personally, I'm still slow with git and hg.
D1/D2 Compatibility Annoyances
An annoying aspect of D for me these days, as a library maintainer, is keeping compatibility between D2 and D1. Derelict is very simple in terms of code. Aside from the shared library loading routines and a bit of logic in the DerelictGL package to help with OpenGL context management and extension loading, the majority of the project is just a bunch of enums and function pointer declarations. When I got started on the Derelict2 branch, the goal of which was to add support for D2 (in addition to the D1-Phobos/Tango support already there) I didn't anticipate any issues. I was wrong.
The first issue I came across was with const/immutable. Some of the Phobos routines I needed to call (particularly from std.string and std.conv) use simple char arrays in D1, but use immutable(char)[] in D2. This was easily fixed with aliases. This is also where the first annoyance came. Because of the nature of version statements, everything inside a version(D_Version2) block is still parsed in D1. I had seen this in the newsgroups before, so I knew the way around it was string mixins. That felt like a hack then, and it still does now. I mean, what's the point of a D_Version2 if we have to use mixins to do anything with it?
Next came the declarations of the C function pointers. At first, I was going to ignore const in the C declarations entirely, just as I do in Derelict 1. Besides, I thought, function parameters in this case are dealt with on the C side, so the declaration on the D side shouldn't matter. But, in the end I was convinced that this is impractical. So, I started making these declarations const correct. For function parameters, decorating with in rather than const does the trick and doesn't require separate declarations for D1/2. The return values are a different story. Fortunately, there are very few functions that declare const return values in the C libraries with which Derelict binds. So for those few cases, I aliased to a common type based on D version, again, however reluctantly, using string mixins.
I was sure that my D2 compatibility issues were behind me at this point and began to port more of the Derelict trunk over to the new branch. Then not so long ago, someone brought up a new issue in the forums. When using multiple threads and D2's shared paradigm, the function pointers in Derelict were being put in Thread Local Storage and were not usable outside of the thread in which they were loaded. The solution is to declare all of the function pointers __gshared. My reflexive, naive solution was to add a mixin("__gshared:") in front of the function pointer declarations in D2, and do nothing in D1. This, of course, doesn't work. As I should have remembered from the extern(C)/extern(Windows) problem a while back.
So now, in order to make sure that all the function pointers and any other global variables actually are global in D2, we need to declare them as __gshared, while declaring them normally in D1. The only solution I see that still fits with Derelict's philosophy of sticking as close to the original, bound APIs as possible, is to wrap all of the global declarations in string mixins. This, to me, is just absolutely ridiculous. It's not difficult to do and doesn't really add much to maintenance, but on principle it's just silly. A large portion of code in Derelict, somewhere around 50% or more I would guess, is about to become string mixins. Yuck. What a hack.
In C it would be a simple matter of a couple of preprocessor defines. We absolutely need something that simple in D, rather than resorting to mixing in all of the code in multiple modules. I think the majority of people who are concerned about D1/2 compatibility are library maintainers. I wonder, with such a hackish solution for maintaining compatibility, how many won't just shrug their shoulders and drop D1 support altogether? I'm tempted to, but I know that's quite impractical given that a number of Derelict users are also Tango users, which is D1-only.
So, am I the only one that would like a nice solution that can allow D1/2 compatibility without resorting to mixins? A D_Version2 block that is ignored in D1 would be a good start. Allowing statements like __gshared: to affect everything following the version block in which it's declared would be another. And if that's not possible, then something that achieves the same result is all I'd want. I'd love to hear opinions on this situation.
Since it's likely just wishful thinking on my part (this has been discussed in the newsgroups more than once and nothing has ever come of it), I'm already looking at ways to get the most bang for my buck. If I'm going to be forced to use string mixins, there's got to be something I can do to take advantage of them. One thing I'm considering is DerelictObjects. Wouldn't it be interesting if you could configure Derelict to use free functions, __gshared free functions, or to wrap the functions in classes based on your own preferences or project needs? Hmmm.
Derelict2 Updates
I've been making steady progress on Derelict2 lately. I still don't have a heck of a lot of time, but I'm sparing it when I can. Things are starting to shape up.
One big thing I'm happy about is that I've finally settled in to a nice pattern with the makefile build system. I abandoned makefiles in frustration during the early days of the project, but that was mostly because I wasn't used to using them. Now, they sit next to the dsss.conf files and do their job admirably. You can read about how to build the Derelict2 libraries, and how it differs from Derelict1 even when using DSSS, in a new sticky I put up in the forums. Proper documentation will come later on down the line.
Also, if you have an eye toward using DerelictGL in Derelict2, you'll want to know how it differs from Derelict1, as I've changed the extension loading interface.
Off the top of my head, I'm not sure what does and doesn't work on Linux/Mac/FreeBSD at the moment. I think DerelictSDL, at least, should be ready to roll across all platforms. With the exception of DerelictGL, if a package is working on multiple platforms in Derelict1, it should be ready to go in Derelict2 as well. And usually the only thing that's missing is the appropriate library name strings in the specific loaders.
I encourage anyone using Derelict to give Derelict2 a go and put it through the paces. My goal is to have everything implemented and tested well before D2 ships. That's when I hope to move it over to the trunk.
And since I haven't posted the URLs anywhere yet:
To browse the source: http://www.dsource.org/projects/derelict/browser/branches/Derelict2
And to check out via svn: http://svn.dsource.org/projects/derelict/branches/Derelict2
News Roundup
Finally, I have the time to sit down and knock up a post with a number of D news items I've failed to report over the past couple of weeks. If you've made an announcement in the NG or elsewhere and I fail to mention it here, I promise it's not intentional!
First, let's start with the dstats library. This project "attempts to provide some basic (and some not-so-basic) statistics functionality as a plain old library on top of D, as well as a bunch of utility code useful for implementing higher level statistical functionality." dsimcha announced that it was recently updated to make use of the new Phobos features introduced in DMD 2.029, such as ranges.
dsimcha also made available the RangeExtra module. It's a collection of ranges missing from the new Phobos, some of which might find their way into a future Phobos release. A documentation page and the source module are available under the Phobos license or BSD.
And there's still more from dsimcha. In the SVN trunk for dstats, you'll find a file named random.d. This is a port to D of the NumPy random number generators.
Moritz Warning announced a new version of Web-GMUI, which is "a remote web interface for MLDonkey, aMule, rTorrent, Transmission and giFT." This new release includes numerous bugfixes and an Italian translation.
Moritz also announced that he ported one of Kenta Cho's awesome little shooters, Titanion, to use Tango and Derelict. The original was written against Phobos using bindings from D-porting. Visit the Titanion project page at Sourceforge for more on the port.
Clay Smith managed to get h3r3tic's Hybrid GUI working on top of his ArcLib. You can read about it on Clay's blog.
Leandro Lucarella posted about his Naive Garbage Collector implementation. Read more about it and follow links of interest on his blog.
Uwe Keller announced the release of Evanescent v0.1. It's "a collection of tools for reasoning in propositional logic and provides implementations of inference engines in the D programming language." The project is hosted with Google Code, but also has a prescence at DSource.
Frank Fischer announced a completely rewritten version of LYLA, "a matrix and linear-algebra library for large scale matrices."
Steve Teale has updated the DCat web application server to build with both DMD 1.043 and 2.029.
And the last bit of news for this roundup: Mike Wey has announced GtkD 1.2.
Derelict2 Branch Begun
I've just committed to the Derelict repository the beginnings of the Derelict2 branch, in case anyone is interested. Currently it only contains DerelictAL, DerelictSDL, and DerelictUtil. Aside from a rearchitecting and some improvements internally, Derelict2 also adds D2 support.
You can read more about it over at the Derelict forums.