In the makefile, I had what I figured will be the most commonly used extension families on by default: EXT, ARB, NV, ATI and AMD. The user had only to add other needed families when building the library, or just comment out the line and uncomment another line that specifies DerelictGL_ALL for all extensions. It all seemed like a fine idea at the time. Unfortunately, given that I haven’t had any projects at all going on for a while, I haven’t been eating my own dog food.

A few days ago there was a post in the Derelict forums specifying a problem that seemed puzzling on the surface, but to which I thought I had a simple solution. When compiling anything using OpenGL extensions, errors like the following were popping up:

Error: undefined identifier glGenFramebuffersEXT
Error: undefined identifier GL_FRAMEBUFFER_EXT
Error: undefined identifier glActiveTextureARB
…

Obviously, thought I, he’s compiling DerelictGL without specifying any versions, meaning he’s not using the provided makefiles or DSSS.  Of course, when I pointed out what the problem might be, I received a reply that shot it down. If you are a regular D user, you will understand where this is going. Unless you’re having a brain cramp like I did.

I was puzzled. Confused. Bewildered. So I took the test case and compiled it myself. I got the same errors. Luckily, I already knew somewhere in my subconsciousness what was going on, so it didn’t take that long to bubble to the surface. But I did go through a couple of false starts, thinking first that there was a problem with version statements and string mixins. Then, after posting about it in the forums, I realized that string mixins have nothing to do with it. So it was a problem with version statements. Then, after starting a new post in the forums, I realized that it wasn’t a problem at all. This is how version statements work.

If you’re a newcomer to D, or just haven’t caught on yet, the problem is simple. When you create a D program using a library that has been compiled with specific version statements, the library binary will contain the symbols defined by those versions. However, when compiling your program, the compiler has no idea which versions were used to compile the library. So if you add any types or function calls that are declared in version statements in the library modules, the compiler will rightly complain that they don’t exist. They aren’t visible when compiling your program unless you add the same versions to your command line. The converse of this is that if you use versions to compile your program that weren’t used to compile the library, you’ll get errors when linking–anything defined in version statements from imported modules will be visible to the compiler, but the symbols won’t be present in the library.

For some reason, I had it in my head that simply compiling the library with the appropriate version statements would be all that is required. Boy, do I feel silly. As a result of this, I’ve turned on support for all OpenGL extensions by adding version=DerelictGL_ALL to the relevant source modules. Doing it in the makefile would result in the same problem, requiring users to add -verison=DerelictGL_ALL to the command line when building their apps. So if you pull down Derelict2 from svn and want to use OpenGL extensions, you shouldn’t be surprised with compilation errors from Derelict anymore.

Now, the question I have is this. Is this behavior desirable? This is analogous to issues with libraries using #define to turn features on and off  in C and C++ (silly you, forgetting to add #define USE_DOUBLES_INSTEAD_OF_FLOATS when linking with that nifty, precompiled physics library). But despite the fact that I’m a long time C user accustomed to this sort of behavior (not to mention the number of years I’ve been around D), I still expected versions in D to behave differently. Wouldn’t it be cool if it did? What if there were a way to automatically pull in the versions a library uses when compiling your app, so that you don’t have to bother adding anything to the command line? Food for thought.

The first issue I came across was with const/immutable. Some of the Phobos routines I needed to call (particularly from std.string and std.conv) use simple char arrays in D1, but use immutable(char)[] in D2. This was easily fixed with aliases. This is also where the first annoyance came. Because of the nature of version statements, everything inside a version(D_Version2) block is still parsed in D1. I had seen this in the newsgroups before, so I knew the way around it was string mixins. That felt like a hack then, and it still does now. I mean, what’s the point of a D_Version2 if we have to use mixins to do anything with it?

Next came the declarations of the C function pointers. At first, I was going to ignore const in the C declarations entirely, just as I do in Derelict 1. Besides, I thought, function parameters in this case are dealt with on the C side, so the declaration on the D side shouldn’t matter. But, in the end I was convinced that this is impractical. So, I started making these declarations const correct. For function parameters, decorating with in rather than const does the trick and doesn’t require separate declarations for D1/2. The return values are a different story. Fortunately, there are very few functions that declare const return values in the C libraries with which Derelict binds. So for those few cases, I aliased to a common type based on D version, again, however reluctantly, using string mixins.

I was sure that my D2 compatibility issues were behind me at this point and began to port more of the Derelict trunk over to the new branch. Then not so long ago, someone brought up a new issue in the forums. When using multiple threads and D2′s shared paradigm, the function pointers in Derelict were being put in Thread Local Storage and were not usable outside of the thread in which they were loaded. The solution is to declare all of the function pointers __gshared. My reflexive, naive solution was to add a mixin(“__gshared:”) in front of the function pointer declarations in D2, and do nothing in D1. This, of course, doesn’t work. As I should have remembered from the extern(C)/extern(Windows) problem a while back.

So now, in order to make sure that all the function pointers and any other global variables actually are global in D2, we need to declare them as __gshared, while declaring them normally in D1. The only solution I see that still fits with Derelict’s philosophy of sticking as close to the original, bound APIs as possible, is to wrap all of the global declarations in string mixins. This, to me, is just absolutely ridiculous. It’s not difficult to do and doesn’t really add much to maintenance, but on principle it’s just silly. A large portion of code in Derelict, somewhere around 50% or more I would guess, is about to become string mixins. Yuck. What a hack.

In C it would be a simple matter of a couple of preprocessor defines. We absolutely need something that simple in D, rather than resorting to mixing in all of the code in multiple modules. I think the majority of people who are concerned about D1/2 compatibility are library maintainers. I wonder, with such a hackish solution for maintaining compatibility, how many won’t just shrug their shoulders and drop D1 support altogether? I’m tempted to, but I know that’s quite impractical given that a number of Derelict users are also Tango users, which is D1-only.

So, am I the only one that would like a nice solution that can allow D1/2 compatibility without resorting to mixins? A D_Version2 block that is ignored in D1 would be a good start. Allowing statements like __gshared: to affect everything following the version block in which it’s declared would be another. And if that’s not possible, then something that achieves the same result is all I’d want. I’d love to hear opinions on this situation.

Since it’s likely just wishful thinking on my part (this has been discussed in the newsgroups more than once and nothing has ever come of it), I’m already looking at ways to get the most bang for my buck. If I’m going to be forced to use string mixins, there’s got to be something I can do to take advantage of them. One thing I’m considering is DerelictObjects. Wouldn’t it be interesting if you could configure Derelict to use free functions, __gshared free functions, or to wrap the functions in classes based on your own preferences or project needs? Hmmm.

So I don’t really need, or want, DSSS to manage my libraries for me. I’ve got my own system that works just fine. That, actually, is one of the reasons I never bothered to use DSSS before nor wanted to use it when I downloaded it. I also didn’t like that it generated .di files for everything. Nor did I like the idea that I had to learn how to create dsss.conf files in order to make it do what I wanted with my own projects. I was quite happy with using the simple response files I use for Bud and Rebuild.

Recently, I came to realize that it would be a good idea to add a dsss.conf file to the Derelict trunk. Since more and more people are using Tango with Derelict, and since the provided build script cannot be run by DMD when using Tango (at least on Windows, it has to be compiled into an executable first), DSSS seemed like it could potentially be a solution to the problem. So it was time to buckle down and dig into dsss.conf files.

As it turns out, creating a dsss.conf file is not nearly as complicated as the docs make it seem (Gregor did say doc writing isn’t one of his strengths). It wasn’t long before I had DSSS doing my bidding and building all of the Derelict libraries. There are still a few nits to work out, like I’m fairly certain that the DerelictSDL stuff is being compiled into every single DerelictSDL* library. I’d prefer that not to happen, but I haven’t yet seen any obvious way to exclude entire packages (nor have I really dug around for one or asked Gregor). There’s also a bug with library names that Gregor has already fixed for the next release. So when Gregor does release the next version, Derelict will be getting a dsss.conf file.

My intent was to provide dsss.conf alongside buildme.d, so that people who prefer can still run the build script. Some time ago I had refactored it to allow support for multiple build tools. That idea has now gone the way of the dodo, so I streamlined it and cleaned it up a bit just to support bud only. But now, I’m thinking, do I really need the build script? Providing two different methods of building the same project could potentially lead to confusion. I get enough confusion from just the single build script as it is. The majority of Derelict users don’t bother reading the docs, judging by the majority of support requests I get in the forum. I may very well drop the build script and just require dsss if you want to build the libraries.

Today, I set some time aside to get started on the new, reinvented Smoothie. The first thing I would normally do is set up a couple of response files for Rebuild (or Bud as the case may be) for debug/release configurations. This time, I decided to go with a dsss.conf file instead. I still don’t like that it generates .di files (is there a way to turn that off?) or creates extra files that I don’t need, but it’s not a big deal. Besides, a ‘dsss distclean’ solves that problem.

Still, if that were all, I’d see no real reason to give up Rebuild for DSSS. The big things for me are the ability to set up multiple builds in a single config file and the variety of hooks and commands available. I tend to configure my projects in such a way that I often have multiple test apps set up in their own packages. Using Rebuild or Bud, I’d need to configure a separate response file for each app. Also, I can already see how some of the hooks are going to be very useful for setting up a complex build process that involves manipulating more than just D source files.

I’ve said before in a NG post that DSSS solves problems I don’t have. That’s true, to an extent, but I see now that there’s so much more to it than just library management and easy project compilation. To say I’ve warmed up to it is an understatement. I’ll be replacing the response files for my game, and any other projects I have lying around, with DSSS configuration files. Color me converted.

Technorati Tags: D Programming Language, DSSS

Personally, I don’t see it as a big deal either. For my purposes, I don’t really need true closures in D. I do like to see them implemented in scripting languages, though. They are very handy for game AI. Since Lua added closures, that has become one of its major selling points for game developers. If I do come across a case where a true closure would come in handy, I have no problem making use of an inner class instead. D just has so many other useful features, such as anonymous delegates and lazy evaluation (two among many), that not having true closures doesn’t bother me a bit.

Technorati Tags: D Programming Language, delegates, closures

Unfortunately, this issue goes beyond Derelict. There are several OpenGL bindings out there that are affected by this. If you are using one of them and have not yet upgraded your compiler, you will be in for a bit of a shock when you do. Here’s a list of the bindings I know of that use the same, no-longer-legal technique:

Bindings – both the gl and glu modules are affected

D-porting – both opengl.d and openglu.d, found in the opengl.zip file, are affected

GLFW – three modules (in support/d/imports in the directory tree) are affected: gl.d, glu.d and glfw.d

If you are using any of the above modules, you will have to update them (or ask a maintainer to) to use them with DMD 1.018 and later. The old OpenGL binding at DedicateD is not affected, as it is not cross-platform (and is actually quite outdated by now, anyway).

There may be other D modules out there, bindings to libraries other than OpenGL or perhaps OpenGL bindings I’m unaware of, that are affected. If you find yourself using one of them, the offending code looks like, or similar to, this:

version(Windows)
{
    extern(Windows):
}
else
{
    extern(C):
}

// function, or function pointer, declarations follow here

Walter outlines some alternative solutions in a newsgroup post. Choose an option that works for you and roll with it.

Technorati Tags: D Programming Language, DMD, Digital Mars, Derelict, OpenGL bindings

Not too long ago, Kevin Glass let loose the first point release of his 2D Java game library, Slick. I’ve been following his progress with Slick for a while now and thought it would be nifty to port to D. I didn’t bother with it though, because I didn’t see it as something I had time for. Besides, the D community already has Clay Smith’s ArcLib. When I decided to sit down and work on a test case as described in the post linked above, I thought about what would constitute a good project. Slick immediately came to mind. So I worked on it for a couple of days and then had the epiphany I wrote about.

All projects need a name. I thought about this for a bit and wanted to derive something from Slick. Eventually, I settled on Smoothie. I couldn’t find any other game libraries with the same name. Plus, I love drinking them.

While Smoothie is based on Slick, it is not a direct port. I’m going in a different direction with much of the design, though I am certainly using some of Slick verbatim. Like Derelict, I’m primarily writing this for me. I’ll also be developing a commercial game in tandem with the library, so that’s going to drive most of the design decisions for a while. But I do have milestones set out where I will make point releases of Smoothie under the BSD license.

The first milestone will allow the creation of simple 2D games, such as Pong, Asteroids, or Space Invaders. The second milestone will add special effects, simple GUI components, support for tile-based games, and more. Each milestone is set up to be an incremental improvement over the last. Though I haven’t gotten more than the first two milestones completely fleshed out, I am looking farther down the road. I’ve structured the framework such that it can, eventually, support different windowing APIs for OpenGL context creation (SDL, DFL, SWT, or what have you). It will also be able to support 3D rendering in addition to 2D, together or separately. For example, in a 2D game you could easily create a GUI component that rendered a 3D model such that it’s not part of the game world, but part of the UI. Or in a 3D game you could easily create a 2D GUI component for the player to play a mini-game (like, maybe, tetris) while waiting for the game to load, or as part of a challenge. You could then use the same code for the mini-game and release it as a completely separate game with very few changes.

I’m a long way off from the 3D bits yet, so you won’t be seeing that any time soon. I’ve got a couple of 2D game ideas I want to pump out first. But this should give you an idea of what Smoothie is and what my goals for it are. Once I’m ready for the first point release, I’ll ask Brad for a DSource project. I expect that to be early next month. Starting soon, I’ll be keeping a devlog detailing the progress of Smoothie and the games I make with it, over at my game blog.

Comments and questions are welcome!
 

Technorati Tags: D Programming Language, Slick, Smoothie

For a long while now I’ve been working sporadically on a commercial game project using C. Progress has been slow primarily because I only spend a few hours a week working on it (if that). But there have been other speed bumps along the way, things that any C developer worth his salt will have encountered more often than he would like (as an example, I spent three days chasing down a memory corruption bug not too long ago). I love C, with all of its foibles and weaknesses, but there certainly are days when I feel like throwing my computer out of the window.

Not too long ago, with one of the more recent DMD releases, I started to get this nagging feeling that the time for D had arrived. The more I ignored it, the stronger it got. Seeing in the NG those glowing reviews by the team0xf guys made it stronger still. So I decided to put it to the test on something other than Derelict and more than just a 10 minute test case. I quickly came up with a project idea and got to work.

After a several hours scattered over 2 days, I’ve already made great progress. Derelict requires only a small subset of D features, so though I’ve been aware of the benefits of using many of them, I’ve not actually seen any in production code until now. The more D I use on this project, the more I never want to see a line of C code again. In fact, I’ve already come to two decisions. For one, I’m going to move this test project forward into a production library. Second, I’m going to drop the C game project completely. I’ll eventually start over using D (it may seem to others like a waste of months of effort, but it’s really not a big deal). Before that, though, this new project has given me a couple of other ideas to work on.

Yeah, I’m going all out on this. I’ve typed up design documents, set up milestones and tasks on my local copy of activeCollab, and I actually sat down and blocked off large chunks of time on my calendar for development. My game business has been something that I wanted to get up and running sometime in the next few years (i.e., no solid plan), but now the worm has turned and I’m focusing on it exclusively. I’ll continue teaching English, but I’m going to stop taking contract software development work completely. Contract work was the real time killer for me and it’s never been something I really enjoy (mostly J2EE stuff). There are a few other minor distractions I’m cutting out as well. From here on, I’ll consider myself a full time D game programmer!

There are two side-effects. The first is that I’ll be able to set aside more time to work on Derelict than I ever have been able to before. Since I’ll now be a Derelict user myself, I’ll get to see first-hand any warts that crop up (at least, on a subset of the packages). The second is that the little test project that started this enlightenment will be made publicly available under a BSD license.

The point of this post is to proclaim to the world that D has set me free! My old reluctance to use it for production code was warranted way back when, but I can see no justifiable reason not to use D in its current state. I can’t remember the last time I had this much fun typing code into a text editor. Thanks, Walter, for bringing D to where it is today.

Technorati Tags: D Programming Language

There is a natural tendency when learning a new language to, initially, attempt to use it the same as a language you are more familiar with. This is especially true when the two languages are similar in syntax. Since most newcomers to D seem to primarily come from C++ and Java, and since D is similar to both languages in many respects, people are naturally trying to shoehorn C++ and Java idioms into D, sometimes becoming frustrated when things don’t work as they expect.

Feature requests for functionality found in other languages are not uncommon on the D newsgroup. I personally think Walter has done a great job of sticking to his guns for the most part. Granted, some of the features that he relented on after the community harrassed him have turned out to be incredibly useful. Some have even given D a unique flavor that really sets it apart.

I would love to see reflection/introspection capabilities similar to Java or C#. That would really put D in a unique position, being the statically compiled systems language that it is. I wouldn’t mind seeing a more explicit syntax for properties, but it’s not something I consider a deal breaker. I can do without assignment overloads, but if such a thing ever did make it in, I wouldn’t cry about it.

So I have my opinions and everyone else has their own. But it’s not the feature requests and wishlists in and of themselves that prompted this post. It’s the reasoning behind some of them. I get the impression that some feature requests are made not because they are great ideas that would improve the language, but because they would bring D more in line with language X.

I shall repeat myself: D is not C++ and D is not Java!

One of the reasons so many C++ programmers have such a disdain for Java is because they can not, or will not, take their C++ hats off long enough to learn that Java isn’t C++. I have seen enough microbenchmarks, which are generally useless anyway, coded in Java by C++ programmers to know that C++ programmers can’t write Java code. When you whip out a Java compiler, you have to be wearing a Java hat.  Operator overloading is not a feature in Java, so if you are thinking in Java you won’t miss it. The same holds true in reverse, of course — Java programmers need to be wearing a C++ hat when Stroustrup comes off of the shelf.

One more time: D is not C++ and D is not Java!

When you come to the Land of D, drop whatever hat you are wearing at the gate. Keep your fundamentals, because you’ll need those. Keep the memory of what it was like to wear that other hat, because sometimes those C++ or Java tricks may come in handy in the D world. But under no circumstance should you expect to find yourself on familiar terrain. Sometimes you will, but familiar does not mean identical.

The next time you see yourself looking for that C++ or Java feature that D doesn’t have, ask yourself which hat you are wearing. When you have the D hat on, most of the time you’ll find that you just don’t need that feature at all.

Technorati Tags: D Programming Language, C++, Java