Migration Complete

I ditched my LEMP stack and set up a LAMP stack instead. I moved everything over to a cheaper VPS plan in the process. I was only using a tiny fraction of the resources I was paying for on the old plan. Anyway, I’m on more familiar ground now.

I was unable to do everything all at once, so it took longer than I anticipated because of interruptions (the biggest one being sleep). I’ve still got a couple of plugins to install and configure. But all of the critical stuff is done, as far as I can see. If you notice any issues, please let me know.

Oh, and I’ve got email notifications working this time around. I was never able to get it functioning on the other system. So if you do leave a comment, it shouldn’t languish for days without a response.

Blog Downtime and Derelict Updates

If you’ve dropped by here any time and gotten a “502 Bad Gateway” error, my inexperience at managing a LEMP stack is probably to blame. As soon as I can make the time for it, I’m going to move to something I’m much more familiar with and eliminate the problem once and for all. Time is something in short supply right now, but I really need to get this done as it’s annoying the hell out of me.

My time shortage also means I’m going to be slower than usual in responding to Derelict issues. I generally prefer to handle them as soon as I see them in my inbox if I can. Right now, I’m knee deep in a new project that’s going to require most of my free time for the next few months. I intend to make time for Derelict every weekend (and whenever I can find breathing room during the week), so please keep that in mind if you report any new issues. Pull requests are greatly appreciated and are guaranteed to get pulled immediately if I have no objections to them, but anything that requires more than a few minutes for me to fix is most likely going to be near the bottom of my priorities.

Preparing For DerelictSDL2 2.0

SDL 2.0.4 appears to be in the final push for release. Once it is out, I intend to update the SDL-2.0.x branch of DerelictSDL2 (where I’ve tagged the 1.9 series so far) and tag a 2.0 release. From that point forward, I will no longer be maintaining the other branches (corresponding to tags 1.0.x, 1.1.x, and 1.2.x).

Toward that end, I’d really like to strongly recommend anyone using DerelictSDL2 start using the 1.9 series today and, more importantly, let me know how you are getting on with it — especially if it’s working for you. There are effectively three API versions of SDL 2 right now: 2.0.0, 2.0.1 and 2.0.2/3 (2.0.3 was a bugfix release and has no API differences over 2.0.2). If you are require any of those as a minimum, then you can load it with the 1.9 series as described in the post where I introduced the SharedLibVersion feature, e.g.:

import derelict.sdl2.sdl;
void main() {
    // Require 2.0.0 as a minimum so that all SDL2 releases
    // can be loaded successfully.
    DerelictSDL2.load( SharedLibVersion( 2, 0, 0 ));
}

I also strongly recommend that you pass in 2.0.0 as above, allowing any SDL version to load, unless you absolutely require API functions from a later version. Particularly if you are distributing on Linux or one of the BSDs, which usually means relying on the SDL libraries being installed on the system. On Windows, where it’s customary to ship DLLs with the app, it’s not so important.

So, please, change your dub dependency to >=1.9.0 and let me know how it works out for you. If there are any problems, I’d really like to hear about them before I tag a 2.0 release.

Thanks!

DerelictAllegro5

I had a binding to Allegro 5 in an older iteration of Derelict, but it was problematic. There were unexpected crashes when using some Allegro dlls on Windows and there was a big issue with how to handle Allegro on Mac (it requires a special main function to run). So a while back I learned from reading about SeigeLord’s DAllegro5 that the crash issues were specific to dlls compiled with MinGW and have disappeared with newer versions of the compiler. I also discovered that he had come up with a way to deal with Mac. With those issues gone, I decided to start on a new DerelictAllegro5 binding for DerelictOrg. I worked on it sporadically, then somehow forgot all about it.

Recently, it popped back into my head so I dug up the source that I last touched in March of last year. To my surprise, I had managed to finish every module I had intended to finish. It only required a bit of tweaking to get my little allegtest app to compile and run. I’ve done a bit more work on it to make my life easier in maintaining it and now I think it’s ready for a work out, so I’ve pushed it to GitHub and will register derelict-allegro5 version 0.0.1 with dub as soon as I publish this post (I want to link here from the README until I get some better documentation).

If you are using or thinking about using Allegro 5, I’d appreciate if you’d test this out for me. Right now, it binds to the 5.0 series rather than the wip 5.1. There are a couple of things that make this binding different than other packages in Derelict and you need to be aware of them before using it.

First up, on Mac you need to launch the app with a special function in the loader. Allegro requires some special handling on OSX in order for things to work properly. I’ve adapted SeigeLord’s solution and created a function in the loader called “run”. It takes a pointer to a function that returns int and has no parameters. It will launch a separate thread and call the function pointer. So on OSX your app should call DerelictAllegro5.run(&foo) after you’ve loaded the libraries.

I’ve put up a sample at DPaste that shows how to load all of the Allegro libraries and handle the Mac workaround. Note that although the sample versions out the Mac stuff, you don’t need to do that if you don’t want. You can call the run method on any platform and it will work just fine. It’s up to you how to handle it. Also note that I’m catching any thrown Throwable in macMain, saving it in a module-scope variable, then rethrowing it in the main function. That’s because the function pointer passed to DerelictAllegro5.run will ultimately be called from the C side, so I want to make sure no exceptions are lost across the C/D boundary. If someone discovers that exceptions can cross that boundary on Mac without problem, then that isn’t necessary.

Second, the naming convention of the Allegro dlls on Windows is just outright annoying. Most C libraries out there do not include patch versions in the dll names, but all of the Allegro dlls do so. That means Derelict has to specifically ask for the 5.0.10 version of the libraries or the 5.0.11 version of the libraries and so on. On top of that, the dlls are compiled to either statically link with system libraries or not. If there’s an ‘-mt’ in the name, it’s the former, while ‘-md’ means the latter. Then there’s the option to use a monolithic dll which bundles all of the separate addons into one monster library. That’s not including the debug versions!

So, I had to make a choice: support every variation of the libraries or only rely on a specific set? I opted for the latter. By default, the loader for the core library and each addon expects the -mt versions of each library. Each loader will first look for the monolithic dll; next it will try 5.0.11; finally it will attempt to load 5.0.10. If you prefer to use the -md versions, or would like to support any 5.0.x version of the library, for now I recommend you just rename your dlls to something simple (like allegro5.dll, allegro5-audio.dll or whatnot) and pass those names to the load method for each loader to override the default behavior.

I have no idea if the Allegro5 build system is set up to build and install monolithic versions of the library on Mac and Linux, so the loaders currently don’t attempt to load those. However, the situation is better there with the 5.0.x versioning scheme, since most systems will probably be configured with a Mac framework or a dylib/so with the generic 5.0 prefix. This will allow any 5.0.x version to be loaded, though the loaders do look for 5.0.11 first on Linux (it’s intended to be the last release of the 5.0.x series, anyway). If any Allegro users on Mac or Linux could let me know about what exactly is installed on your system, that would be great. I’m particularly interested in whether or not there’s a monolithic version of the library available. Next time I boot into my Linux partition, I’ll see about installing Allegro and check it out myself (but I’m in no hurry to do that).

It seems like I’m missing something, but I can’t remember what it is. So that’s all for now. If you try DerelictAllegro5, please let me know even if it works flawlessly. Prebuilt Windows binaries for 5.0.10 are available from the Allegro Wiki. Linux users should be able to get 5.0.10 from your package manager. I’ve heard that SeigeLord updated MacPorts with 5.0.11. If you want to build it yourself, source packages for the recently released 5.0.11 and older versions are up at Sourceforge.

DerelictUtil 1.9.1

Having used the new SharedLibVersion stuff in DerelictUtil myself for a bit now, I’ve realized the need for a particularly useful enhancement.

If a loader does not support loading based on SharedLibVersion, attempting to do so will trip a compile-time assert. I don’t want users to be confused, having not yet read the docs I haven’t yet written, if they find their apps failing to load older versions of a lib in the wild even though they specified the minimum version to load. Because that’s exactly what would happen. Say, for example, you tell DerelictGLFW3 that you’re perfectly fine with release 1.1.x (which supports GLFW 3.1) going ahead and loading GLFW 3.0.x. The loader would silently ignore your request and fail to load any 3.0.x shared libraries. Now, you’ll get an assert message telling you that the loader does not support SharedLibVersion. I assume you all develop with asserts on like I do.

This brings me one step closer to a 2.0 release of the package. Please keep in mind that I may add, remove or change things in the 1.9.x series before I declare 2.0. If you’re using the new SharedLibVersion stuff and it’s working for you, please let me know. I’m sure you’ll let me know if it isn’t.

Support for GLFW 3.1 in Derelict

I’ve added support for GLFW 3.1 to DerelictGLFW3. To make use of it, use “derelict-glfw3″: “~>1.1.0″ as your DUB dependency. If  you don’t yet need the new functions and want to continue working with GLFW 3.0, then make sure your DUB dependency is “~>1.0.0″ and not “>=1.0.0″. The former will make sure you only pick up updates to the 1.0.x series, whereas the latter will update you with 1.1.0 if you run dub upgrade. Of course, specifying specific versions is always safe (“==1.0.0″ or “==1.0.1″), but doesn’t give you any updates. The 1.0.x branch has been quite stable, though, so using “==” shouldn’t hurt you at all.

I may add support for SharedLibVersion to DerelictGLFW3 at some point, but right now I have no plans to do so. Pick the version of GLFW3 you want to use and make sure you use the appropriate dependency version so that you aren’t trying to load a 3.0 DLL with the 3.1 binding.

DerelictCEF – Binding to Chromium Embedded Framework

I recently found myself wanting to play around with the Chromium Embedded Framework. I wanted to use it from D, of course, but no bindings exist for it that I’m aware of. So I did what I always do in that situation and created one.

DerelictCEF is a binding to the C API of CEF. I’m not about to try a dynamic binding to any C++ API any time soon. Unfortunately, the C API is not for the faint of heart. It requires a good deal of boilerplate, uses the ugly_underscored_functions_names that didn’t used to bother me until I started regularly using D, and has a plethora of struct types full of function pointers. Worse, the documentation on the C API is scattered and scarce AFAICT.

I’ve added the binding to DerelictOrg and the DUB registry (“derelict-cef”: “>=0.0.1″) for anyone who wants to give it a spin. But be forewarned: don’t expect too much help from me at this point. I’ve never used CEF before I implemented this binding. So far, all I’ve done is grabbed the CEF binaries, taken the sample C code from this github project (something anyone who wants to try DerelictCEF should study) and slapped something sloppily together in D to verify that the binding works on Windows. I’ve uploaded a screenshot to imgur for your viewing pleasure.

I’ll be playing around with this off and on when the whim strikes me, so I’ll be able to kick the tires a bit and look for issues, but I’m sure I won’t be getting anywhere near full coverage. At some point, I’ll boot into my Linux partition and make sure it compiles and runs there. If someone else gets there first, please let me know. I’d also love to hear the same about Mac. Eventually, when I’m more familiar with CEF and confident the binding is doing what it’s supposed to do, I’ll tag a 1.0.0 release.

In the meantime, I highly recommend that if you aren’t familiar with CEF or aren’t willing to put the time investment into learning how to use it, that you stay away from DerelictCEF for now. If you don’t know CEF have a limited knowledge of C, you’re going to be asking for trouble. I’m currently learning my way around the API and until I know what I’m doing, I’m just unable to help anyone else with it. I can already see that this is something that’s going to require more documentation than any other Derelict package. Until I’m able to write it, consider this as being for the experienced and adventurous only.

I’m really looking forward to playing with this.

DerelictFI Gets SharedLibVersion Support

For the longest time, DerelictFI did not support FreeImage 3.15.0. Between that version and 3.15.4, only one function was added in 3.15.1, so I just didn’t see the benefit in adding an entire branch to support it. Unfortunately, I’ve seen more than one report of DerelictFI.load failing because that function was missing, indicating the user’s system had 3.15.0 installed. The most recent mention of it was what prompted me to add the SharedLibVersion feature to DerelictUtil.

Now, I’ve implemented support for that in DerelictFI. I’ve added a new branch and tagged a 1.9.0 release. This will be bumped up to 2.0 once I’m sure all is well. The binding API is implemented against FreeImage 3.15.4, which doesn’t differ all that much from 3.15.0 — a few additional types and one new function. The loader will attempt to load 3.15.4 by default, but it will happily load 3.16.0 if it is available instead (just without the new API features of that version — I’ll have a new branch for that eventually). If you don’t need any of that newer stuff, you can load any version of FreeImage from 3.15.0 on up with the following:

import derelict.freeimage.freeimage;
void main() {
    DerelictFI.load( SharedLibVersion( 3, 15, 0 ));
}

If you are using DerelictFreeImage in your project, please change your derelict-fi dependency to “>=1.9.0″ and give it a spin. It should work out of the box.

New Derelict Feature

These days, a common issue new Derelict users have is the SymbolLoadException. Once upon a time, the exception message was enough for the user to figure out what was going on and act appropriately. Nowadays, there are more inexperienced programmers finding their way to D and Derelict who haven’t quite realized what exception messages are telling them. Nor can they be expected to be able to go out and determine the differences between different versions of a C library such that they can implement a MissingSymbolCallback for the Derelict loader.

The SymbolLoadException appears to be particularly common on Linux, since the package managers often are a release or two behind on some libraries, or the user hasn’t yet realized he needs to upgrade. The new Derelict user can find the versioning scheme for packages like DerelictSDL2 to be confusing, choosing to use the highest version when they really can use a lower one. The documentation, which is far from complete, doesn’t currently help as well as it should.

After giving this issue some though, I’m fairly confident that documentation isn’t going to be enough. The versioning scheme I came up with in the beginning seemed rational. But in addition to needlessly confusion new users, it has also become a bit of a maintenance hassle for me.

I’ll pick on DerelictSDL2 again. In addition to the master branch, there are three release branches: 2.0.0, 2.0.1, and 2.0.2. Each branch name corresponds to one or more SDL releases. I decided this configuration was necessary since the SDL maintainers have no qualms with introduction new functions in the API with a patch release. For example, SDL 2.0.1 and 2.0.2 both added new functions, while 2.0.3 did not. Since DerelictSDL2 can’t load what isn’t there, the 2.0.1 or 2.0.2 binding will fail to load a 2.0.0 shared library. This is rather unacceptable.

Then there are the git tags for dub management. The 2.0.0 branch is tagged with 1.0.x; 2.0.1 is tagged with 1.1.x; 2.0.2 is tagged with 1.2.x. This also seemed perfectly reasonable at the time I implemented it. But, again, it has caused some confusion among users.

The maintenance problems should be obvious. Every time someone sends a PR to master that needs to be applied to each branch, I often have to manually update each branch, as there will inevitably be merge conflicts. If I find a bug in one branch, it also usually must be propagated to all the others. When SDL 2.0.4 is released, it will have more new functions, necessitating a 2.0.4 branch and a 1.3 or 1.4 series of tags, and adding to the maintenance burden.

Until today, I hadn’t really given any thought on a better approach. But after reading a post in the newsgroup in which an issue with DerelictFI, the FreeImage binding, was mentioned, I realized it would be useful to implement one or more loaders for older versions of FreeImage. That means DerelictFI would start to have multiple branches and add to the maintenance burden — a demotivating factor if ever there was one. So I determined to do something that could benefit all Derelict bindings, including third-party ones, without breaking backward compatibility.

The result is the SharedLibVersion struct. It’s implemented in derelict.util.loader. It has three int fields: major, minor and patch. I’ve also added three new versions of the load method, one companion for each of the three existing ones, that takes a SharedLibVersion as an additional parameter. The idea is that you set the fields to the minimum version of the library that you require and the loader will load whichever version is present, greater than or equal to the required minimum.

For example, let’s say you are perfectly happy to load SDL 2.0.0 and don’t need the new functions in 2.0.1 or 2.0.2. In that case, you can load like this:

import derelict.sdl2.sdl;
void main() {
    DerelictSDL2.load( SharedLibVersion( 2, 0, 0 ) );
}

As long as any released version of SDL2 (release candidates and alpha/beta versions aren’t supported) is on the system, it should load fine if it isn’t corrupt. Following is an example and short description of each load method.

import derelict.sdl2.sdl;
void main() {
    // Only knows how to load 2.0.2/3 and will fail on 2.0.0 or 2.0.1.
    DerelictSDL2.load();
    DerelictSDL2.load( "MySDL2.dll,SDL2.dll" );
    DerelictSDL2.load( ["MySDL2.dll", "SDL2.dll"] );

    // Will load 2.0.0 and higher.
    DerelictSDL2.load( SharedLibVersion( 2, 0, 0 ));
    DerelictSDL2.load( "MySDL2.dll,SDL2.dll", SharedLibVersion( 2, 0, 0 ));
    DerelictSDL2.load( ["MySDL2.dll", "SDL2.dll"], SharedLibVersion( 2, 0, 0 ));    
}

This same mechanism is available to all Derelict loaders, as the interface is found in the SharedLibLoader class in derelict.util.loader. The versions of the load method which accept a SharedLibVersion call the protected method configureMinimumVersion. SharedLibLoader subclasses can override this method (it is optional — the default does nothing) and configure a default MissingSymbolCallback appropriate for the library being loaded. How the version number is used depends upon the library being loaded. For example, the patch number is important for SDL, but might be ignored for another library. The goal is to be as flexible as possible and allow each loader to handle things appropriately.

Again, this is entirely optional and no Derelict loader is required to implement this. DerelictGL3 likely never will, for example, since OpenGL is a rather different beast from other libraries and the binding already makes use of a special reload method. I do intend to implement this on the existing DerelictOrg packages where appropriate and encourage third-party maintainers to do the same.

To use this in your own loaders, you’ll need to add version 1.9.0 of DerelictUtil as a dependency to your dub.json. You can see use the DerelictSDL2 implementation as an example. To use this feature with SDL2 in an app, you’ll need version 1.9.0 of DerelictSDL2 as a dependency. Eventually, I’ll bump the versions up to 2.0.0. I have another major feature I want to add and test before then.

This will eventually be in the documentation, which I hope to finish up before the end of January. By the way, any web design wizards who are are willing to submit a PR to the docs with a nice-looking and responsive stylesheet will be much appreciated!