DerelictUtil and DerelictSDL2 Updates

DerelictSDL2 2.0.1 has been pushed to github and is ready in the DUB repository. Nothing major here, just some function signature corrections. If you aren’t using those functions, no need to worry.

DerelictUtil 2.1 is also ready to go. The minor version bump is due to a potential binary incompatibility for people who manage their projects manually (with DUB, it would only be a problem for people using >= in their dependency declarations). There’s one change here: DerelictException now correctly reports the file name and line number. This is something I have included with every custom exception class I’ve written since D’s __FILE__ and __LINE__ constants were made to apply at the call site when used as default function parameter values. DerelictException predates that change by a considerable margin and I never paid any attention to it after. A PR has rectified that oversight.

DerelictUtil 3.0 is currently sitting in a github branch, but not yet available in the DUB repository. It removes some cruft and adds some new utilities that previously were implemented in specific Derelict packages, but have proven to be useful in the general case. To avoid as much breakage as possible, I’m not going to tag a new release until I have updated every Derelict package with a new version the depends on it. This is a gradual process. The biggest work is on DerelictGL3, which is getting a massive overhaul and support for compile-time OGL version configuration and custom context objects. This default behavior will be the same as it is now (with a new package name), but the new features will allow unused bits of the OGL API, including extensions, to be eliminated from the binary. The lion’s share of that work is done and I’ll hopefully have time to wrap it up later this month. Then I can turn my attention to the other packages. I expect to have DerelictUtil 3.0 ready to go by the end of the year, but I readily admit I have a dismal track record with such predictions.

DerelictOGL: DerelictGL3 Reborn?

I’m currently experimenting with a new approach to loading OpenGL in Derelict. What I’ve done is to put all of the declarations into manifest constant strings according to GL version and set things up so that a loading function for each is automatically registered when the string is mixed in. So then you can do this:

import derelict.ogl;

// This is the new thing here
mixin(DerelictGL33);

void main()
{
    // Load the shared library + OpenGL 1.1
    DerelictOGL.load();

    // Create an OpenGL context
    ...

    DerelictOGL.reload();
}

So the approach for loading the shared library and then the extended versions and extensions is the same. Aside from the name changes, the only difference is the mixin at the top of the file. The idea is that you would create your own OpenGL module (e.g. mygl), publicly import derelict.ogl, then mixin the version you want along with any extensions you intend to use.

One goal is to reduce the binary size by getting rid of the cruft you don’t want to use. Another is to make it possible to actually mix the function pointers in to a context object rather than having them as globals. Whether or not this proves practical remains to be seen.

At the moment, I’ve hit a brick wall with OpenGL 3.0+. Given that some of the ARB extensions were promoted to core with the same function and type names (names that had no ARB suffix even in the extension specification) For example, the declarations in GL_ARB_framebuffer_object are the same in the extension as they are in OGL 3.0. I haven’t yet figured a way to implement the mixins such that the user can mixin both the extension and the 3.0 functions without a conflict.

I have no idea if this is going to prove feasible in the long run. If it does, this will almost certainly become a new package rather than DerelictGL3 2.0. If it doesn’t, well, it was an interesting diversion.

Loading Conditionally Compiled GLFW 3 Functions

Only a few days after the release of DerelictGLFW3 3.0, I’ve tagged a 3.1 release. Four functions that were loaded automatically no longer are and a new public function was added to the DerelictGLFW3 loader, hence the version bump. The reason for the change is because of a new feature that no other Derelict binding currently has.

Throughout the binding’s existence, I have largely ignored the conditionally compiled native functions in the GLFW API. I added two Windows-specific and two Mac-specific functions to the binding as part of a pull request some time ago. This was fine, as, even though they were conditionally compiled, they were always present on the given platforms. I was also able to implement the return values in a way that didn’t require any platform-specific type declarations that might conflict with, say, custom bindings to the Cocoa API on Mac.

More native functions were added in GLFW 3.1 and, in GLFW 3.2, functions that depend on the Vulkan API were added. I thought I could get away without implementing them initially and put it off to later, but a couple of days after I announced the new version, an issue was reported on github wiht a possible solution (using a subclass of DerelictGLFW3Loader specifically for Vulkan functions). In my previous post, I already had a solution in mind using string mixins. In the end, I implemented using template mixins instead and, after it was tested, tagged the 3.1 release.

The problem is that the functions are declared using types specific to external APIs, such as the Vulkan API, or the Win32 API. I would either have to use C-style generic types (void*) where possible or declare my own types in Derelict. That, as has been shown in the past, will only lead to conflicts when a user imports types declared in a different binding that and try to use the functions from DerelictGLFW3.

Moreover, while I can be guaranteed that the Windows- and Mac-specific functions will be present on (almost) any GLFW binaries on those platforms, the same cannot be said of EGL, Vulkan, Mir, Wayland, or even X11. Adding these directly to the loader by default will result in exceptions when they are not present, forcing nearly every GLFW user to work around that with the selective loading mechanism. Quite far from an ideal situation.

By using template mixins, client code can import any binding with the types they need and then use the relvant GLFW functions without fail. For example:

import core.sys.windows.windows : HWND, HGLRC;
import derelict.glfw3;

mixin DerelictGLFW3_WindowsBind;

With this, the three Windows-specific functions declared in glfw3native.h will be available in client code. When compiling DerelictGLFW3 as a static binding, that’s all that need be done. For the default dynamic binding, the functions still need to be loaded.

void main() {
    DerelictGLFW3.load();
    DerelictGLFW3_loadWindows();
}

The loaders load method must be called first, then the free fucniton that loads the mixed in functions can be called.

Obviously, core.sys.windows.windows is part of DRuntime and I could depend on that in Derelict without resorting to mixins, but for consistency the Windows stuff uses the same approach as the other conditionally compiled functions where I don’t have that luxury. You can read more about which mixins and loaders all available, including which ones alias to DerelictGLFW3_NativeBind and DerelictGLFW3_loadNative, in the DerelictGLFW3 documentation.

Note that the system-specific stuff in DerelictGL3 has the same potential problem with type conflicts. However, since I moved the to using the DRuntime Win32 bindings, that potential has dramatically decreased (seems no one really uses any of the platform-specific stuff there anyway). I will most likely apply this technique there anyway to help shrink the size of the bindings a bit. I also intend to explore using it with the OpenGL extensions.

But that’s for another day and another post. Now it’s back to bothering people via email in my pursuit of content for the new D Blog.

GLFW 3.2 Support in DerelictGLFW3

GLFW 3.2 was released very recently. I’ve (mostly) added support for it to DerelictGLFW3, available now in the DUB repository under the 3.0.0 tag. There are a couple of caveats, though. First, the bindings for the functions in glfwnative.h are still incomplete. Second, I did not add bindings to the new Vulkan-related functions. Both the native stuff and the Vulkan stuff have a couple of potential issues I need to work around.

The first problem is that they are all conditionally compiled, meaning they may not be available in any give GLFW3 binary. Properly supporting them means segmenting the specific declaration on the D side in version blocks. In order for client code to make use of the version blocks, I would need to add subconfigurations for each possible version combination for both the dynamic and static builds of DerelictGLFW3. Not pretty.

The second problem is that all of the affected functions rely on types defined outside of GLFW. If I were to declare any of those types, for example VkInstance, in DerelictGLFW3, it would conflict with any Vulkan bindings the client might be using, making it a completely distinct type (i.e. derelict.glfw3.types.VkInstance vs. whatever.types.VkInstance). Again, not pretty.

I suspect the solution to this will be a set of string mixins that the user use to declare everything in the appropriate context, so that the types are picked up from whatever binding is used to Vulkan, XWindows, Wayland, and all of that (for the Windows stuff at least, it’s now possible to rely on Phobos for the appropriate types, but for consistency I’ll probably add it to the mixin anyway). The mixins will also need to implement a custom loader for these specific functions. I have simple approach to this in mind, so that’s not a problem.

I’ll get a solution implemented at some point, but I’m in no hurry. For now, please enjoy using the core GLFW 3.2 API in D.

DerelictAllegro5 Now Supports Allegro 5.2

DerelictAllegro5 has been buggy for a while, but the latest 1.0.1 tag seems to work as intended. It binds to Allegro 5.2, which was released a couple of months back. I finally got around to the update earlier this week.

If you want to use Allegro 5.0 with Derelict, you still can with the 0.0.8 release of DerelcitAllegro5, but be aware that it’s almost certainly buggy. I neglected it for a long time and found some issues with it when I was updating and I didn’t bother to fix them. I’ll pull any PRs made against that branch that are sent my way, but new projects should be using Allegro 5.2 anyway. In other words, I tagged the 5.2 branch as 1.0 because I consider the 5.0 branch dead.

On a related note, GLFW 3.2 was officially announced not long before I started on this post. I expect to update DerelictGLFW3 at some point in the next few days.

DerelictPHYSFS 2.0.0

DerelictPHYSFS spent quite some time languishing at the bottom of the second page of DerelictOrg projects, starving for attention. I created it on a whim a couple years ago when I wanted to use PhysicsFS for a project that I subsequently abandoned. In the time since, there have been no PRs for the project, no emails asking for help, no enhancement requests, nothing. So there it sat, forgotten.

Recently, I’ve begun work on assembling a codebase from the numerous abandoned games and graphics projects scattered around my dev projects folder. That eventually led me back to DerelictPHYSFS and a simple wrapper I had made for it (see Defile). The README for DerelictPHYSFS was still pointing to the old, defunct documentation page on this blog, rather than to the pages at derelictorg.github.io (which are still far from complete, but should currently be informative enough to use any Derelict package without much difficulty). The binding itself was stable. The 2.0.x API of PhysicsFS had not changed since I first made the binding (the last stable release, 2.0.3, was in October, 2012!), but the 2.1 API is what you get when building PhysicsFS from the development branch.

I normally don’t like to update Derelict bindings against unreleased versions of a C library, but in this case I think an exception is warranted. Any shared libraries built from the development branch of PhysFS would be compatible with the existing binding, but it would be nice to be able to make use of the newer API. So I’ve updated DerelictPHYSFS to the unreleased PhysicsFS 2.1. You can get that in the 2.1 branch on github, or through a dependency version of ~>2.0.0 in your dub configurations. The original binding is still there in the 2.0 branch, currently at version 1.0.3 as a dub dependency. I have no plans to update it further, unless someone is using it and really needs a bugfix (but I don’t think many people actually are using it).

I’ve also updated Defile to depend on the new version of DerelictPHYSFS. I intend to expand its API as I need it. It’s been available in the dub registry for quite some time and is still there for anyone who wants to use it. If you do, I make no promises about support. It’s a personal project that I decided to share with others and not something I intend to make room in my schedule to support for anyone other than me. Simple bug fixes or enhancements, maybe (definitely those that benefit me!), but anything time consuming? Forget it. Don’t let that discourage you from using it, though, if it’s something you need. It’s Boost licensed, so feel free to take it and do what you want with it.

There’s a lot of work I intend to get done with Derelict in the coming months. I need to update DerelictAllegro5 to Allegro 5.2 and get it into what I consider the right state for a 1.0.0 tag. I need to add support to DerelictLUA for Lua 5.1 so it can be used with LuaJIT. I also want to implement a static binding configuration for it like I did for DerelictGLFW3. I intend to do the same for a few other Derelict packages. I also need to work more on the documentation. That’s all competing with my plans for learningd.org and the example project from the book, the game-oriented stuff I’ve been working on the past few weeks, and some writing projects (not related to D or anything tech) I’ve got underway. Anyone got any spare time for sale?

DerelictUtil 2.0.6

I’m currently working on a project where I decided to make use of the Derelict’s ability to load specific file names in some cases, but the default in others. That’s when I discovered something about DerelictUtil I had forgotten — it doesn’t allow for empty or null library name strings.

For reasons lost in the mists of time, I originally chose to use an empty arg overload that calls the single string arg version, which then asserts the string is not empty or null. If I were starting from scratch today, I would make it so the single string arg overload takes an empty string as a default argument. Instead, I’ve modified the single string version to use the default library names in case of null or empty strings. If you also would like to make use of this, you should require a minimum dependency of version 2.0.6.

New POSIX-Specific Feature for DerelictUtil

I’ve been persuaded to add a new feature to DerelictUtil for systems that use dlopen to load shared libraries. Unlike LoadLibrary on Windows, the function accepts certain flags that determine when symbol resolution occurs and whether the symbols should be visible to other objects. For as long as DerelictUtil has existed, it has called dlopen in a way that matches the behavior of LoadLibrary on Windows, with immediate resolution and local (as opposed to global) visibility.

I’ve just added a new property that allows those who know what they are doing to change the dlopen flags as required:

import derelict.util.sharedlib;
version(linux) derelictLDFlags = LDFlags.rtldLazy;
DerelictFoo.load();
version(linux) derelictLDFlags = LDFlags.rtldNow | LDFlags.rtldGlobal;
DerelictBar.load();

The example uses version(linux), but it’s available anywhere version(Posix) is defined. Note that it’s a global property that affects all subsequent calls to the load method on any loader instance. The default is LDFlags.rtldNow.

DerelictGLFW3 2.0: derelict-glfw3-static

The desire to add static bindings to some of the Derelict packages goes way, way, back (at least as far as June of 2006, probably further). I was dead set against it, then for it, then against it, then for it, then against it… Anyway, the day has finally come when I’ve added static binding support to a Derelict package. In this case, it’s DerelictGLFW3 and it’s available now for your DUB dependencies with the 2.0.0 tag. Note that this is only available for GLFW 3.1. I have no plans to add support for this to the 3.0 binding.

So, DerelictGLFW3 is now both a dynamic and a static binding all in one. Dynamic is the default configuration, so you can configure your projects as you always have and call the DerelictGLFW3.load method. To enable the static binding, add the following to your dub.sdl.

dependency "derelict-glfw3" version="~>2.0.0"
subConfiguration "derelict-glfw3" "derelict-glfw3-static"

You’ll also need to link either dynamically or statically to the appropriate GLFW3 binaries. See more in the package documentation.

The static binding eliminates the DerelictUtil dependency and also produces smaller library files. It also reduces the size of your final executable, since you won’t have all of the loader code and the function pointers compiled in anymore. As much as I love the convenience of a dynamic binding, static bindings on Windows are more convenient than they were when I first started Derelict back in 2004. I’ve found myself wanting to use them on more than one occasion. I do intend to add support for this to some other Derelict packages, if not all. DerelictSDL2 is first on that list, but I make no promises as to when I’ll get around to it.

DerelictFT 1.1.0

I’ve just tagged release 1.1.0 of DerelictFT. This updates the binding to FreeType 2.6.3 (the 1.0.x series was implemented against FreeType 2.5), which should work with any of the 2.6.x releases so far. The changes in FreeType 2.6 compared to 2.5 are minor, but they are of a nature that makes SharedLibVersion support problematic. As such, I have no plans to add support for it to DerelictFT.