Roundup, October 2017: Demo Time!

As you can plainly see above, my little tussle with Glade has borne fruit in the form of a new DFGame demo! The editor module isn't completely finished, but I had enough to work with so I decided to make a video anyway. Now then, let's discuss how the demo (and by extension, the Editor Module) works.

Split Architecture

To understand the current architecture of DFGame, I think it's useful to first look at the original architecture of Halberd, my WIP RPG engine. Halberd's codebase spawned DFGame originally, so it's useful to see where things came from. As I explained back in 2015, Halberd (and later, DFGame) was split into 5 modules:

• "Common" - Contained all code that worked in the editor and game
• "Editor Backend" - Editor-specific code, without the UI
• "Game Backend" - Game-specific code, without GLFW
• "Editor Frontend" - The editor UI
• "Game Frontend" - The game startup code, with stuff like window creation

When DFGame, I identified a couple weak points in this design. First of all, lumping nearly everything into a common module was a bit silly. Second, the backend/frontend concept felt somewhat unnecessary. To correct this, I created DFGame's current modules:

• Core
• Math
• Audio
• Graphics
• Gameplay
• Application
• Editor

 The first 5 modules are fairly self-explanatory. The application module is the general equivalent to the "Game Frontend" module and the Editor module is the "Editor Frontend" and "Editor Backend" rolled into one. The idea is that both of these modules will implement a common API, allowing the game to run regardless of where it is. This is the most important part of setting up a robust set of development tools, ensuring that the game code and window/input code are seperated.

Vala Interop

Besides the architecture, there's a second detail needed to get DFGame's editor functionality working. Typically, the languages used to write software and the languages used to write games are different. This can lead to many issues getting games to embed nicely inside of their tools, especially if there are ABI incompatibilities between the languages.

This was one of my main reasons for picking Vala. The Vala compiler translates Vala code to C, so getting a Vala frontend to play nice with a C backend is quite simple. To make the triangle in the editor demo controllable from Vala, I only needed to write a simple binding treating it as a class.

Conclusion

To sum things up simply, building a good editor for your game requires two things:

• An architecture that allows the game to run without the window/input code
• A means of bridging the editor's language and the game's language (This could be as simple as using the same language for both)

Once you have both of these things handled, you can work on your editor without worrying much. Once I finish up the Editor module, I'll be able to do much more with DFGame. I'll still be focused on Singularity for the time being, but I'll still be working on this occasionally.

Dusting the Cobwebs out of Singularity: Mistakes and Lessons

I'm pretty tired today, so I'm going to talk a little about my strategy for finishing up this Singularity update and some of the issues I see in my old code.

Homecoming

Getting back to an old codebase is usually a wince-inducing experience, but this instance was worse than usual. Singularity was always a learning process for me, as I taught myself software development over the past few years. Since putting the project down, I've had close to 9 months of software development experience. Naturally, my skills have changed drastically and my old code looks terrible now.

Since I left off mid-refactor, the code is in a bit of a half-finished state. I don't want to go all the way back to the last update, so I'm trying to fix and re-purpose what I've already got to avoid some poor decisions from the past.

Static Shock

For a long time, I regarded globally-accessible data as a very bad thing. As a result, I avoided static functions and variables like the plague in most of my software. You don't want to overuse these types of things, but it's important to remember that there's pretty much nothing in programming that should always be avoided.

In this case, the most obvious use case for statics is Singularity's settings classes. That's right, there were two of them. Depending on how loose your definition of "settings" is, you could even say that there were three (One to act as an interface to the application preferences, one to provide command-line settings, and one to provide the resolved path to the database). None of them were static, so if I needed them in a class I had to make that class hold a reference to the one(s) I needed. Some of those references even had different names!

I have since made the contents of both "main" settings classes static, and nested the command-line settings into the global settings. While I was at it, I also folded the path resolver into the command-line settings (since that's what decides the db path most of the time anyway). The result absolutely violates SOLID, but that honestly doesn't matter because it also greatly simplifies the affected code and makes it much more understandable.

Lesson learned: Trying to follow a particular coding paradigm perfectly at the expense of your actual design is, naturally, a terrible idea. At the end of the day, there's a time and place for everything.

Shameful SQL

I don't think I've ever really enjoyed working with SQL, nor do I expect this to be a particularly unpopular opinion. Within Singularity's SQLite database, there's a table containing all of the feed entries that it has saved. As it turns out, these entries have no unique keys and are subject to change. That makes swapping them out in a clean fashion damn near impossible.

So where did this terrible idea come from, anyway? The answer is GUIDs. RSS/Atom feed entries require a unique ID. However, we can't use that as our key in the database because keys could overlap between two feeds. My current plan is to combine something from the owning feed with the entry's GUID, and then hash the result. Simple and reasonable, right? Well, my OLD solution for handling updates was this:

1. Create a new temporary table and fill it with entries that match our feed.
2. Create a brand new unique index on the GUID (since there's only one feed, it should be unique now).
3. Save the items to this temporary table
4. Copy the entire thing back to its original location
5. Drop the temporary table 
6. Repeat for every individual feed (about 700 times)
7. Put the kettle on the stove, because this is going to take a while

Sometimes, (exactly 50% of the time) it crashes the entire program because a GUID managed to show up twice and database errors are treated as critical. Looking back, all I can do is cry.

Lesson Learned: Just because you just learned a couple fancy SQL tricks doesn't mean you should actually use them. Instead, consider if there's a better solution that doesn't overwork your database for no reason. Or at least use a better database, like PostgreSQL or Lucene.

Think Big! No, Smaller!

Often, it's a good idea to consider how well a solution might scale. After all, projects grow in scope over time and you don't want to leave yourself with a subpar solution down the line. On the other hand, it's also important to consider the current scale of the project you're working on.

Case in point, I made the rather poor decision to load as little as possible into memory. This seems smart, since it saves on RAM, but I don't need to do it! Seriously. Let's look at the numbers:

• I'm subscribed to ~700 active feeds
• My current Singularity database holds ~1 year of feed entries
• My current Singularity database is just over 100Mb in size

At this rate, singularity might end up using 1Gb of RAM...in a decade. Most modern web browsers regularly eat up that much right now! By the time the database gets big enough to fill my current machine's 16 gigs of RAM, I'll be several decades underground already. Unless I decide tomorrow to make Singularity into a big public online service, there's no point to optimizing for space.

More than just saving on RAM, I've repeatedly butted heads with a messy "clean up" system to auto-delete entries after a certain length of time passes. This is even more misguided, considering how cheap and plentiful hard drive space is. On top of adding extra bugs and code, this "feature" is much less useful than, say, a simple archival system that hides old entries while still letting you search for them. Unless a user was tracking a ludicrous number of feeds, I really don't see the point in trying any fancy space-saving tricks that slow things down and introduce bugs.

Lesson Learned: Base your projections on real use-cases, not crazy what-ifs and guesswork.

Conclusion

There are plenty of other problems left, but these are the big ones that I'm working on right now. Besides this I've still got a few other projects that I'm toying around with. Still, this one has my interest right now so you can probably count on more updates as development continues.

DFGame: Glade-ing Along to Victory

I planned to take a break from DFGame, but I happened to be in the mood to play around with the editor module. I ran into some issues while working on a new demo, and ended up having to do a few hours of research to get things working. Then I thought to myself: "Gee, someone else is bound to have this problem. Maybe I should write about it!"

What's New?

Sometime last year, I learned how about this cool feature in Vala, and started using the Glade UI editor with my software projects. However, it was only recently that I learned about a way to add custom widgets (such as DFGame's editor widgets) to Glade for graphical editing. Naturally, I had to give this a shot for the new and improved DFGame!

Let's discuss what went wrong, and how I solved these problems.

Problem 1: Nothing Happens?

It took quite a while for me to actually get any new UI elements to appear in Glade. There were a couple issues that caused this, primarily thanks to the documentation being rather vague about about how templates work... at least until you notice the tiny "next" button. Let's look at the example widget catalog that the documentation shows:

<glade-catalog name="foo" library="foo" depends="gtk+">
  <init-function>my_catalog_init</init-function>

  <glade-widget-classes>
    <glade-widget-class name="FooFrobnicator" generic-name="frobnicator" title="Frobnicator"/>

    ... widget classes go here
  </glade-widget-classes>

  <glade-widget-group name="foo" title="Foo">
    <glade-widget-class-ref name="FooFrobnicator"/>
    ... widget class references go here
  </glade-widget-group>

  ... widget groups go here
</glade-catalog>

(Borrowed from this page)

The difference between "name", "generic-name", and "title" isn't immediately obvious. If you look at a later page, there is an explanation:

• "name" is the name of the widget's class in code. It also seems to be used for class references in widget groups later in the file
• "generic-name" is used internally, I guess? Still not 100% sure on that one
• "title" is what the user will actually see in Glade

Unfortunately, this still wasn't enough because there remained an underlying problem to address: I write my Gtk code in Vala, but Glade only cares about GObject C. For the unacquainted, Vala acts as a wrapper language of sorts, and gets converted to GObject C for compilation. This means that there are two more things to adjust:

First of all, the class names are different. C lacks namespaces, so in order to provide them Vala adds each namespace onto every class that you create. In my case, this meant changing "Viewport" to DFGameViewport.

Next, there's a less obvious issue. Vala generates something referred to as a "get type function" which does something? I assume it either returns something that represents the object's type, or it returns an instance, but I don't have enough experience with GObject to know for certain. Glade overrides this function to create placeholders, but it apparently guesses the function name based on the "name" value that's passed in. In my case, I had to pass in the function name explicitly in order to make the catalog work. This is probably just a quirk in Vala's naming conventions, I suppose.

Once that was out of the way, Glade finally displayed my new viewport widget for DFGame and all was well, right?

Problem 2: Nope, That Doesn't Exist

As any seasoned programmer will tell you, life is never simple.
The fixes above were enough to let me create the UI for a new demo, but once the time came to actually test it out the UI wouldn't load! Instead, I was left with a cryptic message:

"(demo_editor:24355): Gtk-CRITICAL **: Error building template class 'MainWindow' for an instance of type 'MainWindow': .:3:4733 Invalid type function 'df_game_viewport_get_type'"

...???????!!?!??!??!?!!!??!!?!
The class was there. I could see it in the code. The library was definitely getting linked. So why then did it fail? Given a good couple hours, I was unable to answer this question. Luckily, the internet came through for me. All I had to do was slip in a single line of code at the start of the program

typeof(Viewport).ensure();

...and that was enough.

Takeaways

 This little experience further cements my thoughts regarding Gtk and many GNOME technologies, and I thought I'd discuss that briefly before as this post wraps up.

I like working with Gtk, Vala, and so on, but I think many of these technologies share the same problem. At the same time as GNOME works to streamline the design of its software and simplify things for normal users and new devs, there's this weird dropoff point where most docs past the beginner tutorials assume that you understand the technology stack and just need a quick reference.

The result is (presumably) "intermediate" devs like myself getting lost and confused after their gentle start suddenly drops out from under them. Thankfully, I ran into most of those issues a couple years ago, but occurrences like this one remind me that they still exist.

So I guess today's lesson is...maybe I should try writing a couple mid-level Vala/Gtk tutorials? Maybe GNOME needs more acknowledgement of their newer users and non-C APIs? Beats me.

Roundup, September 2017 - It's done!

It took longer than planned, but we're finally there:

Additional Comments

As I said in the video, DFGame v1.0 is now out! After almost 2 years of effort, I'm glad to see this project more-or-less done. There will be more, but for now it's time to wrap things up and move on.

I had an idea for another DFGame demo, as I mentioned last time. However, I wasn't very happy after messing around with it. After giving it some consideration, I decided not to go forward with it.

Next Up

Now that I'm done with this project, I'm going back to my other projects to continue them. I haven't had much time to work since I started working on DFGame again, so I'm going to go back and work on some of my software projects. In particular, Singularity has been sitting around mid-rewrite for a long time, and I'd love to have a new release out later this year (or early next year).