Small Hiatus

If you've been waiting for more roguelike tutorial posts, you may be disappointed. Because of the amount of work I've had on my plate recently, I've been unable to keep up with my schedule. For the moment, I've decided to put the series on hiatus until I have more time freed up. Im guessing that it'll be a couple of months before I really have the free time to devote to this project, but after that I'll try to post more frequently.

I have some standalone tutorials and informative posts planned as well, to add some more work to my portfolio, but there's no ETA on those. In the mean time, this blog will still have plenty of content thanks to class.

Capstone Update 14: The blog post where I actually discuss monster AI

This past week, our team took The Last Light to an expo in Montreal. By all accounts, it went well, but the work leading up to it was intense. I'll probably be taking things a bit easy, given that I spent 17+ hours working on the project last week.

One of the big things that I worked on last week was the monster AI. We now have a rough sketch of the AI implemented, and in the first level. I'm going to go through a quick rundown of the behaviors here, and what I plan to add in the future.

The Monster is based on a basic finite state machine, currently containing 4 states:

Wandering

 This is the default state for the monster. Currently, it chooses a random location within a small radius that isn't illuminated and that it can see, then goes there and repeats. This was implemented before I received the full designer spec for the monster, so it differs somewhat from their design for the moment.

Their system has the monster moving randomly between predetermined points, with a bias towards the player's position. This is partially implemented, but the waypoint EQS generator is still WIP.

Seeking

Previously, the player would die if they stayed in the darkness too long. Now, we have a more concrete and visible threat in the monster. When the player strays into the darkness without a light, the monster is alerted and enters a seeking state.

In this state, the monster finds a direct path to the player and chases them. If the player is caught, they begin to rapidly lose health, shown by a post-processing effect. Their health will regenerate after a few seconds of safety, however.

Currently, this is a pretty dumb state. Because the monster uses UE4's default pathfinding, it finds the most direct route. However, this route usually passing through includes light, which harms it. I haven't decided how to handle this yet, but I'm considering both navigation modifiers and custom navigation. Only time will tell.

Fleeing

 When the monster is lit and begins to take damage, it retreats. This is done using a simple pair of EQS tests on top of the same generator used for wandering. The tests A. ensure that the monster won't be moving into light and B. Heavily favor locations that are more distant from the player. The result is a behavior that feels 'right' when the player shines a light on the monster.

Dead

The monster takes damage from light, so if it spends too much time lit up it will enter a dead state. Once 'dead', it disappears for a few seconds, before reforming and continuing with what it was doing.

One major thing that needs to be done still is making it move in this state. It seems counter-intuitive, but by making it move as if it were fleeing, I can ensure that the monster won't reform directly on the player or inside a light.

My Goals

I've explained all the monster work that remains, but I also have some other goals for this coming week, mostly focused on graphical improvements.

First off, I need to improve the damage effect. Right now, it's not quite "in your face" enough, which probably makes it difficult for new players to realize that they're dying. Some additional blur, contrast, and color tweaks should hopefully make the player more aware of what's happening.

After that, I need to work some more on the interaction shader. Currently, it works well, but the designers have requested more control over it, such as color and width options. I'd like to get that done soon so that they have more time to play with it.

I don't expect to solve the pathfinding issue just yet, but we'll see.

Capstone Update 13: Making a Monster

The time has come for me to return, take up my mantle as an AI programmer, and write a bait-and-switch post about Unix shell scripting. I did work on our buddy the shadow monster this week, but it's not in the game yet and I'll be making much more progress on it this weekend. In the meantime, I'm going to make a post about a specific problem, and how I solved it with a just a tiny bit of scripting.

If you've used Unreal Engine 4, you probably know about the Substance plugin. This plugin is useful for making materials, but also has some issues. First off, it absolutely does not run on Linux. Given that my primary dev machine exclusively runs Linux, this is a problem. If I try loading a project that contains the plugin without installing it, Unreal simply crashes. The second issue is that it spreads. For some reason, installing Substance in one project seems to make it quietly install itself into other projects. So, if a single member installs Substance, the rest of the team will have to install the plugin on every new computer that they load the project onto.
Clearly, we have a problem.

However, I'm not here to complain about a problem, but instead to describe how I solved it. As a linux user, I think I can safely say that I'm more comfortable with shell scripting than your average developer. I'm also quite comfortable with git's command-line interface as well. Lastly, I know that Unreal stores plugins in the its text-based .uproject file. Going with my knowledge and experience, I decided to try and create a script that could auto-remove Substance whenever it appeared, so that no-one would ever need to deal with it on this project again.

Search and Destroy

The first thing that I had to tackle was finding and removing Substance. After all, if I couldn't do that then there would be no point to this exercise. Here's an example of a .uproject file with Substance installed:

{
    "FileVersion": 3,
    "EngineAssociation": "4.9",
    "Category": "",
    "Description": "",
    "Modules": [
        {
            "Name": "MindTreeGames",
            "Type": "Runtime",
            "LoadingPhase": "Default",
            "AdditionalDependencies": [
                "Engine",
                "CoreUObject"
            ]
        }
    ],
    "Plugins": [
        {
            "Name": "Substance",
            "Enabled": true,
            "MarketplaceURL": "com.epicgames.launcher://ue/marketplace/offers/2f6439c2f9584f49809d9b13b16c2ba4"
        }
    ]
}

Keeping in mind that the plugins section only appears if there are plugins installed, and that The Last Light doesn't make use of any plugins. In other words, we're free to simply wipe the Plugins section out entirely. Of course, we still need to find Substance, but we can easily fix that with a simple grep.

# Grep provides us with a line number at the start of the line, so we use cut
# to isolate it
substance_line=`grep -n "Substance" $project_file | cut -f1 -d:`

# Then, all we need to do is check if the resulting variable is empty or not
if [ -z $sub_line ]; then
    echo "No Substance here, but remain vigilant!"
else
    # The +/- parts here are hardcoded, based on the fact that our .uproject
    # file is relatively stable.
    start_l=$((sub_line - 3))
    end_l=$((sub_line + 3))

    # Delete the calculated lines, and get the resulting file
    result=`sed "$start_l,$end_l d" $project_file`

    # Now, just write out the result to the file. I take this extra step so that
    # I can provide a preview before making the change.
    echo -e "$result" > $project_file
fi

With this basic script, we can now find and remove the Substance plugin from a project with a single call!

Hooking it Up

I had a good start with my script, but I wanted a proper automated solution. Luckily, git provides us with the necessary tools to make this happen. By placing scripts in the .git/hooks directory, we can add scripts that are called on certain git-related events. If we add our script as a post-merge hook, then we can make our script run every time we pull changes from a remote.

In addition to adding a hook, we may want to commit our changes, and push them to save other team members the work of removing it themselves. As it turns out, we can simply call git commit and git push from a post-merge hook, and they work completely fine.

# Grep provides us with a line number at the start of the line, so we use cut
# to isolate it
substance_line=`grep -n "Substance" $project_file | cut -f1 -d:`

# Then, all we need to do is check if the resulting variable is empty or not
if [ -z $sub_line ]; then
    echo "No Substance here, but remain vigilant!"
else
    # The +/- parts here are hardcoded, based on the fact that our .uproject
    # file is relatively stable.
    start_l=$((sub_line - 3))
    end_l=$((sub_line + 3))

    # Delete the calculated lines, and get the resulting file
    result=`sed "$start_l,$end_l d" $project_file`

    # Now, just write out the result to the file. I take this extra step so that
    # I can provide a preview before making the change.
    echo -e "$result" > $project_file

    # Commit and push our changes immediately
    git commit -a -m "[bot] Removed Substance Plugin"
    git push origin master
fi

With these changes, all we have to do when we pull in the Substance Plugin is type in a password, and our script handles the rest.

Console Programming 3-Week Project 1 - Texture Loading in DFGame

What is this?

In one of my current classes, students are expected to design and implement coding projects of their choosing, then provide a writeup on their blog or portfolio where they explain the project. This is my first writeup, and you can expect 2 more similar posts later in the year.

As I've mentioned previously, one of my goals for the year is to ditch the DFEngine project and go back to developing games from the ground up. Instead of building on a large, monolithic engine, the goal is to implement lots of helpful code in libraries, which I can use to get started on projects quickly. I gave the library, DFGame, a short mention in a recent Halberd update, but I didn't go particularly in-depth as to what I'm actually doing with the engine. Here, I'll be documenting the work I've done on this engine over the past 3 weeks.

A General Overview of DFGame

Technically speaking DFGame is actually a set of 5 libraries broken up into the following modules:

• common - The common code that all other modules rely on. Contains various low-level systems such as IO and logging.
• editor [empty] - The editor-specific backend. Examples of editor-specific backend code include debug drawing, or handling assets that are baked or repackaged in the final exported game.
• game [empty] - The game-specific backend. Mostly intended for game logic. This is deliberately separated from the frontend code to allow embedding it directly into an editor instance.
• editor-front - The editor-specific frontend. Written in Vala, and provides helpful widgets and classes for building a graphical GTK3-based editor interface.
• game-front [empty] - The game-specific frontend. Examples of game-specific frontend code include graphics context creation, or main loop handling.

Note: Why Vala?

I chose to use Vala for all developer front-end code for the same reasons as with Halberd. Writing GTK code (or any UI code, for that matter) in C is a huge pain. Dealing with inherently hierarchical systems like a complex user interface is one of the few most obvious advantages of object-oriented programming as a whole.

I chose Vala because it is fully compatible with C, even more than C++. Vala uses vapi files and gobject-introspection to convert a C#-like object-oriented syntax directly to GObject C, allowing for it to both use and be used in C code. I consider the flexibility that Vala affords me to be extremely valuable.

Using this system of modules gives me a few specific advantages:

• It physically separates my code, which organizes it and also forces me to decouple distinct sections of my codebase as much as possible.
• I can write functions that do the same things in different ways for different purposes. For instance, I can write an editor asset loader for my editor that reads in an xml file, then write a game loader for the same asset that loads it from compressed binary.
• This same principle can be applied to a number of other uses, such as handling logs. If I had to specifically prefix all of these functions with whether they belonged to the editor or the game, then sort through all of them every time they were needed, I'd probably sob into a pillow before going to sleep at night.
• It allows me to link game logic and rendering into my editor for play-in-editor support, (A necessary feature in any modern engine) without having to perform folder wizardry with multiple main functions.

I've taken the time to write pkg-config files for every module, so linking against DFGame on any *NIX system is a breeze.

The DFGame Texture Loader

In DFGame, textures are stored in a super-simple texture struct:

typedef struct texture
{
    GLuint    handle;
    uint16_t  width;
    uint16_t  height;
} texture;

The handle refers to a texture buffer handled by OpenGL, and is guaranteed by all texture-related functions to be in 32-bit RGBA format.

Note

When I refer to the bit depth of an image, I'm referring to the number of bits per-pixel, not per-channel.

You might notice that I'm using 16-bit unsigned integers to store the texture dimensions. As it turns out, a 32-bit true color image with the maximum dimensions that this struct supports would take up approximately 135 Gigabytes of VRAM. (Also, most OpenGL implementations don't support maximum texture resolutions beyond 4096 anyway)

The Texture Loading Process

Next, I'm going to go over the higher-level part of the texture loader.
The main entry function for the loader is load_resource_to_texture. This function takes in a resource string pair, and returns a newly allocated texture struct containing the image pointed to by the pair. Alternatively, there is also load_resource_to_texture_buffer, which load_resource_to_texture calls. That function gives a nice, 32-bit RGBA buffer for more specialized loaders such as the Halberd tileset loader.

I'm going to go on a quick tangent to explain what a resource string pair is, and how it actually works. If you don't care about that and just want to read about the loaders, feel free to skip to the next heading. The concept of the resource pair goes back to Halberd's development, when I was looking for ways to handle assets in the editor and the game with the same functions. Ultimately, I ended up breaking up file paths into 3 parts: The base path, the resource path, and the filename.

The base path is the path to the actual content directory of the project. In other words, it points to wherever you store all of the game's assets. The base path is kept as a hidden static variable in the C file that handles most resource-related functions. To get the actual filepath to an asset, I have the construct_extended_resource_path function. Most asset loading functions, including the texture loader, use this function to get to the actual file they need. The function simply takes in a resource string pair, then concatenates it with the base path (with some basic error checking, of course). As simple as it is, it gives me a lot of flexibility. I can move the base path all I want, and the engine won't care at all. Furthermore, resource string pairs let me easily separate the actual filename from the path. This is very important for any filename-related functions, because it ensures that I don't accidentally read from/write to the path to the file when messing with the name.

The Loaders

Next, I'll briefly talk about the loaders. They're all based on the lib[type] libraries, but each one is a little different so I think they're worth discussing a little bit.

PNG Loader

My PNG loader is the oldest one that I have, dating back to before this project, and even well before I started Halberd work this summer. Despite that, it's still not a full implementation. Also, because it's so old, it's starting to need some real cleaning up. Notably, there seem to be a few error cases missing still. It works reliably, but could probably benefit the most from another pass.

JPEG Loader

libjpeg is a strange beast. The other libraries were very well documented, but libjpeg contained no API docs at all. From my understanding, this may have been intentional, in order to ensure that users were sufficiently good at working with C to use it. Thankfully, there were some well-commented example applications, so it could've been worse. To distinguish the JPEG test texture, I exported my PNG test with as many artifacts as I could muster (by setting the quality to 5% in the Gimp).

TGA Loader

I started the TGA loader before finishing the JPEG loader, but the end was delayed because of an amusing bug caused by a momentary lapse of judgement. When converting the raw data from a TGA file to the internal standard format that I use, I use the byte-depth to determine where to move bytes around. However, I accidentally passed the bit-depth in instead, resulting in an 8x smaller image! Interestingly, this didn't crash the program. It's definitely worth keeping an eye on memory access, that's for sure.

Tiff Loader

Sadly, the TIFF loader was fairly unremarkable. However, I think it's safe to say that the TIFF and PNG formats are the top two formats that I'm handling, as far as features go. Looking over the documentation surprised me quite a bit, because I don't usually use TIFFs very much. In particular, the ability to store multiple images in a TIFF may be worth revisiting at some point.

Conclusion

So, there you have it. I think this first project went quite well, fitting perfectly into the 3-week slot I carved out for it. My next project is probably going to involve Unix sockets, and be less related to games and more to general software development.

I'll be putting the dfgame project up on my software page soon. Until then, here's a link to the dfgame git repo.

Halberd Update 4 - Watch this!

It's been too long since my last Halberd update, but rest assured: I've been hard at work.

The Asset Registry

The registry can automatically tell when an asset has been changed outside the editor, and can prompt the user for more information.

I was been aware for a while that if I didn't create some method for handling assets and their relationships with each other, I was going to be in a world of hurt later. Over break, I spent a lot of time considering my options and came up with the asset registry.
The asset registry is a special container that records assets and assigns numerical IDs to them. The engine can report changes to the locations/names of registered assets, and the registry will change accordingly. Then, assets that depend on them can refer to them with their ID, and never have to worry about moves or deletions. Whenever you make some sort of connection between an asset and another, the depended asset is automatically registered (unless it exists in the registry, in which case the ID is returned). This makes managing assets extremely simple and straightforward.

New Editors

Adding a reference to an asset is quite simple, thanks to my generic asset-selection widget

For a long time, I've had to make most assets by writing config files, DFEngine style. This is a horribly slow, boring, and imprecise way to actually produce a game, so I've started making views in the editor for designing and editing various game objects. So far, I've finished the first pass on the Actor editor and I've started on the Sprite editor. I've got a lot of work on my plate, so progress here has been a bit slow.

I've also added a tile selection pane to the map editor. Not only can you choose specific tiles to place, you can also add new tilesets to a map and save them into the map file as well! I made this before the asset registry, so it needs some adjustments in order to fall in line with the rest of the asset editors. The tile selector is also a bit slow sometimes. I'll need to look into this in the future.

The Logging System

One thing that I've wanted in my engines for a while is a better way to log information and errors. After taking some time to work out logging in Halberd, I think I have a decent system. Using variadic macros, I've made logging fuinctions that can take arguments in the same way that printf can. This alone is a huge boon, as far as getting information goes. On top of that, I've added a simple way to add a callback function for logging. I'm not using any at the moment, but once I get the chance I'll be adding popups for error messages and a logging window to search, filter, and examine messages from the editor.

DFGame

Of course, what real good is such a nice log, if I can only use it for a single project? As I mentioned in my 2015 retrospective, I want to have a few basic libraries to make development simpler. I've started work on DFGame, a game engine utility library, and so far I'm happy with the results. Much of my resource handling and logging code has been added already, and I'll be adding much more in the near future.

By doing this, I've been able to rip several files straight out of Halberd and into DFGame, which has the added benefit of making Halberd's codebase just a tad smaller and simpler. I'm hoping that in a couple of months I'll have enough in there to give me an easy jumpstart on any new game projects.

Capstone Update 12: Lighting Up

This week, I made the first pass of a very important feature in The Last Light: Light Detection.

One of the central themes in The Last Light is the fear of darkness. Darkness is not only supposed to be scary, but dangerous as well. In darkness, the monster becomes more aggressive and staying outside of the light for long enough will result in an immediate game over.

Of course, to make all of this happen we need to know if the player is standing in darkness or not, and that's where light detection comes into play. Last semester, the team faked this by surrounding segments of the game world with giant "light volumes" that could be turned on and off. While this does solve the problem, it's a pain for the designers. Not only do they need to put these things in place, they then need to hook them all up to the power system along with the lights in each room. They also add more cruft to sift through when searching a level for a particular asset or area. For this reason, I was tasked with finding a solution that relied on the actual lights in the scene.

Ultimately, I came across a solution in the UE4 forums. (Here, for the interested) I'm still not sure about it, but it seems to work well. The general method is this:

1. Iterate every light in the scene
1. If a light is off, it doesn't affect the player, or there's no "line of sight" to the player, ignore it.
2. Otherwise, determine how strongly it's lighting the player up and add the result to a counter
2. If the counter is above a certain threshold, the player is safe. Otherwise, the player is considered to be "in darkness".

My main worry about this method is about performance. Every tick, I need to check every light that's loaded in the level. No matter how fast this is, this still comes with overhead. Furthermore, the shadow monsters will also need to know their light level. It's entirely possible that these checks will negatively affect performance in a big way once multiple actors are involved.

My current hope is that this won't be an issue. If it does, then I'll probably see if I can either cache some information or cheat on distant AIs. For the moment, however, I have a more pressing issue to resolve with this system. When I first made the implementation last week, I was mostly aiming to create a quick proof of concept. As a result, detection is currently very glitchy.

This mostly comes down to the way I'm determining light levels. For the sake of getting the feature to a kind-of working state, I simply divided the brightness of affecting lights by the distance. This doesn't really reflect the way that unreal's lights actually fall off, and as a result you can survive in some very dark areas while sometimes dying well within a light. Clearly, I need a new algorithm for properly determining falloff. Fixing this is going to be my first goal for the weekend, and my first guess for this fix is getting the light's radius value involved. I don't know for sure that this will be enough, but if it's not I can take a look at the UE4 code to see how they determine light falloff.

After that, I'll hopefully start tackling the shadow monsters! Yes, that's right. I'm back for another round of AI programming. Stay tuned.