Capstone Update 8: A real adventure

Note: If you somehow managed to get here in a mad search for information on writing EQS Tests in Unreal 4, there's a complete sample towards the bottom of the page.

My last post was a bit of a downer, but worry not! I have yet another dense AI post ready this time around. Today, I'm going to discuss how I'm making the Hero AI more intelligent and interesting to deal with.

The Problem

After a recent round of testing, our team identified a problem: The Hero in our game was predictable to the point of being boring. At the time, the Hero AI would follow a set path to the level exit, only leaving the path to kill enemies and grab loot. No matter where he ended up, and where the player led them, the Hero would always just make a beeline for the end of the level.
Players didn't like this, because it made the Hero feel very robotic. We wanted our Hero to be more interesting, so we decided to try and make the Hero explore levels properly. However, we had a number of smaller issues to work past first:

1. There is no easy way, to my knowledge, to store and keep track of where an AI actor has been in Unreal. There were lots of little "half solutions" proposed, such as making the Hero spawn zones that affected navigation periodically. None of these ideas really seemed great, though.
2. As much as we wanted the Hero to be more independent, we also still wanted some level of control over them. The game wouldn't be very fun if the player had no idea where the hero was going, after all!

The Solution

We made several changes to the game to make our idea work. First off, we've decided to move back to a more modular way of building levels. We originally planned to build our levels out of pre-made rooms and corridors, but started moving to building levels out of smaller tile objects. However, if we make the rooms ahead of time then it's easier for us to use them as targets for the Hero.

In addition, I re-purposed the path node objects in our game. These nodes were used to make the hero move to the exit without hugging walls and corners, but I've given them a new function using our new 'Point of Interest' component. This component has 2 main uses: To give the Hero a list of places to explore, and to keep track of where the Hero has been. Path nodes have a trigger volume that marks the point of interest as visited once the Hero enters it. The useful part of this is that even if the hero just passes by the node on the way to another, the hero still 'knows' that it has been there already.

With a setup like this, the level only needs nodes near 'things', such as intersections, rooms, and other important objects, rather than having nodes lying around everywhere. As an additional safeguard, the Hero has a case to path directly to the level exit if it runs out of valid nodes.

The basic level geometry and pathing nodes of our test level.

Handling Distractions

So now we have a method of making the Hero wander around our level without doubling back too much. With this alone, though, the Hero won't do anything but walk around. We want to keep the Hero's current method of dealing with interesting things (that is, walking to them and dealing with them in a context-sensitive manner), but we also want to avoid the main problem: If the Hero picks a new target to walk to, it'll overwrite and erase the old one. To get around this, I broke the blackboard entry controlling the Hero's movement in two: a short-term target for 'distractions' like above, and a long-term target based on the Hero's goal of exploration. As long as the Hero's short-term target is set, the Hero will deal with that first. Needless to say, this all took a while to set up nicely. Here's a screenshot of the Hero's new behaviour tree:

The scary thing is that this will be growing soon...

Coolness

At this point,  we have a much smarter Hero than we started with. The only problem is that now that is so much more independent, we have no goods way to influence its decisions.
That's where the new coolness system comes in. Every Point of Interest on the map has a custom 'coolness' value that our designer can set. This value influences the POI's final score in our EQS test, which probably sounds like a huge load of gobbledygook if you've never done AI in Unreal before. Simply put, it's a heuristic that affects which targets the Hero is likely to select first when exploring.
To actually make this work, we needed to write a custom EQS test to use our coolness values. However, it turns out that there's a small problem with that:

To my knowledge, no one has made any resources on how to actually do it.

Ultimately, I ended up referring to the Unreal Engine source code to make our custom test, and that worked out alright. Still, to save any future coders who might find this the hassle, here is the entirety of our coolness EQS test (This test targets Unreal Engine 4.9.2, so keep that in mind if you're using an older/newer version):

NodeCoolnessTest.h:

    1 // Fill out your copyright notice in the Description page of Project Settings.
    2 
    3 #pragma once
    4 
    5 #include "EnvironmentQuery/EnvQueryTest.h"
    6 #include "NodeCoolnessTest.generated.h"
    7 
    8 UCLASS()
    9 class DUNGEONRESTOCKER_API UNodeCoolnessTest : public UEnvQueryTest
   10 {
   11     // Don't forget your constructor! (I did.)
   12  GENERATED_BODY()
   13     UNodeCoolnessTest(const FObjectInitializer& ObjectInitializer);
   14 
   15  virtual void RunTest(FEnvQueryInstance& QueryInstance) const override;
   16 
   17     // A property like this one will appear in the EQS editor's side pane
   18  UPROPERTY(EditDefaultsOnly)
   19  float MinCoolnessMod = 0.5f;
   20 };

NodeCoolnessTest.cpp:

    1 // Fill out your copyright notice in the Description page of Project Settings.
    2 
    3 #include "DungeonRestocker.h"
    4 #include "NodeCoolnessTest.h"
    5 #include "PoinOfInterestBase.h"
    6 #include "EnvironmentQuery/Items/EnvQueryItemType_ActorBase.h"
    7 
    8 UNodeCoolnessTest::UNodeCoolnessTest(const FObjectInitializer& ObjectInitializer) : Super(ObjectInitializer)
    9 {
   10     // This line is important, because it determines what your test actually
   11     // takes as input. Without it, your test will never run.
   12     ValidItemType = UEnvQueryItemType_ActorBase::StaticClass();
   13 }
   14 
   15 // This function is the actual test itself
   16 void UNodeCoolnessTest::RunTest(FEnvQueryInstance& QueryInstance) const
   17 {
   18     // min/max code borrowed from the ue4 source. These return the thresholds
   19     // set in the EQS editor
   20     FloatValueMin.BindData(QueryInstance.Owner.Get(), QueryInstance.QueryID);
   21  float MinThresholdValue = FloatValueMin.GetValue();
   22  FloatValueMax.BindData(QueryInstance.Owner.Get(), QueryInstance.QueryID);
   23  float MaxThresholdValue = FloatValueMax.GetValue();
   24 
   25     // This for loop is ytour friend. It iterates through all of the objects
   26     // returned by the Evironment Query
   27     for (FEnvQueryInstance::ItemIterator It(this, QueryInstance); It; ++It)
   28     {
   29 
   30         UPoinOfInterestBase* cmp;
   31         
   32         // This is how you retrieve an AActor, but you can do the same for
   33         // other types of objects.
   34         AActor* act = QueryInstance.GetItemAsActor(It.GetIndex());
   35         if(act) {
   36             cmp = act->FindComponentByClass<UPoinOfInterestBase>();
   37             if(cmp) {
   38                 // This is how you actually score your objects.
   39                 // The 3rd argument should be replaced with any float value
   40                 // you have for scoring.
   41     It.SetScore(TestPurpose, FilterType, FMath::RandRange(cmp->CoolnessFactor * MinCoolnessMod, cmp->CoolnessFactor), MinThresholdValue, MaxThresholdValue);
   42             }
   43         }
   44     }
   45 }

What's Next?

This is a great start, but there's still plenty to be done before Our Beloved Hero becomes smart enough to be a really fun and challenging opponent. My first goal, which I've begun, is making the Hero smart enough to solve basic puzzle-like challenges. I'm working on getting it to grab keys to open doors, which will probably evolve into switch puzzles and the like later.
My other main issue with the Hero's new AI is that it still can't be guided by the player. You can put down a breadcrumb trail of items and enemies, and the Hero will follow it, but then it'll just turn around and go back to what it was doing. You can't lead it into a certain branch of the dungeon, then double back while it's distracted to fill out the rest, which I think would add a lot of fun to the game and help combat the problem of not knowing where it's headed.

Either way, I have a lot of work ahead of me!