Before diving head-first into creating AI, this chapter goes over the internals and setup of the sandbox. While the AI scripting will all take place in Lua, it is important to understand how Lua interfaces with the sandbox and what responsibilities remain in C++ compared to Lua.

The sandbox is laid out to easily support individual applications while sharing the same media resources between them. A key project, which is demo_framework, provides all the common code used throughout the book. The only difference between each individual chapter's C++ code is the setup of which Lua sandbox script to execute. Even though the entire sandbox framework is available from the beginning of the book, each chapter will introduce new functionality within the sandbox incrementally:

bin
    x32/debug
    x32/release
    x64/debug
    x64/release
build (generated folders)
    projects
    Learning Game AI Programming.sln
decoda
lib (generated folders)
    x32/debug
    x32/release
    x64/debug
    x64/release
media
    animations
    fonts
    materials
    models
    packs
    particle
    programs
    shaders
    textures
premake
    premake
    SandboxDemos
src
    chapter_1_movement (example)
        include
        script
        src
    ogre3d (example)
        include
        src
    ...
tools
    decoda
    premake
vs2008.bat
vs2010.bat
vs2012.bat
vs2013.bat

Now we'll take a look at what each folder within the sandbox project contains:

Premake is a Lua-based build configuration tool. AI sandbox uses Premake in order to support multiple versions of Visual Studio without the need to maintain build configurations across solutions and build configurations.

Executing the vs2008.bat batch file will create a Visual Studio 2008 solution within the build folder. Correspondingly, the vs2010.bat and vs2012.bat batch files will create Visual Studio 2010 and Visual Studio 2012 solutions of AI sandbox.

Building the sandbox requires only one dependency, which is the DirectX SDK. Make sure that your system has the DirectX SDK installed, or download the free SDK from Microsoft at http://www.microsoft.com/en-us/download/details.aspx?id=6812.

The sandbox solution file is located at build/Learning Game AI Programming.sln once one of the vs2008.bat, vs2010.bat, vs2012.bat, or vs2013.bat batch files have been executed. The initial build of the sandbox might take several minutes to build all of the libraries the sandbox uses, and further builds will be much faster.

Lua 5.1.5 is used by the sandbox instead of the latest Lua 5.2.x library. The latest build of the Decoda IDE's debugger only supports up to 5.1.x of Lua. Newer versions of Lua can be substituted into the sandbox, but Lua's debugging support will no longer function.

At the time of writing this, Ogre3D 1.9.0 is the latest stable build of the Ogre3D graphics library. A minimal configuration of Ogre3D is used by the sandbox, which only requires the minimum library dependencies for image handling, font handling, ZIP compression, and DirectX graphics support.

The included dependencies required by Ogre3D are:

The sandbox was created with the DirectX SDK Version 9.29.1962, but any later version of DirectX SDK will work as well. Additional open source libraries providing debug graphics, input handling, physics, steering, and pathfinding are detailed as follows:

Premake dev e7a41f90fb80 is a development build of Premake that is based on the Premake development branch. The sandbox's Premake configuration files utilize the latest features that are only present in the development branch of Premake.

Decoda 1.6 build 1034 provides seamless Lua debugging through an inspection of the sandbox's debug symbol files.

Decoda is a professional Lua integrated development environment (IDE) that was released as open source by Unknown Worlds Entertainment, makers of Natural Selection 2 (http://unknownworlds.com/decoda/).

Decoda takes a unique approach to Lua script debugging, which makes integrating with an application far superior to other Lua debuggers. Instead of using a networked-based approach where an internal Lua virtual machine must be configured to support debugging, Decoda uses the debug symbol files produced by Visual Studio in order to support Lua debugging. The key advantage of this approach is that it supports Lua debugging without requiring any changes to the original application. This key difference in Decoda allows for easy debug support in the sandbox's embedded Lua virtual machines.

Begin by opening this chapter's Decoda project (decoda/chapter_1_movement.deproj). Each sandbox Decoda project is already set up to run the correct corresponding sandbox executable. To run the sandbox within Decoda, press Ctrl + F5, or click on the Start Without Debugging option from the Debug menu.

Setting up a new Decoda project only requires a few initial steps to point Decoda to the correct executable and debug symbols.

A new Decoda project can be configured with the following steps:

The Project Settings screen can be seen as follows:

Setting the Decoda debug executable and working directory

To begin debugging Lua scripts within the sandbox, press F5 within Decoda. F5 will launch the sandbox application and attach Decoda to the running process. Selecting Break from the Debug menu or setting a breakpoint over a running script will pause the sandbox for debugging.

Debugging Lua scripts in Decoda

If you're familiar with the Visual Studio Watch window, the Decoda Watch window is very similar. Type any variable within the Watch window to monitor that variable while debugging. Decoda also allows you to type in any arbitrary Lua statements within the Watch window. The Lua statement will be executed within the current scope of the debugger.

Viewing local Lua variables in the Decoda Watch window

The stack window shows you the currently executing Lua call stack. Double-click on any line to jump to the caller. The Watch window will be automatically updated based on the current scope specified by the call stack.

Analyzing the Lua call stack in the Decoda Call Stack window

The Virtual Machines window shows you each of the Lua virtual machines the sandbox is running. In this case, there is a separate virtual machine for the sandbox and a separate virtual machine for each of the running agents.

Debugging multiple Lua virtual machines within Decoda

To simultaneously debug both the C++ sandbox and any running Lua scripts, there are a few approaches available.

If the sandbox was launched from Decoda, you can always attach it to the running process from Visual Studio through the Debug menu's Attach to Process option.

Simultaneously debug the C++ sandbox when launching from Decoda

Decoda can also attach itself to a running process through the Debug menu. If the sandbox is run through Visual Studio, you can attach Decoda at any time in the same fashion that Visual Studio attaches it to the sandbox.

Simultaneously debug Lua scripts when launching the sandbox from Visual Studio

To automatically attach both Decoda and Visual Studio when the sandbox launches from Decoda, select the Attach System Debugger option from the Debug menu. Upon running the application from Decoda, Windows will prompt you to immediately attach a Just-In-Time (JIT) debugger.

The following screenshot shows you the Debug option that we are accessing in order to attach the system debugger:

Automatically attach the Decoda debugger when launching the sandbox

In order for Decoda to know which Lua file to associate the currently executing Lua script, the luaL_loadbuffer Lua API function must pass in the Lua filename as chunkName during the loading. The luaL_loadbuffer function is an auxiliary Lua helper function provided in lauxlib.h:

lauxlib.h

int luaL_loadbuffer(
    lua_State* luaVM, const char* buffer,
    size_t bufferSize, const char* chunkName);

The Lua virtual machine is represented by lua_State struct, which is defined in the lstate.h header file. Lua states are self-contained structures without the use for any global data, making them practical for threaded applications:

lstate.h

struct lua_State;

The sandbox runs multiple Lua virtual machines simultaneously. One main virtual machine is assigned to the sandbox itself, while each spawned agent runs its own separate virtual machine. The use of individual virtual machines comes to the sandbox at the cost of performance and memory but allows for iterating Lua scripts in real time on a per-agent basis.

As Lua is a weakly typed language that supports functions that can receive an arbitrary amount of parameters as well as return an arbitrary number of return values, interfacing with C++ code is a bit tricky.

To get around the strong typing of C++, Lua uses a First In First Out (FIFO) stack to send and receive data from the Lua virtual machine. For example, when C++ wants to call a Lua function, the Lua function as well as the function parameters are pushed onto the stack and are then executed by the virtual machine. Any return values from the function are pushed back to the stack for the calling C++ code to handle.

The same process happens in reverse when the Lua code wants to call the C++ code. First, Lua pushes the C++ function onto the stack, followed by any parameters sent to the function. Once the code has finished executing, any return values are pushed back to the stack for the executing Lua script to handle.

The Lua call stack indexing scheme

When interfacing with Lua, stack values can either be retrieved bottom-up, or top-down. The top element in the stack can be retrieved with an index of -1, while the bottom element of the stack can be retrieved with an index of 1. Additional elements can be retrieved by indexing -2, -3, 2, 3, and so on.

Lua has 8 basic primitives: nil, Boolean, number, string, function, userdata, thread, and table:

A metatable in Lua is a table primitive that allows custom functions to override common operations such as addition, subtraction, assignment, and so on. The sandbox uses metatables extensively in order to provide common functionality to the userdata that C++ manages.

To retrieve a metatable within Lua, use the getmetatable function on any object:

metatable getmetatable(object);

To set a metatable within Lua, use the setmetatable function that passes both the object to set and the metatable:

nil setmetatable(object, metatable);

A metamethod is a particular entry within a metatable that is called when an overriden operation is being requested by Lua. Typically, all Lua metamethods will begin with two underscores at the beginning of the function's name.

To add a metamethod to a metatable, assign a function to the metatable indexed by the metamethod's name. For example:

local metatable = getmetatable(table);
metatable.__add = function(left, right)
    return left.value + right.value;
end
setmetatable(table, metatable);

Userdata is an arbitrary block of data whose lifetime is managed by Lua's garbage collector. Whenever a code creates userdata to push into Lua, a block of memory managed by Lua is requested from lua_newuserdata:

lua.h

void* lua_newuserdata(lua_State* luaVM, size_t userdataSize);

While the sandbox makes extensive use of userdata, the construction and destruction of the allocated memory is still handled within the sandbox. This allows for Lua scripts to be easily iterated without worrying about Lua's internal memory management. For instance, when an agent is exposed to Lua through userdata, the only memory that Lua manages for the agent is a pointer to the agent. Lua is free to garbage collect the pointer at any time and has no effect on the agent itself.

The sandbox hooks into the Lua script through three predefined global Lua functions: Sandbox_Initialize, Sandbox_Cleanup, and Sandbox_Update. When the sandbox is first attached to the corresponding Lua script, the Sandbox_Initialize function is called. Each update tick of the sandbox will also invoke the Sandbox_Update function in the Lua script. When the sandbox is being destroyed or reloaded, the Sandbox_Cleanup function will have an opportunity to perform any script-side cleanup.

In order for C++ to call a Lua function, the function must be retrieved from Lua and pushed onto the stack. Function parameters are then pushed on top of the stack, followed by a call to lua_pcall, which executes the Lua function. The lua_pcall function specifies the number of arguments the Lua function receives, the number of expected return values, and specifies how to handle errors:

lua.h

int lua_pcall(
   lua_State* luaVM, int numberOfArguments,
   int numberOfResults, int errorFunction);

For example, the Agent_Initialize Lua script function is called in the AgentUtilities class in the following manner:

Agent.lua

function Agent_Initialize(agent)
    ...
end

First, the Lua function is retrieved from Lua by name and pushed onto the stack. Next, the agent itself is pushed as the only parameter to the Agent_Initialize function. Lastly, lua_pcall executes the function and checks whether it succeeded successfully; otherwise, an assertion is raised by the sandbox:

AgentUtilities.cpp

void AgentUtilities::Initialize(Agent* const agent)
{
    // Retrieves the lua virtual machine the agent script is
    // running on.    lua_State* luaVM = agent->GetLuaVM();

    lua_getglobal(luaVM, "Agent_Initialize");

    // Agent_Initialize accepts one parameter, an Agent.
    AgentUtilities::PushAgent(luaVM, agent);

    // Execute the Agent_Initialize function and check for 
    // success.
    if (lua_pcall(luaVM, 1, 0, 0) != 0)
    {
        assert(false);
    }
}

Exposing C++ functions to Lua takes place through a process called function binding. Any bound functions exposed to Lua become accessible either as a global function or as a function available through a package. Packages in Lua are similar to namespaces in C++ and are implemented as a global table within Lua.

typedef int (*lua_CFunction) (lua_State *L);

int Lua_Script_AgentGetRadius(lua_State* luaVM);

int Lua_Script_AgentGetRadius(lua_State* luaVM)
{
    if (lua_gettop(luaVM) == 1)
    {
        Agent* const agent = AgentUtilities::GetAgent(
            luaVM, 1);

        return AgentUtilities::PushRadius(luaVM, agent);
    }
    return 0;
}

const luaL_Reg AgentFunctions[] =
{
    { "GetRadius", Lua_Script_AgentGetRadius },
    { NULL, NULL }
};

void AgentUtilities::BindVMFunctions(lua_State* const luaVM)
{
    luaL_register(luaVM, "Agent", AgentFunctions);
}

While the sandbox uses userdata to pass around agents and the sandbox itself, another use of userdata is to add basic primitives into the sandbox. These primitives are completely controlled by Lua's garbage collector.

The vector primitive added into the sandbox is a good example of using userdata that is completely controlled by Lua. As a vector is essentially a struct that only holds three values, it is a great choice for Lua to completely maintain the creation and destruction of the data. What this means to the C++ code interacting with Lua vectors is that code should never hold on to the memory address returned from Lua. Instead, the code should copy values retrieved from Lua and store them locally.

const luaL_Reg LuaVector3Metatable[] =
{
    { "__add", Lua_Script_Vector3Add },
    { "__div", Lua_Script_Vector3Divide },
    { "__eq", Lua_Script_Vector3Equal },
    { "__index", Lua_Script_Vector3Index },
    { "__mul", Lua_Script_Vector3Multiply },
    { "__newindex", Lua_Script_Vector3NewIndex },
    { "__sub", Lua_Script_Vector3Subtract },
    { "__tostring", Lua_Script_Vector3ToString },
    { "__towatch", Lua_Script_Vector3ToWatch },
    { "__unm", Lua_Script_Vector3Negation },
    { NULL, NULL }
};

#define LUA_VECTOR3_METATABLE "Vector3Type"

void LuaScriptUtilities::BindVMFunctions(lua_State* const luaVM)
{
    ...

    luaL_newmetatable(luaVM, LUA_VECTOR3_METATABLE);
    luaL_register(luaVM, NULL, LuaVector3Metatable);

    ...
}

int LuaScriptUtilities::PushVector3(
    lua_State* const luaVM, const Ogre::Vector3& vector)
{
    const size_t vectorSize = sizeof(Ogre::Vector3);

    Ogre::Vector3* const scriptType =
        static_cast<Ogre::Vector3*>(
            lua_newuserdata(luaVM, vectorSize));

    *scriptType = vector;

    luaL_getmetatable(luaVM, LUA_VECTOR3_METATABLE);
    lua_setmetatable(luaVM, -2);
    return 1;
}

The demo framework follows a very simple update, initialization, and cleanup design shared throughout many of the classes within the sandbox.

The following is a class overview of the BaseApplication.h header:

The BaseApplication abstract class overview

The BaseApplication class has the main responsibility to configure the application window, process input commands, as well as configure and interface with Ogre3D. The Cleanup, Draw, Initialize, and Update functions are stub functions with no implementation within the BaseApplication class itself. Classes that inherit from BaseApplication can overload any of these functions in order to plug in their own logic:

Ogre

Ogre3D handles the general update loop and window management of the sandbox. BaseApplication implements the Ogre::FrameListener interface in order to implement both the Update and Draw calls of the sandbox:

The following is a class overview of the OgreFrameListener.h header:

The Ogre3D FrameListener interface functions

The second interface, which is Ogre::WindowEventListener, enables the sandbox to receive specific callbacks to window events, such as the window movement, resizing, closing, closed, and window focus change:

The following is a class overview of the OgreWindowEventListener.h header:

The Ogre3D WindowEventListener interface functions

Note

Both interfaces specify functions that are called from Ogre's main thread, so no race conditions exist to handle events.

Object-Oriented Input System

The Object-Oriented Input System (OIS) library is responsible for all the keyboard and mouse handling within the sandbox. The BaseApplication class implements both OIS interfaces in order to receive calls for key presses, mouse presses, and mouse movements. Once the BaseApplication class receives these events, it sends these events to the sandbox in turn.

The following is a class overview of the OISKeyboard.h header:

The Ogre3D KeyListener interface functions

The following is a class overview of the OISMouse.h header:

SandboxApplication

The main AI sandbox class, which is SandboxApplication, inherits from the BaseApplication class that implements the Cleanup, Draw, Initialize, and Update functions. The CreateSandbox function creates an instance of a sandbox and hooks up the Lua script specified by the filename parameter.

The following is a class overview of the SandboxApplication.h header:

The SandboxApplication implementation of the BaseApplication abstract class

Sandbox

The sandbox class represents the sandbox data as well as handling calls into the Lua sandbox script. The creation of a sandbox requires an Ogre SceneNode instance to situate the sandbox in the world. The sandbox's SceneNode instance acts as the parent to any additional geometry SceneNodes used for rendering; this also includes the AI agents within the sandbox.

The following is a class overview of the Sandbox.h header:

The sandbox class overview

Agent

The agent class encapsulates the agent data and performs function calls to the corresponding Lua script assigned to the agent from the LoadScript function. A SceneNode is required to construct an agent instance to maintain an orient and position the agent within the world.

The following is a class overview of the Agent.h header:

The agent class overview