CS/INFO 4152: Advanced Topics in Computer Game Development

The CUGL Engine

For years we let students use whatever engine they wanted for this course. It was a support nightmare. In addition, we had to make sure to break people up into Android-only groups and iOS-only groups, which made team formation difficult. We desperately needed a cross-platform solution for this course. We also needed one in which was cleanly designed and was relatively inexpensive to deploy to a mobile device (so not Unity, as it fails on both counts). Our research turned up three viable options: LibGDX, and Cocos2D-X, and Marmalade (the last one is now defunct).

We used Cocos2D-X for several years. While it worked, students struggled with it because a lot of poor design decisions and legacy issues in the Cocos engine. The compile times were long (as long as an hour on some Windows machines) and the file sizes were huge (over 2 GB). In addition, there were a lot of restrictions that prevented students from using modern C++ features like smart pointers.

Several years ago we came to a realization. Looking at the Cocos2D-X projects, we discovered that the students used a very small number of classes from the engine. Therefore, we decided to do something radical. We made our own game engine, which we call the Cornell University Game Library (CUGL). This engine consists of these core classes rewritten from scratch to fully use the features of C++11.

If this seems a little daunting, you should realize that this engine is not completely from scratch. It is built on top of SDL. However, if you look at the SDL API, you will see that it is not really a game library. It is more of a hardware abstraction layer to make cross-platform development easier. The difference is similar to that between LibGDX and LWJGL.

[Download]   [Documentation]   [Apple]   [Android]   [Windows]


download

Unlike other game engines, we do not have any fancy install environments. To get the game engine, you download a sample "Hello World" application, which you then modify to serve as your main project. Note that downloading this engine requires a valid Cornell net-id.

See the instructions below for how to modify the sample project to your needs. We recommend that you modify it for all three platforms (Apple, Android, Windows) even if you are only targeting one of them. That makes it easier to quickly port to the other platforms later.

You will note that sample file includes the source code for the CUGL classes (but not the SDL source). You are welcome to modify these classes as you wish. In addition, we may post bug fixes on Piazza as the semester continues. In addition, if you create classes that you feel are of general purpose, you are welcome to contribute them to the engine for future semesters.


Documentation

The engine consists of three parts: the SDL base layer, Box2D physics, and the CUGL C++ classes. In addition, you have full access to OpenGL on all platforms. While many of you are familiar with OpenGL from CS 4620, you may not have ever used it in C/C++ before. Therefore we have some helpful reading below to help you adjust.

CUGL

The CUGL C++ classes are the primary classes that you will use to write your game. These classes, plus a little Box2D and maybe some OpenGL should be enough for your to develop any kind of (2D mobile) game you want this semester. The API above is a Javadoc-style web page generated from the class headers.

CUGL replied upon a scene graph for 2d graphics. We have a tutorial above (created by a former TA) explaining how this scene graph works. In particular, there are tools for your designers to generate scene graphs without doing any coding at all. This allows you to wire-frame up your user interfaces.

The most important thing to understand about CUGL is how classes are allocated. CUGL makes heavy usage of smart-pointers to make memory management easier. You should almost never use the keyword new in your code (and only if you know what you are doing). You should make use of the static allocators to create smart pointers. For example, instead of

    SpriteBatch* batch = new SpriteBatch();

you should write

    std::shared_ptr<SpriteBatch> batch = SpriteBatch::alloc();

You should look at the CUGL source code to see how the classes are designed. You will notice that we have taken code that typically goes in a constructor and split it across three different methods.

  • A single default constructor, which initializes fields to default values.
  • One or more initializers, which initialize the class from user parameters.
  • One or more allocators, which use an initializer to make a smart pointer.

The exception to this rule are classes that are intended to go on the stack, like the math classes Vec2 and Size. Since those are never to combined with new, they do everything in the constructor and have multiple constructor definitions.

We will see the reason for this architecture later in class. Even though it may violate some C++ style guides on the Internet, it really is the cleanest way to organize a resource-heavy architecture like a game engine.

If you believe that you find a bug in our engine, please let us know via Piazza. Your post should start with the word [BUG]. In addition, you should use the #bug tag.


Box2D

Many of you have experience with Box2D from the introductory course. The C++ version is the native version, so you can follow the iForce tutorials exactly this time. In addition, there is a lot of functionality in the C++ version that was not available in the Java version.

With that said, you do not need to use Box2D directly when writing your game. The 2d/physics package in CUGL is a wrapper around Box2D and provides some basic functionality, at the level of the CS 3152 physics lab.


SDL

The SDL base layer is written in pure C with no C++ features at all. SDL requires manual memory management and cannot use smart pointers. In addition, it uses C-style function pointers instead of C++ closures for callback functions. Programming in SDL is very spartan, to say the least.

For the most part you should not need to program in SDL. We have wrapped the most important features into CUGL classes for you. However, there are some platform-specific features that are not part of CUGL, such as the JNI interface for Android devices. If you want to use these, you will need to access SDL directly.


OpenGL

The key thing to understand about OpenGL programming in CUGL is that there is no one version of OpenGL for all platforms. The desktop platforms support OpenGL 4.x, while the mobile platforms support OpenGLES 3.x.

OpenGLES is a essentially a limited version of OpenGL used on iOS and Android. Hence as long as you target OpenGLES, you should be fine and your code will run on all platforms. However there are some exceptions, particularly with the shader languages. OpenGLES requires some extra keywords in the shaders that OpenGL does not. You should talk to the instructor if you really need to get into the details of OpenGL and custom shaders.


CUGL for Apple Devices

Currently, it is only possible to develop for iOS on a Macintosh. While Windows is pushing some cross-platform tools for iOS, they are not currently compatible with our engine (this is a future MEng project). However, you really only need one MacOS machine on your team to deploy to a device. The cross-platform tools mean that everyone else on your team can develop and test on Windows using Visual Studio.

If no one on your team has access to a Macintosh, you can still develop for iOS. The CIS undergraduate computing lab in Gates G33 has several Macintoshes (because I specifically asked for them for this class). Most CS and INFO majors should already have access to this lab. If you need access to this lab and do not have it, please contact Amy Elser.

Setting Up Your Computer

To develop for MacOS and iOS, you must have XCode 10 While XCode 9 might be okay, you should avoid anything older than this. One reason for this is that it is much simpler to load on to a device in since XCode 9. But more importantly, iOS 12 requires XCode 10. In general, every time you upgrade your version of iOS, you must upgrade your version of XCode.

We recommend that you read the instructions for XCode below for the provided sample HelloWorld project. You should get this project to build for both platforms (MacOS and iOS) before modifying the project. That way you get experience with building a working program before you accidentally break something.


Working With XCode

Because there is a lot of configuration to do, we will not ask you to make a new XCode project. Instead, you should get an existing XCode project and work with that. The engine download will provide you with such a sample project. All of the files that you need to modify will be in the build-apple folder (though you may eventually want to add extra classes and assets to the project). In this folder is an XCode 8 project for building both MacOS and iOS.

When you open up the project, it will look like the image below. The important thing to pay attention to are the Build Targets. There are main two targets available: HelloWorld (Mac) and HelloWorld (iOS). You select the current build target from the drop-down menu in the top left, next to the stop button. You will actually see additional builds in the drop-down menu other than the two main targets, but these come from the CUGL support project and may safely be ignored.

apple-xcode
Figure 1: XCode Build Targets

When you select a build target, you also choose a device. For Mac it will always be your computer. For iOS you will be given a choice of a device or any device currently connected to your computer. That device must be approved for development. You can approve it by choosing Window > Devices. In addition, you must set the build to have proper code signing.

Our sample uses the Cornell University account for signing. This is an official iOS Developer account which we can give you access to if you wish. However, this is no longer necessary as of XCode 7 and higher. You may now load it on to your device by choosing Personal Team, as shown below. There is a limit to the number of apps you can load per month using Personal Team signing, but unless you are a TA, this should not be an issue.

apple-xcode
Figure 2: iOS Code Signing

The XCode Personal Team is as good as anything that Cornell can provide you for free. However, if you want multiplayer support, then one member in your team will need to join the iOS Developer Program, which costs $100 per year. That is because networking requires iTunes Connect, which comes with the developer program and is only available to commercial (not academic) users. This program will also allow you to distribute your game commercially, so it is a good idea to have a membership if you want to develop your game beyond this course.

Once you select a build target, press the play button to compile and run. You should do this for both platforms (MacOS and iOS).


Modifying the HelloWorld Project

When you want to modify a project, the absolutely most important thing to do is to modify the bundle identifier. You can see this for the iOS build target in Figure 2 above. It looks like a Java package. The MacOS build target has a similar bundle identifier, though it is slightly different. The bundler identifier uniquely defines your application. An iOS device can only have one program loaded with the given bundle identifier. A MacOS computer can only run one program with a given bundle identifier at a time.

You should come up with a good bundle identifier for your app and leave it fixed all semester long. We recommend that you follow package naming conventions, but add the App name at the end (as the app name is not part of the identifier). Everytime you change this identifier, it will count as a new App and load separately on to your device.

While it is less important, we also recommend that you rename the application. You will need to quit XCode to rename the file. You can name it whatever you want, but remember to preserve the .xcodeproj suffix. When you do this you will notice that the build targets in the drop-down menus look like white gears, as shown below. This means that the targets are deactivated and need to be fixed.

Before you do this, you may want to rename the targets themselves to match the project. To do this, simply click on the target name as shown below.

apple-rename
Figure 3: Renaming Build Targets

Once you have named the targets appropriately, select Manage Schemes.. from the build target drop-down menu. You will want to delete the old schemes as shown below.

apple-del-scheme
Figure 4: Deleting Defunct Build Schemes

Next hit the plus button. You will see that it will automatically make a scheme for one of the build targets for you. Just select OK. You should create a scheme for both targets (MacOS and iOS). In addition, you will want to drag them to the top so that they always appear at the top of the drop-down menu.

apple-add-scheme
Figure 5: Adding Active Build Schemes

You can now safely build this application and be assured that it will not clash with the original demo.


Adding Files to the Project

Whenever you add new files to sources or assets you will need to add them to the XCode project. In the cases of assets, we recommend that you add a reference to the folder. That way any new assets in that folder are automatically added to the project. For example, if you add a reference to a folder called sounds, any sound file placed in this folder is automatically added to the application when it builds.

The tricky thing about adding files to XCode is paying attention to the build targets. When you add a file, it adds it to one or more build targets. You always want to add to both the MacOS and the iOS build target, as shown below. If you do not pay attention to this, then you will often only add it to one of the two targets, breaking your build. Click on the Options button to verify which builds you are adding to.

apple-add-files
Figure 6: Adding Files to XCode

Modifying the Launch Assets

There are two assets that you cannot control through CUGL: the splash screen and the icon. The splash screen is shown even before the loading screen on the mobile devices, as they have a long initialization time.

These are located in the Resources folder (not be confused with the Assets folder). If you click on the iOS.xassets file, you will see options for the icon, a landscape splash screen and a portrait splash screen. When you click on each one, you will also see that they have several options for each available resolution.

apple-icons
Figure 7: XCode Icon Assets

These items are just PNG files. To set an icon, just drag a PNG on top of a slot, and it will replace it with the new value if possible. The icon must be the correct resolution. A 29pt icon at 3x resolution (the top-right icon) must be 87x87 pixels. If the PNG is not the right size it will be rejected. You must replace all of the icon slots for it to display correctly, so icon creation can be a bit of a chore.

apple-drop
Figure 8: Replacing Icon Assets

The splash screens work the same way. While XCode does not enforce resolution for these, we recommend that you stay the same size. That is 480x320 for 1x Landscape and 320x480 for 1x Portrait. We have code in the project that makes sure this will be displayed properly on all devices, no matter the aspect ratio.

One last word of warning on icons. On an iOS device you will notice that the corners are rounded. However, our icons are square. Apple will cut the corners for you automatically. Furthermore, the App Store will automatically reject any icon with an alpha channel. Keep this in mind when designing icons.


CUGL for Android Devices

Android devices can be developed on either MacOS or Windows. That is because Android Studio is a cross-platform IDE that runs on all devices. With that said, you should not make Android Studio your primary IDE, as it is not good for developing C++ applications. Debugging for Android can be very difficult, as you will not have much more than logging tools (e.g. print statements).

We recommend that you use your desktop platform as a back-up development environment to help with debugging. So if you develop on a Mac, use the Mac desktop build for testing. If you develop on Windows, use the Windows desktop build for testing. This will allow you to use a proper IDE (XCode or Visual Studio) to trace executions and search for memory issues. This is imperfect, as you will not be able to test touch screens or accelerometers on the desktop, but it gives you something to fall back on.

Supposedly, there is some support for Android applications in Visual Studio. Furthermore, Microsoft claims to have a really good simulator. We have not gotten that integration to work properly, however. If you are able to get it to work, let us know.

Setting Up Your Computer

For Android itself, we will be working with Android Studio. This is a Google-fork of the IntelliJ IDE that several of you used in 3152. This is a brand new tool this year. Last year Android Studio did not yet support the NDK (i.e. C++ development on Android), and was limited to Java development. That forced us to use Eclipse, which was horrible. But Android Studio is much better this year, and early testing has been fantastic.

Once you download Android Studio, you will need to get the both the correct Android SDK and the Android NDK. The former is necessary to build Android apps in general, while the latter is necessary to develop in C++. You will get these in Android Studio using the SDK Manager

Unfortunately, Android Studio is one of those stupid applications that requires you to open or create a project before you can even change the settings. So start by opening the HelloWorld application in Android Studio. Choose Open an Existing Project, and navigate to the build-android folder. This is an Android Studio project, so select this folder (unlike LibGDX, you do not need to chose the Gradle file). The IDE will recognize it and display a window like the one shown below.

android-import
Figure 9: Opening an Android Studio Project

Before you do anything else, you need to access the SDK Manager. Choose Tools > Android > SDK Manager. There are two panels that you need to update. You need SDK Platforms to make sure that you support the right Android OS. You need SDK Tools to integrate the NDK. Our settings are shown below. You probably do not need all of those SDKs; we have just been developing for Android for a while. Note that Android KitKat is the absolute oldest version of Android that CUGL supports.

android-import
Figure 10: Supported SDK Platforms

android-import
Figure 11: SDK Tools Including the NDK


Warning: There is an important issue to be aware of. Our build process uses ndk-build, which does not tolerate paths with spaces in them. The entire path from the root folder (C:\ on Windows, / on MacOS) to the project must be free of spaces. On Windows, if you have a space in your user directory, you are in trouble. You will need to make a new user (as you cannot change user directories for an existing user) account on your machine.


Working With Android Studio

Because there is a lot of configuration to do, we will not ask you to make a new Android Studio project. Instead, you should work from the HelloWorld template provided, adding classes to it and making modifications. Therefore, the next you should learn to do is how to compile and run the program.

To compile the project, choose Build > Make Project. It will not look like anything is happening unless you select the Gradle Console shown below. This will let you monitor the state of the build.

android-build
Figure 12: Building an Android Project

It is important that you do that, since the build is a long process. First, Android Studio invokes ndk-build to compile all of the C++ files. Then it compiles the main Java application with links to the C++ classes via JNI. We recommend you go get a drink, as it can take up to 10 minutes the first time you compile an application. That is because the CUGL library is quite involved. In subsequent builds it will only compile the changes, and so it will go much faster.

If there is a problem with the C++ files, then Android Studio will pull up the file with the error immediately after compilation fails. However, it is not guaranteed to display all of the issues. It may just show the first one it found. That is why we always recommend that you compile for a desktop release (MacOS or Window) to catch minor bugs before you try to launch on Android.

If the compilation succeeds, then you are ready to run the program. Plug your Android device into your computer. Select Run > Run 'app'. This will launch a window asking you to chose an Android device. It will include several simulators to choose from. But hopefully it shows your device as well, as shown below.

android-device
Figure 13: Selecting an Android Device

Once you choose the device, you should see "Installing APK" at the bottom of the window. Do not unplug your device until this message disappears and the app launches. Again this can take a while, but nowhere as long as compiling. While you are waiting, you should click on the Logcat tab in the IDE. This is where all of the logging (i.e. print) statements will be displayed. his window is basically the extent of your ability to debug Android. Breakpoints and memory dumps, such as are possible in XCode and Visual Studio, are out of the question.

android-logcat
Figure 14: Android Device Logging


Modifying the HelloWorld Project

When you want to modify a project, the absolutely most important thing to do is to modify the application package. This is defined in the file AndroidManifest.xml show below. For Android, the Java package works just like the bundle identifier in that it uniquely defines your application. An Android device can only have one program loaded with the given package.

android-manifest
Figure 15: The Android Package Manifest

You should come up with a good package name for your app and leave it fixed all semester long. We recommend that you follow package naming conventions, but add the App name at the end (as the app name is not part of the identifier). Everytime you change this identifier, it will count as a new App and load separately on to your device.

When you change the package, you will also need to change it in several other places. You need to change it in the build.gradle (app) file which is under Gradle Scripts. You will also need to change the Java activity. The Java activity is the lone Java class that serves to boot-strap your application and load all of the C++ classes through JNI. The sample activity actually has two activities. The first, SDLActivity.java is a base class that does all of the work and should never be changed. The subclass HelloWorld.java simply sets the initial splash screen and defines the application package. This is the file you should modify.

To change the displayed name in Android, you will need to make another change. The application name is set in res/values/strings.xml. Unlike the package name, this is not a unique identifier and can be the same name as an existing app.

Finally, you may change the name of the activity class HelloWorld.java. However, this is not necessary and will have no visible affect on the application. If you change it, you will need to change the activity name in AndroidManifest.xml as well.


Adding Files to the Project

Adding assets to the project is easy. Anything that you add to the assets folder will automatically be included with the Android build provided that the link to the assets folder is correct in Android Studio. If you are having problems with this folder, look at the build.gradle file for Module: app. The sourceSets attribute links the assets folder into the application.

The challenge is when you want to add a subdirectory to the sources folder. Currently ndk-build is only configured to recognize files in the top-level directory. To add a subdirectory to ndk-build, you will need to modify the jni/source/Android.mk file. This is not the Android.mk shown under External Build Files. That is the top level build file that links your code to CUGL. Try as we might, we were not able to get Android Studio to display the individual make files (sigh).

So instead, you will need to take the text editor of your choice and open the file app/jni/source/Android.mk. You will see the variable LOCAL_SRC_FILES which defines the list of files to compile. You will need to add your new subdirectory to this variable. To see how do this, look at how app/jni/cugl/Android.mk handles multiple directories. The interface is not completely intuitive, but that is nothing a little Stack Overflow cannot solve.


Modifying the Launch Assets

There are two assets that you cannot control through CUGL: the splash screen and the icon. The splash screen is shown even before the loading screen on the mobile devices, as they have a long initialization time.

These are located in the res folder (not be confused with the assets folder). You will see a lot of sub folders of the form drawable-xxx. These represented the various resolutions supported. Each of these folders contains an icon and two splash screens (one for Landscape, the other for Portrait). You will notice that these are just PNG files. You can replace them with your own PNG files.

When assigning new icons or splash screens, the file names should not change. In addition they should be exactly the same resolution as before. If you change the resolution, Android will not crash, but weird things will happen.

Android Studio has some nice tools for making a new icon. You can access them by right clicking on app in Android view and choosing New > Image Asset. However, at this time you cannot use it to make a new splash screen.


CUGL for Windows

CUGL can create traditional 32-bit and 64-bit applications. However, it cannot create Universal Windows applications (e.g. Surface, Windows Phone). That is because SDL support for UWP is not really read for prime time (indeed, we cannot get the core SDL library to build on this platform).

Since we do not support mobile Windows devices, we currently view Windows as a debugging platform. It is like MacOS is for XCode: a way for a developer to quickly test code without having to have a device handy. With that said, it is a very powerful debugging platform, as Visual Studio provides you with a lot of C++ tools not found in Eclipse. If you are getting segmentation faults in your Android App, Visual Studio has some of the best memory tools out there.

With that said, there is nothing preventing you from releasing your game as a Windows app (though you cannot do it through the Windows App Store). You are welcome to create a Windows fork of your game if you wish. However, we will not grade your game on this platform.


Setting Up Your Computer

If you are working on Windows, you should use Visual Studio 2017. Visual Studio 2017 has fairly stable support for C++11, which is the dialect of C++ that we will be using. Unlike previous versions of Visual Studio, the Community version appears to have enough features to use for this class. However, remember that all Cornell students get access to Microsoft products via Azure Dev Tools. Log into this web page with your Cornell netid e-mail and netid password.

Working With Visual Studio

Because there is a lot of configuration to do, we will not ask you to make a new Visual Studio project. Visual Studio is particularly brittle and requires a large number of custom property settings to build. So instead, you should get an existing Visual Studio project and work with that. The engine download provides you with such a sample project. All of the files that you need to modify will be in the build-win10 folder (though you may eventually want to add extra classes and assets to the project). In this folder is a Visual Studio solution for building the Windows application.

If you have never worked with Visual Studio before, you should be aware of the difference between a solution and a project. A project builds a self-contained library or piece of software. A solution combines several projects together, possibly linking the result. In the top-level build-win10 directory, there is a Visual Studio file called HelloWorld.sln. This is the solution. Open this file. When you do, you will see a Visual Studio with three projects: Box2D, CUGL, and the HelloWorld project (which is not the same as the HelloWorld solution).

windows-ide
Figure 16: A Visual Studio Solution with Three Projects

We separated Box2D from CUGL because of some build issues that you do not need to be worried about. The most important project is HelloWorld, which contains all of the new code for the demo application. The solution links this project to CUGL (and hence Box2D). The solution also marks HelloWorld as the main project, so that it is the one that launches whenever you run the application.

To build the application, choose Build > Build Solution. You will need to select a configuration before doing so. These are the two drop-down menus to the left of the play button. The default configuration is Debug x86, which is an unoptimized 32-bit application. To make it a 64-bit application, choose x64 from the second drop-down. You should only choose a Release configuration for packaging your app. The optimizations in Release mode will erase and re-order a lot of code, making the debugger useless.

When you want to run the application, hit the play button. This will run it in a debugger, allowing you to set breakpoints or inspect the program when errors occur. Most of your time with Visual Studio will be doing just this.


Modifying the HelloWorld Project

To modify a Windows Visual Studio project, the only thing that you need to do is rename the project. All other settings (including the name of the application created) are determined by this. You may also rename the solution if you wish, but that is not as important.

You might think that you rename the project by right-clicking on it and choosing rename, as shown below. Unfortunately, that is wrong. That command is for .NET only. For C++ projects it is useless and does not actually rename the project.

windows-rename-fail
Figure 17: The Most Useless Command Ever

A quick search of Stack Overflow shows that the only way to rename a C++ project is to follow three steps: (1) remove it from the solution, (2) rename all of the files manually, and (3) add it back to the solution. For example, suppose you are renaming the HelloWorld project to ShipDemo. Then you need to replace HelloWorld in all of the file names below with ShipDemo.

windows-manual-rename
Figure 18: List of Project Files to Rename


Adding Files to the Project

Adding assets to the project is easy. Anything that you add to the assets folder will automatically be included with the Windows build. The project properties have been configured to include all of the assets that it finds.

Adding a source code file requires just only a little more work. To add a file, right click on the Source Files filter. Select Add Existing Item and add the files that you want. Visual Studio knows to compile an C and C++ files that you add.


Packaging Windows Applications for Release

Visual Studio really wants to run the application in the debugger. Even if you compile in release mode, and navigate to the output directory, you will discover that you cannot double-click on the file. It will tell you right away that a DLL is missing. The debugger knows where to find this DLL, but the application itself does not.

To solve this problem, the application needs to have a lot of files added to it (and some even removed). As an example, build the release for x86 and then navigate to the x86/Release folder. You will see a lot of files in this folder, as many of these files are intermediate results. Pull out the .exe for your application, as it is the only file that matter, and put it in a new directory. Inside this new directory you should add all of the folders inside of assets (but not the assets folder itself). You should also add all of the DLLs from the build-windows/dll/x86. If you did this for the sample project it would look something like the directory below.

windows-release
Figure 19: Packaging HelloWorld for Release

You can now double-click on the Windows .exe and it will work correctly. It is very important to not put the DLLs in any sort of folder. The fact that Windows applications have such messy directories is one of the reasons people always launch Windows apps from the Start Menu.


Modifying the Launch Assets

The only launch asset for Windows is the icon, and that only matters for the release. However, for completeness we will tell you how to modify this. All you have to do is replace the icon1.ico in the project folder, shown below. The next time that you build the application, Visual Studio will automatically integrate this new icon into the executable file.

windows-icon
Figure 20: The Windows ICO File

This means that the only challenge is making an .ico file. This is not a traditional graphics file format. Fortunately, there are a bunch of online conversion tools to turn a PNG file into an ICO file. There are also some icon editting programs. However, we much prefer to create the icons as PNG files first, since that is the format used by Apple and Android.