Making Structs More Useful with Object Handles
Structs can be a great way to keep the garbage collector off your back and to use the CPU’s data cache more effectively. Not everything can be a struct though. At a minimum, you’ll need to use some Unity and .NET classes like MonoBehaviour
and string
. If your struct has any of these as fields, you can no longer use sizeof(MyStruct)
. That really limits its usefulness, so a workaround is needed. Enter object handles: a simple way to represent any object
as a plain old int
which won’t break sizeof
. Read on to see how these work and some code you can easily drop into your project to start using them right away!
Say you want to store an array of structs in unmanaged memory to avoid creating a managed array object that the GC will eventually collect. Making an array is simple in such a case:
struct Enemy { public Vector3 Position; } // Allocate unmanaged memory big enough for 1000 enemies Enemy* enemies = (Enemy*)Marshal.AllocHGlobal(sizeof(Enemy) * 1000);
The sizeof(Enemy)
is critical here. That’s how we know how much space one enemy takes up. Now consider if we add a name string
to the Enemy
:
struct Enemy { public Vector3 Position; public string Name; } // Allocate unmanaged memory big enough for 1000 enemies Enemy* enemies = (Enemy*)Marshal.AllocHGlobal(sizeof(Enemy) * 1000);
Now we get a compiler error on the part where we try to do sizeof(Enemy)
:
error CS0208: Cannot take the address of, get the size of, or declare a pointer to a managed type `Enemy'
That’s because string
is a managed class and we’re not allowed to know the size of its instances. So we need a workaround if we want to use any managed classes. That’s basically a given since classes like string
and MonoBehaviour
are indispensable in everyday Unity programming.
The workaround adds a layer of indirection. Instead of directly storing the string
in the struct, we store something that can be used to get the string
. A simple int
should do for this purpose. So we’ll store managed objects outside the struct and use the int
to identify an individual object
.
Doing this is really quite simple. We just need an array of stored objects and another array of available int
handles that we’ll treat as a stack. To store an object, pop a handle off the stack and use it as an index into the objects array to store the object there. To get a stored object, simply index the array with the handle! To remove an object, index into the array with the handle and set it to null then push the handle onto the stack.
The following code implements the system in about 20 lines of code plus a bunch of comments. Look it over and see how it works and keep in mind it’s MIT licensed so it should be easily incorporated into virtually any project:
/// <summary> /// Stores objects and allows access to them via an int. /// This class is thread-safe. /// </summary> /// /// <author> /// JacksonDunstan, http://JacksonDunstan.com/articles/3908 /// </author> /// /// <license> /// MIT /// </license> public static class ObjectStore { // Stored objects. The first is always null. private static object[] objects; // Stack of available handles private static int[] handles; // Index of the next available handle private static int nextHandleIndex; /// <summary> /// Initialize the object storage and reset the handles /// </summary> /// /// <param name="maxObjects"> /// Maximum number of objects to store. Must be positive. /// </param> public static void Init(int maxObjects) { // Initialize the objects as all null plus room for the // first to always be null. objects = new object[maxObjects + 1]; // Initialize the handles stack as 1, 2, 3, ... handles = new int[maxObjects]; for ( int i = 0, handle = maxObjects; i < maxObjects; ++i, --handle) { handles[i] = handle; } nextHandleIndex = maxObjects - 1; } /// <summary> /// Store an object /// </summary> /// /// <param name="obj"> /// Object to store. This can be null. /// </param> /// /// <returns> /// An handle to the stored object that can be used with /// <see cref="Get"/> and <see cref="Remove"/>. If /// <see cref="Init"/> has not yet been called, a /// <see cref="NullReferenceException"/> will be thrown. /// </returns> public static int Store(object obj) { lock (objects) { // Pop a handle off the stack int handle = handles[nextHandleIndex]; nextHandleIndex--; // Store the object objects[handle] = obj; // Return the handle return handle; } } /// <summary> /// Get the object for a given handle /// </summary> /// /// <param name="handle"> /// Handle of the object to get. If this is less than zero /// or greater than the maximum number of objects passed to /// <see cref="Init"/>, this function will throw an /// <see cref="ArrayIndexOutOfBoundsException"/>. If this /// is zero, not a handle returned by <see cref="Store"/>, /// a handle returned by a call to <see cref="Store"/> with /// a null parameter, or a handle passed to /// <see cref="Remove"/> and not subsequently returned by /// <see cref="Store"/>, this function will return null. If /// <see cref="Init"/> has not yet been called, a /// <see cref="NullReferenceException"/> will be thrown. /// </param> public static object Get(int handle) { return objects[handle]; } /// <summary> /// Remove a stored object /// </summary> /// /// <param name="handle"> /// Handle of the object to Remove. If this is less than /// zero or greater than the maximum number of objects /// passed to <see cref="Init"/>, this function will throw /// an <see cref="ArrayIndexOutOfBoundsException"/>. The /// handle may be be reused. If <see cref="Init"/> has not /// yet been called, a <see cref="NullReferenceException"/> /// will be thrown. /// </param> public static void Remove(int handle) { lock (objects) { // Forget the object objects[handle] = null; // Push the handle onto the stack nextHandleIndex++; handles[nextHandleIndex] = handle; } } }
Now let’s return to our original example with the enemy that needs a name. Instead of adding a string Name
field we’ll add a int NameHandle
field that we can use to get its name:
struct Enemy { public Vector3 Position; public int NameHandle; } // At app startup... ObjectStore.Init(100000); // Later on in the app... // Allocate unmanaged memory big enough for 1000 enemies Enemy* enemies = (Enemy*)Marshal.AllocHGlobal(sizeof(Enemy) * 1000);
Since Enemy
no longer has any managed object fields, sizeof(Enemy)
compiles an works just fine. Now we can use NameHandle
to get and set strings. Here’s an example using a property:
struct Enemy { public Vector3 Position; private int NameHandle; public string Name { get { return (string)ObjectStore.Get(NameHandle); } set { // Remove existing name if (NameHandle != 0) { ObjectStore.Remove(NameHandle); NameHandle = 0; } // Store new name if (value != null) { NameHandle = ObjectStore.Store(value); } } } }
The Name
property makes it appear to users of Enemy
like there is a string Name
as we’d written before. They can freely get an set it and the details are taken care of behind the scenes. The only wrinkle is that users must remember to set Name
to null
before they’re done with the Enemy
or the string
and its handle will be leaked. Object handles should be treated like file handles or any other resource that requires manual cleanup.
That’s all there is for today’s demonstration of object handles. They’re a simple and reasonably-efficient way to make structs more useful any time you need the sizeof(MyStruct)
operator.
#1 by Leucaruth on June 12th, 2017 ·
As always, thanks for a really useful post.
#2 by AleksandarMi on June 14th, 2017 ·
Awesome idea!
Additionally I would try out how big impact on performances would be if we introduce inner struct for object handle?
Something like:
That would simplify writing of properties:
To bad we cannot use generic structs here, it would be event more awesome with implicit conversion. That way user could use field as if it was of original type that we referenced.
#3 by jackson on June 14th, 2017 ·
That’s definitely something you could do! If you wanted to go further, you could make it into a
Nullable<T>
-style type:#4 by Jérémie on June 16th, 2017 ·
What about arrays in structs? The array is a reference type, right?
#5 by jackson on June 16th, 2017 ·
Yes, managed arrays are reference types that you can create object handles for. Unmanaged arrays store in memory allocated by
Marshal.AllocHGlobal
, as shown in the article, are not reference types. They’re just a pointer (e.g.int*
).