JacksonDunstan.com

One of the advantages we get when we use unmanaged memory is a huge increase in flexibility. For example, we can easily allocate a whole array of objects at once instead of one-at-a-time when we new a class instance. We can also create a memory pool with one allocation then divide it up ourselves. It turns out that can really speed up memory allocation and, at the same time, actually reduce memory fragmentation on top of the fragmentation we’re avoiding by not creating any garbage for the GC. Today’s article shows you how memory pools work and provides an implementation of one you can use in your own projects!

A memory pool is a block of memory that we, not the OS or VM, decide to sub-divide into a fixed number of constant-size blocks. For example, we may want a memory pool for small allocations so we’d create a memory pool of 10,000 256-byte blocks. To do so, we just call Marshal.AllocHGlobal(256*10000) and all of a sudden we have enough memory for all the objects. For example, we could treat it like an array and make a simple allocator:

public struct Vector3Pool
{
	private Vector3** pool;
	private int nextVector;
 
	public Vector3Pool(int poolSize)
	{
		pool = (Vector3**)Marshal.AllocHGlobal(poolSize * sizeof(Vector3));
		nextVector = 0;
	}
 
	public Vector3* Allocate()
	{
		return pool[nextVector++];
	}
}
 
Vector3Pool pool = new Vector3Pool(10000);
Vector3* a = pool.Allocate();
Vector3* b = pool.Allocate();
Vector3* c = pool.Allocate();

That’s a really simple allocator, but it illustrates the flexibility we have with unmanaged memory. We can decide to treat the memory we get back from AllocHGlobal however we want. We can treat it as though it was an array of Vector3 objects just by casting it to Vector3**. We don’t have to call AllocHGlobal every time we need room for another Vector3. We can write our own Allocate function!

There is, of course, a huge problem with this kind of pool: there’s no way to release the vectors we allocated when we’re done with them. We need a Free, but that’s impractical with this simple design. So instead we’ll switch to a new design.

In the new design, we’ll keep a linked list of free blocks in the pool. This linked list will be intrusive because we’re going to store it in the blocks themselves. This means we won’t have to allocate any more space to hold the linked list and we certainly won’t be allocating one node of the list at a time.

The next step of the design is that we want some safety for the memory that we allocate, at least in debug mode. So we’ll add a little extra data at the end of each block in the pool to store a sentinel value. It’s just a constant value, but we can check it to make sure that we didn’t accidentally write beyond the start or end of a block.

With those two pieces in place, here’s how the memory pool’s memory will look:

Memory Pool Initial State

We’re keeping the “next” pointer in the list of free blocks at the start of each block. Initially, each block points to the next block and the last block points to null. We keep a Free pointer to the “head” of the list.

So how do we allocate from the pool? Easy! All we have to do is chop the head off of the list. It’s just three lines of code and it’s super fast:

void* Allocate()
{
	void* ptr = Free;
	Free = *Free;
	return ptr;
}

Here’s how it looks after we allocate our first block:

Memory Pool After Allocating First Block

And here’s how it looks after we allocate another block:

Memory Pool After Allocating Second Block

OK, so how do we free a block? Also easy! All we have to do is add the block to the head of the array with three more lines of code:

void Free(void* ptr)
{
	void** pHead = (void**)ptr;
	*pHead = Free;
	Free = pHead;
}

Here’s how the pool looks after we free the first block:

Memory Pool After Freeing First Block

And after we free the second block:

Memory Pool After Freeing Second Block

The “free list” now takes a different path through the blocks in the pool, but that’s OK as long as it gets to all of them.

The real code is a little more complex than the Allocate and Free functions above because it needs to check the sentinel values, do other error checking, and add some useful features. That’s the core of the design though. Feel free to peruse the code below for the full details if you’re curious.

Now let’s put this design to the test. To do so, we’ll use a little test script that allocates five different ways:

Create a class instance with new
Allocate unmanaged memory big enough for a struct with Marshal.AllocHGlobal
#2 and also clear the memory with zeroes
Allocate unmanaged memory from a memory pool with the above design
#4 and also clear the memory with zeroes

Here’s the test script:

using System;
using System.Diagnostics;
 
using UnityEngine;
 
unsafe public class TestScript : MonoBehaviour
{
	class TestClass{public int X; public int Y;}
	struct TestStruct{public int X; public int Y;}
 
	void Start()
	{
		const int reps = 10000000;
		UnmanagedMemory.SetUp(reps*2+1);
		int blockSize = sizeof(TestStruct);
		UnmanagedMemoryPool pool = UnmanagedMemory.AllocPool(blockSize, reps);
		Stopwatch stopwatch = Stopwatch.StartNew();
 
		stopwatch.Reset();
		stopwatch.Start();
		for (int i = 0; i < reps; ++i)
		{
			new TestClass();
		}
		long newClassTime = stopwatch.ElapsedMilliseconds;
 
		stopwatch.Reset();
		stopwatch.Start();
		for (int i = 0; i < reps; ++i)
		{
			UnmanagedMemory.Alloc(blockSize);
		}
		long allocTime = stopwatch.ElapsedMilliseconds;
 
		stopwatch.Reset();
		stopwatch.Start();
		for (int i = 0; i < reps; ++i)
		{
			UnmanagedMemory.Calloc(blockSize);
		}
		long callocTime = stopwatch.ElapsedMilliseconds;
 
		stopwatch.Reset();
		stopwatch.Start();
		for (int i = 0; i < reps; ++i)
		{
			UnmanagedMemory.Alloc(&pool);
		}
		long allocPoolTime = stopwatch.ElapsedMilliseconds;
 
		UnmanagedMemory.FreeAll(&pool);
 
		stopwatch.Reset();
		stopwatch.Start();
		for (int i = 0; i < reps; ++i)
		{
			UnmanagedMemory.Calloc(&pool);
		}
		long callocPoolTime = stopwatch.ElapsedMilliseconds;
 
		UnityEngine.Debug.LogFormat(
			"Allocation Type,Time\n" +
			"New Class,{0}\n" +
			"Alloc,{1}\n" +
			"Calloc,{2}\n" +
			"Pool Alloc,{3}\n" +
			"Pool Calloc,{4}",
			newClassTime,
			allocTime,
			callocTime,
			allocPoolTime,
			callocPoolTime
		);
	}
 
	void OnApplicationQuit()
	{
		UnmanagedMemory.TearDown();
	}
}

If you want to try out the test yourself, simply paste the above code into a TestScript.cs file in your Unity project’s Assets directory and attach it to the main camera game object in a new, empty project. Then build in non-development mode, ideally with IL2CPP. I ran it that way on this machine:

LG Nexus 5X
Android 7.1.1
Unity 5.5.1f1, IL2CPP

And here are the results I got:

Allocation Type	Time
New Class	1152
Alloc	1144
Calloc	1403
Pool Alloc	60
Pool Calloc	211

Memory Allocation Performance

Note: “calloc” is shorthand for “allocate and clear with zeroes”

Using new to create a class instance is just about as fast as calling Marshal.AllocHGlobal. At 10 million iterations, the difference is negligible. There’s no CallocHGlobal, so my C# code has to clear the allocated memory with zeroes and that adds to the time for the “Calloc” category.

Pools are in a completely different league. Those three simple lines of code absolutely crush the performance of non-pool allocations. It’s 5x faster to “calloc” from a pool and a 20x difference difference if you don’t need to clear with zeroes. That’s often the case since you’re just going to write all the fields of a structure anyways.

So the performance advantages are clear when using a memory pool, but how do they fare in other categories? Well, they create no garbage whatsoever, so that’s an obvious win over using new to get class instances. The GC will simply never track, let alone run to collect any of the memory allocated for or by a pool.

They use fixed-size blocks, so it’s impossible to fragment memory by splitting it into tinier and tinier chunks until reasonably-sized objects won’t fit anymore. That can be a big win compared to Unity’s GC which is notorious for causing fragmentation.

They have the safety of sentinels, so we are alerted when we accidentally underflow or overflow a block. That’s not as safe as individual allocations where the OS will crash the app if we write outside the memory allocated for us, so this is definitely a riskier approach than individual AllocHGlobal calls or using new with classes in that respect.

Finally, here’s the code that implements the memory pool. It’s built on the UnmanagedMemory static class from the previous article. Here’s an example of how to use it:

try
{
	// 10k max allocations at a time
	UnmanagedMemory.SetUp(10000);
 
	// Allocate a pool of 1000 blocks, each 256 bytes long
	UnmanagedMemoryPool pool = UnmanagedMemory.AllocPool(256, 1000);
 
	// Allocate a block from the pool
	void* ptr = UnmanagedMemory.Alloc(&pool); // or Calloc
 
	// Free a block back into the pool
	UnmanagedMemory.Free(&pool, ptr);
 
	// Free all the blocks in the pool
	UnmanagedMemory.FreeAll(&pool);
 
	// Free the pool itself
	UnmanagedMemory.FreePool(&pool);
}
// In debug mode there are many exceptions to indicate problems: sentinel overwrites, out of memory, etc.
catch (UnmanagedMemoryException ex)
{
	Debug.LogErrorFormat("Something went wrong with unmanaged memory: {0}", ex);
}
finally
{
	// Done with unmanaged memory. OnApplicationQuit() is a good place to put this.
	UnmanagedMemory.TearDown();
}

And here’s the code itself. It’s about 500 lines, including all the comments and error handling:

using System;
using System.Diagnostics;
using System.Runtime.InteropServices;
 
/// <summary>
/// Types of exceptions that can happen while dealing with unmanaged memory
/// </summary>
/// <author>Jackson Dunstan, http://JacksonDunstan.com</author>
/// <license>MIT</license>
public enum UnmanagedMemoryExceptionType
{
	PointerCanNotBeNull,
	CountMustBeNonNegative,
	AllocationSizeMustBePositive,
	AllocationTableIsFull,
	NullPool,
	UninitializedOrDestroyedPool,
	PoolIsDry,
	SentinelOverwritten,
	BlockSizeBelowMinimum,
	NumberOfBlocksMustBePositive,
	PointerDoesNotPointToBlockInPool
}
 
/// <summary>
/// An exception indicating an error related to unmanaged memory
/// </summary>
/// <author>Jackson Dunstan, http://JacksonDunstan.com</author>
/// <license>MIT</license>
unsafe public class UnmanagedMemoryException : Exception
{
	/// <summary>
	/// Create the exception
	/// </summary>
	/// <param name="type">Type of the exception</param>
	/// <param name="pointer">Pointer related to the exception</param>
	public UnmanagedMemoryException(UnmanagedMemoryExceptionType type, void* pointer = null)
	{
		Type = type;
		Pointer = pointer;
	}
 
	/// <summary>
	/// Get the type of the exception
	/// </summary>
	/// <value>The type of the exception</value>
	public UnmanagedMemoryExceptionType Type { get; private set; }
 
	/// <summary>
	/// Pointer related to the exception
	/// </summary>
	/// <value>The pointer related to the exception</value>
	public void* Pointer { get; private set; }
 
	/// <summary>
	/// Get a string version of this exception
	/// </summary>
	/// <returns>A string version of this exception</returns>
	public override string ToString()
	{
		return string.Format(
			"[UnmanagedMemoryException: Type={0}, Pointer={1}]",
			Type,
			(IntPtr)Pointer
		);
	}
}
 
/// <summary>
/// A pool of unmanaged memory. Consists of a fixed number of equal-sized blocks that can be
/// allocated and freed.
/// </summary>
/// <author>Jackson Dunstan, http://JacksonDunstan.com</author>
/// <license>MIT</license>
unsafe public struct UnmanagedMemoryPool
{
	/// <summary>
	/// Unmanaged memory containing all the blocks
	/// </summary>
	public byte* Alloc;
 
	/// <summary>
	/// Pointer to the next free block
	/// </summary>
	public void* Free;
 
	/// <summary>
	/// Size of a single block. May include extra bytes for internal usage, such as a sentinel.
	/// </summary>
	public int BlockSize;
 
	/// <summary>
	/// Number of blocks
	/// </summary>
	public int NumBlocks;
}
 
/// <summary>
/// Tools for dealing with unmanaged memory
/// </summary>
/// <author>Jackson Dunstan, http://JacksonDunstan.com</author>
/// <license>MIT</license>
unsafe public static class UnmanagedMemory
{
	#if UNITY_EDITOR
		/// <summary>
		/// Hash table that keeps track of all the allocations that haven't been freed
		/// </summary>
		private static void** allocations;
 
		/// <summary>
		/// Size/length of the <see cref="allocations"/> hash table
		/// </summary>
		private static int maxAllocations;
	#endif
 
	/// <summary>
	/// The size of a pointer, in bytes
	/// </summary>
	public static readonly int SizeOfPointer = sizeof(void*);
 
	/// <summary>
	/// The minimum size of an <see cref="UnmanagedMemoryPool"/> block, in bytes
	/// </summary>
	public static readonly int MinimumPoolBlockSize = SizeOfPointer;
 
	#if UNITY_ASSERTIONS || UNMANAGED_MEMORY_DEBUG
		/// <summary>
		/// Value added to the end of an <see cref="UnmanagedMemoryPool"/> block. Used to detect
		/// out-of-bound memory writes.
		/// </summary>
		private const ulong SentinelValue = 0x8899AABBCCDDEEFF;
	#endif
 
	/// <summary>
	/// Prepare this class for use
	/// </summary>
	/// <param name="maxAllocations">Maximum number of allocations expected</param>
	[Conditional("UNITY_EDITOR")]
	public static void SetUp(int maxAllocations)
	{
		#if UNITY_EDITOR
			// Create the allocations hash table
			if (allocations == null)
			{
				int size = maxAllocations * SizeOfPointer;
				allocations = (void**)Marshal.AllocHGlobal(size);
				Memset(allocations, 0, size);
				UnmanagedMemory.maxAllocations = maxAllocations;
			}
		#endif
	}
 
	/// <summary>
	/// Stop using this class. This frees all unmanaged memory that was allocated since
	/// <see cref="SetUp"/> was called.
	/// </summary>
	[Conditional("UNITY_EDITOR")]
	public static void TearDown()
	{
		#if UNITY_EDITOR
			if (allocations != null)
			{
				// Free all the allocations
				for (int i = 0; i < maxAllocations; ++i)
				{
					void* ptr = allocations[i];
					if (ptr != null)
					{
						Marshal.FreeHGlobal((IntPtr)ptr);
					}
				}
 
				// Free the allocations table itself
				Marshal.FreeHGlobal((IntPtr)allocations);
				allocations = null;
				maxAllocations = 0;
			}
		#endif
	}
 
	[Conditional("UNITY_ASSERTIONS"), Conditional("UNMANAGED_MEMORY_DEBUG")]
	public static void Assert(bool condition, UnmanagedMemoryExceptionType type, void* data = null)
	{
		#if UNITY_ASSERTIONS
			if (!condition)
			{
				throw new UnmanagedMemoryException(type, data);
			}
		#endif
	}
 
	/// <summary>
	/// Set a series of bytes to the same value
	/// </summary>
	/// <param name="ptr">Pointer to the first byte to set</param>
	/// <param name="value">Value to set to all the bytes</param>
	/// <param name="count">Number of bytes to set</param>
	public static void Memset(void* ptr, byte value, int count)
	{
		Assert(ptr != null, UnmanagedMemoryExceptionType.PointerCanNotBeNull);
		Assert(count >= 0, UnmanagedMemoryExceptionType.CountMustBeNonNegative);
 
		byte* pCur = (byte*)ptr;
		for (int i = 0; i < count; ++i)
		{
			*pCur++ = value;
		}
	}
 
	/// <summary>
	/// Allocate unmanaged heap memory and track it
	/// </summary>
	/// <param name="size">Number of bytes of unmanaged heap memory to allocate</param>
	public static IntPtr Alloc(int size)
	{
		Assert(size > 0, UnmanagedMemoryExceptionType.AllocationSizeMustBePositive);
 
		IntPtr intPtr = Marshal.AllocHGlobal(size);
		#if UNITY_EDITOR
			void* ptr = (void*)intPtr;
			int index = (int)(((long)ptr) % maxAllocations);
			for (int i = index; i < maxAllocations; ++i)
			{
				if (allocations[i] == null)
				{
					allocations[i] = ptr;
					return intPtr;
				}
			}
			for (int i = 0; i < index; ++i)
			{
				if (allocations[i] == null)
				{
					allocations[i] = ptr;
					return intPtr;
				}
			}
			Assert(false, UnmanagedMemoryExceptionType.AllocationTableIsFull);
		#endif
		return intPtr;
	}
 
	/// <summary>
	/// Allocate unmanaged heap memory filled with zeroes and track it
	/// </summary>
	/// <param name="size">Number of bytes of unmanaged heap memory to allocate</param>
	public static IntPtr Calloc(int size)
	{
		IntPtr intPtr = Alloc(size);
		Memset((void*)intPtr, 0, size);
		return intPtr;
	}
 
	/// <summary>
	/// Allocate a block of memory from a pool
	/// </summary>
	/// <param name="pool">Pool to allocate from</param>
	public static void* Alloc(UnmanagedMemoryPool* pool)
	{
		Assert(pool != null, UnmanagedMemoryExceptionType.NullPool);
		Assert(pool->Alloc != null, UnmanagedMemoryExceptionType.UninitializedOrDestroyedPool);
		Assert(pool->Free != null, UnmanagedMemoryExceptionType.PoolIsDry);
 
		void* pRet = pool->Free;
 
		// Make sure the sentinel is still intact
		#if UNITY_ASSERTIONS || UNMANAGED_MEMORY_DEBUG
			if (*((ulong*)(((byte*)pRet)+pool->BlockSize-sizeof(ulong))) != SentinelValue)
			{
				Assert(false, UnmanagedMemoryExceptionType.SentinelOverwritten, pRet);
			}
		#endif
 
		// Return the head of the free list and advance the free list pointer
		pool->Free = *((byte**)pool->Free);
		#if UNITY_ASSERTIONS || UNMANAGED_MEMORY_DEBUG
			*((ulong*)(((byte*)pRet)+pool->BlockSize-sizeof(ulong))) = SentinelValue;
		#endif
		return pRet;
	}
 
	/// <summary>
	/// Allocate a zero-filled block of memory from a pool
	/// </summary>
	/// <param name="pool">Pool to allocate from</param>
	public static void* Calloc(UnmanagedMemoryPool* pool)
	{
		void* ptr = Alloc(pool);
		Memset(ptr, 0, pool->BlockSize);
		return ptr;
	}
 
	/// <summary>
	/// Allocate a pool of memory. The pool is made up of a fixed number of equal-sized blocks.
	/// Allocations from the pool return one of these blocks.
	/// </summary>
	/// <returns>The allocated pool</returns>
	/// <param name="blockSize">Size of each block, in bytes</param>
	/// <param name="numBlocks">The number of blocks in the pool</param>
	public static UnmanagedMemoryPool AllocPool(int blockSize, int numBlocks)
	{
		Assert(
			blockSize >= MinimumPoolBlockSize,
			UnmanagedMemoryExceptionType.BlockSizeBelowMinimum
		);
		Assert(numBlocks > 0, UnmanagedMemoryExceptionType.NumberOfBlocksMustBePositive);
 
		#if UNITY_ASSERTIONS || UNMANAGED_MEMORY_DEBUG
			// Add room for the sentinel
			blockSize += sizeof(ulong);
		#endif
 
		UnmanagedMemoryPool pool = new UnmanagedMemoryPool();
 
		pool.Free = null;
		pool.NumBlocks = numBlocks;
		pool.BlockSize = blockSize;
 
		// Allocate unmanaged memory large enough to fit all the blocks
		pool.Alloc = (byte*)Alloc(blockSize * numBlocks);
 
		#if UNITY_ASSERTIONS || UNMANAGED_MEMORY_DEBUG
		{
			// Set the sentinel value at the end of each block
			byte* pCur = pool.Alloc + blockSize - sizeof(ulong);
			for (int i = 0; i < numBlocks; ++i)
			{
				*((ulong*)pCur) = SentinelValue;
				pCur += blockSize;
			}
		}
		#endif
 
		// Reset the free list
		FreeAll(&pool);
 
		return pool;
	}
 
	/// <summary>
	/// Free unmanaged heap memory and stop tracking it
	/// </summary>
	/// <param name="ptr">Pointer to the unmanaged heap memory to free. If null, this is a no-op.
	/// </param>
	public static void Free(IntPtr ptr)
	{
		if (ptr != IntPtr.Zero)
		{
			Marshal.FreeHGlobal(ptr);
			#if UNITY_EDITOR
				void* voidPtr = (void*)ptr;
				int index = (int)(((long)voidPtr) % maxAllocations);
				for (int i = index; i < maxAllocations; ++i)
				{
					if (allocations[i] == voidPtr)
					{
						allocations[i] = null;
						return;
					}
				}
				for (int i = 0; i < index; ++i)
				{
					if (allocations[i] == voidPtr)
					{
						allocations[i] = null;
						return;
					}
				}
			#endif
		}
	}
 
	/// <summary>
	/// Free a block from a pool
	/// </summary>
	/// <param name="pool">Pool the block is from</param>
	/// <param name="ptr">Pointer to the block to free. If null, this is a no-op.</param>
	public static void Free(UnmanagedMemoryPool* pool, void* ptr)
	{
		Assert(pool != null, UnmanagedMemoryExceptionType.NullPool);
		Assert(pool->Alloc != null, UnmanagedMemoryExceptionType.UninitializedOrDestroyedPool);
 
		// Freeing a null pointer is a no-op, not an error
		if (ptr != null)
		{
			// Pointer must be in the pool and on a block boundary
			Assert(
				ptr >= pool->Alloc
				&& ptr < pool->Alloc + pool->BlockSize * pool->NumBlocks
				&& (((uint)((byte*)ptr-pool->Alloc)) % pool->BlockSize) == 0,
				UnmanagedMemoryExceptionType.PointerDoesNotPointToBlockInPool
			);
 
			// Make sure the sentinel is still intact for this block and the one before it
			#if UNITY_ASSERTIONS || UNMANAGED_MEMORY_DEBUG
				if (*((ulong*)(((byte*)ptr)+pool->BlockSize-sizeof(ulong))) != SentinelValue)
				{
					Assert(
						false,
						UnmanagedMemoryExceptionType.SentinelOverwritten,
						ptr
					);
				}
				if (ptr != pool->Alloc && *((ulong*)(((byte*)ptr)-sizeof(ulong))) != SentinelValue)
				{
					Assert(
						false,
						UnmanagedMemoryExceptionType.SentinelOverwritten,
						(((byte*)ptr)-sizeof(ulong))
					);
				}
			#endif
 
			// Insert the block to free at the start of the free list
			void** pHead = (void**)ptr;
			*pHead = pool->Free;
			pool->Free = pHead;
		}
	}
 
	/// <summary>
	/// Free all the blocks of a pool. This does not free the pool itself, but rather makes all of
	/// its blocks available for allocation again.
	/// </summary>
	/// <param name="pool">Pool whose blocks should be freed</param>
	public static void FreeAll(UnmanagedMemoryPool* pool)
	{
		Assert(pool != null, UnmanagedMemoryExceptionType.NullPool);
		Assert(pool->Alloc != null, UnmanagedMemoryExceptionType.UninitializedOrDestroyedPool);
 
		// Point each block except the last one to the next block. Check their sentinels while we're
		// at it.
		void** pCur = (void**)pool->Alloc;
		byte* pNext = pool->Alloc + pool->BlockSize;
		#if UNITY_ASSERTIONS || UNMANAGED_MEMORY_DEBUG
			byte* pSentinel = pool->Alloc + pool->BlockSize - sizeof(ulong);
		#endif
		for (int i = 0, count = pool->NumBlocks-1; i < count; ++i)
		{
			#if UNITY_ASSERTIONS || UNMANAGED_MEMORY_DEBUG
				if (*((ulong*)pSentinel) != SentinelValue)
				{
					Assert(false, UnmanagedMemoryExceptionType.SentinelOverwritten, pCur);
				}
				pSentinel += pool->BlockSize;
			#endif
			*pCur = pNext;
			pCur = (void**)pNext;
			pNext += pool->BlockSize;
		}
 
		// Check the last block's sentinel.
		#if UNITY_ASSERTIONS || UNMANAGED_MEMORY_DEBUG
			if (*((ulong*)pSentinel) != SentinelValue)
			{
				Assert(false, UnmanagedMemoryExceptionType.SentinelOverwritten, pCur);
			}
		#endif
 
		// Point the last block to null
		*pCur = default(void*);
 
		// The first block is now the head of the free list
		pool->Free = pool->Alloc;
	}
 
	/// <summary>
	/// Free a pool and all of its blocks. Double-freeing a pool is a no-op.
	/// </summary>
	/// <param name="pool">Pool to free</param>
	public static void FreePool(UnmanagedMemoryPool* pool)
	{
		Assert(pool != null, UnmanagedMemoryExceptionType.NullPool);
 
		// Free the unmanaged memory for all the blocks and set to null to allow double-Destroy()
		Free((IntPtr)pool->Alloc);
		pool->Alloc = null;
		pool->Free = null;
	}
}

Please feel free to let me know what you think about memory pools in the comments!

#1 by benjamin guihaire on March 6th, 2017 · Reply

Looks like you are on a nice quest with memory and C# !

there is few other things that can be done to try to accelerate memory allocations, such as forcing allocation on the stack in an unsafe context:

public static unsafe void Test() 
{
	int* ptr = stackalloc int[100];
}

and, when declaring a struct, you can use the fixed keyword on constant size length array to make the allocation part of the struct itself (so it match the memory layout would would get in c code):

struct MyStruct
{
 public int i; 
 public fixed int j[512];
}

#2 by jackson on March 6th, 2017 · Reply

These are both great examples of the kind of flexibility you get with unsafe code. These techniques can easily lead to performance wins, too.

#3 by ms on March 6th, 2017 · Reply

amazing!

#4 by trungdc on March 13th, 2017 · Reply

keep awesome!

#5 by Gabriel Mota on May 7th, 2019 · Reply

Thank you, good ideia and code.

#6 by Groo on July 3rd, 2019 · Reply

Hi, nice article!

Do you think you could share this on Github too? Adding a couple of Span and Memory overloads and some synchronization would allow this code to be used in a bunch of places.

Two ideas:

1. Pool methods are not thread safe, so that’s one reason why a more general implementation would be slightly slower, but a cheap spinlock would be enough.

2. Casting to long* in Memset would shorten the loop 8x (to ensure alignment, block size + sentinel might be made the next multiple of 8 inside AllocPool).

If you put it on Github, I would happily contribute some lightweight synchronization and add Span/Memory overloads.

In any case, thanks a lot and best regards!

Vedran

#7 by jackson on July 4th, 2019 · Reply

I’m glad you enjoyed the article. :)

I don’t have any plans to create a GitHub project out of this article, but you’re free to create one if you’d like. All code on this site is under the MIT license, so feel free to use it for commercial or open-source purposes such as the modifications you mention here.

#8 by snops on July 17th, 2019 · Reply

Hi,

I have also been looking at implementing this code for a while; and finally did a test today with an implementation.

However, I wonder how practical it is to use. It’s been a really long time since I developed with C++. Does everything have to become pointers in the end (as that’s what the Alloc returns?) Or can I dereference it (eg *Vector3) and then keep using it as a normal variable and store it in places that should remain intact (eg in a .vertices of a Mesh object, or a Vector3[] of some class object)?

Also, would be interesting to allocate a large array in one go, instead having to allocate using a loop.

Still need to experiment a bit more!

#9 by snops on July 17th, 2019 · Reply

Ah, got the array working. Just had to get the **’s a bit back in memory. So this works. However, the normal Alloc with the size parameter just does a normal AllocHGlobal. As mentioned above, using the pool is much faster. I gave this a test by changing BlockSize directly to the required size (obviously this is a bad idea this way). The difference is quite significant.

with 10000000 iterations / Unity 2018.4 and IL2CPP:

– Normal new Vector3(): 116ms
– Alloc using AllocHGlobal (as below): 46ms
– Alloc from pool (as below): 15ms

Still, not sure if I would consider changing my code too much locations as the difference is not that significant in an absolute time (not looping 10000000 items that often, even in my procedural terrain generation) – But just on a few spots might help a tiny bit , next to all the parallel code :-).

// Works!: Vector3* data = (Vector3*)UnmanagedMemory.Alloc(count * blockSize); // Test for fake blocksize. This can only end poorly eventually. // Also works perfect - but dangerous pool.BlockSize = blockSize * count; Vector3* data = (Vector3*)UnmanagedMemory.Alloc(&pool); for (int i = 0; i < count; i++) { data[i].x = 10; data[i].y = i * 2; data[i].z = i + 1; }

#10 by sw on September 12th, 2020 · Reply

Hello, thank you for this article. It is very helpful and instructive. I am quite new to pointers in C# and this way I can learn a lot, also I am already implementing similar pool in my project.

Originally, I thought I might have found a bug in your code because there is no additional space in TestStruct reserved for the pointer to next item in the pool. Therefore the values stored in TestStruct{int X;int Y;} overwrite the original value in memory where is the address of the next block in the pool. The value X shares its memory space with the first 4 bytes of the pointer address. It took me a while to realize it doesn’t matter because the address is not needed until the Free() method is called and in this method the pool rewrites the value back to the address.

It is a nice implementation detail which saves some memory space and it can be understood nicely when looking at the pictures where the “Next Ptr” and “Rest of block data” boxes change to single “All block data” block. Which I missed during first couple of readings… :-)

I hope this helps to some other visitors who decide to implement this pool as well.

Faster Memory Allocation with Memory Pools

Comments