Loop Performance: Part 5 (New Compiler, IL2CPP)
Now that Unity has a new compiler that makes foreach
loops not create garbage with List<T>
, it’s time to re-test all the kinds of loops to see if anything’s changed. This article is the first in the series to test on a real Android device using IL2CPP, so these numbers should be much more accurate for most games. Read on for the results!
Since the last test, a few things have changed:
- Upgraded Unity from 5.3 to 5.6
- Switched from Mono to IL2CPP
- Changed OS from macOS to Android
- Made the test reporting itself create less garbage
Here’s the updated test script:
using System; using System.Collections.Generic; using System.Text; using UnityEngine; public static class ArrayExtensions { public static void ForEach<T>(this T[] array, Action<T> callback) { for (int i = 0, len = array.Length; i < len; ++i) { callback(array[i]); } } } public class TestScript : MonoBehaviour { private static int delegateSum; private static Action<Counter> ForEachDelegate = c => delegateSum += c.Value; private class Counter { public int Value; } void Start() { StringBuilder report = new StringBuilder(10 * 1024); report.Append("Size,"); report.Append("Array For,"); report.Append("Array While,"); report.Append("Array Foreach,"); report.Append("Array ForEach (Uncached),"); report.Append("Array ForEach (Cached),"); report.Append("List For,"); report.Append("List While,"); report.Append("List foreach,"); report.Append("List Foreach (Uncached),"); report.Append("List ForEach (Cached)\n"); const int numIterations = 100000; for (int size = 10; size <= 1000; size *= 10) { var stopwatch = new System.Diagnostics.Stopwatch(); var sum = 0; var array = new Counter[size]; for (int i = 0; i < size; ++i) { array[i] = new Counter() { Value = i }; } stopwatch.Reset(); stopwatch.Start(); for (var iteration = 0; iteration < numIterations; ++iteration) { for (int i = 0, len = array.Length; i < len; ++i) { sum += array[i].Value; } } var arrayForTime = stopwatch.ElapsedMilliseconds; stopwatch.Reset(); stopwatch.Start(); for (var iteration = 0; iteration < numIterations; ++iteration) { var i = 0; var len = array.Length; while (i < len) { sum += array[i].Value; ++i; } } var arrayWhileTime = stopwatch.ElapsedMilliseconds; stopwatch.Reset(); stopwatch.Start(); for (var iteration = 0; iteration < numIterations; ++iteration) { foreach (var cur in array) { sum += cur.Value; } } var arrayForeachTime = stopwatch.ElapsedMilliseconds; stopwatch.Reset(); stopwatch.Start(); for (var iteration = 0; iteration < numIterations; ++iteration) { array.ForEach(c => sum += c.Value); } var arrayForEachUncachedTime = stopwatch.ElapsedMilliseconds; stopwatch.Reset(); stopwatch.Start(); for (var iteration = 0; iteration < numIterations; ++iteration) { array.ForEach(ForEachDelegate); } var arrayForEachCachedTime = stopwatch.ElapsedMilliseconds; var list = new List<Counter>(size); list.AddRange(array); stopwatch.Reset(); stopwatch.Start(); for (var iteration = 0; iteration < numIterations; ++iteration) { for (int i = 0, len = list.Count; i < len; ++i) { sum += list[i].Value; } } var listForTime = stopwatch.ElapsedMilliseconds; stopwatch.Reset(); stopwatch.Start(); for (var iteration = 0; iteration < numIterations; ++iteration) { var i = 0; var len = list.Count; while (i < len) { sum += list[i].Value; ++i; } } var listWhileTime = stopwatch.ElapsedMilliseconds; stopwatch.Reset(); stopwatch.Start(); for (var iteration = 0; iteration < numIterations; ++iteration) { foreach (var cur in list) { sum += cur.Value; } } var listForeachTime = stopwatch.ElapsedMilliseconds; stopwatch.Reset(); stopwatch.Start(); for (var iteration = 0; iteration < numIterations; ++iteration) { list.ForEach(c => sum += c.Value); } var listForEachUncachedTime = stopwatch.ElapsedMilliseconds; stopwatch.Reset(); stopwatch.Start(); for (var iteration = 0; iteration < numIterations; ++iteration) { list.ForEach(ForEachDelegate); } var listForEachCachedTime = stopwatch.ElapsedMilliseconds; report.Append(size); report.Append(','); report.Append(arrayForTime); report.Append(','); report.Append(arrayWhileTime); report.Append(','); report.Append(arrayForeachTime); report.Append(','); report.Append(arrayForEachUncachedTime); report.Append(','); report.Append(arrayForEachCachedTime); report.Append(','); report.Append(listForTime); report.Append(','); report.Append(listWhileTime); report.Append(','); report.Append(listForeachTime); report.Append(','); report.Append(listForEachUncachedTime); report.Append(','); report.Append(listForEachCachedTime); report.Append('\n'); } Debug.Log(report.ToString()); } }
If you want to try out the test yourself, simply paste the above code into a TestScript.cs
file in your Unity project’s Assets
directory and attach it to the main camera game object in a new, empty project. Then build in non-development mode, ideally with IL2CPP. I ran it that way on this machine:
- LG Nexus 5X
- Android 7.1.2
- Unity 5.6.0f3, IL2CPP
And here are the results I got:
Size | Array For | Array While | Array Foreach | Array ForEach (Uncached) | Array ForEach (Cached) | List For | List While | List foreach | List Foreach (Uncached) | List ForEach (Cached) |
---|---|---|---|---|---|---|---|---|---|---|
10 | 5 | 4 | 4 | 51 | 25 | 11 | 11 | 28 | 49 | 27 |
100 | 35 | 34 | 34 | 179 | 225 | 96 | 96 | 239 | 202 | 252 |
1000 | 440 | 433 | 407 | 1767 | 2248 | 964 | 960 | 2359 | 1784 | 2667 |
The results are certainly different this time! With arrays, it doesn’t make any difference if you use for
, foreach
, or while
. You’ll get the same speedy results with any of the three. If you use the ForEach
extension method, you’ll suffer a 4-13x slowdown!
List<T>
has different results. Good old for
and while
loops are still unquestionably the fastest, but foreach
can’t keep up. Even though it’s not creating any garbage anymore, it takes about 2.5x longer to loop with foreach
on a List<T>
! Actually, foreach
is on par with the List<T>.ForEach
method when you use a cached delegate.
Compared to the previous article, it’s now easy to recommend for
or while
loops universally. It doesn’t matter if you’re using List<T>
or a plain array, a few elements or a great many, you’ll always get the best performance with for
or while
. If you’re using a plain array, you can opt for foreach
with no penalty, but make sure to not do this with List<T>
. ForEach
functions that take a delegate should never be used when performance is critical.