Repeatable Random Performance
A couple of years ago I posted a class that generats pseudo-random numbers in a repeatable way. This is useful for a variety of tasks, but a recent comment reminded me that I hadn’t tested its performance. Today I’ll pit my repeatable random function against the standard Math.random
function as well as Skyboy’s repeatable random class. Read on for the results!
The test class is straightforward. Essentially, I just request a lot of random numbers from each random number generator:
package { import flash.display.*; import flash.utils.*; import flash.text.*; public class RandomPerformance extends Sprite { private var __logger:TextField = new TextField(); private function row(...cols): void { __logger.appendText(cols.join(",")+"\n"); } public function RandomPerformance() { stage.align = StageAlign.TOP_LEFT; stage.scaleMode = StageScaleMode.NO_SCALE; var logger:TextField = __logger; logger.autoSize = TextFieldAutoSize.LEFT; addChild(logger); var beforeTime:int; var afterTime:int; var REPS:int = 10000000; var i:int; var num:Number; var jacksonRandom:Random = new Random(33); var skyboyRandom:SkyboyRandom = new SkyboyRandom(33); row("Generator", "Time"); beforeTime = getTimer(); for (i = 0; i < REPS; ++i) { num = Math.random(); } afterTime = getTimer(); row("Math.random", (afterTime-beforeTime)); beforeTime = getTimer(); for (i = 0; i < REPS; ++i) { num = skyboyRandom.extractNumber(); } afterTime = getTimer(); row("Skyboy Random", (afterTime-beforeTime)); beforeTime = getTimer(); for (i = 0; i < REPS; ++i) { num = jacksonRandom.nextNumber(); } afterTime = getTimer(); row("Jackson Random", (afterTime-beforeTime)); } } }
I ran this test in the following environment:
- Flex SDK (MXMLC) 4.5.1.21328, compiling in release mode (no debugging or verbose stack traces)
- Release version of Flash Player 11.1.102.55
- 2.4 Ghz Intel Core i5
- Mac OS X 10.7.2
Here are the results I got:
Generator | Time |
---|---|
Math.random | 278 |
Skyboy Random | 1606 |
Jackson Random | 244 |
Skyboy’s version uses a more sophisticated random technique and as such requires more time to generate its random numbers. In this case the penalty is about a 5x performance loss compared to the other algorithms. Speaking of which, Math.random
performs a little slower than my repeatable random class, perhaps due to the static function call. Regardless, if you’re doing a heavy amount of random number generator (e.g. to generate noise), you should be able to get a small speedup by switching from Math.random
to my repeatable random number generator. As a bonus, your random numbers will be repeatable!
Spot a bug? Have a suggestion? Post a comment!
#1 by guest on February 6th, 2012 ·
Will it help somehow if you save reference to a static function?
beforeTime = getTimer();
const nativeRandom:Function = Math.random;
for (i = 0; i < REPS; ++i)
{
num = nativeRandom();
}
#2 by jackson on February 6th, 2012 ·
Oh how I wish that would work! Unfortunately, because you’d now be making a dynamic function call, the time for
Math.random
viaFunction
object goes to about 4000ms on the same machine.#3 by UnknownGuardian on February 6th, 2012 ·
If you check out skyboy’s code, it calls generateNumbers() on the first pass. Could you call extractNumber() once before throwing it in the loop so you don’t hit the performance take on that on the first iteration and then compare speed?
#4 by jackson on February 6th, 2012 ·
That’s a good idea. I tried it out though and it didn’t change the results for Skyboy’s random number generator. The initial generation is probably so small compared to the 10 million random numbers being generated that the difference is lost. If, however, you were just generating a few random numbers then this might be a big deal. However, any of the random number generators will give sufficient performance (well under 1ms) at just a few iterations.
#5 by skyboy on February 6th, 2012 ·
My implementation is only moderately optimized; my local copy runs substantially better just by turning the static retrieval of int/Number values to local static retrievals. It’s more or less a direct port of what you’d see in C++.
The generateNumbers function is also called every 624 iterations, rather than just on the first, and is the source of most of the overhead. For a while now I’ve been thinking about how to inline that method to execute on every call instead to more evenly distribute the work load (and halve the total number of loop iterations for any external loop over 623).
That Mersenne twister implementation is also guaranteed to not repeat its own sequence before 219937 iterations (4.3154 * 106001).
#6 by skyboy on February 7th, 2012 ·
I just wrote the inline modification algorithm (and fixed some bugs); I now get these results (assigning via modulo to a Vector of 10k elements; http://pastebin.com/2b39m8Cc):
The
ex
test would be the inline uint algorithm.#7 by skyboy on February 7th, 2012 ·
I hate replying to myself so many times. It feels spammy.
After applying the inline method to all of the functions, running the test gets me these results:
#8 by jackson on February 7th, 2012 ·
No worries about the replying.
Thanks for the update on your Mesenne Twister randomizer. It is indeed a high quality random number generator and, with your updates, not that far (~2x) off in performance. For most tasks that don’t involve tons of random number generation, I’m sure the performance would be fine.
#9 by Jota on February 6th, 2012 ·
@guest i don’t think so. If i get this right, your nativeRandom var will be a pointer for the static Math.random method. Correct me if i’m wrong, people.
Cheers!
Jota
#10 by Mark on February 10th, 2012 ·
I mostly use this (seeded) random function, I don’t know if it is that fast.
It is based on this one:
http://www.calypso88.com/?p=524
#11 by Mark on February 10th, 2012 ·
Huh it is missing a line of code (?!), the random function body should be
#12 by Mark on February 10th, 2012 ·
Sorry it is still stripping one line of code.. I don;t know why but it is incomplete. please remove this comment. Check out http://www.calypso88.com/?p=524 for the function body
#13 by skyboy on February 20th, 2012 ·
I just graphed the output of the three RNGs to see how random it is (I was working on testing the performance of a quick particle engine) and I got these as results:
With the particle’s X/Y positions assigned as such:
Math.random: http://megaswf.com/serve/2149330
My PRNG: http://megaswf.com/serve/2149340
Jackson’s PRNG: http://megaswf.com/serve/2149345
Very simple, yet also very revealing. Jackson, the numbers chosen for yours aren’t that great: it results in what appears to be a very sparse, very static image. I think I may have seen a particle jump a few times, but there’s so little movement that I’m not certain it actually happens; It might just be an optical illusion.
An alternative test is to multiply the result by 1,000 or 10,000 then increment the int position of that value in a Vector of the same length. Something like the following:
Again, very simple; and it’s easily turned into a graph to plot the distribution; I suggest the values be normalized to within the range [-1, 1] before graphing, though.
#14 by jackson on February 20th, 2012 ·
Very interesting results you have there. Version, as stated in the original article, was found online somewhere and the Google search still shows it cited by multiple sources. Perhaps there is something incorrect about my implementation though. Any ideas?
#15 by skyboy on February 20th, 2012 ·
It’s the values used; getting proper distribution for a PRNG (particularly in the short term) isn’t easy and usually requires something substantially more advanced (and subsequently slower). There’s also a more rigorous set of tests here.
#16 by skyboy on February 20th, 2012 ·
I just tested again, and it seems like it’s very random in how random it is (somewhat ironic). I decompiled the other SWF and verified the PRNG is correct, but it looks nothing like this: http://megaswf.com/serve/2151370
#17 by skyboy on February 20th, 2012 ·
No, I was wrong: I had made a typo and was using the wrong variable to assign the x position. Correcting the typo brings it back to the very sparse output seen previously.
With 233280.0 there are 130 unique points of data, tweaking that to 233281.0 gives you 1424 unique points of data, 233287.0 gets 10. If you want to really improve it, you’d need to pick out several large (yet under 27 bits) primes from a database such as this: http://www.bigprimes.net/archive/prime/
#18 by jackson on February 20th, 2012 ·
I wonder why there were so many other people using 233280.0 then. How does it look with 233281.0?
#19 by skyboy on February 20th, 2012 ·
233281: http://megaswf.com/serve/2152059
4294967291: http://megaswf.com/serve/2152063
However, because the latter is a huge number and well over 27 bits, you get a minimum of a 2x performance hit.
But it works absolute magic.
#20 by skyboy on February 21st, 2012 ·
After going over a couple hundred (yay arrays) upper-limit prime ints, I found two of great interest.
67110467
, which results in only 3 points after a few thousand iterations at the longest for the best possible seed, or immediately for a bad seed. Not particularly useful, but the lowest number of values I’ve found yet.67111013
, which results in a constantly changing set of points; it’s not as random-looking as the4294967291
mod, but I haven’t seen any repeating patterns in the output. This seems to be the most desirable so far.