# Experiment: The fastest way to iterate over an array

``````For Each tmp As MyClass In someArray
// Do work.
Next tmp
``````

Whilst busy porting some Java code to Xojo, I needed to squeeze as much performance as I could out of my code. The code contained lots of tight loops with multiple iterations and I identified them as one of the bottlenecks in the code.

I had been using `While...Wend` loops with a pre-computed upper limit, thinking that would be faster than one of the many other ways that Xojo lets you iterate over an array but I decided to run an experiment to “definitively” answer the question: “What is the most efficient (in terms of time) method to iterate over an array of custom classes?”. I decided to iterate over an array of objects (rather than Xojo primitives like Integers) as this was the use case I had in the code I was trying to port.

Below is the test code I ran. `Vector` is a simple custom class with two `Double` properties (`X` and `Y`):

``````// Create an array of 100,000 Vector instances to work with.
Var vectors() As Vector
For i As Integer = 1 To 100000
Next i

// We'll now try the seven different ways of iterating over
// an array to see which is fastest.
// 1. While...Wend
Var i As Integer = 0
Var start1 As Double = System.Microseconds
While i < 100000
vectors(i).X = vectors(i).X + 1
vectors(i).Y = vectors(i).Y + 1
i = i + 1
Wend
Var end1 As Double = System.Microseconds - start1

// 2. For...Loop (with hardcoded limit)
Var start2 As Double = System.Microseconds
For j As Integer = 0 to 99999
vectors(j).X = vectors(j).X + 1
vectors(j).Y = vectors(j).Y + 1
Next j
Var end2 As Double = System.Microseconds - start2

// 3. For...Loop (query the array for the limit)
Var start3 As Double = System.Microseconds
For k As Integer = 0 To vectors.LastRowIndex
vectors(k).X = vectors(k).X + 1
vectors(k).Y = vectors(k).Y + 1
Next k
Var end3 As Double = System.Microseconds - start3

// 4. For...Loop (known limit, predefined counter)
Var a As Integer
Var start4 As Double = System.Microseconds
For a = 0 To 99999
vectors(a).X = vectors(a).X + 1
vectors(a).Y = vectors(a).Y + 1
Next a
Var end4 As Double = System.Microseconds - start4

// 5. For...Loop (query array for limit, predefined counter)
Var b As Integer
Var start5 As Double = System.Microseconds
For b = 0 To vectors.LastRowIndex
vectors(b).X = vectors(b).X + 1
vectors(b).Y = vectors(b).Y + 1
Next b
Var end5 As Double = System.Microseconds - start4

// 6. For...Each...Loop
Var start6 As Double = System.Microseconds
For Each v As Vector In vectors
v.X = v.X + 1
v.Y = v.Y + 1
Next v
Var end6 As Double = System.Microseconds - start6

// 7. For...Each...Loop (predefined temp variable)
Var tmp As Vector
Var start7 As Double = System.Microseconds
For Each tmp In vectors
tmp.X = tmp.X + 1
tmp.Y = tmp.Y + 1
Next tmp
Var end7 As Double = System.Microseconds - start7

Break
``````

I found the results really interesting. Below are the results with the fastest method being listed first:

Method Time (microseconds)
(7) For…Each (predefined tmp variable) 13,347
(6) For…Each 13,865
(2) For…Loop (hardcoded limit) 17,541
(4) For…Loop (known limit, predefined counter) 17,573
(3) For…Loop (using `LastRowIndex`) 18,248
(1) While…Wend 20,897
(5) For…Loop (using `LastRowIndex` and a predefined counter) 35,346

As you can see, the fastest way to iterate over a single dimension array of custom classes appears to be using a `For...Each...Next` loop. Whilst technically using a predefined counter (i.e: a variable declared outside the loop) rather than allowing the Xojo framework to create a local temporary variable to house the current element is faster, I would recommend people use method 6.

Compiled App or in the debugger?

As Main thread yields on loop boundaries, what about using pragmas to disable background processing as that can skew testing depending on what else may be going on

-Karen

Good thinking Karen and thanks for joining.

So I’ve re-run the tests (this time doing each method three times and averaging the time taken) and I’ve run the tests with and without the following pragmas:

``````#Pragma BackgroundTasks False
#Pragma BoundsChecking False
#Pragma NilObjectChecking False
#Pragma StackOverflowChecking False
``````

I’ve also done the tests both in the debugger and for compiled (macOS) apps. Essentially, the results are unchanged. Here’s the results for the compiled apps (since they’re more relevant than apps being debugged).

### No pragmas

Method Time (microseconds)
(6) For…Each 3926
(7) For…Each (predefined tmp variable) 3823
(2) For…Loop (hardcoded limit) 7193
(4) For…Loop (known limit, predefined counter) 7283
(3) For…Loop (using `LastRowIndex`) 7534
(1) While…Wend 7717
(5) For…Loop (using `LastRowIndex` and a predefined counter) 33032

### With pragmas enabled

Method Time (microseconds)
(6) For…Each 1853
(7) For…Each (predefined tmp variable) 1976
(2) For…Loop (hardcoded limit) 4925
(4) For…Loop (known limit, predefined counter) 4912
(3) For…Loop (using `LastRowIndex`) 5114
(1) While…Wend 5175
(5) For…Loop (using `LastRowIndex` and a predefined counter) 27502

So in summary - method 6 is still the best.

I do wonder if the mechanism used to access & set the values in the array elements doesnt have a big influence - perhaps skewing the overall times
in cases where you have to do vectors(index) multiple times this will add overhead that might influence the result where the for each access has already cached that one reference thereby avoiding the overhead

by removing those accesses in all cases and making the code

``````#Pragma BackgroundTasks False
#Pragma BoundsChecking False
#Pragma NilObjectChecking False
#Pragma StackOverflowChecking False

// Create an array of 100,000 Vector instances to work with.
Var vectors() As Vector
For i As Integer = 1 To 100000
Next

// We'll now try the seven different ways of iterating over
// an array to see which is fastest.
// 1. While...Wend
Var i As Integer = 0
Var start1 As Double = System.Microseconds
While i < 100000
Dim local As Vector = vectors(i)
i = i + 1
Wend
Var end1 As Double = System.Microseconds - start1
textarea1.AddText "test 1 : " + Str(end1, "######00.00") + EndOfLine

// 2. For...Loop (with hardcoded limit)
Var start2 As Double = System.Microseconds
For j As Integer = 0 To 99999
Dim local As Vector = vectors(j)
Next
Var end2 As Double = System.Microseconds - start2
textarea1.AddText "test 2 : " + Str(End2, "######00.00") + EndOfLine

// 3. For...Loop (query the array for the limit)
Var start3 As Double = System.Microseconds
For k As Integer = 0 To vectors.LastRowIndex
Dim local As Vector = vectors(k)
Next
Var end3 As Double = System.Microseconds - start3
textarea1.AddText "test 3 : " + Str(end3, "######00.00") + EndOfLine

// 4. For...Loop (known limit, predefined counter)
Var a As Integer
Var start4 As Double = System.Microseconds
For a = 0 To 99999
Dim local As Vector = vectors(a)
Next
Var end4 As Double = System.Microseconds - start4
textarea1.AddText "test 4 : " + Str(end4, "######00.00") + EndOfLine

// 5. For...Loop (query array for limit, predefined counter)
Var b As Integer
Var start5 As Double = System.Microseconds
For b = 0 To vectors.LastRowIndex
Dim local As Vector = vectors(b)
Next
Var end5 As Double = System.Microseconds - start4
textarea1.AddText "test 5 : " + Str(end5, "######00.00") + EndOfLine

// 6. For...Each...Loop
Var start6 As Double = System.Microseconds
For Each local As Vector In vectors
Next
Var end6 As Double = System.Microseconds - start6
textarea1.AddText "test 6 : " + Str(end6, "######00.00") + EndOfLine

// 7. For...Each...Loop (predefined temp variable)
Var tmpLocal As Vector
Var start7 As Double = System.Microseconds
For Each tmpLocal In vectors
Next
Var end7 As Double = System.Microseconds - start7
textarea1.AddText "test 7 : " + Str(end7, "######00.00") + EndOfLine
Break
``````

you may find you get different results

Ah theres a bug in this test with a few tweaks to avoid bugs, leaking variables from one test to another, and making it so the test acess each array element as frw tiems as possible this is what I get

``````#Pragma BackgroundTasks False
#Pragma BoundsChecking False
#Pragma NilObjectChecking False
#Pragma StackOverflowChecking False

// Create an array of 100,000 Vector instances to work with.
Var vectors() As Vector
For i As Integer = 1 To 100000
Next

// We'll now try the seven different ways of iterating over
// an array to see which is fastest.
// 1. While...Wend
If True then
Var i As Integer = 0
Var startTime As Double = System.Microseconds
While i < 100000
Dim v As Vector = vectors(i)
v.X = v.X + 1
v.Y = v.Y + 1
i = i + 1
Wend
Var endTime As Double = System.Microseconds - startTime
textarea1.AddText "test 1 : " + Str(endTime, "######00.00") + EndOfLine
End If

If True Then
// 2. For...Loop (with hardcoded limit)
Var startTime As Double = System.Microseconds
For j As Integer = 0 To 99999
Dim v As Vector = vectors(j)
v.X = v.X + 1
v.Y = v.Y + 1
Next j
Var endTime As Double = System.Microseconds - startTime
textarea1.AddText "test 2 : " + Str(endTime, "######00.00") + EndOfLine
End If

If True then
// 3. For...Loop (query the array for the limit)
Var startTime As Double = System.Microseconds
For k As Integer = 0 To vectors.LastRowIndex
Dim v As Vector = vectors(k)
v.X = v.X + 1
v.Y = v.Y + 1
Next k
Var endTime As Double = System.Microseconds - startTime
textarea1.AddText "test 3 : " + Str(endTime, "######00.00") + EndOfLine
End If

If True then

// 4. For...Loop (known limit, predefined counter)
Var a As Integer
Var startTime As Double = System.Microseconds
For a = 0 To 99999
Dim v As Vector = vectors(a)
v.X = v.X + 1
v.Y = v.Y + 1
Next a
Var endTime As Double = System.Microseconds - startTime
textarea1.AddText "test 4 : " + Str(endTime, "######00.00") + EndOfLine

End If

If True then
// 5. For...Loop (query array for limit, predefined counter)
Var b As Integer
Var startTime As Double = System.Microseconds
For b = 0 To vectors.LastRowIndex
Dim v As Vector = vectors(b)
v.X = v.X + 1
v.Y = v.Y + 1

Next b
Var endTime As Double = System.Microseconds - startTime
textarea1.AddText "test 5 : " + Str(endTime, "######00.00") + EndOfLine
End If

If True then

// 6. For...Each...Loop
Var startTime As Double = System.Microseconds
For Each v As Vector In vectors
v.X = v.X + 1
v.Y = v.Y + 1
Next v
Var endTime As Double = System.Microseconds - startTime
textarea1.AddText "test 6 : " + Str(endTime, "######00.00") + EndOfLine
End If

If True then

// 7. For...Each...Loop (predefined temp variable)
Var tmp As Vector
Var startTime As Double = System.Microseconds
For Each tmp In vectors
tmp.X = tmp.X + 1
tmp.Y = tmp.Y + 1
Next tmp
Var endTime As Double = System.Microseconds - startTime
textarea1.AddText "test 7 : " + Str(endTime, "######00.00") + EndOfLine
End If

Break
``````

Compiled with moderate optimization times are very close across 3 runs
test 1 : 2286.89 2119.53 2526.26 avg 2310.89
test 2 : 2190.26 1997.53 2244.74 avg 2144.18
test 3 : 2415.89 2211.95 2401.39 avg 2343.08
test 4 : 2360.11 2240.63 2144.33 avg 2248.36
test 5 : 2385.53 2331.21 2238.51 avg 2318.42
test 6 : 2108.28 2007.67 1945.47 avg 2020.47
test 7 : 2034.90 2048.68 1933.17 avg 2005.58

Doh! As always, excellent investigative work Norm. I had erroneously thought that declaring a local variable in each iteration added an unnecessary burden to the code but it obviously helps. I suppose that’s what the compiler is already do for you being the scene when you use a `For...Each...Next` loop anyways.

Fortunately method 6 is still the fastest .

Ummm … mechanism 7 averages out slightly faster ?
But just barely. just under 15 microseconds across 3 runs

A great piece of information. Enjoying this as a useful reference.
Thanks for sharing!

That seemed within the margin of error. Besides, omitting that fact means my first post remains correct 1 Like

Welcome to the forum @anic297!

Thank you; glad to be here!

of course being right is VASTLY more important 1 Like