There are many reasons why source code compiled for and running on different platforms might not get the same result, even when single threaded.
- Floating point - compiler might chose a different binary pattern for a decimal number (unlikely), order of evaluation (likely), precision (using double for some intermediate results), implementation of functions such as asin or sqrt.
- Sorting algorithms - most sorting algorithms are not stable - items with equal keys may be sorted into either order.
- Random number generators - these are obviously unstable.
Platform here refers to a combination of hardware, compiler version, compiler switches, libraries, and operating systems.
For our purposes we are mostly concerned with only changing the number of threads, which is discussed elsewhere [where?].
However we have seen differences between Linux and OSX caused by these kinds of issues, and we also see them when trying to replace one algorithm with a supposedly equivalent one.
Floating point operations
Most h/w these days uses IEEE floating, so that is not usually the problem.
Most numbers have an closest binary representation, and compilers are good at using it.
Order of evaluation is a big problem. This can change depending on compiler switches. Summing large numbers of floating point and switching between multiply-by-inverse and divide are both suspect.
Floating point functions
Consider picking a maximum of a list by comparing either (a) list[i] and list[j], and (b) their sqrt's. Unfortunately sqrt maps unequal numbers onto equal numbers, so these may not pick the same element. Furthermore sqrt may not get exactly the same value on two different platforms, so values that are equal on one may differ on another.
This is also true of all the trig functions, so when the trig functions are used to control movement or distance the results may vary.
Sometimes sorting algorithms are used to merge work done by several threads into a consistent result. In this, and other, cases when the keys might be equal, and the order of resulting list can affect the result, some deterministic value needs to be used to impose an order on the equal keys.
Random number generators
We are using several and they are all non-portable.