== THIS PAGE HAS BEEN MOVED TO SHAREPOINT! ==
Please refer to this site/make edits here for the most updated information: https://partnershealthcare.sharepoint.com/sites/LCN/SitePages/Morpho-Optimization-Project.aspx

----
<<BR>>
<<BR>>
<<BR>>


Parent: [[MorphoOptimizationProject]]

When replacing one algorithm with another, it is very easy to introduce tiny differences that cause later code to produce visibly different results.

Rather than just comparing the final results, which does not help with debugging, it is best to compare the intermediate results either immediately after the algorithm or, ideally, during the execution of the algorithm - this will greatly assist debugging any differences.

To this end, the following outline of a change seems to work well

=== Add the new code with very few changes to the old ===
eg.

   void algorithm(''args'') { ... }

becomes

   void algorithm_old(''args'') {...}

   void algorithm_new(''args'') {...}

   void algorithm(''args'') {
       bool do_old = !!getenv("FREESURFER_algorithm_old");
       bool do_new = !!getenv("FREESURFER_algorithm_new") || !do_old;
       if (do_old) algorithm_old(''args'');
       if (do_new) algorithm_new(''args'');
   }

=== Add the comparison ===
Comparing the results may vary from (a) the simple comparison of their return value, or (b) comparing many fields that they wrote.  In the later case, sometimes I have resorted to this

   void algorithm_old(''args'') {...}

   void algorithm_new(''args'', bool old_done) {
        ...
        if (old_done) check(p->m == value); else p->m = value;  // if tedious, use a macro...
        ...
   }

   void algorithm(''args'') {
       bool do_old = !!getenv("FREESURFER_algorithm_old");
       bool do_new = !!getenv("FREESURFER_algorithm_new") || !do_old;
       if (do_old) algorithm_old(''args'');
       if (do_new) algorithm_new(''args'', do_old);
   }

However there are worse situations, such as when the algorithm looks like this

   void algorithm_new(''args'', bool old_done) {
        ...
        loop
            call a function that makes changes somewhere inside an MRIS
        ...
   }

By using the mris_hash function, it is possible to compare MRIS, but it may take too long.
To reduce the number of tests, I will do something like

   void algorithm_''new or old''(''args'', bool old_done) {
        static long count, limit = 1;
        ...
        loop
            count++;
            call a function that makes changes somewhere inside an MRIS
            if (count >= limit) {
                limit *= 2;
                output the mris_hash
            }
        ...
   }

Now it is possible to compare the outputs and, by changing the sequence of limit values, narrow in on the iteration causing the difference.

=== When the algorithm changes its own inputs ===
If the algorithm modifies its own inputs, it may not be possible to run both and compare as above.

It may be necessary to either (a) save and restore the inputs around the first call, or (b) use two different runs and compare the outputs as described above.