• MorphoOptimizationProject_ComparingOldAndNew

THIS PAGE HAS BEEN MOVED TO SHAREPOINT!

Please refer to this site/make edits here for the most updated information: https://partnershealthcare.sharepoint.com/sites/LCN/SitePages/Morpho-Optimization-Project.aspx

Parent: MorphoOptimizationProject

When replacing one algorithm with another, it is very easy to introduce tiny differences that cause later code to produce visibly different results.

Rather than just comparing the final results, which does not help with debugging, it is best to compare the intermediate results either immediately after the algorithm or, ideally, during the execution of the algorithm - this will greatly assist debugging any differences.

To this end, the following outline of a change seems to work well

Add the new code with very few changes to the old

eg.

• void algorithm(args) { ... }

becomes

• void algorithm_old(args) {...}

  void algorithm_new(args) {...}

  void algorithm(args) {

  • bool do_old = !!getenv("FREESURFER_algorithm_old");

    bool do_new = !!getenv("FREESURFER_algorithm_new") || !do_old; if (do_old) algorithm_old(args); if (do_new) algorithm_new(args);

  }

Add the comparison

Comparing the results may vary from (a) the simple comparison of their return value, or (b) comparing many fields that they wrote. In the later case, sometimes I have resorted to this

• void algorithm_old(args) {...}

  void algorithm_new(args, bool old_done) {

  • ..

    if (old_done) check(p->m == value); else p->m = value; // if tedious, use a macro...

  • ..
  }

  void algorithm(args) {

  • bool do_old = !!getenv("FREESURFER_algorithm_old");

    bool do_new = !!getenv("FREESURFER_algorithm_new") || !do_old; if (do_old) algorithm_old(args); if (do_new) algorithm_new(args, do_old);

  }

However there are worse situations, such as when the algorithm looks like this

• void algorithm_new(args, bool old_done) {

  • .. loop
    • call a function that makes changes somewhere inside an MRIS
  • ..
  }

By using the mris_hash function, it is possible to compare MRIS, but it may take too long. To reduce the number of tests, I will do something like

• void algorithm_new or old(args, bool old_done) {

  • static long count, limit = 1;
  • .. loop
    • count++; call a function that makes changes somewhere inside an MRIS

      if (count >= limit) {

      • limit *= 2; output the mris_hash
      }
  • ..
  }

Now it is possible to compare the outputs and, by changing the sequence of limit values, narrow in on the iteration causing the difference.

When the algorithm changes its own inputs

If the algorithm modifies its own inputs, it may not be possible to run both and compare as above.

It may be necessary to either (a) save and restore the inputs around the first call, or (b) use two different runs and compare the outputs as described above.