Freesurfer CMake Port

This page is mostly for documenting the process of converting the automake framework to cmake. General documentation for using cmake with freesurfer can be found here here.

Overview of the framework

The top-level CMakeLists.txt file is the main cmake configuration script (which replaces setup_configure, configure.in, and Makefile.am), and all subdirectories are added from here with the add_subdirectory() function. This main script is split into three parts: locating third-party packages, configuring compilation settings, and configuring freesurfer libs and programs.

third-party packages

Most of the packages required by freesurfer are located via custom "find-modules" stored in the cmake subdirectory. These find-modules expect each package to be installed under a common path defined by FS_PACKAGES_DIR. On Martinos machines, this variable automatically defaults to /usr/pubsw/packages, but external developers must provide this path manually:

cmake . -DFS_PACKAGES_DIR="/path/to/packages"

If a package is not found under FS_PACKAGES_DIR, cmake will continue to look through the default search paths. Alternative paths to package installs can also be specified with the <PACKAGE>_DIR variables. For example, to use a non-default ITK version:

cmake . -DITK_DIR="/usr/pubsw/packages/itk/3.16.0"

find modules

In CMakeLists.txt, packages are located by using the find_package() function. Some common, modern projects, like Qt, VTK, ITK, Boost, etc..., distribute cmake config files with, so locating the package's include directory and libraries is a fairly straightforward, automatic process:

find_package(ITK HINTS ${ITK_DIR} REQUIRED)

In this example, if ITK is found by cmake, then ITK_FOUND is set to true, and ITK_INCLUDE_DIR and ITK_LIBRARIES are set accordingly. This syntax and the variables generated by find_package() all follow the same general pattern across packages.

Unfortunately, most freesurfer dependencies don't ship with cmake configuration files, so we have to create our own find-modules... fortunately, this isn't too difficult. Most find-modules look something like this, where we're only searching for an include directory and one or two libraries.

external developers

In general, the goal is to distance ourselves from distributing the pre-built package tarballs since they are difficult to maintain across multiple platforms. The packages/build_packages.py script is a potential alternative to the pre-built archives - it's a utility script to help external developers build freesurfer dependencies on their own.

The packages configured within build_packages.py are built using the tarballs and buildscripts stored in the packages/source dir and will get installed to a destination directory specified on the command-line:

build_packages.py "/path/to/install/destination"

This script loops through each package in the pkgs list variable, extracts the associated tarball to destination/package-name/version/src, and runs the package build script. The individual build scripts (like this one always expect a single commandline argument that points to the desired install directory. After each successful package build, the checksum of the tarball is saved to the src dir - this way, the package is only rebuilt when the source code has been modified. Once the dependencies are compiled and installed, developers can then point FS_PACKAGES_DIR to their local install directory.

For now, we should definitely still offer the prebuilt packages since it's just easier for most developers and we need them for the travis builds anyway.

removing packages from source

The non-FS libraries (like jpeg, glut, xml2, expat, minc, netcdf, etc...) that used to be built within freesurfer no longer get built by cmake - these packages are now expected in FS_PACKAGES_DIR (/usr/packages/pubsw). THis has a few advantages: we can easily test/swap out different versions of packages, and we can test out different compilers/settings without having to worry about modifying third-party source code and compiling packages differently than the way their developers intended to. Doing this, we can build successfully with gcc5 (at least on mac), and hopefully this helps with Bevin's current push to compile all c-code with g++ (since the external packages are a big roadblock).

Validation

On my machine (topaz), the cmake configuration only takes a few seconds, and the multithreaded buildtime is about half:

recons

I built the same version of dev with automake and cmake and ran the buckner40 recons (on sulc) with both distributions. Both tests produced the exact same results for each subject, and the average cmake-distro runtime was actually about 50 minutes faster (still working on figuring out way - could just be a coincidence since there was a lot of background processing happening on sulc).

libraries

I wrote a quick script to traverse through each binary in the automake and cmake installs and compare differences between the linked libraries. All binaries are the same with the exception of a few:

• the fem_elastic and utils binaries previously linked to some unnecessary libraries - those are gone
• libuuid is not needed by all the binaries
• the freeview linked libraries are pretty different. Part of this is probably due to updating to Qt5, but I didn't really want to dig into each missing/added library, especially since most of them are probably configured automatically by the Qt cmake module (which I generally trust to be accurate). I think it might just be easier to run some robust tests on freeview to make sure everything's working correctly. We should probably do this anyway since we've upgraded to Qt5

TODO

• everything builds (and seems to run) successfully on OS X, but this hasn't been fully tested. I'll need to run/compare the buckner40 and check the linked libraries

• implement something like a SMALL_DIST component that would mimic the smalldist build. cmake has a nice feature that let's you tag calls to install() with a particular "component" name. All files get installed under a standard make install, but when you configure with a particular component enabled (i.e. SMALL_DIST), only the files/executables tagged accordingly get copied

Questions

• It looks like the large-file-support compiler flags -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE are only necessary on 32 bit machines - is this correct? In configure.in, LFS flags are applied to all osx builds, but I imagine this is something that can be removed?

• In configure.in, osx flags are set to include -msse2 -mfpmath=sse, but the USE_SSE_MATHFUN conditional is never turned on. Is this a bug or is this intentional?

• It seems that autotools requires the AUTHORS, NOTICE, README, VERSION, etc. files for an install. Should we just remove these from the distribution now that they're not needed. Maybe we should just distribute one general readme file?

• Is there any reason why we distribute lib/petsc? These are static libraries, so we don't link to them, and it doesn't seem like there's a licensing requirement