Differences between revisions 2 and 19 (spanning 17 versions)
⇤ ← Revision 2 as of 2017-04-13 14:12:31 →
Size: 2544
Editor: ZekeKaufman
Comment:
← Revision 19 as of 2019-02-03 12:18:17 → ⇥
Size: 0
Editor: AndrewHoopes
Comment: moved to GitAnnex
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
#acl LcnGroup:read,write,delete,revert

<<TableOfContents>>

This page describes how to deal with adding and tagging data files in the freesurfer source code repository.

== Initial Setup ==

Based in the information included in the [[Freesurfer_github|Freesurfer_github]] page, the remotes of your freesurfer repo working directory should look something like:

 {{{
git remote -v
 
  datasrc file:///space/freesurfer/repo/annex.git (fetch)
  datasrc file:///space/freesurfer/repo/annex.git (push)
  origin git@github.com:zkaufman/freesurfer.git (fetch)
  origin git@github.com:zkaufman/freesurfer.git (push)
  upstream git@github.com:freesurfer/freesurfer.git (fetch)
  upstream git@github.com:freesurfer/freesurfer.git (push)
 }}}

== Adding a data file ==

The following example assumes we want to add a data file called 'testdata.tar.gz' to the 'distribution' directory:

{{{

}}}

== Getting a data file ==

To retrieve the contents of a data file:

{{{
## Retrieve a specific file
git annex get mri_em_register/testdata.tar.gz

## Retrieve everything under the current directory (not recommended)
git annex get .
}}}

To retrieve everything (not recommended):

{{{

== Tagging a data file ==

The data files have been broken down into the following 3 categories, and it is essential that data files get the proper tag(s) so that our servers and diskspace is not overwhelmed when only a known subset of the data is required.:

 1. Those being required for build time checks (tagged '''makecheck''')
 1. Those required for a local installation (tagged '''makeinstall''')
 1. Everything else (untagged)

=== Display metadata ===

To show all the metadata associated with a file:

{{{
git annex metadata mri_em_register/testdata.tar.gz
}}}

=== Assign metadata ===

To assign a tag to an existing datafile.

{{{
git annex metadata mri_em_register/testdata.tar.gz -s fstags=makecheck
git annex sync
}}}

We can also append tags:
{{{
git annex metadata mri_em_register/testdata.tar.gz -s fstags+=makeinstall
}}}

To list all files with a given tag:
{{{
git annex find --metadata fstags=makecheck
}}}

To download all datafiles with a given tag:

{{{
## Get only the data files required for build time checks (1.9 GB)
git annex get --metadata fstags=makecheck .

## Get only the data files required for local installation (4.3 GB)
git annex get --metadata fstags=makeinstall .

## Just give me everything! Not Recommended (6.8 GB)
git annex get .
}}}