Home

"HELP IM A BUG!" -- Calvin and Hobbes

This documentation is also on https://gate.nmr.mgh.harvard.edu/wiki/stufflebeamlab/index.php?title=Connectome_visualization_utility and is probably incomplete

The connectome visualization utility is a program in development in the Stufflebeam lab specifically designed for the visualization of brain-based networks. The primary developer is Roan LaPlante. You can contact him at

==Usage==

From within the Martinos center, the easiest way to run cvu is /cluster/neuromind/rlaplant/mayavi/cvu/run

From outside the Martinos center, you will need to download and install the program and its dependencies. cvu is on the Python Packages Index (PyPI) which enables you to use tools such as easy-install. Currently the listing on PyPI ''DOES NOT'' automatically install the required dependencies. For hackers or developers, you can also get the source from [https://github.com/aestrivex github].

The packages needed are MNE python, numpy/scipy, traits, traitsui, matplotlib, mayavi, bctpy and chaco. i think that installing these will always include tvtk, enable, and pylab, but there may be some edge cases. most of these packages should be available from your scientific python distribution of choice.

In order to display connectivity, cvu requires three things: *A Freesurfer parcellation (freesurfer .annot files) *An ordering file (a text file with ROI names) *A matrix (.npy, .mat, or .txt)

All of these things can be loaded on the command line. If you do not specify command line arguments, the program will display some sample data instead. Inside the GUI, you will be able to load all of the same things as well as tinker with many other options.

cvu accepts various command line arguments. Because these may change with development, run with --help to see the full list. As of 13:22, 29 January 2014 (EST) these are the accepted arguments: -p greg.gii --parc=greg: location of annotations *h.greg.annot -a greg.mat --adjmat=greg.mat: location of adjacency matrix -d greg.nii --subjects-dir=greg/: specifies SUBJECTS_DIR -s greg --surf=greg: loads the surface *h.greg -o greg.txt --order=greg.txt: location of parcellation ordering file --adj-order greg.txt: location of adjmat ordering file --surf-type=pial: specifies type of surface. pial is used by default. white is also a reasonable choice. inflated is ok for visualizing one hemisphere at a time. -q: specifies quiet flag -v: specifies verbose flag (currently does nothing) --max-edges 46000: discards all but the strongest ~46000 connections -h --help: display this help

There is only one thing you can do in the command line that cannot also be done in the GUI: specify the verbosity level. This does not affect anything except how much infomation is printed to stdout or stderr.

===Usage Details===

This section documents the design logic of how cvu chooses to represent much of its data. Selecting an appropriate visualization of any type of data is, in principle, a hard problem to solve. Therefore, some of the choices for design and data representation for brain network data are logical and might constitute the simplest possible representations of that data, but may still not necessarily be intuitive to use or understand.

As such, this section is devoted to explaining topics that are not immediately obvious from using the program. Some of which are critical for using cvu correctly and understanding what it is doing!

=====order files===== THIS SECTION IS IMPORTANT!

Ordering files are text files specifying the order in which labels appear in some data files. As the command line options suggest, there are two completely different types of order files -- a ''parcellation'' ordering and an ''matrix'' ordering.

The parcellation ordering file represents the '''desired''' order of ROIs that you wish for cvu to display in the connection matrix and on the circle plot.

The adjacency matrix ordering file represents the order in which the ROIs actually are presented in the connection matrix. This could differ from the parcellation order. For instance, you might have a matrix that is generated by ordering all regions alphabetically. But you may want to display a matrix with regions in a more intelligently selected order.

Parcellation orderings are always required. Matrix orderings are optional. If a matrix ordering is not specified the corresponding parcellation ordering will be used in its place (i.e., it is assumed the matrix is already in the correct order).

Note that loading other ordered data, such as module communities and scalars, works in the same way as matrices and uses an optional matrix ordering file.

In order for an ROI to be enabled in a cvu dataset, the annotation file must have a label matching an entry in the parcellation ordering. Further, each entry in the ordering file must begin with 'lh_' or 'rh_' as these are not present in the annotation names themselves. Any entry present in the parcellation ordering, but not in the parcellation itself, is ignored (and vice versa). A message will be printed to stdout indicating this.

The delete keyword has special significance in ordering files. It means, "ignore this entry." In a parcellation ordering this does not do anything. In a matrix ordering, it means that the row and column at this position in the matrix will not be included in the cvu dataset.

This behavior has the useful feature that if you have an adjmat with various entries that are not part of the parcellation (such as areas of the brain stem) and you wish to get rid of them, you can specify the same ordering file for both parcellation and adjacency matrix and replace the unwanted regions with delete.

In the GUI, there is also an option to ignore delete entries (e.g. treat the ordering file as though all delete entries were simply not there).

If the adjmat ordering, after deleting the "delete" entries, is not the same length as the loaded parcellation ordering, cvu will stop the operation and return an error.

Our lab uses a parcellation scheme called the Lausanne parcellations which were developed at Ecole Polytechnique de Lausanne. These are divisions of the standard freesurfer aparc, with regions of roughly equal size. Several parcellations of this type exist at different resolutions, with the smallest one simply equivalent to freesurfer aparc. Ordering files corresponding to these parcellations are included with the program, along with a parcellation of the fsaverage5 brain. You can morph the labels from fsaverage5 to fit your subjects if you want to extract data for these parcellations. See the freesurfer_morph tutorial for more details.

The _cmp suffix for these parcellations denotes the canonical connectomemapper ordering which begins at roughly frontal pole and wraps around the top of the cortex, proceeding to the parietal and then occipital lobe and ending at the temporal pole. This is a decent choice of ordering display for a matrix. The _alph suffix provides these ROIs in alphabetical order.

There are a number of other ordering files available, and various other parcellation types. If you would like common atlases to be supported, or need help getting your data to display at all, please feel free to contact me personally for support at rlaplant @ nmr.mgh.harvard.edu (I will just be happy that someone is using the program).

=====--max-edges and thresholds===== THIS SECTION IS IMPORTANT!

A little bit of explanation is required to understand the purpose of the --max-edges option. When the number of connections is too large, several things happen. One thing that can happen is that there are too many connections for effective visualization; there is too much data to usefully see or make sense of anything. This occurs at around roughly 500 connections (play around with the threshold slider to examine this effect). Another thing that can happen is that cvu has to repeatedly process a very large number of connections, and so becomes very slow. This occurs at around roughly 40000 connections. In order to improve performance, by default the program (i.e. when --max-edges is not specified or left as 0) will discard all but the first 20000 connections. This is a soft cap that examines the value of the 20000th strongest connection and includes all connections with an equal value to it, so in practice the real max may be slightly higher. Note that if the soft max considerably exceeds the hard max, the excess connections will be discarded -- again, to override this behavior, increase max-edges.

Nonetheless, there may be purposes for which users want to work with more connections than that. There are two reasons I can imagine to do that: generation of extremely high-resolution images, or calculation of graph statistics on high-resolution data.

IMPORTANT TL;DR If you do nothing, and ignore max-edges whenever you load data, only the 20000 strongest connections will be displayed!

After the roughly 20000 or max-edges connections are removed, there is an additional threshold which limits the number of connections to display. This threshold is different from the max-edges cutoff in that the program keeps track of all of the connections that are not being displayed because they are under threshold, but all connections below the max-edges cutoff are discarded entirely and cannot be recovered without loading the adjacency matrix file again with a higher value of max-edges. As such, any graph statistics that you might calculate with the new threshold will take into account all of the invisible connections up to the max-edges cutoff (but many of these statistics will examine only a proportion of these connections).

The display threshold can be either proportional or absolute. If it is proportional, it will be a number between 0 and 1 -- for instance, if pthresh is .90, 10% of connections will be displayed and 90% will be thresholded. If the threshold is absolute, any connections with an edge value exceeding the value of the threshold will be preserved. As a rule of thumb, the proprtional threshold is more valuable for exploratory analysis, and the absolute threshold is more valuable for generating figures and visualizing statistics.

=====Datasets and View manager=====

cvu is divided into multiple ''datasets.'' When the program starts, only one dataset is shown. It is called "sample data" because it usually represents the sample dataset except when cvu is started with command line arguments. Even if the program is started with command line arguments such that another matrix is displayed and not the sample data, the initial dataset is still called "sample data."

When you load new parcellations, you have the option to select "create new dataset." If you do, the currently selected dataset is ignored and instead a new dataset will be created in a new window. Other than being in a new window that has a different shape, this dataset functions exactly the same as the sample dataset.

When a dataset window is closed, the dataset is not deleted from cvu. cvu is still keeping track of the state of that visualization and the data itself. To delete the dataset, open "manage views", select the dataset from the table at left, and click delete. The "Sample data" dataset that lives on the main GUI cannot be deleted, but you can load a new parcellation and/or matrix over the data that it contains.

=====custom parcellations and surfaces====

cvu uses freesurfer surfaces to create the surface meshes and freesurfer annotations (clusterings of label files) to position the cortical nodes. By default, the program uses the fsaverage5 brain for all the node locations and the pial surface for display. However, it is possible to use other surfaces and other brain registrations. The most likely use of other brain registrations is for clinical data; it may be of interest to visualize the connectivity and node locations on the brain of a patient with a lesion, for instance. Users familiar with freesurfer will find it straightforward to specify this morphology by specifying SUBJECT and SUBJECTS_DIR. The parcellation must be encoded as an annotation (.annot) file. There are other surfaces that may be used as opposed to the pial. The orig and white surfaces (generated by freesurfer) are ok, but the inflated surface doesn't display properly because its hemispheres are centered differently. This makes it unsuitable for visualizing interhemispheric connections. I strongly advise against using custom surfaces because getting them to display in the right place is more trouble than it is worth.

=====circle plot labels=====

The circle plot is designed to be readable. As such, in high resolution parcellations, many labels must be omitted from the plot in order to make it actually be readable. This is inescapable, but this behavior is undesirable if you are interested in a particular region and would like to create a figure that helps to visualize that region specifically. Such a figure is not useful if the region of interest is not even labeled. In this case, you can specify a number of regions that are required to be among those labeled on the plot, by pressing the "force display of ROIs" button from the load adjmat menu. You may only make this specification prior to loading the connectivity data (e.g. the adjacency matrix). Doing this will relax the spatial restrictions slightly, but it is possible that this operation may fail (imagine the extreme case in which the user specified that every region must be labeled). If cvu cannot figure out how to spatially organize all of the labels that are required, it will just fail with an error message. This has never actually happened to me in practice except when I was explicitly testing this functionality.

Regardless of which labels are omitted from the text, mousing over a region will display a tooltip containing that region.

=====Tractography=====

Tractography is broken! Very sorry. I hope to fix it sometime in 2014.

===Tools and Features=== =====Plotting windows=====

cvu currently has three main windows in which data is displayed for visualization -- an interactive 3D brain model rendered with mayavi displaying the centroids of the ROIs and the connectivity between these regions as a glass model, an interactive adjacency matrix rendered with chaco, and a circle plot displaying the connectivity rendered in matplotlib. Mostly, whenever you change any views in one of these windows, the view and color scheme in the other windows will also change.

To rotate the 3D brain, drag and click. By default, this will rotate the scene in 3 dimensions about its center. Hold down SHIFT and CONTROL to alter this behavior. SHIFT will allow you to pan in any direction without zooming (dragging while holding down middle click has the same effect). CONTROL allows you to rotate the scene about its center for only the 2 dimensions currently defining the field of view. To zoom, scroll the mouse wheel or hold and drag the right mouse button. Clicking on the nodes, or selecting a node from the 'show node' button will display the connectivity only originating at that node. To restore all of the connections, right click on any of the plots or click the 'show all' button. Note also that you can specify many other parameters of the 3D brain display by clicking on the configuration button in the top left hand corner.

The adjacency matrix can be panned by left clicking and dragging. Clicking anywhere on the adjacency matrix will show the connectivity for the node corresponding to the row selected from the matrix. Zooming is done with the mouse wheel or the right mouse button. Right click to restore the entire matrix of connections.

The circle plot cannot be panned or zoomed due to fundamental limitations in matplotlib. This may not be fixed for some time -- the technical limitation is so deep that in order to fix this polar plots would have to be rewritten completely in matplotlib. However, you can click on the nodes on the edges, which will highlight the connections originating from only that node. Right clicking will restore all of the connections in their original glory. Note that the text on the circle plots is not exact with respect to the nodes being shown; what is exact is the color. You can find a listing of colors to brain regions by clicking on the 'color legend' button.

Whenever a node is highlighted/displayed in any one of the plots, each of the other plots shows the same data. In this way it is easy to compare the same data in several different views.

Each of these plots can be captured in a high resolution image. Click on the 'snapshot' buttons and specify the desired dpi. Technically speaking, high values of dpi will not translate to images with high dots per inch, but they will generate much larger images so the effective image quality will be roughly as specified.

=====Modules=====

cvu will calculate graph partitions and display modules based on minimal cut sets. How to optimally partition a graph into communities is an NP-hard problem. cvu uses a modularity maximization strategy from the brain connectivity toolbox for python (my translation of the brain connectivity toolbox for matlab). However, in the event that you prefer a different algorithm, you can also load the community structure from a file. The file should be an Nx1 numpy, matlab, or txt matrix with values representing community affiliations ranging between 1 and M for some M<=N. This is the format returned by all functions in BCT and bctpy. bctpy has several alternative algorithms for generating community structure that may be of interest.

To use the default algorithm, click on "calculate stats" and select the "modules" calculation type. You'll be able to threshold the data at whatever threshold you choose before the calculation. To load your own vector, click on "load stats" and select "modules".

After loading, cvu will display each module in a different, random color. These colors can show low contrast because they are selected randomly. If you can't see, try going to "View Module -> View All Modules" to recolor them. Alternatively, you can view each individual module one at a time in the "View Module" tab.

=====Scalars and Graph Theoretical Quantities=====

cvu is meant to be a visualization tool for networks. As such, it wouldn't be complete without visualization of graph theoretical quantities that describe these networks.

cvu can calculate a variety of measures using bct for python, which is a port of the Brain Connectivity Toolbox (written in matlab) to python, for which I am the maintainer. bct is a rich toolbox that has a variety of interesting graph theory features. I have only integrated several of the most important ones into cvu directly. The graph theory measures that are calculated natively are: *Global/local efficiency *Average node strength *Clustering coefficient *Eigenvector/betweenness centrality *Modularity (requires modules) *Participation coefficient (requires modules) *Within-module degree (requires modules)

To calculate these metrics, first click on the "options" button and go to the "graph statistics" tab. Here you can select which of the graph metrics you'd like to calculate. Some of them (particularly local efficiency) take a long time to calculate and are disabled by default.

Then, go to "calculate stats" and select the "statistics" calculation type. As with modules, you can threshold the matrix before performing this calculation.

The statistics are then displayed in the "show statistics" panel. You can also visualize these statistics on the brain. To do this, select "save to scalar" and give the statistic a unique name. Then, click on the "show scalars" button in cvu. You'll be able to choose a value for each of several different visualization dimensions.

If you want to visualize some statistics that are not available, you can again load Nx1 matrices from files from the "load stats" tab.

=====Movies=====

cvu is able to make movies of the 3D brain. These movies are created by having the X server grab the screen output from cvu and record it to a video file (using ffmpeg). This grabs the screen output. This means that if you were to place another window in front of the spot where the brain is, the other window would be captured in your movie. There is no straightforward way to get around this.

You can enable or disable a simple animation while making movies. This animation automatically rotates the current view. You can change the axis of rotation by interacting with the brain as you would any other mayavi instance (e.g. clicking and dragging). The frequency of this animation is controlled by the animation speed parameter.

Additionally, you can specify the framerate and bitrate of the output movie.
Increasing these parameters above their default values will increase the image quality, at the cost of additional disk space.

=====Colors=====

By clicking on the 'colors' tab within the options panel, you will be able to select different color schemes for each dataset. Each of these color schemes can be selected from a list of preexisting colormaps, or specified from a file.

In addition, you can create your own colormaps in a custom colormap editor. This editor specifies ((RGB or HSV) and (alpha)) values. There can be arbitrarily many segmentations -- that is, colors defining the colormap gradients. To create a segmentation, left click. To delete a segment, right click. RGB (red green blue), HSV (hue spectrum value) and alpha (transparency) values can be adjusted for each segmentation by dragging the little box. When done using the editor, three files will be saved -- a .lut, .png, and .grad file. The one that the cvu reads and understands is the .lut file.

WHEN USING THE COLOR EDITOR MAKE SURE TO USE NONZERO ALPHA VALUES!

The color editor seems to make the default transparency rather low -- i.e. fully transparent. Not every visualization respects transparency values -- for instance, the lines on the circle plot ignore this value. But for any visualizations that do respect transparency (e.g. the 3D brain), if you don't make these values higher, you won't see anything. Chances are, you '''will''' make this mistake if you use the color editor. If you do, this is the problem; set your transparency higher! Sorry that I can't fix this.

=====Python Shell=====

There is an interactive python shell in the program. It is fantastic for looking up data or doing math on the fly.

===Options=== =====Floating 3D text=====

Setting this off removes the text floating around the 3D brain. Note that the text is always disabled when making snapshots due to technical limitations.

=====Thresholds=====

See the --max-edges and thresholds section of this manual

=====Surface Style=====

Offers several visualizations for the surface triangular mesh (in VTK terms, several different visualizations of the connectivity). Purely cosmetic differences.

=====Surface Opacity=====

Changes the opacity of the 3D brain between 0 and 1. At 1, the 3D brain is completely opaque, effectively making it impossible to see any connections. At 0, the brain surface is not present at all. Low values provide a good balance between anatomical specificity and transparency, while high values provide some amusingly useless images.

=====Module Connection Style=====

When viewing single modules, select which connections are shown *Intramodular (default): only connections where both nodes are in the module are shown *Intermodular: only connections where exactly one node is in the module are shown *Both: connections with at least one node in the module are shown

=====Interhemispheric/LH/RH connections=====

There are three types of connections; intrahemispheric connections within only the left hemisphere, intrahemispheric connections within only the right hemisphere, and interhemispheric connections. Selecting these options will selectively disable connections of the specific type.

Note that you will have to recalculate the view (e.g. by right clicking any plot) before these changes will take effect; clicking the "force render" button is not sufficient. The reason that this step is necessary is that recalculating the plot with a new set of connections is a costly (time-consuming) operation. Imagine accidentally ticking that box, and having to wait for several seconds (potentially upwards of 30 seconds for large parcellations), only to just turn it off afterwards and wait for it to compute again. This potential inconvenience outweighs the inconvenience of right clicking once to effect the desired changes. I may change this, though

=====LH/RH nodes===== Selecting these options will remove the spherical nodes from the 3D brain display. Unlike the options to display connection types, this operation is computationally fast and will update the 3D brain immediately.

=====LH/RH surfaces===== Selecting these options will remove the gray surfaces from the 3D brain display. Unlike the options to display connection types, this operation is computationally fast and will update the 3D brain immediately.

=====Colorbar (connections)===== Provides a colorbar in the 3D brain view showing the values associated with the connections.

=====Scalar colorbar===== Provides a colorbar in the 3D brain view showing the values associated with scalars (i.e. graph measures). These measures are always visualized between 0 and 1 and the scalar colorbar has to reflect this for technical reasons.

==Acknowledgments==

I did not write every line of code executed by cvu. I used libraries. Most of these libraries are BSD-like licensed, such that a copy of the license must accompany the basic documentation. These copies are contained in the licenses folder of cvu. If your copy of cvu did not contain the licenses folder, you may peruse the licenses on cvu's github page at https://github.com/aestrivex/cvu