Yearbook of Medical Informatics 2002. In Press
Imaging
informatics and the Human Brain Project:
the Role of
Structure
James F. Brinkley and
Cornelius Rosse
Structural Informatics
Group, Department of Biological Structure
University of Washington,
Seattle, USA
August 21, 2001
The human brain is arguably the most complex and least understood of all organs in the body, yet relatively recent technological advances are rapidly opening up entirely new avenues for understanding its structure and function. Primary among these new technologies are images, not only of structure, but also of function, which provide increasingly detailed views of the thinking brain. These and other technologies have led to an explosion of research results in neuroscience, such that over 15,000 abstracts are presented at the annual meeting of the Society for Neuroscience (http://www.sfn.org).
As in other biomedical fields this proliferation of data has led to an information glut that makes it impossible for any one individual to comprehend more than a small fraction of the available results. Yet it is often argued that the only way we will truly understand the brain is to develop an integrated view that ties together data at levels ranging from genes to behavior.
As a response to this dilemma the Human Brain Project (HBP) [1-3] was initiated in 1993 as a result of an Institute of Medicine Report [4]. The goals of the HBP are to 1) develop re-usable, generalizable and widely-available software tools that are specialized for neuroscience data and knowledge, 2) develop methods for integrating diverse forms of raw and processed neuroscience information, 3) develop Internet-based methods for sharing and disseminating the integrated information to promote knowledge discovery and the development of distributed, large scale models of brain function, and 4) apply these tools and information systems to research, clinical medicine and education. The hope is that by applying informatics tools and techniques to the fragmented data and knowledge that currently characterize neuroscience, it will be possible to regain a sense of wholeness from the ever-diversifying parts. The aggregate research endeavor that results from these and similar goals is called neuroinformatics [5].
One of the many neuroinformatics research questions that arise from these goals is how to integrate diverse forms of raw and processed information. Neuroscience data collected from humans alone come in multiple forms (e.g., sequence, image-based, electrophysiological, behavioral) at multiple levels (gene, molecular, ultrastructural, cellular, neural circuit, whole brain), and from multiple individuals. The fact that data come from multiple individuals is particularly difficult to address since no two human brains are exactly alike, let alone the brains of non-human species from which a large amount of data are obtained. Much of the research effort in the HBP and other neuroscience labs deals with the problem of relating multiple brains.
Anatomy is the common frame of reference for nearly all HBP efforts at integration, since anatomy in its broadest definition embraces all levels of structure from the molecular to the macroscopic [6]. (Neuro)anatomy not only provides an understanding of the physical organization of the brain, it also can serve as a framework for organizing all forms of neuroscience data. This postulate is consistent with a central tenet of modern biology, namely that function can only be understood in terms of the physical structure that underlies it.
This central role of anatomy is not limited to neuroscience. In fact, an understanding of the structure of the body is essential for virtually all biomedical endeavors since both normal and abnormal functions can be regarded as attributes of anatomical structures. We therefore argue that anatomy is a prime candidate for organizing and integrating not only neuroscience information but virtually all other biomedical information as well.
In order to develop such an anatomical (or structural) information framework many informatics research problems must be solved in areas such as representation, analysis, management, visualization and dissemination of anatomical information. Solutions to these problems require the application and invention of new methodologies rooted in computer science. These problem areas include, for instance, knowledge representation, image understanding, graphics, visualization, databases and user interfaces.
The richness of these problem areas, their broad applicability, and the commonality of anatomical patterns at multiple levels of organization have prompted us to define structural informatics as a field for dealing with the broad range of issues arising from the representation, management and use of information that pertains to the physical organization of the body [7]. We use the term structural as opposed to anatomical informatics to avoid the connotation of the term “anatomy” which, despite its definition to the contrary, is often limited to the macroscopic (gross) level.
The subject of this edition of the Yearbook is imaging informatics [8], which can be defined as the development of methods for organizing, managing, retrieving, analyzing and visualizing images. Images of all sorts obtained from any or all regions of the body are the central focus of imaging informatics.
From the point of view of structural informatics images are only one source (though probably the most important one) of data about anatomical structures. Other sources include, for example, gene sequences, nuclear magnetic resonance spectroscopy, X-ray crystallography, the physical exam, endoscopy, and auscultation.
The focus of neuroinformatics is understanding the brain in all its aspects – anatomy, pathology, function (including behavior). Thus, images and anatomy are important components of neuroinformatics research, but they are not the only ones. Others include, for example, genetics, biochemistry, physiology, psychology, pathology, neurology, radiology and neurosurgery.
The subject of this review is the intersection of these three fields (structural-, imaging- and neuro informatics) within the context of the HBP. Of the 26 projects currently listed on the HBP research grants page (http://www.nimh.nih.gov/neuroinformatics/researchgrants.cfm) 19 use images as a primary source of data. We limit our review primarily to these and related projects because 1) we are most familiar with HBP work, 2) the HBP provides exemplary research projects in many relevant areas, 3) the HBP represents the primary national effort in the application of informatics to neuroscience, and 4) we wish to make the wider informatics community more aware of the HBP. However, we point out that a large amount of image related research deals with the brain, as evidenced by any issue of journals such as IEEE Transactions on Medical Imaging, and a large amount of non-HBP neuroscience research involves the use of images and anatomical information.
The paper is organized into three basic sections: structural imaging, functional imaging, and image-based brain information systems. Structural imaging provides the anatomical substrate on which the functional data can be mapped, analogous to geographic information systems, which map various kinds of data to the earth. However, for brain mapping the problem is complicated by the fact that no two brains are alike.
Images are almost exclusively the source of data for visualizing and reconstructing the anatomy of the brain. Different imaging modalities provide complementary and often highly detailed anatomical information. All modalities are either inherently digital or can be converted to digital form by film scanning.
Traditional image sources are photographs of gross dissections, or microscopic sections that may be frozen (cryosections) or histochemically stained to emphasize certain structural components such as myelin [9]. Electron microscopy reveals the ultrastructure of the brain at the level of synaptic connections and cytoplasmic inclusions [10]. Immunocytochemical and DNA-hybridization techniques depict the distribution of specific proteins or messenger RNA, thereby allowing the expression of specific genes to be observed in different parts of the brain during development, maturity and senility [11]. From the image processing point of view all these image sources can be regarded as 2-D image sections.
In the living brain, computed tomography (CT) distinguishes different structures by virtue of their radio-density, magnetic resonance imaging (MRI) distinguishes structures by their differential response to radio frequency pulses applied within a graded magnetic field, and magnetic resonance venography (MRV), and arteriography (MRA) emphasize veins and arteries by altering the parameters of the radio frequency pulses [12]. An HBP-funded effort at Caltech is developing advanced methods for in vivo MR microscopic imaging that is being used to generate high resolution images of the developing embryo [13].
Traditional image sources provide 2-D views of parts of the brain. However, because the brain is three-dimensional, the most informative data come from techniques that either directly or indirectly image the entire 3-D volume of interest. Therefore, most current brain imaging research is concerned with 3-D image volume data.
Informatics issues that arise when dealing with 3-D structural brain images include image registration, spatial representation of anatomy, symbolic representation of anatomy, integration of spatial and symbolic anatomic representations in atlases, anatomical variation, and characterization of anatomy. All but the first of these issues deal primarily with anatomical structure, and therefore fall in the field of structural informatics. They could also be thought of as being part of imaging informatics and neuroinformatics. Depends on the point of view.
Image volume data are represented in the computer by a 3-D volume array, in which each voxel (volume-element, analogous to pixel in 2-D) represents the image intensity in a small volume of space. In order to accurately depict brain anatomy, the voxels must be accurately registered (or located) in the 3-D volume, and separately acquired image volumes from the same subject must be registered with each other.
Technologies such as CT, MRI, MRV and MRA (section 2) are inherently 3-D: the scanner generally outputs a series of image slices that can easily be reformatted as a 3-D volume array, often following alignment algorithms that compensate for any patient motion during the scanning procedure. Confocal microscopy [14], which generates a 3-D image volume through a tissue section, is also inherently 3-D, as is electron tomography, which generates 3-D images from thick electron-microscopic sections using techniques similar to those used in CT [15].
Two-dimensional images can be converted to 3-D volumes by acquiring a set of closely spaced parallel sections through a tissue or whole brain. In this case the problem is how to align the sections with each other. For whole brain sections (either frozen or fixed) the standard method is to embed a set of thin rods or strings in the tissue prior to sectioning, to manually indicate the location of these fiducials on each section, then to linearly transform each slice so that the corresponding fiducials line up in 3-D [16]. A popular current example of this technique is the Visible Human, in which a series of transverse slices were acquired, then reconstructed to give a full 3-D volume [17].
It is difficult to embed fiducial markers at the microscopic level, so intrinsic tissue landmarks are often used as fiducials, but the basic principle is similar. However, in this case tissue distortion may be a problem, so non-linear transformations may be required. For example Fiala and Harris [18] have developed an interface that allows the user to indicate, on electron microscopy sections, corresponding centers of small organelles such as mitochondria. A non-linear transformation (warp) is then computed to bring the landmarks into registration.
An approach being pursued (among other approaches) by the National Center for Microscopy and Imaging Research (http://ncmir.ucsd.edu/) combines reconstruction from thick serial sections with electron tomography [19]. In this case the tomographic technique is applied to each thick section to generate a 3-D digital slab, after which the slabs are aligned with each other to generate a 3-D volume. The advantages of this approach over the standard serial section method are that the sections do not need to be as thin, and fewer of them need be acquired.
A alternative approach to 3-D voxel registration from 2-D images is stereo-matching, a technique developed in computer vision that acquires multiple 2-D images from known angles, finds corresponding points on the images, and uses the correspondences and known camera angles to compute 3-D coordinates of pixels in the matched images. The technique is being applied to the reconstruction of synapses from electron micrographs by a HBP collaboration between computer scientists and biologists at the University of Maryland [20].
A related problem to that of aligning individual sections is the problem of aligning separate image volumes from the same subject, that is, intra-subject alignment. Because different image modalities provide complementary information, it is common to acquire more than one kind of image volume on the same individual. For example, in our own HBP work, we acquire an MRI volume dataset depicting cortical anatomy, an MRV volume depicting veins, and an MRA volume depicting arteries [21]. By “fusing” these separate modalities into a single common frame of reference (anatomy, as given by the MRI dataset), it is possible to gain information that is not apparent from one of the modalities alone. In our case the fused datasets are used to generate a visualization of the brain surface as it appears at neurosurgery, in which the veins and arteries provide prominent landmarks.
When intensity values are similar across modalities, linear alignment can be performed automatically by intensity-based optimization methods [22, 23]. When intensity values are not similar (as is the case with MRA, MRV and MRI), images can be aligned to templates of the same modalities that are already aligned [24, 25]. Alternatively, landmark-based methods can be used. The landmark-based methods are similar to those used to align serial sections, but in this case the landmarks are 3-D points. The Montreal Register Program [26] (which can also do non-linear registration, as discussed in section 2.5.1) is an example of such a program.
The reconstructed 3-D image volume can be visualized directly using volume rendering techniques [27]. It can also be given as input to image-based techniques for warping the image volume of one brain to other, as described in section 2.5.1. However, more commonly the image volume is processed in order to extract an explicit spatial (or quantitative) representation of brain anatomy. Such an explicit representation permits improved visualization, quantitative analysis of brain structure, comparison of anatomy across a population, and mapping of functional data. It is thus a component of most research involving brain imaging.
Extraction of spatial representations of anatomy, in the form of 3-D surfaces or volume regions, is accomplished by segmenting (or isolating) brain structures from the 3-D image volume. Fully automated segmentation is an unsolved problem, as attested to by the number of papers about this subject in IEEE Transactions on Medical Imaging. However, because of the high quality of MRI brain images, a great deal of progress has been made in recent years; in fact, several software packages do a credible job of automatic segmentation, particularly for normal macroscopic brain anatomy in cortical and sub-cortical regions [28-34]. The HBP-funded Internet Brain Segmentation Repository [35] is developing a repository of segmented brain images to use in comparing these different methods.
Popular segmentation and reconstruction techniques include reconstruction from serial sections, region-based methods, edge-based methods, model or knowledge-based methods, and combined methods.
The classic approach to extracting anatomy is to manually or semi-automatically trace the contours of structures of interest on each of a series of aligned image slices, then to “tile” a surface over the contours [36]. The tiled surface usually consists of an array of 3-D points connected to each other by edges to form triangular facets. The resulting 3-D surface mesh is then in a form where it can be further analyzed or displayed using standard 3-D surface rendering techniques [37].
Neither fully automatic contour tracing nor fully automatic tiling has been satisfactorily demonstrated in the general case. Thus, semi-automatic contour tracing followed by semi-automatic tiling remains the most common method for reconstruction from serial sections, and reconstruction from serial sections itself remains the method of choice for extracting microscopic 3-D brain anatomy [18].
This and the following sections primarily concentrate on segmentation at the macroscopic level.
In region-based segmentation voxels are grouped into contiguous regions based on characteristics such as intensity ranges and similarity to their neighbors [38]. A common initial approach to region-based segmentation is first to classify voxels into a small number of tissue classes such as gray matter, white matter, cerebrospinal fluid and background, then to use these classifications as a basis for further segmentation [39, 40]. Another region-based approach is called region-growing, in which regions are grown from seed voxels manually or automatically placed within candidate regions [21, 41]. The regions found by any of these approaches are often further processed by mathematical morphology operators [42] to remove unwanted connections and holes [43].
Edge-based segmentation is the complement to region-based segmentation: intensity gradients are used to search for and link organ boundaries. In the 2-D case contour-following connects adjacent points on the boundary. In the 3-D case isosurface following or marching cubes [44] connects border voxels in a region into a 3-D surface mesh.
Both region-based and edge-based segmentation are essentially low-level techniques that only look at local regions in the image data.
The most popular current method for medical image segmentation, for the brain as well as other biological structures, is the use of deformable models. Based on pioneering work called “Snakes” by Kass, Witkin and Terzopoulos [45], deformable models have been developed for both 2-D and 3-D. In the 2-D case the deformable model is a contour, often represented as a simple set of linear segments or a spline, which is initialized to approximate the contour on the image. The contour is then deformed according to a cost function that includes both intrinsic terms proscribing how much the contour can distort, and extrinsic terms that reward closeness to image borders. In the 3-D case a 3-D surface (often a triangular mesh) is deformed in a similar manner. There are several examples of HBP-funded work that use deformable models for brain segmentation [28, 30, 31, 41].
An advantage of deformable models is that the cost function can include knowledge of the expected anatomy of the brain. For example, the cost function employed in the method developed by MacDonald [30] includes a term for the expected thickness of the cortical sheet. Thus, these methods can become somewhat knowledge-based, where knowledge of anatomy is encoded in the cost function.
An alternative knowledge-based approach explicitly records shape information in a geometric constraint network (GCN) [46], which encodes local shape variation based on a training set. The shape constraints define search regions on the image in which to search for edges. Found edges are then combined with the shape constraints to deform the model and reduce the size of search regions for additional edges [47, 48]. One potential advantage of this sort of model over a pure deformable model is that knowledge is explicitly represented in the model, rather than implicitly represented in the cost function.
Most brain segmentation packages use a combination of methods in a sequential pipeline. For example, in our own recent work we first use a GCN model to represent the overall cortical “envelope”, excluding the detailed gyri and sulci [32]. The model is semi-automatically deformed to fit the cortex, then used as a mask to remove non-cortex such as the skull. Isosurface following is then applied to the masked region to generate the detailed cortical surface. The model is also used on aligned MRA and MRV images to mask out non-cortical veins and arteries prior to isosurface following. The extracted cortical, vein and artery surfaces are then rendered to produce a composite visualization of the brain as seen at neurosurgery.
MacDonald et al. describe an automatic multi-resolution surface deformation technique called ASP (Anatomic Segmentation using Proximities), in which an inner and outer surface are progressively deformed to fit the image, where the cost function includes image terms, model-based terms, and proximity terms [30]. Dale et al. describe an automated approach that is implemented in the FreeSurfer program [28, 49]. This method initially finds the gray-white boundary, then fits smooth gray-white (inner) and white-CSF (outer) surfaces using deformable models. Van Essen et al. describe the SureFit program [31], which finds the cortical surface midway between the gray-white boundary and the gray-CSF boundary. This mid-level surface is created from probabilistic representations of both inner and outer boundaries that are determined using image intensity, intensity gradients, and knowledge of cortical topography. Other software packages also combine various methods for segmentation [33, 41, 50, 51].
Given segmented brain structures, whether at the macroscopic or microscopic level, and whether represented as 3-D surface meshes or extracted 3-D regions, it is often desirable to attach labels (names) to the structures. If the names are drawn from a controlled terminology they can be used as an index into a database of segmented structures, thereby providing a qualitative means for comparing brains from multiple subjects.
If the terms in the vocabulary are organized into symbolic qualitative models (“ontologies”) of anatomical concepts and relationships, they can support systems that manipulate and retrieve segmented brain structures in “intelligent” ways. For example, a dynamic scene generator could assemble 3-D scenes of various segmented brain structures, overlaying them with anatomic names [52, 53].
If the anatomical ontologies are linked to other ontologies of physiology and pathology they can provide increasingly sophisticated knowledge about the meaning of the various images and other data that are increasingly becoming available in online databases (section 4) It is our belief that this kind of knowledge (by the computer, as opposed to the neuroscientist) will be required in order to achieve the seamless integration of all forms of data envisioned by the HBP.
As in other biomedical fields the HBP has recognized the need for controlled vocabularies and ontologies to relate multiple sources of data. This recognition is evidenced by the keynote speeches at the 2001 spring meeting of the HBP [54, 55]. As in the spatial case it is commonly accepted that neuroanatomy provides the most logical organizational framework; in this case, however, neuroanatomy is represented symbolically rather than spatially.
At the most fundamental level Nomina Anatomica [56] and its recent successor, Terminologia Anatomica [57] provide a classification of officially sanctioned terms that are associated with macroscopic and microscopic brain structures. This canonical term list, however, has been substantially expanded by synonyms that are current in various fields of the neurosciences, and has also been augmented by a large number of new terms that designate structures omitted from Terminologia Anatomica. Many of these additions are present in clinical controlled terminologies (MeSH [58], SNOMED [59], Read Codes [60], GALEN [61]). Unlike Terminologia, which only exists in hard copy, these vocabularies are entirely computer-based, and therefore lend themselves for incorporation in HPB related applications.
The most complete primate neuroanatomical terminology is NeuroNames, developed by Bowden and Martin at the University of Washington [62]. NeuroNames, which is included as a knowledge source in the National Library of Medicine’s Unified Medical Language System (UMLS) [63], is primarily organized as a part-of hierarchy of nested structures, with links to a large set of ancillary terms that do not fit into the strict part-of hierarchy. Other neuroanatomical terminologies have also been developed [64-67]. A challenge for the HBP is to either come up with a single consensus terminology or to develop Internet tools that allow transparent integration of distributed but commonly-agreed on terminology, with local modifications.
Classification and ontology projects to-date have focused primarily on arranging the terms of a particular domain in hierarchies. As we noted with respect to the evaluation of Terminologia Anatomica [68], insufficient attention has been paid to the relationships between these terms. Terminologia, as well as anatomy sections of the controlled medical terminologies, mix -is a- and -part of- relationships in the anatomy segments of their hierarchies. Although such heterogeneity does not interfere with using these term lists for keyword-based retrieval, these programs will fail to support higher level knowledge (reasoning) required for knowledge-based applications.
In our own Structural Informatics Group at the University of Washington we are addressing this deficiency by developing a Foundational Model of Anatomy (FMA), which we define as a comprehensive symbolic description of the structural organization of the body, including anatomical concepts, their preferred names and synonyms, definitions, attributes and relationships [6, 69].
The FMA is being implemented in Protégé-2000, a frame-based knowledge acquisition system developed at Stanford [70, 71]. In Protégé anatomical concepts are arranged in class-subclass hierarchies, with inheritance of defining attributes along the isa link, and other relationships (e.g., parts, branches, spatial adjacencies) represented as additional slots in the frame. The FMA currently consists of over 60,000 concepts, represented by 85,000 terms arranged in 75 types of relationships that represent all structures except the brain visible to 1 mm, and many microscopic and molecular structures as well. We are currently in the process of integrating NeuroNames with the FMA as a Foundational Model of Neuroanatomy (FMNA) [72].
Our belief is that the FMNA, as an integral component of the FMA for the entire body, will prove useful for symbolically organizing and integrating neuroscience information. But in order to answer non-trivial queries in neuroscience and to develop “smart tools” that rely on deep knowledge, additional ontologies must also be developed, among other things, for physiological functions mediated by neurotransmitters, pathological processes and their clinical manifestations as well for the radiological appearances, with which they correlate. The relationships that exist between these concepts and anatomical parts of the brain must also be explicitly modeled. Next generation HBP efforts that link the FMNA and other anatomical ontologies with separately developed functional ontologies such as the biophysical description markup language (BDML) being developed at Cornell [73] will be needed in order to accomplish this type of integration.
Spatial representations of neuroanatomy, in the form of segmented regions on 2-D or 3-D images, or 3-D surfaces extracted from image volumes, are often combined with symbolic representations to form digital atlases. A digital atlas (which for this review refers to an atlas created from 3-D image data taken from real subjects, as opposed to artists’ illustrations) is generally created from a single individual, which therefore serves as a “canonical” instance of the species. Traditionally, atlases have been primarily used for education, and most digital atlases are used the same way.
For example, the Digital Anatomist Interactive Atlas of the brain [74] was created by outlining regions of interest on 2-D images (many of which are snapshots of 3-D scenes generated by reconstruction from serial sections) and labeling the regions with terminology from NeuroNames. The atlas, which is available both on CD-ROM and on the web, permits interactive browsing, where the names of structures are given in response to mouse clicks; dynamic creation of “pin diagrams”, in which selected labels are attached to regions on the images; and dynamically-generated quizzes, in which the user is asked to point to structures on the image [75].
An example of a 3-D brain atlas created from the Visible Human is Voxelman [76], in which each voxel in the Visible Human head is labeled with the name of an anatomic structure in a “generalized voxel model” [77], and highly-detailed 3-D scenes are dynamically generated. Several other brain atlases have also been developed primarily for educational use [78, 79].
In keeping with the theme of anatomy as an organizing framework, atlases have also been developed for integrating functional data from multiple studies [65, 80-85]. In their original published form these atlases permit manual drawing of functional data, such as neurotransmitter distributions, onto hardcopy printouts of brain sections. Many of these atlases have been or are in the process of being converted to digital form . The Laboratory of Neuroimaging (LONI) at UCLA has been particularly active in the development and analysis of digital atlases [86], and the Caltech HBP has recently released a web-accessible 3-D mouse atlas acquired with micro-MR imaging [87].
The most widely used human brain atlas is the Talairach atlas, based on post mortem sections from a 60-year-old woman [88]. This atlas introduced a proportional coordinate system (often called “Talairach space”) which consists of 12 rectangular regions of the target brain that are piecewise affine transformed to corresponding regions in the atlas. Using these transforms (or a simplified single affine transform based on the anterior and posterior commissures) a point in the target brain can be expressed in Talairach coordinates, and thereby related to similarly transformed points from other brains. Other human brain atlases have also been developed [89-93].
Brain information systems often use atlases as a basis for mapping functional data onto a common framework, much like geographic information systems (GIS’s) use the earth as the basis for combining data. However, unlike GIS’s, brain information systems must deal with the fact that no two brains are exactly alike, especially in the highly folded human cerebral cortex. Thus, not only do neuroinformatics researchers have to develop methods for representing individual brain anatomy, they also must develop methods for relating the anatomy of multiple brains. Only by developing methods for relating multiple brains will it be possible to generate a common anatomical frame of reference for organizing neuroscience data. Solving this problem is currently a major focus of work in the HBP.
Two general approaches for quantitatively dealing with anatomic variation can be defined: 1) warping to a template atlas, and 2) population-based atlases. Variation can also be expressed in a qualitative manner, as described in section 2.6.1.
The most popular current quantitative method for dealing with anatomic variation is to deform or warp an individual target brain to a single brain chosen as a template. If the template brain has been segmented and labeled as an atlas (section 2.4), and if the registration of the target brain to the template is exact, then the target brain will be automatically segmented, and any data from other studies that are associated with the template brain can be automatically registered with the target brain by inverting the warp [94, 95]. Such a procedure could be very useful for surgical planning, for example, since functional areas from patients whose demographics match that of the surgical patient could be superimposed on the patient’s anatomy [96].
The problem of course comes with the word, “exact”. Since no two brains are even topologically alike (sulci and gyri are present in one brain that are not present in another) it is impossible to completely register one brain to another. Thus, the research problem, which is very actively being pursued by many HBP researchers [94], is how to register two brains as closely as possible. Methods for doing this can be divided into volume-based warping and surface-based warping.
Volume-based warping. Pure volume-based registration directly registers two image volumes, without the pre-processing segmentation step. Whereas intra (single)-patient registration (section 2.1.2) establishes a linear transformation between two datasets, inter (multiple)-patient registration establishes a non-linear transformation (warp) that takes voxels in one volume to corresponding voxels in the other volume. Because of the great variability of the cerebral cortex pure volume-based registration is best suited for sub-cortical structures rather than the cortex. As in the linear case there are two basic approaches to non-linear volume registration: intensity-based and landmark-based, both of which generally use either physically-based approaches or minimization of a cost function to achieve the optimal warp.
The intensity-based approach uses characteristics of the voxels themselves, generally without the segmentation step, to non-linearly align two image volumes [29, 95, 97, 98]. Most start by removing the skull, which often must be done manually.
The landmark-based approach is analogous to the 2-D case: the user manually indicates corresponding points in the two datasets (usually with the aid of three orthogonal views of the image volumes). The program then brings the corresponding points into registration while carrying along the intervening voxel data. The Montreal Register program [26] can do non-linear 3-D warps, as can the Edgewarp-3D program [99], which is a generalization of the Edgewarp program developed by Bookstein [100].
A variation of landmark-based warping matches curves or surfaces rather than points, then uses the surface warps as a basis for interpolating the warp for intervening voxels [101, 102].
Surface-based warping. Surface-based registration is primarily used to register two cortical surfaces. The surface is first extracted using techniques described in section 2.2, then image-based or other functional data are “painted” on the extracted surface where they are carried along with whatever deformation is applied to the surface. Since the cortical surface is the most variable part of the brain, yet the most interesting for many functional studies, considerable research is currently being done in the area of surface-based registration [103].
It is very difficult if not impossible to match two surfaces in their folded up state, or to visualize all their activity. (The cerebral cortex gray matter can be thought of as a 2-D sheet that is essentially crumpled up to fit inside the skull). Therefore, much effort has been devoted to “reconfiguring” [31] the cortex so that it is easier to visualize and register. A prerequisite for these techniques is that the segmented cortex must be topologically correct. The programs FreeSurfer [28], Surefit [31], ASP [30] and others all produce surfaces suitable for reconfiguration.
Common reconfiguration methods include inflation, expansion to a sphere, and flattening. Inflation uncrumples the detailed gyri and sulci of the folded surface by partially blowing the surface up like a balloon [31, 33, 49] . The resulting surface looks like a lissencephalic (smooth) brain, in which only the major lobes are visible, and the original sulci are painted on the surface as darker intensity curves. These marks, along with any functional data, are carried along in the other reconfiguration methods a well.
Expansion to a sphere further expands the inflated brain to a sphere, again with painted lines representing the original gyri and sulci. At this point it is simple to define a surface-based coordinate system as a series of longitude-latitude lines referred to a common origin. This spherical coordinate system permits more precise quantitative comparison of different brains than 3-D Talairach coordinates because it respects the topology of the cortical surface. The surface is also in a form where essentially 2-D warping techniques can be applied to deform the gyri and sulci marked on the sphere to a template spherical brain.
The third approach is to flatten the surface by making artificial cuts on the inflated brain surface, then spreading out the cut surface on a 2-D plane while minimizing distortion [31, 49, 104]. Since it is impossible to eliminate distortion when projecting a sphere to a plane, multiple methods of projection have been devised, just as there are multiple methods for projecting the earth’s surface [94]. In all cases, the resulting flat map, like a 2-D atlas of the earth, is easier to visualize than a 3-D representation since the entire cortex is seen at once. Techniques for warping one cortex to another are applicable to flat maps as well as spherical maps, and the warps can be inverted to map pooled data on the individual extracted cortical surface.
The problem of warping any of these reconfigured surfaces to a template surface is still an active area of research because it is impossible to completely match two cortical surfaces. Thus, most approaches are hierarchical, in which larger sulci such as the lateral and central sulcus are matched first, followed by minor sulci.
The main problem with warping to a template atlas is deciding which atlas to use as a template. Which brain should be considered the “canonical” brain representing the population? The widely used Talairach atlas is based on a 60 year-old woman. The Visible Human male was a convict and the female was an older women. What about other populations such as different racial groups? These considerations have prompted several groups to work on methods for developing brain atlases that encode variation among a population, be it the entire population or selected subgroups. The International Consortium for Brain Mapping (ICBM), a collaboration among several brain mapping institutions headed by Mazziotta at UCLA (http://www.loni.ucla.edu/ICBM), is collecting large numbers of normal brain image volumes from collaborators around the world [105]. To date several thousand brain image volumes, many with DNA samples for later correlation of anatomy with genetics, are stored on a massive file server. As data collection continues methods are under development for combining these data into population-based atlases.
A good high-level description of these methods can be found in a review article by Toga and Thompson [94]. In that article three main methods are described for developing population-based atlases: density-based, label-based and deformation-based approaches.
In the density-based method, a set of brains is first transformed to Talairach space by linear registration. Corresponding voxels are then averaged, yielding an “average” brain that preserves the major features of the brain, but smoothes out the detailed sulci and gyri. The Montreal average brain, which is an average of 305 normal brains [106], is constructed in this way. Although not detailed enough to permit precise comparisons of anatomical surfaces, it nevertheless is useful as a coarse means for relating multiple functional sites. For example, in our own work we have mapped cortical language sites from multiple patients onto the average brain, allowing a rough comparison of their distribution for different patient subclasses [107].
In the label-based approach, a series of brains are segmented, and then linearly transformed to Talairach space. A probability map is constructed for each segmented structure, such that at each voxel the probability can be found that a given structure is present at that voxel location. This method has been implemented in the Talairach Demon, an Internet server and Java client developed by Fox et al. as part of the ICBM project [108]. A web user inputs one or more sets of Talairach coordinates, and the server returns a list of structure probabilities for those coordinates.
In the warp-based method, the statistical properties of deformation fields produced by non-linear warping techniques (section 2.5.1) are analyzed to encode anatomical variation in population subgroups [109, 110]. These atlases can then be used to detect abnormal anatomy in various diseases.
The main reason for finding ways to represent anatomy is to examine the relationship between structure and function in both health and disease. For example, how does the branching pattern of the dendritic tree influence the function of the dendrite? Does the pattern of cortical folds influence the distribution of language areas in the brain? Does the shape of the corpus callosum relate to a predisposition to schizophrenia? Can subtle changes in brain structure be used as a predictor for the onset of Alzheimer’s disease? These kinds of questions are becoming increasingly possible to answer with the availability of the methods described in the previous sections. However, in order to examine these questions methods must be found for characterizing and classifying the extracted anatomy. Both qualitative and quantitative approaches are being developed.
The classical approach to characterizing anatomy is for the human biologist to group individual structures into various classes based on perceived patterns. This approach is still widely used throughout science since the computer has yet to match the pattern recognition abilities of the human brain.
An example classification at the cellular level is the 60-80 morphological cell types that form the basis for understanding the neural circuitry of the retina (which is an outgrowth of the brain) [111]. At the macroscopic level Ono has developed an atlas of cerebral sulci that can be used to characterize an individual brain based on sulcal patterns [112].
If these and other classifications are given systematic names and are added to the symbolic ontologies described in section 2.3 they can be used for “intelligent” index and retrieval, after which quantitative methods can be used for more precise characterization of structure-function relationships.
Quantitative characterization of anatomy is often called morphometrics [113] or computational neuroanatomy [114]. Quantitative characterization permits more subtle classification schemes than are possible with qualitative methods, leading to new insights into the relation between structure and function, and between structure and disease [94, 115].
For example, at the ultrastructural level stereology, which is a statistical method for estimating from sampled data the distribution of structural components in a volume [116], is used to estimate the density of objects such as synapses in image volumes reconstructed from serial electron micrographs [18].
At the cellular level Ascoli et al. are developing the L-neuron project, which attempts to model dendritic morphology by a small set of parameterized generation rules, where the parameters are sampled from distributions determined from experimental data [114]. The resulting dendritic models capture a large set of dendritic morphological classes from only a small set of variables. Eventually the hope is to generate virtual neural circuits that can simulate brain function.
At the macroscopic level landmark-based methods have shown changes in the shape of the corpus callosum associated with schizophrenia that are not obvious from visual inspection [117]. Probabilistic atlas-based methods are being used to characterize growth patterns and disease-specific structural abnormalities in diseases such as Alzheimer’s and schizophrenia [118]. As these techniques become more widely available to the clinician they should permit early diagnosis and hence potential treatment for these debilitating diseases.
Perhaps a greater revolution than structural imaging has come about with methods that reveal the functioning of the brain, particularly cognitive function at the macroscopic level (i.e., the thinking brain). It is now routinely possible to put a normal subject in a scanner, to give the person a cognitive task, such as counting or object recognition, and to observe which parts of the brain light up. This unprecedented ability to observe the functioning of the living brain opens up entirely new avenues for exploring how the brain works.
Functional modalities can be classified as image-based or non-image based. In both cases it is taken as axiomatic that the functional data must be mapped to the individual subject’s anatomy, where the anatomy is extracted from structural images using techniques described in the previous section. Once mapped to anatomy, the functional data can be integrated with other functional data from the same subject, and with functional data from other subjects whose anatomy has been related to a template or probabilistic atlas. Techniques for generating, mapping and integrating functional data are part of the field of Functional Brain Mapping, which has become very active in the past few years, with several conferences [119] and journals [120, 121] devoted to the subject.
Image-based functional data generally come from scanners that generate relatively low resolution volume arrays depicting spatially-localized activation. For example, positron emission tomography (PET) [122, 123] and magnetic resonance spectroscopy (MRS) [124] reveal the uptake of various metabolic products by the functioning brain; and functional magnetic resonance imaging (fMRI) reveals changes in blood oxygenation that occur following neural activity [123]. The raw intensity values generated by these techniques must be processed by sophisticated statistical algorithms to sort out how much of the observed intensity is due to cognitive activity and how much is due to background noise.
As an example, one approach to fMRI imaging is the boxcar paradigm applied to language mapping [125]. The subject is placed in the MRI scanner and told to silently name objects shown at 3 second intervals on a head-mounted display. The actual objects (“on” state) are alternated with nonsense objects (“off” state) , and the fMRI signal is measured during both the on and the off states. Essentially the voxel values at the off (or control) state are subtracted from those at the on state. The difference values are tested for significant difference from non-activated areas, then expressed as t-values. The voxel array of t-values can be displayed as an image.
A large number of alternative methods have been and are being developed for acquiring and analyzing functional data [126]. The output of most of these techniques is a low-resolution 3-D image volume in which each voxel value is a measure of the amount of activation for a given task. The low-resolution volume is then mapped to anatomy by linear registration to a high-resolution structural MR dataset, using one of the linear registration techniques described in section 2.1.2.
Many of these and other techniques are implemented in the SPM program [127], the AFNI program [128], the Lyngby toolkit [129], and several commercial programs such as Medex [51] and BrainVoyager [33]. The FisWidgets project at the University of Pittsburgh is developing a set of Java wrappers for many of these programs that allow customized creation of graphical user interfaces in an integrated desktop environment [130].
In addition to the image-based functional methods there are an increasing number of techniques that do not directly generate images. The data from these techniques are generally mapped to anatomy, then displayed as functional overlays on anatomic images.
For example, cortical stimulation mapping (CSM) is a technique for localizing functional areas on the exposed cortex at the time of neurosurgery. In our own work the technique is used to localize cortical language areas so that they can be avoided during the resection of a tumor or epileptic focus [131]. Following removal of a portion of the skull (craniotomy) the patient is awakened and asked to name common images shown on slides. During this time the surgeon applies a small electrical current to each of a set of numbered tags placed on the cortical surface. If the patient is unable to name the object while the current is applied the site is interpreted as essential for language and is avoided at surgery. In this case the functional mapping problem is how to relate these stimulation sites to the patient’s anatomy as seen on an MRI scan.
Our approach, which we call visualization-based mapping [21, 32], is to acquire image volumes of brain anatomy (MRI), cerebral veins (MRV) and cerebral arteries (MRA) prior to surgery, to segment the anatomy, veins and arteries from these images, and to generate a surface-rendered 3-D model of the brain and its vessels that matches as closely as possible the cortical surface as seen at neurosurgery. A visual mapping program then permits the user to drag numbered tags onto the rendered surface such that they match those seen on the intraoperative photograph. The program projects the dragged tags onto the reconstructed surface, and records the xyz image-space coordinates of the projections, thereby completing the mapping.
The real goal of functional neuroimaging is to observe the actual electrical activity of the neurons as they perform various cognitive tasks. fMRI, MRS and PET do not directly record electrical activity. Rather, they record the results of electrical activity, such as (in the case of fMRI) the oxygenation of blood supplying the active neurons. Thus, there is a delay from the time of activity to the measured response. In other words these techniques have relatively poor temporal resolution. Electro-encephalography (EEG) or magnetoencephalography (MEG), on the other hand, are more direct measures of electrical activity since they measure the electromagnetic fields generated by the electrical activity of the neurons. Current EEG and MEG methods involve the use of large arrays of scalp sensors, the output of which are processed in a similar way to CT in order to localize the source of the electrical activity inside the brain. In general this “source localization problem” is under constrained, so information about brain anatomy obtained from MRI is used to provide further constraints [132].
The goal of many of the techniques described in the previous sections is to develop methods for integrating structural and functional brain image data through spatial and symbolic representations of anatomy. As described in section 1 this is one of the major goals of the HBP. Another goal described in that section is to develop Internet-based methods for sharing and disseminating the integrated information.
One way information can be shared is through remote visualization and manipulation of raw and processed images. For example, in our own work we have created a web-based visualization applet that permits 3-D viewing of the results of our visualization-based approach to brain mapping [133]. Similar remote image viewers are being developed by other members of the HBP [134-137].
Two groups permit Internet control of expensive microscopy systems. The Iscope project at the University of Tennessee permits control of a light microscope for viewing slides of a mouse brain atlas [83], whereas the National Center for Microscopy and Imaging Research is implementing web control of an electron microscope [138].
A more comprehensive way for sharing information is to develop backend database systems that allow Web-based queries of the processed and integrated data. As these systems are developed the hope is that links can be established between individual brain information systems so as to promote knowledge discovery and the development of distributed, large-scale models of brain function that will help establish a “wholeness” in neuroscience.
This research area is also active in the HBP, but not as much progress has been made as in the other areas of tool development and methods for integrating data. There seem to be four main reasons for this: 1) the development of information systems depends on progress in tool development and on methods for integrating data in a common anatomical framework, 2) not enough informatics and database experts have become involved in the HBP, 3) not enough content has yet been made available for database experts to “play” with, and 4) the development of information systems raises additional non-trivial issues related to security and intellectual property.
As shown in the previous sections a large amount of effort is going into solving the first problem (tools and integration). We believe that the second problem (not enough informatics experts) arises partly because informatics and computer science investigators are not sufficiently aware of the rich set of problems posed by the HBP. Hopefully, this review article will help in this area. The third problem (not enough content) is also slowly being addressed by ongoing efforts. More content will help attract more database and informatics experts. The fourth problem (security and intellectual property), which is very familiar to clinical informatics workers, is starting to be addressed by those who are developing brain information systems. That this problem is not at all trivial has been noted in several recent articles about the HBP [139, 140].
The information systems that are currently in active development in the HBP can more or less be classified as experiment management systems for local data, systems for handling published results, and raw data repositories analogous to GenBank for gene sequences [141]. This last is the most controversial. A listing of many of the current neuroscience database systems is available [142].
In our work we use the term, “Experiment Management System” (EMS) to refer to an information system that keeps track of the results and protocols for specific experiments of interest to an individual or lab [143]. At the least such a system should permit organization of and access to data of interest to the local individual or group. An EMS usually evolves from a collection of computer files or paper records that has become too unwieldy for even local management. An EMS can therefore be appealing to neuroscientists because it solves an immediate problem of interest to them. If the data are made available on the web, and if appropriate safeguards are implemented to prevent unauthorized access to the data, an EMS can permit data sharing among distributed collaborators. In addition, if at least some of the data integration methods described in the previous sections are implemented, the local EMS will be more amenable to wider sharing in a federated database.
Our HBP work follows this approach: we are developing image processing tools and an EMS of interest to a specific set of neuroscience users, while developing or incorporating integration methods that will later permit more widespread data sharing. We believe that this “bottom-up” approach is a viable complement to the top-down approaches of other HBP efforts if the tools and methods can be “cloned” for use by other groups, and if “hooks” can be provided for later integration of these and other efforts in federated information systems.
The main idea of an EMS is that metadata (data about data) provide indices into individual data files, such as images or segmented anatomy, which are the input or output of various image-processing tools. A simple spreadsheet is often the first place where these metadata are stored. As the need for better search becomes evident the spreadsheet may be imported into a local database such as Microsoft Access, and as the need for remote sharing and more robust data management becomes clear the data may be imported to a higher-end database system that is interfaced to the web. Many commercial database systems provide web-accessible views of the database.
In our own work we have developed an open source Experiment Management System Building Environment (EMSBE), and have used the toolkit to implement an EMS for our HBP work [143, 144]. The toolkit, which is called WIRM (Web Interfacing Repository Manager) is a set of perl APIs that can be interfaced to any back-end relational database, and that can be called by a perl programmer to dynamically generate web views of metadata and associated datafiles [145]. Any of the extensive set of perl modules in the comprehensive perl archive network (CPAN, www.cpan.org) can be used in conjunction with WIRM to provide extensive backend processing of data, including image conversion, import of spreadsheet data, and XML parsing. When coupled with Java applets for viewing 3-D or time varying data located on the server, the resulting systems can provide remote access, visualization, and manipulation of most data of interest to neuroscientists. A similar open source toolkit called Zope (www.zope.org) [146], which is written in Python as opposed to perl, is the basis for a project to develop an open source medical record system (www.freepm.org).
We have used WIRM to create a web-accessible experiment management system for organizing, visualizing and sharing language map data, much of which is in the form of 2-D and 3-D images [143, 147]. The system is currently in use in three widely scattered labs at the University of Washington.
A similar EMS called SUMS (Surface Management System) is being developed at Washington University to handle images processed by the Surefit and Caret programs [31], and a system being developed by Wong et al. at UCSF handles images and other data associated with neurosurgical treatment of epilepsy [148].
Another example of what we call an EMS (our terminology) is the Brain Image Database (BRAID) [137, 149, 150] being developed at Johns Hopkins for management and evaluation of “Image-based clinical trials” [150]. The system, like some others in the HBP [151, 152], is implemented in the Illustra (now Informix now IBM) object-relational database system, which permits the development of specialized “datablades” for image processing and analysis. BRAID is being developed to facilitate lesion-deficit studies in large clinical trials. Patient MR image volumes are warped to one of several labeled human atlases [102], thereby permitting automatic identification of anatomical structures (subject to the limitations discussed in section 2.5). Lesions from patient MR images are manually delineated and stored in the database, along with the warped and labeled images. Analytical tools embedded in the database, and accessed through extended SQL, permit rapid computation of structure-function correlations, as for example, a correlation between lesions in the optic radiations and contra lateral visual field defect [149], or a correlation between traumatic injuries to the right putamen and an increase in attention deficit disorders in children [153].
Other groups in the HBP are also developing what we call EMS’s, but these generally do not involve images to much extent [73, 154, 155]. Of particular relevance for eventual data sharing is the electrophysiological EMS under development by Gardner et al. [73]. As part of that effort Gardner has proposed BDML (Biophysical Description Markup Language), an XML-based common format for data exchange. Although initially in use for sharing of electrophysiological data, BDML was designed from the start to encompass other kinds of brain data, including images. A few other HBP groups have begun experimenting with BDML to see if it is relevant to their own data.
There are also some initial efforts to develop federated database systems that can tie together individual EMS’s [156], although there appear to be few if any published efforts to explore advanced database issues such as intelligent retrieval or content-based retrieval. We believe that these kinds of efforts represent the next stage of the HBP. They will become more widespread as individual EMS’s are developed, as the thorny problems of data integration and intellectual property become ironed out, and as mainstream database experts become interested in the HBP.
At the other end of the spectrum from individual EMS’s are efforts to essentially index published literature in more meaningful ways than simple term searches in Medline. Like individual EMS’s, which deal only with data that the individual researcher wants to share with his or her collaborators, this kind of effort is not controversial because it simply provides enhanced access to public data. The enhancements generally make use of some of the integration methods described in section 2.5 to provide anatomically based queries based on a template atlas, often coupled with a controlled vocabulary.
An early example of such an atlas-based system was the Brain Browser, a Mac HyperCard application that permitted scientists to map experimental results onto a rat brain atlas template [65]. A more recent effort is the Mouse Brain Library at Tennessee, which contains atlas sections and metadata from inbred mouse strains, for use in mapping genetic data [83].
An early, and still one of the few Web-accessible atlas systems that includes mapped data as well as images, is the BrainMap database developed by Fox et al. at the University of Texas [157]. In this system data are integrated primarily according to Talairach coordinates, which are in turn linked to anatomical names. Web forms are used to enter a query as a Boolean combination of constraints such as Talairach coordinates, anatomical names, publication source, laboratory of origin, and imaging protocol. The system returns references to published literature that meet the search constraints. Registered users can retrieve experimental data associated with the data, and an author mode permits authors to input their published results into the system.
The Fox database uses linear Talairach coordinates to integrate data. In contrast, the Bowden brain information system uses the Bookstein landmark-based non-linear registration method [100] to warp 2-D images from the literature to a brain atlas template, which has been labeled by terms from NeuroNames [62]. The template atlas takes the place of the earth in a commercial Geographic Information System (GIS) [158]. When complete the system will permit a web user to type a NeuroName or click on an area of the template atlas to specify a given structure, to add additional constraints such as neurotransmitter type, and to retrieve all maps that have been warped to the template. These maps in turn will contain links to the original articles.
The most controversial HBP efforts are aimed at the establishment of raw data repositories that are widely accessible, in analogy to highly successful bioinformatics efforts such as GenBank [141] or the protein data bank (PDB)[159]. One reason for the controversy is that brain data are seen by most neuroscientists as being much more complex than the relatively simple linear sequences or 3-D coordinate files represented in GenBank or PDB, and in fact it is not even clear how the data should be represented and which data should be shared. As evidenced by section 2.5 it is not clear how to relate data from multiple subjects, let alone at different levels of anatomical granularity. In addition many neuroscientists express concern that public data will not have adequate quality control, and that data will not be adequately protected from unauthorized use.
Perhaps because of these issues there are only a few attempts to establish raw data repositories. One example of such an attempt is the Dartmouth fMRI Data Center [152], which is being developed as a repository for organizing fMRI image datasets submitted by multiple authors. When the project was first discussed it was proposed that authors of articles to certain journals be required to submit their fMRI images to the repository as a condition of publication, again in analogy with the requirement for authors of papers about gene sequences to submit their sequences to GenBank. This proposal generated a fierce reaction from other HBP and neuroscience researchers [140], with the result that most journals retracted the requirement. Nevertheless, there are many researchers, including the director of the HBP [139], who feel strongly that neuroscience must begin to share raw data if the field is to advance. It may be that more advanced database methods, such as federated databases [156] or peer-to-peer databases ala Napster [160], will be required in order to achieve this goal.
In this review we have tried to summarize many of the projects in the Human Brain Project, emphasizing the ubiquity of images in most of them. The resulting imaging informatics problems of image generation, management, processing and visualization are not unique to the brain, yet because of the variety and sheer numbers of brain images, the problems are at least as varied and challenging as any that arise from other areas of the body. Therefore, solutions to these problems should have widespread applicability outside the brain or even biomedicine.
Similarly, we hope we have demonstrated the central role that neuroanatomy plays as an organizational framework, not only for brain images, but also for most other neuroscience data as well. As we noted earlier, a case for this central role of anatomy can be made throughout all of biomedicine, which has prompted to us to define structural informatics as a sub field of biomedical informatics for dealing specifically with information about the physical organization of the body.
As noted in section 1 the brain presents very challenging research problems in structural informatics, in the areas of spatial and symbolic representation, brain segmentation, and especially anatomic variation, yet considerable progress has been made in these areas by HBP and other brain researchers. Since a central tenet of structural informatics is that patterns of physical organization repeat themselves throughout the hierarchy from macroscopic anatomy to molecules, it is highly likely that these results will find use in other areas of the body. One of the main reasons to define a field is to promote this kind of cross-fertilization of techniques.
This potential for cross-fertilization is one of the main motivators for defining the field of neuroinformatics, which is the field that has the most interest in achieving the goals of the HBP. The goals of the HBP to “database the brain” [2] are so ambitious as to practically dwarf the goals of the Human Genome Project. Many have argued (and they may be right) that the goals are too ambitious to be practical, and that resources would be better spent on specific neuroscience-driven projects that involve the use of computers. But the critics may also be wrong. Whether we get to the moon or not may be less important than the side effects that can result from such an endeavor. Just as medical informatics has evolved to promote cross-fertilization among informaticists and health scientists, so too could neuroinformatics promote cross-fertilization among informaticists and neuroscientists. National initiatives such as the HBP can foster these kinds of collaborations by funding interdisciplinary projects that bring together experts in areas such as imaging informatics, structural informatics, neuroscience, radiology, computer science, and information science.
For these kinds of efforts to succeed each kind of expert needs to become educated in the research problems of the other field, in enough detail so that they see how the problems apply to their own field. This paper is as much as anything an attempt to educate the wider biomedical and health informatics community, and the computer scientists and other technology experts that are associated with this community, in just a few of the informatics and computer science challenges associated with this, the problem of understanding the most complex entity known. The paper will have succeeded if it inspires just a few of them to become involved in this grand challenge for the 21st century.
This work was funded by Human Brain Project grant MH/DC02310. In preparing this review we found several web sites to be of particular use as starting points for further exploration. These sites are the HBP home page http://www.nimh.nih.gov/neuroinformatics/index.cfm, the HBP list of funded grants http://www.nimh.nih.gov/neuroinformatics/researchgrants.cfm, a list of software tools developed by Brain Project grantees, maintained by David Kennedy at Harvard http://www.cma.mgh.harvard.edu/tools/index.php, and a list of neuroscience databases maintained by Rolf Kotter at Dusseldorf, for publication in the autumn 2001 issue of the Philosophical Transactions of the Royal Society, Series B: Biological Sciences http://www.hirn.uni-duesseldorf.de/rk/neurodat.htm.
1. Koslow SH, Huerta MF, editors. Neuroinformatics: an overview of the Human Brain Project. Mahwah, New Jersey: Lawrence Erlbaum; 1997.
2. Chicurel M. Databasing the brain. Nature 2000;406:822-825.
3. Kahn J. Let's make your head interactive. Wired 2001 August:107-115.
4. Pechura C, Martin J. Mapping the brain and its functions: integrating enabling technologies into neuroscience research. Institute of Medicine Pub 91-08: National Academy Press; 1991.
5. Heurta M, Koslow S. Neuroinformatics: opportunities across disciplinary and national borders. Neuroimage 1996;4:S4-S6.
6. Rosse C, Mejino JL, Modayur BR, Jakobovits RM, Hinshaw KP, Brinkley JF. Motivation and organizational principles for anatomical knowledge representation: the Digital Anatomist symbolic knowledge base. Journal of the American Medical Informatics Association 1998;5(1):17-40.
7. Brinkley JF. Structural informatics and its applications in medicine and biology. Academic Medicine 1991;66(10):589-591.
8. Kulikowski CA. Medical imaging informatics: challenges of definition and integration. Journal of the American Medical Informatics Association 1997;4(3):252-253.
9. Ambrogi L. Manual of Histologic and Special Staining Techniques. 2nd ed. New York: Mc-Graw-Hill; 1960.
10. Peters A, Palay S, Webster H. The Fine Structure of the Nervous System: Neurons and their Supporting Cells. 3rd ed. New York: Oxford Press; 1991.
11. Crusio WE, Gerlai RT, editors. Handbook of molecular-genetic techniques for brain and behavior research. Amsterdam; New York: Elsevier; 1999.
12. Zimmerman RA, Gibby WA, Carmody RF, editors. Neuroimaging: clinical and physical principles. New York: Springer; 2000.
13. Jacobs RE, Ahrens ET, Dickenson ME, Laidlaw D. Towards a microMRI atlas of mouse development. Comput. Med. Imaging Graph. 1999;23(1):15-24 http://waggle.gg.caltech.edu/hbp/index.html.
14. Wilson T. Confocal Microscopy. San Diego: Academic Press Ltd.; 1990.
15. Perkins G, Renken C, Martone ME, Young SJ, Ellisman M, Frey T. Electron tomography of neuronal mitochondria: Three-dimensional structure and organization of cristae and menbrane contacts. J. Structural Biology 1997;119(3):260-272.
16. Prothero JS, Prothero JW. Three-dimensional reconstruction from serial sections IV. The reassembly problem. Computers and Biomedical Research 1986;19(4):3610373.
17. Spitzer VM, Whitlock DG. The Visible Human Dataset: the anatomical platform for human simulation. Anat Rec 1998;253(2):49-57.
18. Fiala JC, Harris KM. Extending unbiased stereology of brain ultrastructure to three-dimensional volumes. J Am Med Ass 2001;8(1):1-16.
19. Soto GE, Young SJ, MArtone ME, Deerinick TJ, Lamont SL, Carragher BO, et al. Serial section electron tomography: a method for three-dimensional reconstruction of large structures. Neuroimage 1994;1:230-243 http://ncmir.ucsd.edu/abstracts.html#Neuroimage_1.
20. Agrawal M, Harwood D, Duraiswami R, Davis LS, Luther PW. Three-dimensional ultrastructure from transmission electron micropscope tilt series. In: Proceedings, Second Indian Conference on Vision, Graphics and Image Processing. Bangaore, India; 2000. http://www.umiacs.umd.edu/~mla/tem/icvgipfinal.pdf.
21. Modayur B, Prothero J, Ojemann G, Maravilla K, Brinkley JF. Visualization-based mapping of language function in the brain. Neuroimage 1997;6:245-258.
22. Collins DL, Neelin P, Peters TM, Evans AC. Automatic 3-D intersubject registration of MR volumetric data in standardized Talairach space. J. Computer Assisted Tomography 1994;18(2):192-205.
23. Woods RP, Cherry SR, Mazziotta JC. Rapid automated algorithm for aligning and reslicing PET images. J. Comp. Assisted Tomogr. 1992;16:620-633.
24. Woods RP, Mazziotta JC, Cherry SR. MRI-PET registration with automated algorithm. J. Comp. Assisted Tomogr. 1993;17:536-546.
25. Ashburner J, Friston KJ. Multimodal image coregistration and partitioning - a unified framework. Neuroimage 1997;6(3):209-217.
26. MacDonald D. Register: McConnel Brain Imaging Center, Montreal Neurological Institute; 1993.
27. Lichtenbelt B, Crane R, Naqvi S. Introduction to Volume Rendering. Upper Saddle River, N.J.: Prentice Hall; 1998.
28. Dale AM, Fischl B, Sereno MI. Cortical surface-based analysis. I. Segmentation and surface reconstruction. Neuroimage 1999;9(2):179-94.
29. Collins DL, Holmes DJ, Peters TM, Evans AC. Automatic 3-D model-based neuroanatomical segmentation. Hum Brain Mapp 1995;3:190-208.
30. MacDonald D, Kabani N, Avis D, Evans AC. Automated 3-D extraction of inner and outer surfaces of cerebral cortex from MRI. Neuroimage 2000;12(3):340-56.
31. Van Essen DC, Drury HA, Dickson J, Harwell J, Hanlon D, Anderson CH. An integrated software suite for surface-based analysis of cerebral cortex. J Am Med Ass. In Press 2001 http://stp.wustl.edu.
32. Hinshaw KP, Poliakov AV, Martin RF, Moore EB, Shapiro LG, Brinkley JF. Shape-based cortical surface segmentation in a workflow environment for visualization brain mapping. Submitted 2001.
33. Brain Innovation B.V. BrainVoyager. http://www.BrainVoyager.de/; 2001.
34. Subramaniam B, Hennessey JG, Rubin MA, Beach LS, Reiss AL. Software and methods for quantitative imaging in neuroscience: the Kennedy Krieger Institute Human Brain Project. In: Koslow SH, Huerta MF, editors. Neuroinformatics: an overview of the Human Brain Project. Mahwah, New Jersey: Lawrence Erlbaum; 1997. p. 335-360. http://www-cap.stanford.edu/research/neuroimaging/projects/hbp.html.
35. Kennedy D. Internet Brain Segmentation Repository. http://neuro-www.mgh.harvard.edu/cma/ibsr/; 2001.
36. Prothero JS, Prothero JW. Three-dimensional reconstruction from serial sections: I. A portable microcomputer-based software package in Fortran. Computers and Biomedical Research 1982;15:598-604.
37. Foley JD. Computer graphics: Principles and Practice. Reading, Mass.: Addison-Wesley; 2001.
38. Shapiro LG, Stockman GC. Computer Vision. Upper Saddle River, N.J.: Prentice Hall; 2001.
39. Choi HS, Haynor DR, Kim Y. Partial volume tissue classification of multichannel magnetic resonance images - a mixel model. IEEE Trans. Med. Imaging 1991;10(3):395-407.
40. Zijdenbos AP, Evans AC, Riahi F, Sled J, Chui J, Kollokian V. Automatic quantification of multiple sclerosis lesion volume using stereotactic space. In: Proc. 4th Int. Conf. on Visualization in Biomedical Computing. Hamburg; 1996. p. 439-448.
41. Davatzikos C, Bryan RN. Using a deformable surface model to obtain a shape representation of the cortex. IEEE Trans. Medical Imaging 1996;15(6):785-795.
42. Haralick RM. Mathematical Morphology: University of Washington; 1988.
43. Sandor S, Leahy R. Surface-based labeling of cortical anatomy using a deformable atlas. IEEE Trans. Med. Imaging 1997;16(1):41-54.
44. Lorensen WE, Cline HE. Marching cubes: a high resolution 3-D surface construction algorithm. ACM Computer Graphics 1987;21(4):163-169.
45. Kass M, Witkin A, Terzopoulos D. Snakes: active contour models. International Journal of Computer Vision 1987;1(4):321-331.
46. Brinkley JF. Hierarchical geometric constraint networks as a representation for spatial structural knowledge. In: Proceedings, 16th Annual Symposium on Computer Applications in Medical Care; 1992. p. 140-144.
47. Brinkley JF. Knowledge-driven ultrasonic three-dimensional organ modelling. PAMI 1985;PAMI-7(4):431-441.
48. Brinkley JF. A flexible, generic model for anatomic shape: application to interactive two-dimensional medical image segmentation and matching. Computers and Biomedical Research 1993;26:121-142.
49. Fischl B, Sereno MI, Dale AM. Cortical surface-based analysis. II: Inflation, flattening, and a surface-based coordinate system. Neuroimage 1999;9(2):195-207.
50. Wellcome Department of Cognitive Neurology. Statistical Parametric Mapping. http://www.fil.ion.ucl.ac.uk/spm/; 2001.
51. Sensor Systems Inc. MedEx. http://medx.sensor.com/products/medx/index.html; 2001.
52. Brinkley JF, Wong BA, Hinshaw KP, Rosse C. Design of an anatomy information system. Computer Graphics and Applications 1999;19(3):38-48.
53. Wong BA, Rosse C, Brinkley JF. Semi-automatic scene generation using the Digital Anatomist Foundational Model. In: Proceedings, American Medical Informatics Association Fall Symposium. Washington, D.C.; 1999. p. 637-641.
54. Rosse C, Tuttle MS. Explaining the brain to a computer. In: Human Brain Project Annual Meeting; 2001. http://www.nimh.nih.gov/neuroinformatics/rosse2001.cfm.
55. Gardner D, Abato M, Knuth KH, DeBellis R, Gardner EP. A functional ontology for neuroinformatics. In: Human Brain Project Annual Meeting; 2001. http://www.nimh.nih.gov/neuroinformatics/gardner2001.cfm.
56. International Anatomical Nomenclature Committee. Nomina Anatomica. 6th ed. Edinburgh: Churchill Livingstone; 1989.
57. Federative Committee on Anatomical Terminology. Terminologia Anatomica. Stuttgart: Thieme; 1998.
58. National Library of Medicine. Medical Subject Headings - Annotated Alphabetic List. Bethesda, MD: U.S. Department of Health and Human Services, Public Health Service; 1999.
59. Spackman KA, Campbell KE, Cote RA. SNOMED-RT: A reference terminology for health care. In: Masys DR, editor. Proceedings, AMIA Annual Fall Symposium. Philadelphia: Hanley and Belfus; 1997. p. 640-644.
60. Schultz EB, Price C, Brown PJB. Symbolic anatomic knowledge representation in the Read Codes Version 3: Structure and application. J. Am. Med. Inform. Assoc. 1997;4:38-48.
61. Rector AL, Nowlan WA, Glowinski A. Goals for concept representation in the GALEN project. In: Safran C, editor. Proceedings of the 17th Annual Symposium on Computer Applications in Medical Care (SCAMC 93). New York: McGraw Hill; 1993. p. 414-418.
62. Bowden DM, Martin RF. Neuronames brain hierarchy. Neuroimage 1995;2:63-83.
63. Lindberg DAB, Humphreys BL, McCray AT. The unified medical language system. Methods of Information in Medicine 1993;32(4):281-291.
64. Paxinos G, Watson C. The rat brain in stereotaxic coordinates. San Diego: Acedemic Press; 1986.
65. Bloom FE, Young WG. Brain Browser. New York: Academic Press; 1993.
66. Swanson LW. Brain maps: structure of the rat brain. Amsterdam; New York: Elsevier; 1992.
67. Franklin KBJ, Paxinos G. The mouse brain in stereotactic coordinates. San Diego: Academic Press; 1997.
68. Rosse C. Terminologia Anatomica; considered from the perspective of next-generation knowledge sources. Clinical Anatomy 2000;14:120-133 http://sig.biostr.washington.edu/share/pubs/CRTAnat.pdf.
69. Rosse C, Shapiro LG, Brinkley JF. The Digital Anatomist foundational model: principles for defining and structuring its concept domain. In: Proceedings, American Medical Informatics Association Fall Symposium. Orlando, Florida; 1998. p. 820-824.
70. Musen MA. Domain ontologies in software engineering: use of Protege with the EON architecture. Methods Inf Med 1998;37(4-5):540-50.
71. Mejino JLV, Noy NF, Musen MA, Brinkley JF, Rosse C. Representation of structural relationships in the foundational model of anatomy. In: Proceedings, AMIA Fall Symposium. In Press. Washington, DC; 2001.
72. Martin RF, Mejino JLV, Bowden DM, Brinkley JF, Rosse C. Foundational model of neuroanatomy: implications for the Human Brain Project. In: Proc AMIA Annu Fall Symp. Washington, DC In Press; 2001.
73. Gardner D, Knuth KH, Abato M, Erde SM, White T, DeBellis R, et al. Common data model for neuroscience data and data model exchange. J Am Med Ass 2001;8(1):17-33.
74. Sundsten JW, Conley DM, Ratiu P, Mulligan KA, Rosse C. Digital Anatomist web-based interactive atlases. http://www9.biostr.washington.edu/da.html; 2000.
75. Brinkley JF, Bradley SW, Sundsten JW, Rosse C. The Digital Anatomist information system and its use in the generation and delivery of Web-based anatomy atlases. Computers and Biomedical Research 1997;30:472-503.
76. Hohne KH, Pflesser B, Riemer M, Schiemann T, Schubert R, Tiede U. A new representation of knowledge concerning human anatomy and function. Nature Medicine 1995;1(6):506-510.
77. Hohne K, Bomans M, Pommert A, Riemer M, Schiers C, Tiede U, et al. 3-D visualization of tomographic volume data using the generalized voxel model. The Visual Computer 1990;6(1):28-36.
78. Stensaas SS, Millhouse OE. Atlases of the Brain. http://medstat.med.utah.edu/kw/brain_atlas/credits.htm; 2001.
79. Johnson KA, Becker JA. The Whole Brain Atlas. http://www.med.harvard.edu/AANLIB/home.html; 2001.
80. Swanson LW. Brain Maps: Structure of the Rat Brain. 2nd ed. New York: Elsevier Science; 1999.
81. Martin RF, Bowden DM. Primate Brain Maps: Structure of the Macaque Brain. New York: Elsevier Science; 2001.
82. Fougerousse F, Bullen P, Herasse M, Lindsay S, Richard I, Wilson D, et al. Human-mouse differences in the embryonic expression of developmental control genes and disease genes. Human Molecular Genetics 2000;9(2):165-173.
83. Rosen GD, Williams AG, Capra JA, Connolly MT, Cruz B, Lu L, et al. The Mouse Brain Library @ www.mbl.org. In: Int. Mouse Genome Conference 14; 2000. p. 166. http://www.nervenet.org/papers/MBLabst2000.html.
84. Toga AW, Ambach KL, Schluender S. High-resolution anatomy from in situ human brain. Neuroimage 1994;1(4):334-344.
85. Toga AW, Santori EM, Hazani R, Ambach K. A 3-D digital map of rat brain. Brain Research Bulletin 1995;38(1):77-85.
86. Toga AW. UCLA Laboratory for Neuro Imaging (LONI). http://www.loni.ucla.edu/; 2001.
87. Dhenain M, Ruffins SW, Jacobs RE. Three-dimensional digital mouse atlas using high-resolution MRI. Dev. Biol. 2001;232(2):458-470 http://mouseatlas.caltech.edu/.
88. Talairach J, Tournoux P. Co-planar stereotaxic atlas of the human brain. New York: Thieme Medical Publishers; 1988.
89. Van Essen DC, Drury HA. Structural and functional analysis of human cerebral cortex using a surface-basec atlas. J. Neuroscience 1997;17(18):7079-7102.
90. Schaltenbrand G, Warren W. Atlas for Stereotaxy of the Human Brain. Stuttgart: Thieme; 1977.
91. Drury HA, Van Essen DC. Analysis of functional specialization in human cerebral cortex using the visible man surface based atlas. Hum Brain Mapp 1997;5:233-237.
92. Hohne KH, Bomans M, Riemer M, Schubert R, Tiede U, Lierse W. A volume-based anatomical atlas. IEEE Computer Graphics and Applications 1992:72-78.
93. Caviness VS, Meyer J, Makris N, Kennedy DN. MRI-based topographic parcellation of human neocortex: an anatomically specified method with estimate of reliability. Journal of Cognitive Neuroscience 1996;8(6):566-587.
94. Toga AW, Thompson PW. Maps of the brain. Anatomical Record (New Anat.) 2001;265:37-53.
95. Christensen GE, Miller MI, Vannier MW. Individualizing neuroanatomical atlases using a massively parallel computer. IEEE Computer 1996;29(1):32-38.
96. Kikinis R, Shenton ME, Iosifescu DV, McCarley RW, Saiviroonporn P, Hokama HH, et al. A digital brain atlas for surgical planning, model-driven segmentation, and teaching. IEEE Trans. Visualization and Computer Graphics 1996;2(3):232-241.
97. Gee JC, Reivich M, Bajcsy R. Elastically deforming 3D atlas to match anatomical brain images. J. Computer Assisted Tomography 1993;17(2):225-236.
98. Kjems U, Strother SC, Anderson JR, Law I, Hansen LK. Enhancing the multivariate signal of 15O water PET studies with a new nonlinear neuroanatomical registration algorithm. IEEE Trans. Med. Imaging 1999;18:301-319 http://hendrix.imm.dtu.dk/software/kjemswarp/kjemswarp.html.
99. Bookstein FL, Green WDK. Edgewarp 3D: A preliminary manual. ftp://brainmap.med.umich.edu/pub/edgewarp3.1/manual.html; 1998.
100. Bookstein FL. Principal warps: thin-plate splines and the decomposition of deformations. IEEE Trans. Pattern Anal. Mach. Intell. 1989;11(6):567-585.
101. Thompson P, Toga AW. A surface-based technique for warping three-dimensional images of the brain. IEEE Trans. Medical Imaging 1996;15(4):402-417.
102. Davatzikos C. Spatial transformation and registration of brain images using elastically deformable models. Computer Vision and Image Understanding 1997;66(2):207-222 http://ditzel.rad.jhu.edu/papers/cviu97.pdf.
103. Van Essen DC, Drury HA, Joshi S, Miller MI. Functional and structural mapping of human cerebral cortex: solutions are in the surfaces. Proc. National Academy of Sciences 1998;95:788-795.
104. Hurdal MK, Stephenson K, Bowers P, Sumners DW, Rottenberg DA. Coordinate systems for conformal cerebellar flat maps. Neuroimage 2000;11(5):S467 http://www.pet.med.va.gov:8080/papers/abstracts_posters/HBM2000/mhurdal_HBM2000.html.
105. Mazziotta J, Toga A, Evans A, Fox P, et al. A four-dimensional probabilistic atlas of the human brain. J Am Med Ass. In Press 2001.
106. Evans AC, Collins DL, Neelin P, MacDonald D, Kamber M, Marrett TS. Three-dimensional correlative imaging: applications in human brain mapping. In: Thatcher RW, Hallett M, Zeffiro T, John ER, Heurta M, editors. Functional Neuroimaging: technical foundations. San Diego: Academic Press; 1994. p. 145-162.
107. Martin RF, Poliakov AV, Mulligan KA, Corina DP, Ojemann GA, Brinkley JF. Multi-patient mapping of language sites on 3-D brain models. Neuroimage 2000;11(5):S534.
108. Lancaster JL, Woldorff MG, Parsons LM, Liotti M, Freitas CS, Rainey L, et al. Automated Talairach atlas labels for functional brain mapping. Hum Brain Mapp 2000;10(3):120-131 http://ric.uthscsa.edu/projects/talairachdaemon.html.
109. Thompson PM, Toga AW. Detection, visualization and animation of abnormal anatomic structure with a deformable probalistic brain atlas based on random vector field transformations. Med Image Anal 1997;1:271-294.
110. Christensen GE, Rabbitt RD, Miller MI. Deformable templates using large deformation kinematics. IEEE Trans. Image Processing 1996;5(10):1435-1447.
111. Dacey D. Primate retina: cell types, circuits and color opponency. Prog Retin Eye Res 1999;18(6):737-63.
112. Ono MS, Kubik S, Abernathy CD. Atlas of the Cerebral Sulci. New York: Thieme Medical Publishers; 1990.
113. Bookstein FL. Biometrics and brain maps: the promise of the morphometric synthesis. In: Koslow SH, Huerta MF, editors. Neuroinformatics: An Overview of the Human Brain Project. Malwah, New Jersey: Lawrence Erlbaum; 1997. p. 203-254.
114. Ascioli GA. Progress and perspectives in computational neuroanatomy. Anatomical Record (New Anat.) 1999;257(6):195-207 http://www.krasnow.gmu.edu/ascoli/CNG/TNA/index.htm.
115. Toga AW. Brain Atlases. http://www.loni.ucla.edu/Research_Loni/atlases/index.html; 2001.
116. Weibel WR. Stereological Methods. New York: Academic Press; 1979.
117. DeQuardo JR, Keshavan MS, Bookstein FL, Bagwell WW, Green WDK, Sweeney JA, et al. Landmark-based morphometric analysis of first-episode schizophrenia. Biological Psychiatry 1999;45(10):1321-1328.
118. Thompson PM, Mega MS, Toga AW. Disease-specific brain atlases. In: Mazziotta JC, Toga AW, editors. Brain Mapping III: The Disorders. New York: Academic Press; 2001. http://www.loni.ucla.edu/~thompson/PDF/DisChptWeb.pdf.
119. Organization for Human Brain Mapping. Annual Conference on Human Brain Mapping. Brighton, United Kingdom; 2001 http://www.academicpress.com/www/journal/hbm2001/.
120. Toga AW, Frackowiak RSJ, Mazziotta JC, editors. Neuroimage: A Journal of Brain Function. New York: Academic Press; 2001.
121. Fox PT, editor. Human Brain Mapping. New York: John Wiley & Sons; 2001.
122. Heiss WD, Phelps ME, editors. Positron emission tomography of the brain. Berlin; New York: Springer-Verlag; 1983.
123. Aine CJ. A conceptual overview and critique of functional neuroimaging techniques in humans I. MRI/fMRI and PET. Critical Reviews in Neurobiology 1995;9:229-309.
124. Ross B, Bluml S. Magnetic resonance spectroscopy of the human brain. Anatomical Record (New Anat.) 2001;265(2):54-84.
125. Corina DP, Steury K, Poliakov AV, Martin RF, Mulligan KA, Maravilla K, et al. A comparison of language function derived from cortical stimulation mapping and fMRI: data from object naming. Submitted 2001.
126. Frackowiak RSJ, Friston KJ, Frith CD, Dolan RJ, Mazziotta JC, editors. Human Brain Function. New York: Academic Press; 1997.
127. Friston KJ, Holmes AP, Worsley KJ, Poline JP, Frith CD, Frackowiak RSJ. Stastical parametric maps in functional imaging: a general linear approach. Hum Brain Mapp 1995;2:189-210 http://www.fil.ion.ucl.ac.uk/spm/.
128. Cox RW. AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research 1996;29:162-173 http://afni.nimh.nih.gov/afni/index.shtml.
129. Hansen LK, Nielsen FA, Toft P, Liptrot MG, Goutte C, Strother SC, et al. Lyngby - modeler's Matlab toolbox for spatio-temporal analysis of functional neuroimages. Neuroimage 1999;9(6):S241 http://www.pet.med.va.gov:8080/distrib/lyngby.html.
130. Cohen JD. FisWidgets. http://neurocog.lrdc.pitt.edu/fiswidgets/; 2001.
131. Ojemann G, Ojemann J, Lettich E, Berger M. Cortical language localization in left, dominant hemisphere. J. Neurosurgery 1989;71:316-326.
132. George JS, Aine CJ, Mosher JC, Schmidt DM, Ranken DM, Schlitz HA, et al. Mapping function in human brain with magnetoencephalography, anatomical magnetic resonance imaging, and functional magnetic resonance imaging. Joutnal of Clinical Neurophysiology 1995;12(5):406-431.
133. Poliakov AV, Albright E, Corina D, Ojemann G, Martin RF, Brinkley JF. Server-based approach to web visualization of integrated 3-D medical image data. In: Proc. AMIA Fall Symposium; 2001.
134. Hurdal MK. A demonstration of cortical flat mapping. http://www.pet.med.va.gov:8080/incweb/circlepack/; 2001.
135. Drury H, West B, Van Essen D. CARET daemon. http://stp.wustl.edu/CARETdaemon/CARETdaemon.html; 1997.
136. Sereno MI. Webcortex: Web interface to cortical surface database. http://cogsci.ucsd.edu/~sereno/webcortex.html; 2001.
137. Herskovits EH. BRAID: Brain imaging database. http://braid.rad.jhu.edu/; 2001.
138. Hadida-Hassan M, Young SJ, Peltier ST, Wong M, Lamont S, Ellisman MH. Web-based telemicroscopy. J. Structural Biology 1999;125:235-245 http://ncmir.ucsd.edu/CMDA/jsb99.html.
139. Koslow SH. Should the neuroscience community make a paradigm shift to sharing primary data? Nature Neuroscience 2000;3(9):863-865.
140. Nature Neuroscience Editorial. A debate over fMRI data sharing. Nature Neuroscience 2000;3(9):845-846.
141. Benson DA, Karsch-Mizrachi I, Lipman DJ, Ostell J, Rapp BA, Wheeler DL. GenBank. Nucleic Acids Res 2000;28(1):15-18.
142. Kotter R. Neuroscience databases - tools for exploring brain structure-function relationships. Philosophical Transactions of the Royal Society, Series B: Biological Sciences. In Press 2001 http://www.hirn.uni-duesseldorf.de/rk/neurodat.htm.
143. Jakobovits R, Soderland S, Taira RK, Brinkley JF. Requirements of a web-based experiment management system. In: Proceediings, AMIA Symposium 2000. Los Angeles; 2000. p. 374-378.
144. Jakobovits RM, Brinkley JF, Rosse C, Weinberger E. Enabling clinicians, researchers, and educators to build custom web-based biomedical information systems. In: Proc AMIA Annual Fall Symposium. In Press; 2001.
145. Jakobovits R. WIRM: A perl-based application server. Web Techniques 2000(September):97-100 http://www.webtechniques.com/archives/2000/09/jakobovits/.
146. Pelletier M, Latteier A. The Zope Book: New Riders; 2001 http://www.zope.org/Members/michel/ZB/.
147. Brinkley JF, Jakobovits RM. UW Brain Project Language Map Experiment Management System. http://tela.biostr.washington.edu/cgi-bin/repos/bmap_repo/main-menu.pl; 2001.
148. Wong STC, Hoo KS, Knowlton RC, Laxer KD, Cao X, Hawkins RA. Design and applications of a multimodality image data warehouse framework. J Am Med Ass. In Press 2001.
149. Letovsky SI, Whitehead SHJ, Paik CH, Miller GA, Gerber J, Herskovits EH, et al. A brain-image database for structure-function analysis. Am. J Neuroradiology 1998;19:1869-1877.
150. Herskovits EH. An architecture for a brain-image database. Methods of Information in Medicine 2000;39(4-5):291-297.
151. Arbib M. Neural plasticity: data and computational structure. http://www-hbp.usc.edu/; 2001.
152. Gazzaniga MS. The fMRI data center. http://www.fmridc.org/; 2001.
153. Herskovits EH, Megalooikonomou V, Davatzikos C, Chen A, Bryan RN, Gerring JP. Is the spatial distribution of brain lesions associated with closed-head injury predictive of subsequent development of attention-deficit/hyperactivity disorder? Analysis with brain-image database. Radiology 1999;213(2):389-394.
154. Beeman DE, Bower JM, De Schutter E, Efthimiadis EN, Goddard N, Leigh J. The GENESIS simulator-based neuronal database. In: Koslow SH, Huerta MF, editors. Neuroinformatics: An Overview of the Human Brain Project. Malwah, New Jersey: Lawrence Erlbaum; 1997. p. 57-81. http://www.bbb.caltech.edu/hbp/.
155. Miller PL, Nadkarni P, Singer M, Marenco L, Hines M, Shepard G. Integration of multidisciplinary sensory data: a pilot model of the Human Brain Project approach. J Am Med Ass 2001;8(1):34-48 http://ycmi-hbp.med.yale.edu/senselab/.
156. Dashti AE, Ghandeharizadeh S, Stone J, Swanson LW, Thompson RH. Database challenges and solutions in neuroscientific applications. Neuroimage 1997;5(2):97-115.
157. Fox PT, Mikiten S, Davis G, Lancaster JL. BrainMap: A database of human functional brain mapping. In: Thatcher RW, Hallett M, Zeffiro T, John ER, Heurta M, editors. Functional Neuroimaging. San Diego: Academic Press; 1994. p. 95-106. http://ric.uthscsa.edu/projects.
158. Bowden DM, Robertson JE, Martin RF, Dubach MF, Wu JS, McLean MR, et al. Web-tools for neuroscience based on NeuroNames, a template brain atlas, edgewarp and geographic information systems software. In: Fifth international confernce on functional mapping of the human brain. Heinrich-Heine University, Dusseldorf, Germany; 1999. http://braininfo.rprc.washington.edu/.
159. Berman HM, Westbrook J, Feng Z, Gilliland G, Bhat TN, Weissig H, et al. The Protein Data Bank. Nucleic Acids Res 2000;28:235-242.
160. Bly BM, Rebbechi D, Grasso G, Hanson SJ. A peer-to-peer database for brain imaging data. In: Human Brain Mapping; 2001. http://www.academicpress.com/www/journal/hbm2001/11785.htm.