New Research In
Physical Sciences
Social Sciences
Featured Portals
Articles by Topic
Biological Sciences
Featured Portals
Articles by Topic
 Agricultural Sciences
 Anthropology
 Applied Biological Sciences
 Biochemistry
 Biophysics and Computational Biology
 Cell Biology
 Developmental Biology
 Ecology
 Environmental Sciences
 Evolution
 Genetics
 Immunology and Inflammation
 Medical Sciences
 Microbiology
 Neuroscience
 Pharmacology
 Physiology
 Plant Biology
 Population Biology
 Psychological and Cognitive Sciences
 Sustainability Science
 Systems Biology
Timeresolved analysis and visualization of dynamic processes in living cells

Communicated by David C. Ward, Yale University School of Medicine, New Haven, CT (received for review January 4, 1999)
Abstract
Recent development of in vivo microscopy techniques, including green fluorescent proteins, has allowed the visualization of a wide range of dynamic processes in living cells. For quantitative and visual interpretation of such processes, new concepts for timeresolved image analysis and continuous time–space visualization are required. Here, we describe a versatile and fully automated approach consisting of four techniques, namely highly sensitive object detection, fuzzy logicbased dynamic object tracking, computer graphical visualization, and measurement in time–space. Systematic model simulations were performed to evaluate the reliability of the automated object detection and tracking method. To demonstrate potential applications, the method was applied to the analysis of secretory membrane traffic and the functional dynamics of nuclear compartments enriched in premRNA splicing factors.
The development of in vivo microscopy techniques and fluorescent reagents has stimulated interest in studying the dynamics of cellular processes (for review, see refs. 1 and 2). These types of experiments generate large and complex data sets and require tools for visual and quantitative analysis of the observed dynamic processes in space and time.
Imaging fast moving vesicles in living cells at high speed and high spatial resolution generally implies a low signaltonoise ratio, hampering accurate object detection. As a consequence of the optical aperture problem, tracking of small objects based on visual similarity criteria is difficult because many objects appear very similar (3). Highly sensitive object detection and tracking has been recognized as crucial for an accurate evaluation of such data. However, a quantitative interpretation of trafficking vesicles has been generally based on manual evaluation of a userbiased selection of objects with apparently highest motility. Such an evaluation is very timeconsuming and also is limited by the perception of the manual inspector.
Processes in the cell nucleus are much slower and need to be observed over a longer period of time. To avoid disruptions of nuclear processes, the total light exposure during in vivo observation must be minimized. Thus, the signaltonoise ratio and, more importantly, the number of time series taken in a particular experiment is considerably reduced, leading to a loss in spatiotemporal resolution. Displaying time series as movies is a widely used method for visual interpretation. However, this approach does not improve temporal resolution; that is, additional information about the continuous development of the observed processes between the imaged time steps (subpixel resolution in time) is not obtained. Furthermore, quantitative information is not revealed by such a visual approach. In a first approach to quantitatively describe nuclear dynamics in vivo, single particle tracking (4) has been used to estimate the diffusion of chromatin in living cells of different species (5).
Here, we describe a novel technique for fully automated analysis and time–space visualization of time series from living cells, which involves segmentation and tracking of cellular structures as well as continuous visualization and measurement of quantitative parameters with subpixel resolution in time and space. It will be demonstrated that this approach can reconstruct resolution both in time and space. Hence, quantitative and visual information unachievable with traditional methods can be accessed.
METHODS
Highly Sensitive Object Detection.
Imaging structures in living cells at high speed and high spatial resolution generally implies a low signaltonoise ratio (Fig. 1A), hampering accurate object detection. We developed a twostep procedure for object detection. First, a constrained image smoothing by anisotropic diffusion is performed that removes noise without disturbing essential edge information (Fig. 1B). Anisotropic diffusion selectively diffuses an image I in regions where the signal is of constant mean, in contrast to those regions where a rapid signal change occurs. The smoothing process within the PeronaMalik model is monitored by an abstract timescale t; that is, higher values imply stronger filtering (6). The diffusion process depends solely on local image properties and is governed by the shape of the socalled “edgestopping” function g. Here, an object scaledependent edgestopping function based on Tukey’s biweight robust estimator ρ was applied because diffusion with the Tukey norm produces sharper boundaries than the Lorentzian (PeronaMalik) norm (7). For segmentation, an edgeoriented technique using a concept of local orientation was applied. Based on the smoothed representation, candidate edge pixels are determined by a modified form of the standard nonmaximumsuppression algorithm (8), including a weak hysteresis formulation. A pixel is classified as an edge pixel if it has a potential predecessor and successor (hysteresis) and if the magnitude of the gradient is maximal compared with the two neighbors in direction of the gradient (nonmaxima suppression). The first condition assists in the formation of unbroken contours whereas the second inhibits multiple responses to a single edge present in the data (Fig. 1C).
To obtain closed borderlines, edges were assumed to separate two neighboring homogeneous regions. A region was considered homogenous if its intensity values could be modeled by a Gaussian distribution. Edges were traced based on two parameters, namely local orientation and equal probability of belonging to adjacent regions. For small scale structures, local orientation can be approximated by the direction of gradient. This induces a local coordinate system such that the orthogonal axis aligns with the isophote (line of constant intensity) parallel to the borderline. Because borderlines induce a partition of the image, the second condition implies that edge pixels need to belong to adjacent partitions with equal probability. As a result, closed borderlines enclosing homogeneous regions are obtained. Based on the image partition, a region neighborhood graph is built. Each node of the graph is associated with one region and is assigned morphological parameters such as mean intensity, shape, and size of the respective region. Regions of interest are finally detected as regions with locally maximal intensity (Fig. 1D).
Dynamic Object Tracking.
For dynamic analysis, cellular objects need to be tracked in time–space. Based on object features such as size, shape, total intensity, or texture, tracking an object amounts to finding its best match in consecutive images. According to the continuity equation of optical flow, corresponding objects in consecutive images should be similar. However, distortions in the imaging process, such as noise, bleaching, and illumination differences, as well as changes in focal position might considerably distort the time–space continuity assumption. Standard regionbased matching techniques (9) do not give satisfying results in general. Our methods use a fuzzy logicbased system for image sequence analysis (10) based on the assumption that object features are conserved in an indistinct (fuzzy) sense.
Fuzzy theory assumes that all things are matter of degree (11). Fuzzy systems behave as associative memories mapping close inputs to close outputs without requiring a mathematical description of how the output functionally depends on the input. A fuzzy system relies on a linguistic “rule” encoded in a numerical fuzzy associative memory mapping, the fuzzy associative memory rule. According to a dynamic particle model, the velocity of an object is assumed to remain relatively constant. To compare two objects in consecutive images, differences in velocity and deviation of expected extrapolated position from the potential new position are measured. In addition, differences in total intensity and area are computed and translated into fuzzy rules.
Each of these four parameters activates each fuzzy associative memory rule to different degree m_{ANT}. The scalar activation value act_{j} of the fuzzy associative memory rules’ consequent equals the minimum of the four antesequent conjuncts’ values. With correlationproduct, encoding the value of the consequent is multiplied by the activation value. By computing the fuzzy centroid, the output is defuzzified to a single numerical value, the composite similarity measure c_{sim}. Objects in one image correspond to the object with the highest “defuzzified” similarity measure in the consecutive image. In case no corresponding object with a similarity value below a certain threshold has been found, this object remains unmatched, and the respective object track ends. A correspondence map together with the binarized images are the output of the image sequence analysis module and are used for continuous visualization and quantification in time–space.
Continuous Time–Space Reconstruction.
We developed two different approaches for continuous time–space reconstruction. The dynamics of objects with constant shape over time are well described by the dynamic repositioning of their gravity centers. According to the correspondence map provided by the particle, tracking module discrete trajectories are formed in time–space. By cubic bspline interpolation between corresponding gravity centers, they are subsequently transformed into continuous trajectories (Fig. 2A). Cubic bsplines were chosen for interpolation between corresponding points because they are stable in a geometric sense: That is, they do not tend to oscillations even for a large number of sample points. Furthermore, continuous derivatives exist up to the order of two. Note that the first and second derivative correspond to velocity and acceleration, respectively, which are crucial for quantification of dynamics (see below). The reconstruction procedure was embedded in a powerful multidimensional viewer (Open Inventor Scene Viewer, Silicon Graphics, Mountain View, CA; available for almost all hardware platforms), providing a differentiated visualization of a large number of trajectories in time–space (Fig. 2B).
The second approach was designed for visualization of objects dynamically changing their shape over time. This problem accounts for a continuous shape reconstruction from series of twodimensional images in time–space (12, 13). In a first step, the binarized object representation is transformed into a parameterized contour representation (Fig. 3 A–C). Thereafter, corresponding boundary points in adjacent time sections are found by a global optimization scheme. Under the assumption that the transformation of one contour k_{1}(u) in its adjacent contour k_{2}(v) is sufficiently smooth and leaves the order of contour points, unchanged, the optimal transformation is found by minimizing the integral over all Euclidean distances between corresponding contour points.
For minimization of this energy term, a recursive contour splitting approach was chosen. A continuous surface reconstruction in time–space is obtained by bspline interpolation of corresponding boundary points (Fig. 3D) and is visualized within the graphical scene viewer. Triangles are essential primitives for computer graphical display. Hence, continuous surfaces need to be approximated by triangular meshes. To reduce the complexity of the triangle mesh while maintaining a close approximation to the original bspline interpolation, a multiresolution strategy for visualization was developed. At each level of resolution, triangles are formed according to the correspondence map in time–space. To obtain a homogenous triangulation in time–space, a triangle is further subdivided only if its maximal displacement from the bspline interpolated surface exceeds a preset threshold (14).
Quantitative Measurements.
Although multidimensional visualization is crucial for a qualitative evaluation of dynamic processes, morphological and dynamic parameters are required for quantitative measurements. Based on object outlines and the time–space correspondence map, a quantification module was developed. Within this module, morphological parameters such as size and shape as well as dynamic parameters such as path length, velocity, acceleration, mean squared distances, and diffusion coefficients are computed in a fully automated way. An interface to standard statistic software facilitates further evaluation and display of parameters (see below).
RESULTS
Dynamics of Secretory Membrane Traffic.
In a first study, we examined the motility of secretory vesicles mediating biosynthetic transport from the transGolgi network to the plasma membrane (15). Green fluorescent protein (GFP) was tagged to the secretory protein human chromogranin B in Vero cells followed by in vivo timelapse microscopy. After adaptive smoothing and segmentation (Fig. 1), the object information was passed to the image sequence analysis tool for object tracking. Image analysis, graphical preprocessing, and computation of dynamic parameters involving >500 vesicles in 40 time sections was performed in a fully automated way within 60 min on a standard Pentium PC or Silicon Graphics workstation. In comparison, manual evaluation consumed >4 hours of user interaction for a small subset of 40 user selected vesicles with highest motility.
Within the time–space reconstruction module, the resulting 391 trajectories were categorized as stationary, unidirectional, or bidirectional according to their degree of motility (Fig. 2B). A trajectory was considered as stationary if its mean velocity was below the average velocity of all trajectories. The group of remaining vesicles was subdivided into two classes: vesicles performing unidirectional motion and vesicles reversing their direction of motion. A vesicle was considered as reversing its direction at a particular time step if the velocity vector averaged over three time steps before it differed at least 130° in direction as compared with three time steps afterward.
Trajectories were colorencoded for easy interpretation of motility (Fig. 2C). Dynamic parameters such as mean squared distances, diffusion coefficients, and velocities were computed (Fig. 4A). The vast majority (64%) of vesicles were found to be stationary whereas 31% showed periods of fast directed movements interspersed by periods of slow random motion. The average velocity of nonstationary vesicles was 0.587 μm/sec as compared with 0.252 μm/sec for stationary vesicles. The maximum velocity measured for this cell was 1.23 μm/sec (i.e., 6 pixels/time frame; see Table 1). A smaller fraction (5%) showed a reverse in direction, where half of these vesicles reverted their direction by 176–180°.
The plots of mean squared distances and diffusion constants show that stationary vesicles in normal cells as well as vesicles in cells with disrupted microtubules after addition of nocodazole undergo Brownian motion (Fig. 4A). These findings support a trialanderror model for transport of secretory vesicles. According to this model, vesicles are transported fast but on random tracks simply trying any direction to find a target membrane. The model further suggests that vesicles perform fast directed movements if associated with microtubules and slow random search movements toward the next microtubule after dissociation. Of interest, we also observed small clusters of vesicles apparently performing the same kind of random and directed movement bringing up the question whether those vesicles are associated with larger transport intermediates. The described techniques provide a fast and powerful tool to investigate how secretory traffic is regulated: for example, to study the effect of cell motility or cell–cell contact formation on the transport of secretory vesicles.
Comparison with Model Simulations.
In an attempt to quantitatively access the accuracy of the dynamic object tracking module, we performed a systematic simulation of trafficking vesicles under different dynamic parameter settings. Vesicles were simulated as spots with a Gaussian gray value distribution. In an initial configuration, a fixed number of vesicles was randomly placed into the image. The localization of a vesicle in the next time step depends on four parameters: its initial velocity (v_{ini}), its acceleration (a), its direction of motion, and the maximally allowed velocity (v_{max}). Although settings for initial and maximal velocity as well as acceleration differed among vesicles and experiments, a vesicle was allowed to perform any form of change in its directional motion in all simulations.
A sequence of 40 time steps was simulated under different parameter settings (Table 1). After automated object detection, vesicles were tracked by the object tracking module as described above (see Methods). Automatically found trajectories were systematically compared with simulated trajectories (Fig. 4B). Simulation of vesicles with dynamic parameters set to the experimentally determined maximal velocities and acceleration for vesicles in membrane traffic (Table 1) showed that 72.5% of the simulated vesicles could be exactly tracked by the dynamic object tracking module. Considering the high dynamics of vesicles and the frequent intersection of trajectories in time–space (Fig. 5), only 17.5% of the simulated trajectories were unmatched; the remaining 10% of trajectories were subdivided into two or three independent trajectories:, that is, at one or two time steps, a vesicle was lost. Notably, the overall dynamics as reflected by mean velocities of vesicles was accurately estimated by the automated tracking module (deviation to exact mean velocities <3%; see Table 1).
Dynamics of Transcription and PremRNA Splicing.
In a second study, the described technique was applied to study gene expression events in vivo, in particular, the tempospatial and functional relationship between transcription and premRNA splicing. In the mammalian cell nucleus, most splicing factors are concentrated in 20–40 distinct domains called speckles. GFP was fused inframe to the amino terminus of the essential splicing factor 2/alternative splicing factor (SF2/ASF) and was visualized by timelapse microscopy. Timelapse microscopy shows that those speckles are highly dynamic structures (16). BKT1B cells, transfected with GFP–essential splicing fator 2/alternative splicing factor cAMPinducible early genes of BK virus, were triggered in vivo, followed by timelapse microscopy.
After automated image analysis, outlines of speckles and BKinduced RNA were computed (Fig. 3 A–C), followed by a continuous time–space reconstruction and computation of surface dynamics for these speckles (Fig. 3D). The continuous reconstruction shows that one of the speckles extends toward the BK virus gene and intersects the gene signal (Fig. 3E). Notably, the surface dynamics of all neighboring speckles measured by the average acceleration of surface points (data not shown) at each interpolated time step increases after transcriptional activation and reaches its peak when the first speckle hits the gene. Thereafter, the speckles show a rapid slowdown in surface dynamics and a slight reduction in surface area.
In a different cell, speckles were imaged after addition of αamanitin, a specific inhibitor of RNA polymerase II. Notably, the speckles rounded up and showed only low surface dynamics (Fig. 3F). These findings suggest that speckles supply premRNA splicing factors to nearby activated sites of transcription and that the dynamics of speckles is directly related to nearby transcriptional events. We are currently using these tools to quantitatively characterize the surface dynamics and intranuclear positioning of the splicing factor compartments.
CONCLUSIONS
We have developed a widely applicable technique for accurate analysis of dynamic processes in living cells. In the first application, we quantitatively characterized vesicular motility in secretory membrane traffic. Here, the method proved particularly useful because it allowed for a fully automated analysis of even complex data involving >500 vesicles in a series of 20 images. In a comparison with simulated data, we quantitatively accessed the reliability of the dynamic object tracking module. Under experimental conditions, the vast majority of vesicles could be accurately tracked. Furthermore, the small population of unmatched vesicles did not influence the overall dynamics, as reflected by the velocity distribution of vesicles. For the study of nuclear processes, the time–space interpolation technique proved particularly useful for investigating the continuous assembly of functional compartments in response to metabolic requirements.
Photon count limitations and/or limitations in total exposure time during in vivo imaging implies restrictions both in spatial and temporal resolution. Our results indicate that the developed methods help to reconstruct tempospatial resolution. For fastmoving vesicles, the highly sensitive object detection module reveals vesicles that are difficult or even impossible to detect by visual interactive inspection. For nuclear processes, the visualization module provides a time–space interpolation of intermediate time steps, thus allowing the prediction of dynamic processes between measured image steps.
The mechanisms and forces involved in the assembly and dynamics of functional subcellular compartments in response to metabolic requirements are not well understood at present and need further investigations. The present method has been shown to be highly suited for an accurate quantitative and qualitative study of nuclear and cellular processes with improved tempospatial resolution.
Acknowledgments
We thank B. Jähne for generously supporting the development of the particle tracking program and M. Markoutsakis for technical assistance. We are particularly grateful to W. Jäger for his support and helpful comments concerning mathematical image analysis. This work was supported by Bundesministerium für Bildung und Forschung Grant 01 KW 9621 and Deutsche Forschungsgemeinschaft Grant Ja 395/62. D.L.S. is supported by National Institute of General Medical Sciences Grant 42694, W.T. is supported by the Graduiertenkolleg at the Interdisziplinäres Zentrum für Wissenschaftliches Rechnen, and T.M. is supported by the Roche Research Foundation.
Footnotes

↵‡ Present address: National Cancer Institute, National Institutes of Health, Bethesda, MD 20982.

↵¶ To whom reprint requests should be addressed at: University of Heidelberg, Interdisciplinary Center of Scientific Computing, Im Neuenheimer Feld 368, D69120 Heidelberg, Germany. email: eils{at}iwr.uniheidelberg.de.
ABBREVIATION
 GFP,
 green fluorescent protein
 Received January 4, 1999.
 Accepted May 5, 1999.
 Copyright © 1999, The National Academy of Sciences
References
 ↵
 Lamond A I,
 Earnshaw W C
 ↵
 ↵
 ↵
 ↵
 ↵
 Perona P,
 Malik J
 ↵
 ↵
 Nevatia R,
 Babu K R
 ↵
 Anandan P
 ↵
 ↵
 Nauck D,
 Klawonn F,
 Cruse R
 ↵
 Eils R,
 Dietzel D,
 Bertin E,
 Schröck E,
 Speicher M R,
 Ried T,
 RobertNicoud M,
 Cremer C,
 Cremer T
 ↵
 Jähne B,
 Haussecker H,
 Geissler P
 Eils R,
 Sätzler K
 ↵
 Hoppe H
 ↵
 ↵
Citation Manager Formats
Sign up for Article Alerts
Jump to section
You May Also be Interested in
More Articles of This Classification
Biological Sciences
Cell Biology
Related Content
Cited by...
 Tracking and Quantitative Analysis of Dynamic Movements of Cells and Particles
 Computational Image Analysis of Cellular Dynamics: A Case Study Based on Particle Tracking
 Single livecell imaging for systems biology 9
 Automated local bright feature image analysis of nuclear protein distribution identifies changes in tissue phenotype
 Intracellular Mechanics of Migrating Fibroblasts
 Visualizing telomere dynamics in living mammalian cells using PNA probes
 Computational imaging in cell biology
 Development and Use of Fluorescent Protein Markers in Living Cells
 Myosin Va facilitates the distribution of secretory granules in the Factin rich cortex of PC12 cells
 An evolutionarily conserved NPC subcomplex, which redistributes in part to kinetochores in mammalian cells
 Quantitative Imaging of PremRNA Splicing Factors in Living Cells