The mathematical theory of compressed sensing (CS) asserts that one can acquire signals from measurements whose rate is much lower than the total bandwidth. Whereas the CS theory is now well developed, challenges concerning hardware implementations of CS-based acquisition devices—especially in optics—have only started being addressed. This paper presents an implementation of compressive sensing in fluorescence microscopy and its applications to biomedical imaging. Our CS microscope combines a dynamic structured wide-field illumination and a fast and sensitive single-point fluorescence detection to enable reconstructions of images of fluorescent beads, cells, and tissues with undersampling ratios (between the number of pixels and number of measurements) up to 32. We further demonstrate a hyperspectral mode and record images with 128 spectral channels and undersampling ratios up to 64, illustrating the potential benefits of CS acquisition for higher-dimensional signals, which typically exhibits extreme redundancy. Altogether, our results emphasize the interest of CS schemes for acquisition at a significantly reduced rate and point to some remaining challenges for CS fluorescence microscopy.
AUTHOR SUMMARY
In conclusion, we present the principles and implementation of compressive sensing in fluorescence microscopy, together with its applications in biomedical imaging. Our approach readily allows for substantial decrease in the sampling rate when compared to traditional approaches. Furthermore, we set forth a distinctive prospect for hyperspectral acquisition, which has great potential for multicolor single-molecule imaging. From a general standpoint, the acquisition of 3, 4, or even 5D signals puts major constraints on system resources. Because high-dimensional signals are often characterized by extreme sparsity (in other words, they have low information content compared to the total number of voxels), we expect the advantages of CFM to become paramount in biological imaging.
Here, we present a unique modality for fluorescence imaging based on the concepts of the CS theory. Our compressive fluorescence microscope illuminates the whole sample in a patterned manner as obtained by a digital micromirror device, and collects the fluorescence on a fast and sensitive point detector. The images are then computationally reconstructed from measurements corresponding to a set of appropriately chosen patterns (Fig. P1). Compressive fluorescence microscopy (CFM) is therefore a wide-field technique that benefits from the advantages (low cost, high dynamic range, spectral detectors) usually restricted to raster scanning microscopes where one pixel of the image is illuminated at a time and its fluorescence is recorded on a single point detector. With our microscope, we are able to acquire images of fluorescent beads, live cells, and pollen tissues with undersampling ratios of up to 32, that is, with the number of measurements 32 times lower than the total number of pixels in the image. We further demonstrate how CS acquisition and reconstruction schemes can be favorably used for hyperspectral imaging, a highly sought after biological imaging modality where the full fluorescence emission spectrum is recorded for each pixel in the image (5). We finally discuss the influence of different factors, such as shot noise or sample thickness, on the performance of CS microscopy, and suggest different strategies to overcome present limitations. Overall, our work provides a solid basis for future development of CS-based methods in fluorescence imaging.
Fig. P1.
Experimental setup of the compressive fluorescence microscope. The microscope provides a computer-controlled patterned laser illumination based on a digital micromirror device (DMD). For each pattern in the DMD, the fluorescence from the sample is recorded on a single point detector (B) or on a spectral detector. One typical pattern is shown in (C). The conventional fluorescence camera image of a sparse distribution of 2-μm-diameter multicolor fluorescent beads is shown in (D). A multicolor image of the same sample obtained with our compressive fluorescence microscope is shown in (D). A compression ratio of 16 is achieved for this image.
In all the conventional methods employed to record images of biological samples, data acquisition is independent of the information content of the image. Data sampling in space, time, and color is performed according to the Shannon-Nyquist theorem, which states that the sampling frequency must be twice the bandwidth of the signal, even though postacquisition compression is usually performed to reduce the storage size of data. The recent CS theory challenges this dogma and states that one can exploit sparsity or compressibility when acquiring signals of general interest, and that one can design nonadaptive sampling techniques that condense the information contained in a compressible signal into a small amount of data (1, 2). In short, reliable acquisition with far fewer measurements than were thought necessary is possible. With a number of compressive samples well below the limit given by the Shannon-Nyquist theorem, the original image can subsequently be reconstructed without much loss—if any at all—by computational processing. Practical implementations of CS-based acquisition strategies are emerging in fields as diverse as medical resonance imaging, analog-to-digital conversion, and astronomy. While the benefits of CS acquisition have just begun to be investigated in optical imaging (3), little progress has been made in microscopy (4), especially in biological imaging.
Fluorescence microscopy is a crucial tool for fundamental and applied biomedical research. However, the need to acquire images with ever-increasing size and spatial resolution and spectral information puts considerable constraints on the acquisition process, particularly on the number of sensors and/or duration. This paper presents a unique approach to fluorescence bioimaging based on the recent mathematical theory of compressive sensing (CS), explained later in the summary. We aim to dramatically reduce the acquisition time of sparse fluorescent samples with very limited loss of information. We demonstrate the potential of CS microscopy for fluorescent images of beads, cells, and tissues as well as hyperspectral images, in which we could decrease the number of samples by factors of up to 32.
This Direct Submission article had a prearranged editor.
E Candes, J Romberg, T Tao, Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE T Inform Theory52, 489–509 (2006).
J Trzasko, A Manduca, E Borisch, Highly undersampled magnetic resonance image reconstruction via homotopic l0 minimization. IEEE T Med Imaging28, 106–121 (2009).
Y Wu, P Ye, IO Mirza, GR Arce, DW Prather, Experimental demonstration of an optical-sectioning compressive sensing microscope (CSM). Opt Express18, 24565–24578 (2010).
Fluorescence microscopy is a fundamental tool in basic and applied biomedical research. Because of its optical sensitivity and molecular specificity, fluorescence imaging is employed in an increasing number of applications which, in turn, are continuously driving the development of advanced microscopy systems that provide imaging data with ever higher spatio-temporal resolution and multiplexing capabilities. In fluorescence microscopy, one can schematically distinguish two kinds of imaging approaches, differing by their excitation and detection modalities (1). In wide-field (WF) microscopy, a large sample area is illuminated and the emitted light is recorded on a multidetector array, such as a CCD camera. In contrast, in raster scan (RS) microscopy, a point excitation is scanned through the sample and a point detector is used to detect the fluorescence signal at each position.
While very distinct in their implementation and applications, these imaging modalities have in common that the acquisition is independent of the information content of the image. Rather, the number of measurements, either serial in RS or parallel in WF, is imposed by the Nyquist-Shannon theorem. This theorem states that the sampling frequency (namely the inverse of the image pixel size) must be twice the bandwidth of the signal, which is determined by the diffraction limit of the microscope lens equal to λ/2NA (λ is the optical wavelength and NA the objective numerical aperture). Yet, most images, including those of biological interest, can be described by a number of parameters much lower than the total number of pixels. In everyday’s world, a striking consequence of this compressibility is the ability of consumer cameras with several megapixel detectors to routinely reduce the number of bits in a raw data file by an order of magnitude or two without substantial information loss. To quote from David Brady: “if it is possible to compress measured data, one might argue that too many measurements were taken” (2).
The recent mathematical theory of compressed or compressive sensing (CS–see refs. 3, 4) has addressed this challenge and shown how the sensing modality could be modified to reduce the sampling rate of objects which are sparse in the sense that their information content is lower than the total bandwidth or the number of pixels suggest. The fact that one can sample such signals nonadaptively and without much information loss—if any at all—at a rate close to the image information content (instead of the total bandwidth) has important consequences, especially in applications where sensing modalities are slow or costly. To be sure, the applications of CS theory to data acquisition are rapidly growing in fields as diverse as medical resonance imaging (5, 6), analog-to-digital conversion (7), or astronomy (8).
In optics, the interest in CS has been originally spurred by the demonstration of the so-called “single-pixel camera” (9). Since then, reports have explored the potential of CS for visible and infrared imaging (10, 11), holography (12), or ghost imaging (13). In microscopy, the feasibility of CS measurements has recently been demonstrated (14). Altogether, these results open exciting prospects, notably for the important case of biomedical imaging. Yet, there are very few results about the performance of CS hardware devices on relevant biological samples. As such samples often have low fluorescence, it is especially important to understand how the associated noise will affect the acquisition and reconstruction schemes.
In this paper, we describe Compressive Fluorescence Microscopy (CFM), a unique modality for fluorescence biological and hyperspectral imaging based on the concepts of CS theory. In CFM, the sample is excited with a patterned illumination and its fluorescence is collected on a point detector. Images are computationally reconstructed from measurements corresponding to a set of appropriately chosen patterns. Therefore, CFM benefits from many advantages associated with RS techniques, namely, high dynamic range, facilitated multiplexing, and wide spectral range (from the UV to the IR). In truth, the benefits of CS are particularly appealing in biology where fast, high-resolution, and multicolor imaging is highly sought after.
The paper is organized as follows. We begin by recalling the principles of CS theory for optical imaging. We then turn to the description of the practical implementation of CFM and of the sensing protocol. Our techniques are subsequently applied to image several relevant samples, including fluorescent beads, cultured cells and tissues. By extending our implementation, we further demonstrate the possibility of hyperspectral acquisition with up to 128 different spectral channels. A final contribution is a careful study of various noise trade-offs for CFM. We conclude the paper with a discussion of prospective CFM developments.
Compressed Sensing Framework
We wish to image a two-dimensional sample x = {x[i]}, a distribution of fluorescent probes, in which x[i] is the value of x at the pixel/location i (thus one can view pixel intensities x[i] as the coefficients of the image x in a basis of localized functions, namely, the Dirac basis). We represent this object in a basis W of our choosing and write
where the wp’s are (orthogonal) basis functions and the c[p]’s are the coefficients of x in the expansion. We say that the signal is K-sparse if at most K of these coefficients are nonzero. An empirical fact is that most images of interest are well approximated by K-sparse expansions with K much less than the number of pixels N, and this is the reason why data compression is effective; one can store and transmit quantizations of the large coefficients, ignore the small ones, and suffer little distortion.
In our imaging setup, we measure correlations between the image of interest x and sensing waveforms ϕk taken from another basis Φ; that is, we measure
[1]
Here, ϕk is an illumination or intensity pattern so that yk is obtained by collecting all the fluorescence corresponding to those pixels that have been illuminated on a single-point detector. Wide-field and point-like excitation are two extreme cases, corresponding respectively to a uniform sensing waveform (ϕk[i] = 1 for all i) and to a spike or Dirac waveform.
In its simplest form, CS theory asserts that if the signal x is sparse in the representation W, then only few measurements of the form [1] are sufficient for perfect recovery provided the sensing and representation waveforms, respectively ϕk and wp, are incoherent (4, 15). Two systems are said to be incoherent if any element in one of the two cannot be expressed as a sparse linear combination of elements taken from the other. Formally, the coherence between two orthobases W and Φ of is measured by the parameter μ(W,Φ) ranging between 1 and N:
[2]
The Fourier and Dirac bases are in this sense maximally incoherent (we need many spikes to synthesize a sinusoid and vice versa) and μ = 1. On the opposite, two identical bases are maximally coherent and, in this case, μ = N. Hence, incoherence expresses the idea of the level of dissimilarity between any two representations of a signal. With this notion in mind, one perceives how each incoherent measurement—a projection on an element of the basis Φ—conveys a little bit of information about all the entries of the coefficient vector c. An important result in CS theory states that K-sparse signals can be recovered exactly from comparably few measurements in an incoherent system. Further, recovery is achieved by solving a tractable optimization program—a linear program. One solves
[3]
When M measurements are chosen uniformly at random from the basis Φ, the recovery is exact with very high probability; that is, the solution sequence obeys , provided that
[4]
where C is a constant on the order of unity. This result emphasizes both the role of the coherence and the potential gain for large images due to the logarithmic dependence in the pixel size. For incoherent pairs, we only need on the order of K log N random samples.
We have discussed sparse signals above for ease of exposition. However, the theory extends to approximately sparse signals and to noisy data. For instance, if the signal is well approximated by a K-sparse signal (some would say that it is compressible), then the reconstruction error is shown to be small. Further, the recovery is not sensitive to noise in the sense that the error degrades gracefully as the signal-to-noise ratio decreases. We refer to ref. 16 and references therein for quantitative statements.
Our setup is based on a standard epifluorescence inverted microscope (Nikon Ti-E) as shown in Fig. 1A. To generate spatially modulated excitation patterns, we incorporated a Digital Micromirror Device (DMD) in a conjugate image plane of the excitation path. The DMD is a 1,024-by-768 array of micromirrors (Texas-Instrument Discovery 4100) of size 13.68 × 13.68 μm each, and which can be shifted between two positions oriented at +12° or -12° with respect to the DMD surface. The micromirrors are all independently configurable at frequency up to 20 kHz. The DMD is carefully positioned so that the optical axis [defined by the microscope lens and the dichroic mirror DM (Fig. 1A)] is orthogonal to the plane of the DMD.
Fig. 1.
(A) Experimental setup. The dotted and plain segments correspond to planes respectively conjugated to the pupil and sample planes. (B) Slice of lily anther (endogenous fluorescence with epifluorescence microscopy image recorded on a CCD camera). (C) Projection of a Hadamard pattern on a uniform fluorescent sample. (D) Projection of the same Hadamard pattern on the biological sample. (E) Fluorescence intensity during an acquisition sequence.
As light source, we used a laser bench (Roper Scientific) equipped with two superimposed continuous-wave laser (λ = 488 nm, Coherent and λ = 561 nm, Cobolt). The laser beam first passed through a rotating diffuser or a phase scrambler (Dyoptika) in order to reduce the spatial coherence and was then coupled to a 200 μm multimode fiber. At the fiber output, the laser beam was expanded into a 2 cm diameter collimated beam. This beam was oriented toward the DMD at an angle of incidence corresponding to twice the tilting angle of the DMD mirrors (approx. 24°); a micromirror oriented at +12° would reflect the light into the microscope and appear as a bright pixel in the sample plane and, inversely, micromirrors oriented at -12° appear as dark pixels. Depending on the samples, we used an air (Nikon, 20X, Plan Apo VC NA 0.75) or an oil-immersion objective (Nikon, 60X Plan Apo Tirf, NA 1.45). When the 20X air lens was used, the imaging lenses (lenses f1, f2, f3 in Fig. 1A) were chosen to introduce a 1.5X reduction. The overall magnification of the image of the DMD on the sample was 1/30 and the size of a single micromirror equal to 456 nm. When the 60X lens was used, a different set of imaging lenses was chosen which only served as a 1x relay; here, the image size of a single micromirror in the sample plane was equal to 228 nm. Upon illumination with an intensity pattern (excitation intensity approximately 20 to 60 W/cm2), the sample fluorescence was detected on a photomultiplier tube PMT (Hamamatsu) and sampled using an analog-digital converter board (PCI-4462, National Instruments) (Fig. 1B–D). Depending on the brightness of the sample, the pattern projection frequency was 1 or 10 kHz, while the sampling frequency of the PMT signal was 96 kHz. This oversampling of the PMT signal allowed us to remove the portion of the signal corresponding to the transition of the micromirrors between two patterns. In CS measurements, the information on the sample is thus contained in the variations of the intensity signal as a function of the illumination pattern (Fig. 1E). The WF image of the sample could also be directly formed on a camera (ImagEM) placed on the output port of the microscope. For hyperspectral imaging, the PMT was replaced by a fast and sensitive spectral detector described later in the paper. Note that the role of the DMD in our set-up differs from that in the “single-pixel camera” (9) or in some other microscopy setups (14). In the latter, the modulator is placed between the sample and the detector, meaning that it is used to select some of the light within the total signal, rather than to control the excitation pattern. Our choice is motivated by the low level of fluorescence encountered in biological samples such as living cells labeled with fluorescent proteins. Indeed, the overall efficiency of a DMD is 68% and falls down to 34% when only one half of the mirrors are tilted. In our case, the photon collection efficiency is only limited by the numerical aperture of the microscope lens and the quantum yield of the detector as in conventional epifluorescence microscopy.
Choice of the Illumination Patterns.
For the practical implementation of a CS-based image acquisition system, it is essential to determine which incoherent basis should be used when no prior information on the signal is available. There are measurement ensembles, such as the partial Fourier or Hadamard systems, known to be highly incoherent with the bases in which most natural images are sparse. When excitation patterns are generated by micromirrors, ϕk[i] is a binary waveform taking on the two values 0 or 1. An appealing choice for Φ is then the Hadamard system known to be incoherent with the Dirac basis and fine scale wavelets. Since each entry of a Hadamard pattern hk is either -1 or +1, one defines ϕk as a shifted and rescaled version of hk via ϕk = (hk + 1)/2, which can be simply encoded on the DMD. We used patterns of size 256 × 256 and 128 × 128 obtained by binning 2 × 2 and 4 × 4 groups of micromirrors. The actual pattern formed in the sample plane is in fact the convolution of the ideal pattern hk with the point spread function of the microscope Pexc in the excitation path. Fig. 1C and D represent WF images of a Hadamard pattern projected on a uniform and on a biological sample. A specificity of optical imaging is that the sensing elements ϕk represent light intensities and are thus nonnegative which, as discussed later, has important practical implications.
Hadamard waveforms have a sort of spatial frequency (like sinusoids) which grossly depends on the typical block size of the patterns. As the power spectrum of most biological images is generally concentrated at low frequencies, the flexibility in frequency selection is important. We introduce two distinct pattern selection strategies based on the expected spatial content of the sample:
•
When the sample we wish to acquire is sparse in the pixel domain as in the case of single molecule or bead imaging, no typical frequency range needs to be favored and Hadamard patterns are selected uniformly at random.
•
More complex samples have a power spectrum typically decaying like a power law. This a priori information suggests that we should balance low—and high-frequency measurements in order to accurately acquire the low-frequency part of the image, which accounts for a significant part of the total variance. The half-half strategy then projects the m/2 patterns with the lowest spatial frequencies to acquire a low-resolution image of the sample; the high-resolution content of the image is randomly sampled by choosing m/2 measurements among the N - m/2 remaining high-frequency Hadamard patterns. Such an adaptive strategy guarantees an accurate determination of the low-frequency content while allowing for the estimation of details at a finer scale.
It should be noted that Hadamard matrices are not the only possible choice. For instance, some recent work in compressed sensing theory has focused on special cases of structured matrices inspired by low-density parity check codes (32) or statistical physics (33). While preliminary results seem very promising in terms of sensing/reconstruction performance, it is not yet clear how it can be efficiently adapted to acquire biologically relevant samples.
Computational Reconstruction.
In CS, it is essential to enforce the sparsity of the reconstructed signal in some representation W that is chosen a priori. The choice of W highly depends on the spatial structures of the signal to be reconstructed. One would typically use a Fourier representation for oscillatory features, wavelets for pointwise singularities, curvelets for contour-like or filamentary structures (17), and so on. One could also use a concatenation of all these representations. (If one intends on using the Fourier basis as a sparsity basis, one would need to scramble the columns of the Hadamard basis because it would otherwise be coherent with sinusoids.)
After recording the fluorescence intensity during a sequence of up to 65,536 consecutive patterns (Fig. 1C), one can imagine recovering the signal x from these data by solving the optimization problem [3]. Because our measurements are noisy, it is actually better to relax the constraints into
[5]
we ask that the fit holds up to the noise level. In the following, W will be either an orthonormal basis (e.g., Dirac basis) or an overcomplete signal representation (e.g., undecimated wavelet frame or curvelet frame). The corresponding choice will be clearly specified for each individual reconstruction result. For computational reasons, we find it convenient to solve a relaxed version of this problem, namely,
[6]
As is well known, there is a value α(ϵ) such that the two programs coincide. For our experiments, we used the NESTA solver (31) and the regularization parameter α is chosen empirically depending on the noise level. When the signal is nearly sparse and the noise level low, it is known that this program finds a reconstruction with a low mean squared error (MSE).
Sparse Fluorescence Images: Beads, Cells and Tissues
Fluorescent Beads.
We first tested our CS microscope (with the 20x objective) on a sample of fluorescent beads (diameter 2 μm, peak emission at 520 nm, Fluorospheres Invitrogen) deposited on a glass coverslip. At a low density of beads, the WF image is the superposition of a few fluorescence spots on a dark background, a signal similar to that of single molecule imaging data in biology (18). As for the sparsity basis W, we obtained nearly equivalent results using the Dirac basis or a wavelet transform. Here we show images reconstructed with the wavelet transform and using a number of random 256 × 256 Hadamard patterns decreasing from 16,384 down to 512. (To be complete, we used a weighted ℓ1 norm in [6] where the weight of each coefficient is inversely proportional to scale.) In the following, the undersampling ratio is the ratio between the number N of pixels and the number M of measurements. As shown in Fig. 2, most of the bead positions are recovered with undersampling ratios up to 64, corresponding to M ∼ 1.5% of N. At higher undersampling ratios, beads with low intensities are lost.
Fig. 2.
Top left to bottom right: camera snapshot and reconstructed 256-by-256 bead images for values of the undersampling ratio equal to 8, 16, 32, 64, and 128. (A) Plot of the PSNR (see text) for a nominal illumination level (blue curve) and for the same level reduced by a factor 10 (red curve) and a factor of 100 (green curve) (simulated data). The solid lines correspond to the PSNR in raster scan for the same surfacic illumination. (B) Same as (A) for the experimental data.
To quantify the distorsion of the reconstructed image as a function of the undersampling ratio, we calculated the Peak Signal-to-Noise Ratio, PNSR = 10 log(d2/MSE) where , the squared distance between the reconstructed image from all the 256 × 256 possible measurements and that which only uses a fraction. Above, d is the dynamical range of the reconstruction obtained from a full sample. As shown in Fig. 2A, the PSNR decreases with the undersampling ratio (blue curve) and seems to reach a plateau at ratios above 64 where most of the beads are lost. Because our CS microscope is based on an active patterned illumination and the fluorescent signal emitted by the sample varies linearly (at least for moderate illumination powers) with the excitation intensity, we can adjust the SNR of our CS measurements in a very straightforward manner by controlling the excitation laser power. This situation is different from other CS-based imaging techniques such as the single pixel camera. In particular, the ability to vary the excitation intensity allows us to test the effect of the SNR of the measurements on the image reconstruction. We thus acquired a second set of measurements with an excitation light intensity divided by 100 to assess the effect of illumination on compression efficiency.
The red curve in Fig. 2A represents the PSNR of the reconstructed images as a function of the undersampling ratio. As expected, the PSNR is lower than that for the nominal illumination and reaches a plateau at an undersampling ratio of about 10, where almost all the beads are lost, clearly showing that the distortion of the reconstructed image is strongly affected by the amount of detected fluorescent photons. Indeed at such low intensities, photon noise (also termed, shot noise) may be significant.
To further explore the impact of photon noise on the compression efficiency, we performed numerical simulations on an artificial image of fluorescent beads made of 50 Gaussian spots (FWHM 3 pixels) randomly positioned in the field of view of size 256 × 256 pixels. The simulated nominal illumination intensity I0 was set so that the resulting flux (i.e., the sum of the signals over all the pixels) was equal to f0 = 6.4e3. Each measurement yk was then computed as one realization of a Poisson process with mean .
Reconstructions are processed with intensities I0, I0/10 and I0/100 for a range of undersampling ratios between 2 and 64. As shown by the PSNR curves (Fig. 2B), these simulations qualitatively reproduce the loss of compression efficiency for low-light levels but fail to quantitatively estimate the PSNR of the reconstructed images. This result suggests that photon noise is not the only source of image degradation in our imaging system. A possible additional cause is the discrepancies between the theoretical patterns and the effective illumination profiles in the sample plane.
Lily Anther Slice.
In order to investigate the potential of CFM for biological samples, we imaged slices of endogeneously fluorescent lily anther (Carolina Biological Supply). A conventional epifluorescence image of a slice (Excitation 488 nm/Emission 520 nm) recorded on a CCD camera can be seen in Fig. 3 (upper left). The resolution of this image has been sampled down to 128 × 128 pixels. We recorded the same image with the CFM setup by illuminating the sample with 16,384 = 1282 different Hadamard patterns (complete basis). For this experiment, we used a 20x lens with a 0.75 NA. Further, we used curvelets as sparsity basis W because they are known to sparsely encode contour-like structures together with the half-half strategy described earlier to account for both low-spatial frequencies shapes and higher frequencies details of the sample structure. (We again used a weighted ℓ1 norm with weights inversely proportional to scale.) Reconstructed images with varying undersampling ratios from 1 to 8 are displayed in Fig. 3 (top). Here, the method reconstructs images satisfyingly up to an undersampling ratio of about 8. Compared to fluorescent beads, this lower figure can be primarily attributed to the lesser sparsity.
Fig. 3.
Upper line: CS imaging of a slice of a lily anther. Left: Original image (128 × 128 pixels) by conventional epifluorescence microscopy. Left to right: the same sample imaged by CFM with undersampling ratios between 1 and 8. Lower line: CS imaging of COS7 cells expressing Zyxin-mEOS2. Left: superposition of the conventional epifluorescence images of the native (green) and converted form (red) of the markers. Left to right: CFM images of the converted form of the markers at undersampling ratios equal to 2, 4, 8, and 15.
Another important issue is that this sample is not fully two-dimensional. Due to the thickness of the slice (about 50 μm in this case, compared to the focal depth approximately 1 μm), the contrast of Hadamard patterns diminishes away from the focal plane. As a result, there is a nonmodulated background signal and, as further discussed below, the photon noise associated to this signal affects the image reconstruction by “hiding” the useful information contained in the intensity fluctuations due to the variations in the illumination patterns (Fig. 1E).
Zyxin-mEOS2 COS7 Cells.
In many biological applications, it is essential to use high magnification and high NA optics and we thus aimed at testing CFM in these imaging conditions (oil-immersion objective 60x, NA 1,45). To overcome the limitations due to the short focal depth of a high NA objective, we used photoactivation techniques. COS7 cells were transfected with Zyxin-mEOS2 (20). Zyxin is a protein mainly expressed in the cellular focal adhesions, at the surface on which the cells are plated. The protein was fused to mEOS2, a genetically-encoded photoconvertible fluorescent protein tag (19) widely used in superresolution microscopy, that has green fluorescence in its native state (Excitation 506 nm/Emission 519 nm) and can be converted to a red-emitting state (Excitation 573 nm/Emission 584 nm) upon illumination with violet light. The COS7 cells were plated at density of 100,000 cell/mL on 18 mm coverslips on a 12 well plate. The cells were transfected with Eos-Zyxin using chemical transfection (Fugene) 4–5 h after plating and experiments were performed on live cells 18–30 h after the transfection. By using an evanescent wave excitation with a laser at 405 nm, we could convert proteins situated at the vicinity (approximately 100 nm) of the glass coverslip. Therefore, in our sample, the green emitting fluorophores are located within the 3D cellular volume while red-emitting proteins constitute a 2D sample. The superimposed epifluorescence images in the green and red channels (256 × 256 pixels) are shown in Fig. 3 (bottom, left). The same 2D ensemble of photoconverted proteins was subsequently imaged with the CFM setup with 32,768 different 256 × 256 pixels Hadamard patterns (half of the full basis). For this set of data, the pixel size in the sample plane of each Hadamard pattern is 430 nm, about twice the diffraction limit. As for the bead images, we used a wavelet transform as sparsity basis W.
The reconstructed images for undersampling ratios varying between 2 and 15 are displayed in the second row of Fig. 3. For direct comparison with the conventional epifluorescence WF image, a dual color image obtained by superposition of the red converted Zyxin-mEOS2 image and the green native Zyxin-mEOS2 image is shown in Fig. 3. It is noteworthy that even if the fluorescence emission of transfected COS7 cells is low compared to the lily anther slice (by about a factor of 10), the quality of the reconstructed CS images is good for undersampling ratios up to 8 and starts to be degraded at a ratio of about 15. The very low background of our 2D sample clearly enables a better reconstruction. Because fluorescent proteins are very sensitive to photobleaching, one has to use low illumination to minimize this effect during acquisition. Thus, photon noise effects, which manifested themselves only at reduced illumination intensities in the bead images, appear to be a limiting factor for CS imaging of less fluorescent and/or photo-damageable samples. The impact of photon noise on CFM is discussed in more details in a later section.
Hyperspectral Imaging in CFM
Hyperspectral imaging is defined as the combined acquisition of spatial and spectral information. In biological imaging, a growing range of applications such as the study of protein localization and interactions require quantitative approaches that analyze several distinct fluorescent molecules at the same time in the same sample (22). These applications are in fact becoming ever more common with the availability of an increasing panel of fluorescent dyes and proteins with emission ranging from the UV to the far red (23).
Multicolor data are usually acquired by selecting a few distinct spectral bands. However, in many cases, the incomplete separation of the different color channels due to the presence of autofluorescence, along with cross-excitation and emission “bleed-through” of one color channel into the others render the interpretation of multiband images difficult and/or ambiguous. To overcome these limitations, it would often be preferable to record the full spectral information at each pixel of the image.
In this context, two elements make the potential benefits of CS particularly appealing for hyperspectral measurements. First, a full data acquisition can take up a very long time because the number of voxels N quickly gets very large. Second, the signal becomes comparably sparser as the dimension increases. To demonstrate the possibility of CFM for hyperspectral fluorescent imaging, we modified the setup and replaced the point detector by a spectrometer coupled to an EMCCD camera (Evolve 512, Photometrics USA). The entire spectrum between 520 nm and 640 nm is recorded on 128 × 1 pixels. We spin-coated on a glass coverslip a mixture of three types of fluorescent beads (TransFluo Beads, Invitrogen) with different emission spectra in our detection band (see Fig. 4A for a gray WF image of the sample). A complete set of 256 × 256 Hadamard patterns was subsequently projected on the sample and, for each projected pattern, we recorded the fluorescence spectrum.
Fig. 4.
(A–E) Camera snapshot and reconstructed 256-by-256 bead images for undersampling ratios equal to 1, 8, 16, and 32. (F) Normalized spectra (128 spectral lines) of three individual different beads circled in (B) for undersampling ratios equal to 1 (plain circles), 32 (squares), and 64 (triangles). The gray area in the spectrum represents a rejection band of the dichroic mirror used in our setup.
The computational reconstruction of hyperspectral data can be performed in two different manners. The simplest one is a direct extension of the monochromatic case and consists in reconstructing each spectral band independently from the others. This approach, however, does not fully account for the particular structure of the hyperspectral data. Rather, it is worthwhile to exploit sparsity in both the spatial and spectral domains. Hence, we propose a computational reconstruction by solving the same problem as in [6] with the following modification: x is now the full 2D–λ data cube and Φ and W are waveforms ϕk[i,λ] of both space and wavelength. In these experiments, W was obtained by tensorizing the Dirac basis in space—well adapted to point-wise structures like beads—with a wavelet basis along the spectral dimension which is well suited for smooth variations and occasional transients. A slice of the 2D–λ sensing matrix at a fixed λ is the same 2D partial Hadamard transform.
We obtained full color images by pooling the data cube—see Fig. 4—into three spectral bands: blue (500–530 nm), green (530–560 nm) and red (560–630 nm). Such multicolor images are shown for varying values of the undersampling ratio in Fig. 4. We observe that almost no degradation is seen for undersampling factors up to 16. Furthermore, hyperspectral reconstructions provide the spectrum of each individual bead from the reconstructed 2D–λ cube (the normalized spectra of three different beads are shown in Fig. 4F). The spectra are correctly reconstructed for undersampling factors up to 64. Interestingly, when reducing the number of measurements, the distortion primarily affects the low intensity parts of the spectra, similar to the effect of increasing undersampling on the dimmer beads in monochromatic images.
Discussion and Perspectives
We have developed an imaging approach based on the concepts of CS theory which, on samples relevant for biological imaging, allows the reconstruction of fluorescence images with undersampling ratios up to 64. While our results constitute a significant gain over undersampling ratios achieved in prior CS-based imaging approaches, several factors (computational, instrumental, or noise-related) still contribute to limit the current performances of CFM. Below, we discuss these factors as well as the prospects for future developments and applications.
Point Spread Function and Its Modeling.
In the current implementation of CFM, we neglected the role of the point spread function (PSF) of the excitation pathway (corresponding to the lenses f1,f2 and f3 and the objective) and used the idealized matrix with zeros and ones (depending whether a pixel is illuminated or not) for the inversion. In reality, the PSF acts as a low pass filter, meaning that those patterns illuminating the sample are spatially smoother than their theoretical counterparts. In the framework of CS, it is well known that recovering a compressed signal from a sensing matrix that departs from the real one may, in general, dramatically degrade the reconstruction quality. In our case, however, this approximation only has minor consequences for essentially two reasons. In our experiments, the individual pixel size of the Hadamard patterns was at least twice as large as the diffraction-limit of the microscope. In other words, the PSF only has a minor filtering impact on the patterns. Second, neglecting the effect of the PSF in the reconstruction does not dramatically reduce the quality of reconstruction. To further emphasize this fact, let f be the PSF of the lens so that the collected fluorescence is of the form (we measure the dot product between the object of interest and the theoretical patterns convolved with the PSF). If the PSF is symmetric around the origin, we have
in which F is the linear convolution with the PSF. Hence, neglecting the PSF recovers a signal obeying ; in other words, one gets which is x at the resolution of the microscope. Indeed, assuming that the lenses the excitation and emission PSFs are identical, the CSM has the same resolution as its wide-field equivalent. Now if one wishes to account for the PSF, one would need to solve a joint decompression–deconvolution problem. If the deconvolution should provide a higher resolution image, it is well known that it is usually at the cost of noise amplification. Here again, the sparsity prior used for decompression should of course help regularizing this deconvolution step.
Noise and MSE.
It is crucial to examine noise figures in our setup. There are two important facts affecting the quality of reconstruction: the noise distribution associated with CS-type data and the undersampling ratio.
We assume below that photonic noise is the limiting source of noise, an assumption which sets CS-based optical systems apart from the common theoretical CS framework (9, 24, 25). To understand the important trade-offs, we compare the respective situations of RS and CS, two point detection imaging techniques. Below, we put IRS and ICS for the excitation intensity per unit area and per unit time during RS and CS acquisition. Likewise, TRS and TCS are durations of excitation (for a single measurement). Finally, we set λRS = IRS × TRS and similarly for λCS. In practice, all these parameter values are adjusted according to factors that one wants to optimize (acquisition speed, sensitivity, photobleaching, and so on) and they must be evaluated on a case by case basis. Hence, rather than an exhaustive comparison of the relative merits of RS and CS acquisition, the discussion below aims at providing a general framework to understand the nature of the noise in CS measurements.
In the case of RS, the ith pixel measurement is distributed as a Poisson random variable with mean and variance λRS × x[i]. Using the scaled observed data yields a per-pixel MSE equal to
where . For CS, suppose first that we collect all Hadamard measurements (no undersampling). Each measurement is an independent Poisson variable with mean . With patterns of the form where hk is a Hadamard sequence, one can decompose the mean value of yk, as
Hence, this is the sum of a DC offset and a Fourier-like component. The presence of the DC offset (which prevents the optical patterns from being negative) impacts the data SNR. Indeed, a possible source of concern is that for many high-frequency components, may be much greater than the magnitude of —the DC component dominates the high-frequency coefficients. Therefore, when measuring a high-frequency component, we need to deal with a large amount of noise coming from the average fluorescence of the sample under study. This situation is arguably very different than in other applications—for instance, the acquisition of radio-frequency signals—where one can use sensing waveforms that take on negative values by switching the phase of the object we wish to acquire (21). Inverting the Hadamard matrix gives a noisy image obeying
(see Appendix for details). In contrast to RS microscopy, we see that CFM yields a spatially invariant noise level in the pixel domain. In other words, by measuring the image projection on an incoherent basis, the noise gets spread equally over all the pixels. Hence, before applying any processing, the CS situation is more favorable for recovering brighter areas but less so for dimmer regions. Summing up gives
This analysis also shows that it is essential to minimize the sources of signal which could contribute to a constant background and increase . This offset could be due to the nature of the sample itself but can also originate from stray light or out-of-focus fluorescence. These considerations explain, at least qualitatively, why the beads sample, where the background is low compared to the bright spots and is purely bidimensional, is favorable for CFM.
To consider the effect of statistical estimation procedures or data processing, consider the beads sample again in which sparsely distributed beads are located in the field of view. Each bead is an isolated bright spot surrounded by wide nonfluorescent areas. Suppose then that we were to apply a thresholding estimator, setting to zero all intensities below a certain level, and keeping those above threshold. Then one would obtain a very low MSE in the CS setting because dark pixels would be correctly set to zero while bright pixels would have a variance that is orders of magnitude lower than that achievable in the RS case. In short, thresholding would effectively filter out the off-support noise and the situation would be extremely favorable to the CS approach. Quantitatively, if there are K bright pixels, the error after estimation for RS and CS would behave like
[7]
Note the potentially enormous reduction in MSE by the factor K/N. Conversely, RS would be more effective for smooth and bright images (sparse images in the frequency domain).
The comparisons between RS and CS microscopy above are valid as long as the same illumination intensity per pixel is assumed and the number of measurements M is equal to the number of pixels. We now discuss the effect of undersampling. Now the CS recovery is both an inversion and a denoising algorithm and the recovery error depends on the compressibility of the signal (on how sparse it is). For instance, in the fully sampled case where sparsity is assumed in the spatial domain, the CS recovery would essentially invert the Hadamard matrix and then apply soft-thresholding to the output. Now suppose for simplicity that the signal is K-sparse and that the number M of measurements is sufficient for perfect recovery from noiseless data. Then the squared recovery error from M noisy measurements as above would roughly scale like
[8]
where C0 is a small numerical constant (typically on the order of unity). Hence, the main difference with the MSE available from a full sample is a loss of a factor N/M—the undersampling ratio—in the MSE, compare [7]. In other words, halving the number of samples—everything else, namely, intensity and duration of excitation remaining the same—doubles the MSE. The same conclusion applies for approximately sparse signals for which the variance component of the MSE dominates the squared bias. Here again, the more compressible the signal (the smaller K), the better the performance (e.g., the beads sample is favorable to CFM). We would like to also note that K/M is still much smaller than one so that even though we are sampling less, we may still end up with a much better MSE than in RS.
To summarize, CFM is effective when: (i) the most informative parts of the sample are brighter than its mean value and (ii) the sample is highly compressible. Notice that while these two requirements are sample dependent, the second gives some flexibility because one can select a representation in which a class of signals has an optimally sparse representation.
Impact of the Sample Thickness.
One important issue in CFM has to do with the patterned illumination of thick fluorescent samples. Indeed, with a wide-field linear excitation, the illumination propagates throughout the sample and causes the entire volume to fluoresce. Because the optical transfer function of a circular aperture (such as a microscope lens) has a bandwidth that decreases with defocusing (1), the contrast of the pattern diminishes away from the focal plane. As a result, the fluorescence coming from out-of-focus planes is not modulated as a function of the patterns. In fact, this property serves as basis for optical sectioning in structured illumination microscopy (26). In the case of CFM, the off-focus signal contributes to an offset signal on the detector, which, as explained above, tends to significantly degrade the quality of the reconstruction. A few strategies can be considered to add sectioning capabilities to CS based imaging systems (14, 27, 28). One recently demonstrated approach is based on the rejection of the off-focus signal, in a way similar to that of a programmable array microscope (14). Another method is to avoid generating any off-focus signal at all. We demonstrated this method for photoactivation using an evanescent-wave (Fig. 3) and this can be extended to 2D activation within the sample volume with two-photon temporal focusing activation (29) or light sheet illumination (30). In the long term, an even better strategy is to illuminate with an incoherent basis of 3D patterns and, subsequently, to directly reconstruct the sample in 3D.
Conclusions and Prospect
This paper presented the principles and implementation of compressive sensing in fluorescence microscopy together with its applications in biomedical imaging. Our approach, which is based on a patterned excitation of the sample combined with a point-detection of the emitted fluorescence, readily allows for substantial undersampling gains when compared to traditional raster-scanning approaches. Given the rapidly diminishing price of DMDs, CFM also constitutes a low-cost alternative to conventional wide-field imaging techniques based on onerous cameras. CS microscopy could also be useful in situations, such as a diffusing media, where direct imaging on a multipixel detector is not possible. Furthermore, we have set forth a distinctive prospect for hyperspectral acquisition, which has great potential for multicolor single molecule imaging. Interestingly, other multidimensional imaging modalities could be implemented in CFM. This is in particular the case of Fluorescence Lifetime Imaging, which holds great potential in biomedical imaging. Current wide-field approaches suffer from a limited sensitivity and rely on expensive custom-made equipment (34). A CFM-based approach combining a wide-field illumination, a single-point detector in the time domain and a dedicated reconstruction algorithm could provide a powerful alternative.
More generally, the acquisition of 3D, 4D (three spatial dimensions and one spectral or temporal dimension) or even higher-dimensional signals puts unrealistic constraints on system resources. It is indeed hard to imagine that one would want to sample such huge data cubes at rates anywhere close to the Shannon rate. The key is that multidimensional signals become increasingly redundant in the sense that their information content grows at a much lower rate than the number of voxels. For example, movies are comparably far more compressible than still pictures. Likewise, hyperspectral movies are far more redundant than monochromatic movies, and so on. Expressed differently, the ratio between the number of degrees of freedom and the number of voxels decreases very rapidly as the dimension increases. The extreme sparsity of higher-dimensional signals cannot be ignored and we expect the advantages of CFM to become paramount in such applications.
Appendix
This short appendix justifies our SNR calculations, and we begin by introducing some notation. We denote by H the Hadamard matrix and by [1] the vector with all entries equal to one. Hence, acquiring all Hadamard patterns gives us independent Poisson variables with means
Here, we set λCS = 1 as the general case can be obtained via a simple rescaling. Hence, our estimate is of the form
and it is easy to verify that
in which e1 = (1,0,…,0). (Observe that the first entry is a bit special here—a pixel at this location is always illuminated—and that we could always shift the Hadamard matrix as to select any special pixel.) Because , we have . Further,
where is the covariance matrix of the random vector . Because Cov(y) is the diagonal matrix with entries d = (d1,d2,…,dN), we have
To proceed, one verifies that
and that together with . Plugging in gives
(again this highlights the role played by the special pixel). Now suppose that the special pixel is chosen so that there is no probe at this location (which can always be arranged). Then we would know that x[1] = 0 (and would not bother estimating the density at that location) and for i ≠ 1, we would have
as claimed.
Note
The authors declare no conflict of interest.
Acknowledgments.
Deepak Nair is gratefully acknowledged for providing the Zyxin-mEOS2 transfected COS7 cells and for his careful reading of the manuscript. We thank an anonymous reviewer for pointing out the potential benefits of CFM-based methods for lifetime imaging. This work was supported by a grant from the European Aeronautic Defence and Space Foundation. M.C. is supported by the EADS foundation and the conseil régional Aquitaine. E.C. is partially supported by National Science Foundation via grant CCF-0963835 and the 2006 Waterman Award, by Air Force Office of Scientific Research under Grant FA9550-09-1-0643 and by Office of Naval Research under Grant N00014-09-1-0258.
References
1
J Mertz Introduction to Optical Microscopy (Roberts and Company Publishers, Greenwood Village, CO, 2010).
E Candes, J Romberg, T Tao, Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE T Inform Theory52, 489–509 (2006).
J Trzasko, A Manduca, B Borisch, Highly undersampled magnetic resonance image reconstruction via homotopic ℓ0 minimization. IEEE T Med Imaging28, 106–121 (2009).
Y Wu, P Ye, IO Mirza, GR Arce, DW Prather, Experimental demonstration of an optical-sectioning compressive sensing microscope (CSM). Opt Express18, 24565–24578 (2010).
E Candes, D Donoho, New tight frames of curvelets and optimal representations of objects with piecewise C2 singularities. Commun Pur Appl Math57, 219–266 (2004).
J Yoo, et al., Design and implementation of a fully integrated compressed-sensing signal acquisition system. Proc IEEE Int Conf Acoust Speech Signal Process, in press. (2012).
J Huisken, J Swoger, F Del Bene, J Wittbrodt, EH Stelzer, Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science305, 1007–1009 (2004).
Cover image: Pictured is the giant nudibranch, Dendronotus iris, with brightly colored spines and gills. This soft-bodied sea slug feeds on the burrowing anemone and sequesters toxins from its prey, which are then used as a defense. These animals are also capable of swimming to escape from predators. James M. Newcomb et al. review the evolution of swimming behaviors in shell-less mollusks at the behavioral and neuronal levels, as discussed at the Arthur M. Sackler Colloquium, “In the Light of Evolution VI: Brain and Behavior.” See the article by Newcomb et al. on pages 10669–10676 in supplement 1. Image courtesy of Charuni A. Gunaratne.
Deepak Nair is gratefully acknowledged for providing the Zyxin-mEOS2 transfected COS7 cells and for his careful reading of the manuscript. We thank an anonymous reviewer for pointing out the potential benefits of CFM-based methods for lifetime imaging. This work was supported by a grant from the European Aeronautic Defence and Space Foundation. M.C. is supported by the EADS foundation and the conseil régional Aquitaine. E.C. is partially supported by National Science Foundation via grant CCF-0963835 and the 2006 Waterman Award, by Air Force Office of Scientific Research under Grant FA9550-09-1-0643 and by Office of Naval Research under Grant N00014-09-1-0258.
Notes
This Direct Submission article had a prearranged editor.
Commissariat à l’Energie Atomique Saclay, Institut de Recherche sur les lois Fondamentales de l'Univers/Service d’Electronique des Détecteurs et d’Informatique–Service d’Astrophysique, F-91191 Gif sur Yvette Cedex, France;
Laboratoire Kastler Brossel, Centre National de la Recherche Scientifique, Unité Mixte de Recherche 8552, École Normale Supérieure, Université Pierre et Marie Curie–Paris 6, 75005 Paris, France
Author contributions: V.S., J.B., E.C., and M.D. designed research; V.S., J.B., M.C., H.S.M., and M.D. performed research; V.S., J.B., and E.C. contributed new reagents/analytic tools; V.S., J.B., E.C., and M.D. analyzed data; and V.S., J.B., E.C., and M.D. wrote the paper.
Competing Interests
The authors declare no conflict of interest.
Metrics & Citations
Metrics
Note: The article usage is presented with a three- to four-day delay and will update daily once available. Due to ths delay, usage data will not appear immediately following publication. Citation information is sourced from Crossref Cited-by service.
Citation statements
Altmetrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
This paper demonstrates how AI accelerates discoveries in archaeology, even in a region
as well known as the United Nations Educational, Scientific and Cultural Organization
(UNESCO) World Heritage site of Nazca. Our vastly improved account of ...It took nearly a century to discover a total of 430 figurative Nazca geoglyphs, which
offer significant insights into the ancient cultures at the Nazca Pampa. Here, we
report the deployment of an AI system to the entire Nazca region, a UNESCO World ...
Common sense, while often portrayed as universal, is paradoxically also often claimed
not to exist. Here, we resolve this puzzling situation by introducing a formal methodology
to empirically quantify common sense both at individual and ...The notion of common sense is invoked so frequently in contexts as diverse as everyday
conversation, political debates, and evaluations of artificial intelligence that its
meaning might be surmised to be unproblematic. Surprisingly, however, neither the
...
Origami-inspired mechanisms can transform flexible sheets into lightweight and efficient
structures. Inspired by curved-crease origami, here, we design a deployable tubular
structure featuring geometric self-locking and stiffening effects ...Deployable tubular structures, designed for functional expansion, serve a wide range
of applications, from flexible pipes to stiff structural elements. These structures,
which transform from compact states, are crucial for creating adaptive solutions ...
where the wp’s are (orthogonal) basis functions and the c[p]’s are the coefficients of x in the expansion. We say that the signal is K-sparse if at most K of these coefficients are nonzero. An empirical fact is that most images of interest are well approximated by K-sparse expansions with K much less than the number of pixels N, and this is the reason why data compression is effective; one can store and transmit quantizations of the large coefficients, ignore the small ones, and suffer little distortion.
Here, ϕk is an illumination or intensity pattern so that yk is obtained by collecting all the fluorescence corresponding to those pixels that have been illuminated on a single-point detector. Wide-field and point-like excitation are two extreme cases, corresponding respectively to a uniform sensing waveform (ϕk[i] = 1 for all i) and to a spike or Dirac waveform.
is measured by the parameter μ(W,Φ) ranging between 1 and N:
When M measurements are chosen uniformly at random from the basis Φ, the recovery is exact with very high probability; that is, the solution sequence 
obeys 
, provided that
where C is a constant on the order of unity. This result emphasizes both the role of the coherence and the potential gain for large images due to the logarithmic dependence in the pixel size. For incoherent pairs, we only need on the order of K log N random samples.
formed in the sample plane is in fact the convolution of the ideal pattern hk with the point spread function of the microscope Pexc in the excitation path. Fig. 1 C and D represent WF images of a Hadamard pattern projected on a uniform and on a biological sample. A specificity of optical imaging is that the sensing elements ϕk represent light intensities and are thus nonnegative which, as discussed later, has important practical implications.
we ask that the fit holds up to the noise level. In the following, W will be either an orthonormal basis (e.g., Dirac basis) or an overcomplete signal representation (e.g., undecimated wavelet frame or curvelet frame). The corresponding choice will be clearly specified for each individual reconstruction result. For computational reasons, we find it convenient to solve a relaxed version of this problem, namely,
As is well known, there is a value α(ϵ) such that the two programs coincide. For our experiments, we used the NESTA solver (31) and the regularization parameter α is chosen empirically depending on the noise level. When the signal is nearly sparse and the noise level low, it is known that this program finds a reconstruction with a low mean squared error (MSE).
, the squared distance between the reconstructed image from all the 256 × 256 possible measurements and that which only uses a fraction. Above, d is the dynamical range of the reconstruction obtained from a full sample. As shown in Fig. 2A, the PSNR decreases with the undersampling ratio (blue curve) and seems to reach a plateau at ratios above 64 where most of the beads are lost. Because our CS microscope is based on an active patterned illumination and the fluorescent signal emitted by the sample varies linearly (at least for moderate illumination powers) with the excitation intensity, we can adjust the SNR of our CS measurements in a very straightforward manner by controlling the excitation laser power. This situation is different from other CS-based imaging techniques such as the single pixel camera. In particular, the ability to vary the excitation intensity allows us to test the effect of the SNR of the measurements on the image reconstruction. We thus acquired a second set of measurements with an excitation light intensity divided by 100 to assess the effect of illumination on compression efficiency.
.
(we measure the dot product between the object of interest and the theoretical patterns convolved with the PSF). If the PSF is symmetric around the origin, we have
in which F is the linear convolution with the PSF. Hence, neglecting the PSF recovers a signal 
obeying 
; in other words, one gets 
which is x at the resolution of the microscope. Indeed, assuming that the lenses the excitation and emission PSFs are identical, the CSM has the same resolution as its wide-field equivalent. Now if one wishes to account for the PSF, one would need to solve a joint decompression–deconvolution problem. If the deconvolution should provide a higher resolution image, it is well known that it is usually at the cost of noise amplification. Here again, the sparsity prior used for decompression should of course help regularizing this deconvolution step.
where 
. For CS, suppose first that we collect all Hadamard measurements (no undersampling). Each measurement is an independent Poisson variable with mean 
. With patterns of the form 
where hk is a Hadamard sequence, one can decompose the mean value of yk, as
Hence, this is the sum of a DC offset and a Fourier-like component. The presence of the DC offset (which prevents the optical patterns from being negative) impacts the data SNR. Indeed, a possible source of concern is that for many high-frequency components, 
may be much greater than the magnitude of 
—the DC component dominates the high-frequency coefficients. Therefore, when measuring a high-frequency component, we need to deal with a large amount of noise coming from the average fluorescence of the sample under study. This situation is arguably very different than in other applications—for instance, the acquisition of radio-frequency signals—where one can use sensing waveforms that take on negative values by switching the phase of the object we wish to acquire (21). Inverting the Hadamard matrix gives a noisy image 
obeying
(see Appendix for details). In contrast to RS microscopy, we see that CFM yields a spatially invariant noise level in the pixel domain. In other words, by measuring the image projection on an incoherent basis, the noise gets spread equally over all the pixels. Hence, before applying any processing, the CS situation is more favorable for recovering brighter areas but less so for dimmer regions. Summing up gives
This analysis also shows that it is essential to minimize the sources of signal which could contribute to a constant background and increase 
. This offset could be due to the nature of the sample itself but can also originate from stray light or out-of-focus fluorescence. These considerations explain, at least qualitatively, why the beads sample, where the background 
is low compared to the bright spots and is purely bidimensional, is favorable for CFM.
Note the potentially enormous reduction in MSE by the factor K/N. Conversely, RS would be more effective for smooth and bright images (sparse images in the frequency domain).
where C0 is a small numerical constant (typically on the order of unity). Hence, the main difference with the MSE available from a full sample is a loss of a factor N/M—the undersampling ratio—in the MSE, compare [7]. In other words, halving the number of samples—everything else, namely, intensity and duration of excitation remaining the same—doubles the MSE. The same conclusion applies for approximately sparse signals for which the variance component of the MSE dominates the squared bias. Here again, the more compressible the signal (the smaller K), the better the performance (e.g., the beads sample is favorable to CFM). We would like to also note that K/M is still much smaller than one so that even though we are sampling less, we may still end up with a much better MSE than in RS.
Here, we set λCS = 1 as the general case can be obtained via a simple rescaling. Hence, our estimate is of the form
and it is easy to verify that
in which e1 = (1,0,…,0). (Observe that the first entry is a bit special here—a pixel at this location is always illuminated—and that we could always shift the Hadamard matrix as to select any special pixel.) Because 
, we have 
. Further,
where 
is the covariance matrix of the random vector 
. Because Cov(y) is the diagonal matrix with entries d = (d1,d2,…,dN), we have
To proceed, one verifies that
and that 
together with 
. Plugging in gives
(again this highlights the role played by the special pixel). Now suppose that the special pixel is chosen so that there is no probe at this location (which can always be arranged). Then we would know that x[1] = 0 (and would not bother estimating the density at that location) and for i ≠ 1, we would have
as claimed.
