Nonlinear structured-illumination microscopy: Wide-field fluorescence imaging with theoretically unlimited resolution

Principle

The classical resolution limit specifies a maximum spatial frequency k ₀ that can be observed through the microscope. For a light microscope, k ₀ = 2NA/λ_em, where λ_em is the observation wavelength and NA is the numerical aperture of the objective lens (26). If the frequency content of the sample is described in a 2D frequency space, then this limit defines an “observable region” (a circle with radius k ₀ around the origin). Information outside of this region is fundamentally unobservable in the conventional microscope. The goal of resolution extension is to render some of that information observable, that is, to enlarge the observable region.

Structured-illumination microscopy extends resolution beyond the cutoff by moving information into the observable region, from elsewhere in frequency space, in the form of moiré fringes. Moiré fringes are produced by frequency mixing whenever two signals are multiplied. In this case, the multiplication is the one inherent in fluorescence: the observed emission intensity is the product of the local density of fluorescent dye (i.e., the sample) and the local intensity of excitation light.

If the illumination contains a spatial frequency k ₁, then each sample frequency k gives rise to moiré fringes at the difference frequency k – k ₁. Those fringes will be observable in the microscope if |k – k ₁| < k ₀, that is, if k lies within a circle of radius k ₀ around k ₁ (Fig. 2). Thus, the information within that circle has been made indirectly observable; it can be extracted by using a phase-shift method (17). The new information increases the highest observable spatial frequency (the resolution) from k ₀ to k ₀+ k ₁. Thus, to maximize the resolution, it is desirable for the illumination to contain as high spatial frequencies k ₁ as possible. Unfortunately, the set of spatial frequencies that can be generated in a light field is limited by diffraction in the same way as the set of frequencies that can be observed. Thus, k ₁ cannot be made larger than 2NA/λ_exc ≈ k ₀, so the new resolution limit k ₀ + k ₁ can be at most ≈2k ₀. Hence, normal, linear, structured-illumination microscopy (and confocal microscopy, which is a particular case of structured illumination) can extend resolution only by a factor of ≈2.

Even this new limit can be exceeded, however, if the emission rate can be made to depend nonlinearly on the illumination intensity. Such a nonlinearity would cause the effective illumination pattern to contain harmonics with spatial frequencies that are multiples of k ₁. A component with frequency 2k ₁ would extend resolution by twice as much as the linear method, a 3k ₁ component by three times as much, etc. If the nonlinearity can be described by a polynomial of order s, it will give rise to s–1 new harmonics. A nonlinearity that is nonpolynomial (i.e., has an infinite Taylor series) thus produces an infinite number of harmonics, corresponding to theoretically infinite resolution. The resolution in practice will be finite, of course, but bounded only by “soft” matters of signal-to-noise ratio and photostability, not by any hard limit.

The particular nonlinear phenomenon discussed in this article is saturation of the excited state S ₁, as suggested in ref. 25. In the normal fluorescence process, a fluorophore molecule in its electronic ground state S ₀ is excited to the first excited state S ₁ by absorbing a photon at λ_exc and decays back to S ₀ after an average time τ (the fluorescence lifetime) as it emits a photon at λ_em. Clearly, on average each molecule can emit at most one photon per lifetime τ. Thus, it cannot respond linearly to illumination intensities above one photon per absorption cross section per lifetime (Fig. 3a ). If the sample is illuminated by a sinusoidal light pattern with a peak intensity that is near or above this threshold, then the pattern of emission rate per fluorophore takes on a nonsinusoidal shape (Fig. 3b ), which contains the desired series of harmonics with higher spatial frequency than the illumination pattern itself (Fig. 3c ).

If a sample is illuminated with such a distorted pattern, the observed image will be a superposition of several information components, each displaced in frequency space by an amount corresponding to the spatial frequency of each Fourier component of the pattern (Fig. 4d ). The number of pattern harmonics, and thus the number of contributing information components, is infinite, in principle. In practice, however, only a finite number m of information components are strong enough to rise above the noise level.

The observed image is the sum of these m displaced information components. To move the components back to their true positions in frequency space, they must first be separated, which cannot be done based on a single image. If, however, the pattern is shifted (i.e., its phase is changed by some angle ϕ) and a second image is acquired, then that image is a sum of the same components multiplied by varying powers of e^{i ϕ}. The second image represents an independent linear combination of the m components. If m or more images are acquired at different phase shifts, there are as many equations (images) as there are unknowns (information components); thus, the components can be separated by inverting an m × m matrix and applying the inverse pixelwise to the stack of m images. With the natural choice of m phases equally spaced from 0 to 2π, the separation matrix becomes equivalent to a discrete Fourier transform with respect to the phase-shift variable ϕ.

If an illumination photon flux I is applied to a fluorophore with absorption cross section σ, the emission rate E will increase exponentially with time t toward an equilibrium: E ∝ [Iσ/(Iσ + k _f)]{1 – exp[–(Iσ + k _f)t]}, where k _f = 1/τ is the fluorescent decay rate. Thus, for steady illumination or long pulses (pulse length t _p >> τ), the intensity dependence of E is the rational function [Iσ/(Iσ + k _f)], whereas short pulses (t _p << τ) produce an exponential dependence E ∝ {1 – exp[–(Iσ + k _f) t _p]}, which is somewhat more effective at generating harmonics. For this work, a pulse length of 0.64 ns was chosen, which is shorter than typical lifetimes of 2–4 ns but is not so short as to involve extreme field strengths that could increase the risk of nonlinear damage processes.

Measurements

The light source was a frequency-doubled passively Q-switched diode-pumped solid-state laser (Northrop Grumman PolyScientific, Charlotte, NC), a microchip device that is only 22 mm long. The laser produced 3.6-μJ pulses with 640-ps duration at a wavelength of 532 nm and a repetition rate of 6 kHz. The pulse energy reaching the sample was regulated by a polarization-based variable beam splitter.

In all experiments the illumination pattern was projected onto the sample by an ×100/1.4-numerical aperture planapochromatic oil-immersion objective, which was also used to image the sample onto a cooled charge-coupled device camera through a dichroic mirror and a 580- to 630-nm band-pass filter.

Demonstration of Saturation. An initial experiment was conducted to demonstrate that saturation can generate harmonics. A pattern of parallel lines was produced by passing the laser light through a transmission phase grating and projecting a demagnified image of the grating onto the sample. The pattern period (2.5 μm) was chosen to be much coarser than the resolution limit (0.19 μm), to make it possible to observe the pattern and several of its harmonics directly. The line pattern was made purely sinusoidal (apart from the overall Gaussian profile of the laser beam) by allowing only diffraction orders +1 and –1 from the grating to reach the sample. These two beams intersected at an angle in the sample to produce a sinusoidal interference pattern (which is equivalent to the demagnified grating image).

An effectively uniform thin-layer sample was prepared by diluting a suspension of 120-nm-diameter microspheres (Molecular Probes) in water and allowing a drop of suspension to air-dry on a cover glass. The concentration and volume of the drop were adjusted such that areas of close-packed monolayer formed in the drying step. Such a monolayer appears featureless in the conventional microscope, because its periodicity of nm is unresolvable. Once dry, the sample was mounted in a glycerol-based medium.

The fluorescent layer sample was illuminated by the line-pattern-modulated Gaussian beam described above. Images were exposed for 0.1 s, corresponding to ≈600 excitation pulses. At a relatively low peak energy density per pulse of 0.58 mJ/cm², the observed pattern of emission was approximately sinusoidal (Fig. 5a , bottom trace), not deviating far from the illumination intensity pattern. At a higher peak energy density of 37 mJ/cm², however, the emission pattern changed drastically, taking on broad, flat peaks and sharp, narrow valleys (Fig. 5a , top trace) as fluorescence saturated in the high-intensity regions. This behavior is in good qualitative agreement with expectations (Fig. 3b ). Fourier analysis of this pattern revealed five detectable harmonics at higher spatial frequencies than that of the illumination (Fig. 5b , top trace). The lowest harmonic is seen also at the lower pulse energy (Fig. 5b , bottom trace), indicating that the fluorescent response is noticeably nonlinear because of saturation effects already at that energy density.

SSIM Resolution Test. The hardware used for testing actual SSIM microscopy was similar to the one described above except that the period of the line pattern was decreased by increasing the demagnification from the grating to the sample. Apart from the pulsed light source, the apparatus was similar also to that used earlier for linear structured illumination (17, 18). The two beams corresponding to the +1 and –1 diffraction orders were refocused near opposite edges of the back focal-plane aperture of the objective lens and thus crossed each other in the sample at a maximal angle, generating an interference pattern of maximal spatial frequency. The period of the resulting illumination pattern was 0.20 μm, close to the theoretical limit of 0.19 μm at this wavelength. The orientation and phase of this pattern were controlled by rotating and laterally translating the grating. Capacitive sensing of the grating position, referred to a fixed cylindrical counterelectrode surrounding the piezoelectrically movable grating holder, allowed the pattern phase to be controlled with subnanometer reproducibility at any pattern orientation. To maximize the pattern contrast, the interfering beams were s-polarized by using a corotating linear polarizer on the rotation stage; the incoming beam was polarized circularly so that rotation of the stage would not affect the pulse energy.

A resolution test sample was prepared by depositing 50-nm-diameter fluorescent polystyrene microspheres on a cover glass and covering with a glycerol-based mounting medium. This sample was illuminated by the line pattern at a pulse energy density of 5.3 mJ/cm² and an exposure time of 0.15 s (900 pulses). At these parameters, only the first three harmonics of the pattern were expected to be nonnegligible, corresponding to a total of nine superposed information components (Fig. 4 c and d ). Therefore, a sequence of nine images was acquired, with the pattern phase advanced by 2π/9 radians between images. Applying the separation matrix to these nine phase images produced nine separated information components originating from nine regions of frequency space, as indicated in Fig. 4d . This procedure was repeated for 12 different pattern orientations separated by π/12 radians to retrieve complete information from a nearly circular region of frequency space (Fig. 4e ).

The precise orientation, line spacing, and starting phase of the pattern were determined from the data by comparing the different information components where they overlapped. The modulation depth of the pattern and each harmonic (or, equivalently, the relative scaling factors between the various information components) could also be determined this way, as has been done in linear structured-illumination microscopy (17, 18). In practice, it was found preferable to enter the modulation depth values manually on an empirical basis, because the signal-to-noise level in the highest information components was insufficient for reliable automatic determination of modulation depths. With a well characterized microscope, it may be possible to treat all these parameters as known system constants.

Processing was also similar to that described in ref. 17 except for the increased numbers of phase images, information components, and pattern orientations. The separated information components were computationally moved to their true position in frequency space. This translation by a vector k ₁ in reciprocal space was implemented as a multiplication with the corresponding complex wave e^{i k} ₁ ^·x in real space so as not to constrain the values of k ₁ to integer numbers of pixels of the Fourier transformed data. The components were compensated for the known optical transfer function of the microscope and combined by a weighted average (with weights inversely proportional to the local noise variance) where they overlapped. An additive constant was introduced into the denominator of the average (17, 18) to suppress low signal-to-noise regions as in a Wiener filter (26). When all components had been reassembled into a single data set in this way, a triangular apodization function was imposed to minimize the ringing that would otherwise result from the hard edge of the enlarged observable region. The final reconstruction was produced by retransforming the apodized data set to real space. The pixel size was decreased during processing, from 90 nm in the raw data images to 9 nm in the reconstructions, to accommodate the increased resolution. Comparison reconstructions were also calculated by using fewer orders of information components, with the apodization window resized accordingly.

A subset of the resulting reconstruction is shown in Fig. 6d , which demonstrates the dramatic resolution increase compared with conventional microscopy (Fig. 6a ). For additional comparison, Fig. 6c shows a reconstruction from the same data but includes only the center- and first-order information components; thus, the result represents the resolving power of linear structured-illumination microscopy without the saturation effects. Fig. 6b similarly shows a reconstruction in which only the central information component was used, representing conventional microscopy followed by a linear filter that makes the result directly comparable with the processed data.

The successive resolution enhancement is immediately apparent in Fig. 6. The small cluster in the center of the field, for example, appears nearly shapeless in the conventional image, but is seen as a distinct branched rod shape with the help of linear structured-illumination microscopy, and is resolved into its 19 individual beads when the nonlinear saturation effect is used.

To quantify the resolution of the method, intensity profiles were measured across isolated 50-nm beads (Fig. 7), and their full width at half maximum (FWHM) was determined. The average FWHM of 100 such profiles was 58.6 ± 0.5 nm, an improvement by a factor of 5.5 over the 265-nm lateral FWHM of unfiltered conventional microscopy (Fig. 7, dashed trace). The FWHM value underestimates the resolving power, however, because the object is not a point source but rather a bead of finite diameter (specified by the manufacturer as 51 nm). To allow the true point resolution to be estimated, the known bead shape was removed from the reconstruction by linear deconvolution (i.e., the Fourier transform of the reconstruction was divided by the Fourier transform of a 51-nm sphere and retransformed), and 100 bead profiles were measured as described above. The average FWHM in this corrected data set (i.e., the estimated point resolution) was 48.8 ± 0.5 nm.

Discussion

Applicability. Can resolution at the level seen in Figs. 6 and 7 be achieved in cells? In the case of SSIM, the major obstacle is that the photobleaching rate of at least some fluorophores seems to accelerate (in other words, the number of photons that each molecule can emit before being destroyed seems to be smaller) under saturating conditions compared with normal illumination intensities. At the same time, the number of detected photons required for a given signal-to-noise-ratio is greatly increased compared with a single conventional image [by the number of pattern orientations times the squared inverse of the Fourier coefficient (as plotted in Fig. 3c ) of the highest harmonic used]. Photostabilities seen in preliminary measurements [for example, ≈50,000 emitted photons per molecule of the DNA stain Po-Pro-3 (Molecular Probes), mounted in an oxygen-depleted medium and exposed to a light pattern with peak pulse energy nine times its estimated saturation threshold] would theoretically support a two-point resolution of 50 nm in 2D with a signal-to-noise ratio above unity even if each point consisted of only a single molecule. In practice, however, SSIM will probably require samples to be brightly labeled as well as photostable. Highly photostable fluorescent labels do exist, including several types of nanoparticles (27–29). Diamond nanocrystals containing nitrogen-vacancy color centers, for example, are known to fluoresce under continuous saturation conditions for hours without bleaching (28).

We have discussed 2D imaging, but the concept of SSIM, and nonlinear structured-illumination microscopy in general, is easily generalized to 3D in the same manner as in the linear case (17, 18) and, in principle, can improve resolution in all three directions. The requirement for photostability of the fluorophore, of course, increases further in proportion to the number of sections. Because the method relies on clean intensity zeros, it will also be more sensitive than conventional microscopy to sample-induced aberrations.

An important question is to what extent the method can be applied to live samples. Apart from photobleaching, the main concern is sample motion: the simple data processing described here assumes that the sample structure is stationary, within a tolerance given by the improved resolution, during acquisition. For a data set as shown in Fig. 6, this would mean <50 nm of motion over many seconds. Many intracellular processes involve much higher speeds. Thus, live imaging will most likely be restricted to slow-moving structures.

Photodamage is always a concern for live specimens. The light intensities used here are lower by >3 orders of magnitude than those commonly used for two-photon microcopy of living specimens, but the saturation conditions may increase phototoxicity.

Alternative Nonlinear Phenomena. It should be stressed that the nonlinearity used here, saturation of the first excited state S ₁, is only one example of the general concept of a nonlinear dependence of emission rate on illumination intensity. The same processing and acquisition scheme as described above could be applied equally well to other nonlinear effects, the only requirement being that to provide high resolution the nonlinearity must be nonpolynomial or of high order.

Most classical nonlinear-optics phenomena (21) such as harmonic generation are coherent in the sense that light waves produced at different points of the sample have a fixed relative phase and therefore interfere with each other. This coherence leads to a less direct relationship of the observed image to the sample structure than for fluorescence, where light waves emitted from different sample points are mutually incoherent, so that their intensities simply add. This article covers only the fluorescence case.

Multiphoton absorption fluorescence (20) is one classical nonlinear phenomenon that does produce incoherent emission light and thus fits into the framework described in this article. Two-photon fluorescence has a quadratic nonlinearity and therefore could extend resolution by twice as much as linear structured illumination of the same wavelength. That factor is misleading, however, because it refers to a comparison at constant illumination wavelength. In practice, two-photon microscopy uses nearly twice the illumination wavelength that would be used for linear excitation of the same fluorophore, and that factor of two cancels the resolution doubling from the nonlinearity. Thus, two-photon fluorescence produces no significant resolution improvement, and the same argument applies to higher orders of multiphoton absorption.

One proposed microscopy method called “ground-state depletion” is related to the present one but saturates the triplet state instead of the singlet excited state (5). Because the triplet state is associated with photobleaching (30), it is unattractive to purposely accumulate molecules in this state. In the present method, on the other hand, accumulation into the triplet state can be largely avoided by operating at a pulse-repetition rate that is slow compared to the triplet-state lifetime.

Several nonlinear effects exist, however, that do hold promise for very high resolution, including stimulated emission depletion (1–3), energy transfer in specially designed fluorophores (6), and reversible photoactivation of molecules (4, 31–35). The last category is particularly intriguing in that some promising molecules are proteins of the GFP family (32–35), suggesting the exciting possibility that the extreme resolution of nonlinear structured-illumination microscopy could be combined with the in vivo labeling power of GFP-like genetically encoded protein tags.

Conclusions

SSIM provides fluorescence imaging with in principle unlimited resolution using a single, inexpensive laser. Its ultimate resolution in practice is determined by the measurement signal-to-noise ratio, which in turn is limited by photobleaching. A 2D point resolution of <50 nm was achieved on a bead sample. The method is an example of a general concept of nonlinear structured-illumination microscopy, which has the potential to open a new, nanoscale-sized domain to investigation by wide-field light microscopy.