Introduction

Projective, or von Neumann, measurements collapse the observed quantum system on eigenstates of a Hermitian operator, while more general measurements, described by positive operator-valued measures, collapse the system on states from an overcomplete set1. A gradual transition between the two situations can be realized in continuous measurements using meter fields, a canonical example of which is an optical interferometric measurement of the lab-frame position of a harmonic oscillator2. Such measurements are associated with mechanical resonators3, collective atomic spins4,5, ferromagnetic solid-state media6, single molecules7, or density waves in liquids8, that are linearly probed by traveling optical or microwave fields. The boundary between generalized and von Neumann measurements occurs at a certain value of the measurement rate9,10,11. When the rate is slower than the oscillation, measurements with the meter in the vacuum state project the oscillator on coherent states. When the rate is faster than the oscillation, they project the oscillator on position-squeezed states.

The output state of the meter field changes with the measurement regime as much as the oscillator state itself10. The quadratures of the meter are correlated to the extent that their fluctuations can be below the vacuum level at certain quadrature angles12,13. In the slow-measurement regime, due to the time-averaged response of the oscillator to the measurement backaction, the correlations and the associated squeezing exist in a narrow frequency band near the resonance. When the measurement is faster than the oscillation, the oscillator responds to the backaction nearly instantaneously, and the correlations and squeezing are broadband. The detection of such broadband squeezing of light implies observing the backaction-driven motion of the oscillator at frequencies well below the resonance frequency, which is a necessary condition for position squeezing10.

The squeezing of the meter light is both a valuable quantum resource and a figure of merit for the purity of the light–oscillator interaction. In the slow regime, we realize a measurement of a collective spin of a room-temperature atomic ensemble at a rate fifteen times higher than the rate of thermal decoherence. The generated squeezing of the meter light reaches \(11.{5}_{-1.5}^{+2.5}\,{{{{{{{\rm{dB}}}}}}}}\) at the output of the cell, substantially exceeding the squeezing demonstrated previously using collective atomic spins14,15,16, optomechanical cavities17,18,19, levitated nanoparticles20,21, and compact on-chip sources utilizing material nonlinearity22, while approaching the results achievable using bulk nonlinear crystals23. In the fast-measurement regime, we detect broadband squeezing in a bandwidth of several MHz while keeping the backaction-imprecision product24 within 20 % from the value saturating the Heisenberg uncertainty relation. These results enable new regimes for sensing surpassing the standard quantum limit25,26, tests of uncertainty relations for past quantum states27,28, quantum control of material oscillators29,30,31,32,33, and links between collective spins and other material systems16,34,35,36.

Results

Measurements of spin oscillators

Linearly polarized light traveling through an oriented atomic medium (as illustrated in Fig. 1a, b) continuously measures the projection of the total spin on the propagation direction, \({\hat{J}}_{z}\), via polarization rotation. This measurement acts back on the spin via quantum fluctuations of optical torque. When the input light is in a strong coherent state, and the spin satisfies the Holstein–Primakoff approximation37, the process can be described in terms of linearly coupled pairs of canonically conjugate position and momentum variables. The canonical variables of the spin, \({\hat{X}}_{{{{{{{{\rm{S}}}}}}}}}\) and \({\hat{P}}_{{{{{{{{\rm{S}}}}}}}}}\), are the normalized projections defined as \({\hat{X}}_{{{{{{{{\rm{S}}}}}}}}}={\hat{J}}_{z}/\sqrt{\hslash \langle \,{J}_{x}\rangle }\) and \({\hat{P}}_{{{{{{{{\rm{S}}}}}}}}}=-{\hat{J}}_{y}/\sqrt{\hslash \langle \,{J}_{x}\rangle }\), which satisfy the commutation relation \([{\hat{X}}_{{{{{{{{\rm{S}}}}}}}}},\,{\hat{P}}_{{{{{{{{\rm{S}}}}}}}}}]=i\). The variables of the light, \({\hat{X}}_{{{{{{{{\rm{L}}}}}}}}}\) and \({\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}\), are the quadratures proportional to the amplitude and phase differences between the circularly polarized components, respectively. Their commutator is \([{\hat{X}}_{{{{{{{{\rm{L}}}}}}}}}(t),\,{\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}({t}^{{\prime} })]=(i/2)\delta (t-{t}^{{\prime} })\). The fluctuations in this measurement are characterized by the spectrum of the detection noise (including shot noise) normalized by the total spin noise power, which is called imprecision and denoted Simp, and the spectrum of the generalized measurement backaction force acting on the spin, SBA. The Heisenberg uncertainty principle constrains the imprecision and the backaction spectra as \(\sqrt{{S}_{{{{{{{{\rm{imp}}}}}}}}}\,{S}_{{{{{{{{\rm{BA}}}}}}}}}}\ge \hslash /2\) (the spectra are two-sided; see SI Sec. C and ref. 24).

Fig. 1: Quantum measurements of spin quadratures via Faraday rotation.
figure 1

a An optical probe spatially shaped in a square tophat beam travels through an atomic ensemble with the total spin J in a magnetic field B, and is detected using balanced polarization homodyning. The detected quadrature is selected using the λ/4 and λ/2 waveplates. The total spin is oriented by the repump beam traveling along x. PBS: polarization beam splitter. b The polarization angle δα of the probe as a meter for the spin projection \({\hat{J}}_{z}\). c A photograph of an anti-spin-relaxation coated cell. The channel with probed atoms is indicated by the blue rectangle. d The orange curves show power spectral densities (PSD) of homodyne signals recorded at Δ/(2π) = 7 GHz at different quadratures. The trace showing the largest squeezing is highlighted by the blue curve. The black curve is the theoretical prediction based on the global fit, including all quadratures (see the SI). The gray curve is the shot-noise level. The red curve is the theoretical optimum-quadrature squeezing spectrum.

Full size image

When the probing light is far detuned from optical transitions in the ensemble, the total spin couples to the probe via the position-measurement Hamiltonian \({\hat{H}}_{{{{{{{{\rm{int}}}}}}}}}=-2\hslash \sqrt{{{\Gamma }}}\,{\hat{X}}_{{{{{{{{\rm{L}}}}}}}}}{\hat{X}}_{{{{{{{{\rm{S}}}}}}}}}\), and modifies the probe variables according to the input–output relations16,38

$${\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}^{{{{{{{{\rm{out}}}}}}}}}(t)={\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}^{{{{{{{{\rm{in}}}}}}}}}(t)+\sqrt{{{\Gamma }}}\,{\hat{X}}_{{{{{{{{\rm{S}}}}}}}}}(t),\,{\hat{X}}_{{{{{{{{\rm{L}}}}}}}}}^{{{{{{{{\rm{out}}}}}}}}}(t)={\hat{X}}_{{{{{{{{\rm{L}}}}}}}}}^{{{{{{{{\rm{in}}}}}}}}}(t),$$
(1)

where Γ ∝ ΦN/(AΔ2) is the measurement rate expressed as a function of the photon flux Φ, the number of atoms N, the cross section area of the beam A, and the probe detuning from the optical transition Δ34. The measurement rate determines the imprecision \(\hat{z}={\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}^{{{{{{{{\rm{in}}}}}}}}}/\sqrt{{{\Gamma }}}\), and the quantum backaction force \({\hat{F}}_{{{{{{{{\rm{QBA}}}}}}}}}=2\hslash \sqrt{{{\Gamma }}}{\hat{X}}_{{{{{{{{\rm{L}}}}}}}}}^{{{{{{{{\rm{in}}}}}}}}}\). Ideally, the product of their spectral densities saturates the Heisenberg uncertainty relation, \(\sqrt{{S}_{{{{{{{{\rm{imp}}}}}}}}}{S}_{{{{{{{{\rm{BA}}}}}}}}}}=\hslash /2\). When the detection efficiency is below unity, or there are classical noises in the experiment, the backaction-imprecision product is always higher.

The response of the spin to the measurement backaction is described by the Fourier-domain susceptibility \(\chi [{{\Omega }}]={{{\Omega }}}_{{{{{{{{\rm{S}}}}}}}}}/({{{\Omega }}}_{{{{{{{{\rm{S}}}}}}}}}^{2}-{{{\Omega }}}^{2}-i{{\Omega }}{\gamma }_{0})\), where ΩS is the resonance Larmor frequency and γ0 is the intrinsic decay rate. The response induces correlations between \({\hat{X}}_{{{{{{{{\rm{L}}}}}}}}}^{{{{{{{{\rm{out}}}}}}}}}\) and \({\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}^{{{{{{{{\rm{out}}}}}}}}}\) that can be observed by detecting intermediate quadratures of light, \({\hat{Q}}_{{{{{{{{\rm{L}}}}}}}}}^{\phi }=\sin (\phi ){\hat{X}}_{{{{{{{{\rm{L}}}}}}}}}^{{{{{{{{\rm{out}}}}}}}}}+\cos (\phi ){\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}^{{{{{{{{\rm{out}}}}}}}}}\). The spectra of those quadratures, detected by a homodyne detector with efficiency η, are given by

$${S}_{\phi }[{{\Omega }}]=1/4+(\eta {{\Gamma }}/2)\,{{{{{{{\rm{Re}}}}}}}}\left(\chi [{{\Omega }}]\right)\sin (2\phi )+\eta {{\Gamma }}({{\Gamma }}+{\gamma }_{{{{{{{{\rm{th}}}}}}}}})| \chi [{{\Omega }}]{| }^{2}\cos {(\phi )}^{2},$$
(2)

where γth = (2 nth + 1)γ0 is the thermal decoherence rate, and nth is the temperature of the intrinsic bath expressed via the number of excitations (see SI Sec. B3). The term \(\propto \cos {(\phi )}^{2}\) is due to the spin oscillator motion, and the term \(\propto \sin (2\phi )\) is due to the cross-correlation between \({\hat{X}}_{{{{{{{{\rm{S}}}}}}}}}\) and \({\hat{X}}_{{{{{{{{\rm{L}}}}}}}}}^{{{{{{{{\rm{out}}}}}}}}}\).

Negative cross-correlation can squeeze Sϕ[Ω] below the vacuum level of 1/4, with the maximum squeezing being at frequencies around ΩS, where the measurement backaction is resonantly enhanced. The bandwidth around the resonance in which the squeezing is produced is proportional to the measurement rate. As the measurement rate reaches the oscillation frequency, the squeezing band extends to dc.

While the single-oscillator model of the collective spin is sufficient for many purposes, the actual spin has multiple modes, whose annihilation operators can be introduced using the multilevel Holstein–Primakoff approximation39 as

$${\hat{b}}_{m}=\frac{1}{\sqrt{{{\Delta }}{N}_{m}}}\sum\limits_{j=1}^{N}{\left\vert m+1\right\rangle }_{j}{\left\langle m\right\vert }_{j}.$$
(3)

Here, j = − F, . . , F − 1, where F is the ground-state angular momentum number of the atomic species, m is the projection quantum number of the single-atom angular momentum on the x axis, \({\left\vert m+1\right\rangle }_{j}{\left\langle m\right\vert }_{j}\) are the jump operators between the states \({\left\vert m\right\rangle }_{j}\) and \({\left\vert m+1\right\rangle }_{j}\) of the individual atoms, and ΔNm = Nm+1 − Nm are the differences in the mean numbers of atoms in the corresponding states. The frequencies of the oscillators are the energy differences between \({\left\vert m\right\rangle }_{j}\) and \({\left\vert m+1\right\rangle }_{j}\), controlled by an external static magnetic field. The oscillator–light interaction is described by the Hamiltonian (as derived in SI Sec. B, see also refs. 16,40,41)

$${\hat{H}}_{{{{{{{{\rm{int}}}}}}}}}=-2\hslash \sum\limits_{m=-F}^{F-1}\sqrt{{{{\Gamma }}}_{m}}\left({\hat{X}}_{m}{\hat{X}}_{{{{{{{{\rm{L}}}}}}}}}+{\zeta }_{m}{\hat{P}}_{m}{\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}\right),$$
(4)

where the quadratures of the modes satisfy \([{\hat{X}}_{m},{\hat{P}}_{m}]=i\), Γm are the measurement rates, and ζm = ζ(2m + 1)/7 determine the strengths of dynamical backaction (which is a coherent feedback from the light, SI Sec. B and refs. 42,43,44,45). The factor ζ depends on the atomic level structure and is inversely proportional to the detuning from the optical transition Δ. The effect of the dynamical backaction is optical damping with rates γDBA,m = 2ζmΓm, and an increase in the quantum backaction-imprecision product by an amount proportional to ζ2 (see the SI), a small number in all our experiments.

The multimode structure of the spin can complicate the interpretation of measurements using the spin as a probe because multiple modes respond to external signals and the measurement backaction in a more complex way than a single mode. Under the conditions of our experiment, however, the response difference is only significant at frequencies close to ΩS, while far away from ΩS the spin always behaves as a single oscillator with \({\hat{X}}_{{{{{{{{\rm{S}}}}}}}}}=\sum\limits_{m}\sqrt{{{{\Gamma }}}_{m}/{{\Gamma }}}{\hat{X}}_{m}\) measured at the total rate Γ = ∑mΓm and experiencing the decoherence rate γth = ∑mγth,mΓm/Γ, where γth,m are the individual decoherence rates of the modes.

Experimental setup

An ensemble of N ≈ 2 × 1010 cesium-133 atoms at 52 °C is contained in the 1 mm × 1 mm × 4 cm channel of a glass chip, shown in Fig. 1c. The channel is coated with paraffin to reduce the spin decoherence from wall collisions46, and is positioned in a homogeneous magnetic field directed along the x axis (Fig. 1a). The ensemble is continuously probed by a y − polarized laser beam propagating in the z direction that has the wavelength 852.3 nm, blue-detuned from the \(F=4\to {F}^{{\prime} }=5\) transition of the D2 line by Δ/(2π) = 0.7 − 7 GHz. The ensemble is also continuously repumped using circularly polarized light resonant with the \(F=3\to {F}^{{\prime} }=2,\,3,\,4\) transitions of the D2 line (see the SI for details). The combination of spontaneous scattering of probe photons and repumping maintains a steady-state distribution of atoms over the magnetic sublevels of the F = 4 ground state, which has the macroscopic spin orientation along the magnetic field with polarization \(\langle \,{\hat{J}}_{x}\rangle /(NF)\, \approx \,0.78\). The steady-state populations are independent of the probe power in our regime, and correspond to the occupancy of the thermal bath nth = 0.9 ± 0.1 (see SI Sec. B3, and refs. 47,48). The resonance frequencies of the oscillators are set by the Larmor frequency and split by 0–40 kHz in different regimes by the quadratic Zeeman and tensor Stark effects. The Larmor frequency can be positive or negative depending on the orientation of the magnetic field, setting the signs of the effective oscillator masses. We work in the negative-mass configuration34, but the effects that we observe, in particular the squeezing levels, do not change upon the reversal of the sign of the mass (see the SI). The output light is detected using balanced polarization homodyning, which enables shot-noise-limited detection at frequencies down to 10 kHz.

High squeezing of light in a tunable narrow frequency band

In Fig. 1d, we present homodyne spectra recorded at the optical detuning Δ/(2π) = 7 GHz over a range of detection quadratures ϕ. In this measurement, dynamical backaction effects are small (ζ ≈ 0.01), and the probed spin behaves as a single oscillator subjected to position measurements. The data in Fig. 1d show squeezing down to 7.5 dB, attained by the highlighted blue trace. From a global fit of the spectra at all quadratures using an expression that generalizes Eq. (2) to the case of finite ζ and extra detection noise (see SI Sec. D), we infer the measurement rate Γ/(2π) = 13 kHz and the quantum cooperativity \({{{{{{{{\mathcal{C}}}}}}}}}_{{{{{{{{\rm{q}}}}}}}}}={{\Gamma }}/{\gamma }_{{{{{{{{\rm{th}}}}}}}}}=11\).

The measurement rate can be estimated directly from Fig. 1d via the bandwidth ΔΩ over which squeezing is present in any of the traces. This bandwidth is the range of optical sideband frequencies for which the nonlinearity of the atomic medium is high enough to modify the variance of the input optical states at the scale of vacuum fluctuations. In the backaction-dominated regime relevant to our experiments, ΔΩ ~ Γ.

The envelope of the traces in Fig. 1d is described by the spectrum given by Eq. (2) minimized over the detection quadrature ϕ at each frequency. Neglecting the imaginary part of the response, the optimum-quadrature spectrum is given by

$${S}_{\min }[{{\Omega }}]=\frac{1}{4}-\frac{\eta }{2}\frac{{{\Gamma }}}{{{\Gamma }}+{\gamma }_{{{{{{{{\rm{th}}}}}}}}}}\,D\left(\frac{{{\Omega }}-{{{\Omega }}}_{{{{{{{{\rm{S}}}}}}}}}}{{{\Gamma }}+{\gamma }_{{{{{{{{\rm{th}}}}}}}}}}\right),$$
(5)

where \(D(x)=1/\left(1+\sqrt{1+4{x}^{2}}\right)\). The red curve plotted in Fig. 1d additionally accounts for 0.7 shot noise units of excess \({\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}\)-quadrature noise from the thermal motion of fast-decaying spin modes (see II F). This noise is the main limitation for the backaction-imprecision product in this measurement, which equals 1.5 × (ℏ/2).

Due to the scaling Γ ∝ 1/Δ2, higher measurement rates are achievable with the probe laser tuned closer to the atomic transition. In Fig. 2a, we present data obtained at the optical detuning of 3 GHz using 8.4 mW of probe power. In this measurement ζ = 0.054, and the dynamical backaction results in optical damping and hybridization of the oscillator modes (SI Sec. B), as well as optical squeezing in the \({\hat{X}}_{{{{{{{{\rm{L}}}}}}}}}\)-quadrature (see the green trace in Fig. 2a). Since the thermal decoherence of the oscillators is due to baths at a temperature close to zero, the optical damping improves the maximum magnitude of squeezing by about 0.5 dB. The minimum noise shown by the blue trace in Fig. 2a is \(8.{5}_{-0.1}^{+0.1}\,{{{{{{{\rm{dB}}}}}}}}\) below the shot-noise level. The overall detection efficiency of our setup is η = (91 ± 3) %, and the transmission loss at the exit window of the cell is 1.6 %, which means that the magnitude of the squeezing at the exit of the cell is \(11.{5}_{-1.5}^{+2.5}\,{{{{{{{\rm{dB}}}}}}}}\). The backaction-imprecision product in this measurement is 1.9 × (ℏ/2), which is higher than in the measurement at 7 GHz detuning due to the higher excess \({\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}\)-quadrature noise (two shot-noise units).

Fig. 2: Squeezed light generation by collective spin modes.
figure 2

a Homodyne signal PSDs at Δ/(2π) = 3 GHz and different detection angles ϕ indicated in the figure. The points are experimental data. The green and orange traces are obtained close to \({\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}\) and \({\hat{X}}_{{{{{{{{\rm{L}}}}}}}}}\), respectively, and the olive, blue and purple—at intermediate quadratures. The gray trace shows the shot-noise level. The black curves are theoretical predictions based on the global fit over 15 quadratures (only 5 are shown, the rest are presented in the SI). The red curve is the optimum-quadrature squeezing spectrum predicted by the single-oscillator model. b The spectra of classically driven motion of the collective spin. The eight peaks visible at low probe powers correspond to bare oscillator modes due to the transitions between adjacent mF levels. Their frequencies are determined by the linear and quadratic Zeeman energies, and magnitudes are determined by the macroscopic populations of the mF levels as shown in the inset. The spectra at high powers expose the hybridized oscillator modes.

Full size image

Hybridization of spin modes via coupling to light

The experimental spectra in Fig. 2a can be understood as arising from the coupled dynamics of two nearly degenerate bright modes of the spin, which we refer to as modes a and b. To extract their effective parameters, we globally fit the set of spectra recorded over an extended range of quadrature angles (SI Sec. D). We find the total measurement rate Γ/(2π) = 52 kHz, the individual quantum cooperativities \({{{{{{{{\mathcal{C}}}}}}}}}_{{{{{{{{\rm{q}}}}}}}}}^{a}=12\) and \({{{{{{{{\mathcal{C}}}}}}}}}_{{{{{{{{\rm{q}}}}}}}}}^{b}=4\), and the total cooperativity \({{{{{{{{\mathcal{C}}}}}}}}}_{{{{{{{{\rm{q}}}}}}}}}=15\). The lower envelope of the experimental traces is in agreement with the optimum-quadrature spectrum predicted by the single-oscillator model using the same Γ and \({{{{{{{{\mathcal{C}}}}}}}}}_{{{{{{{{\rm{q}}}}}}}}}\).

The bright modes a and b emerge due to the coupling of the individual spin oscillators via the common reservoir of the probe optical modes with coupling rates proportional to ζm and Γm. This coupling approximately scales with the laser detuning as 1/Δ3 at fixed probe power. To illustrate the effect of intermodal coupling, we reduce the laser detuning to 0.7 GHz, where ζ = 0.18. We excite the oscillators with classical white magnetic field noise, and record their driven motion by measuring the P quadrature of the output light. The spectra at different probe powers are shown in Fig. 2b. At the lowest power, the eight bare spin oscillators due to the transitions between adjacent mF levels are individually resolved. As the probe power is increased, the resonances first merge in two (the a and b modes) and then into three peaks. The macroscopic occupancies of different mF levels in the atomic ensemble remain the same at all powers, as is separately checked via magneto-optical resonance signal47, which means that the change in the output spectrum is only due to the coupled dynamics of the collective oscillators.

Measurements of the spin at the rate of its precession

At the detuning of 0.7 GHz from the optical transition, the measurement rate of the spin motion can be as high as the oscillation frequency, and the quantum measurement backaction manifests itself via broadband squeezing of light. While the coupling between individual spin oscillators is pronounced at this detuning, at frequencies much lower than ΩS the spin behaves as a single oscillator. In Fig. 3a, we present spectra recorded using 12.8 mW of optical probe power at two resonance frequencies, 1.09 MHz and 1.79 MHz, in which the bandwidth of low-frequency squeezing extends down to 30 kHz. The minimum noise levels of the homodyne signals (6.5 dB below the shot noise for the 1.09 MHz data) are consistent with the quantum cooperativity \({{{{{{{{\mathcal{C}}}}}}}}}_{{{{{{{{\rm{q}}}}}}}}}=8\). The measurement rate can be estimated from the signal-to-shot-noise ratio on the P quadrature in Fig. 3a using Eq. (2) in the limit Γ ≫ γth,

$${S}_{\phi=0}[{{\Omega }}=0]=1/4+\eta {\left({{\Gamma }}/{{{\Omega }}}_{{{{{{{{\rm{S}}}}}}}}}\right)}^{2},$$
(6)

which yields Γ/(2π) ≈ 2 MHz, a value higher than the resonance frequencies. To further corroborate the measurement rate, we perform a quadrature sweep with the resonance frequency set to 5 MHz and using 10.2 mW of probe power (Fig. 3b). From fitting this data, we find Γ/(2π) = 1.77 MHz, which is consistent within ten percent with the previous estimate corrected for the difference in the probe powers. Theoretically, the optimum-quadrature noise levels should saturate as the Fourier frequency approaches zero, to a value around 0.22 shot-noise units for the 1.09 MHz data in Fig. 3a. In the experiment, the noise levels increase at low frequencies due to excess noise from the atomic ensemble.

Fig. 3: Measurements of spin oscillators outside the rotating-wave regime.
figure 3

a, b Homodyne signal PSDs at Δ/(2π) = 0.7 GHz. The gray curves show the experimental shot-noise levels, and the red curves are the theoretical optimum-quadrature squeezing spectra derived from Eq. (2). a Spectra for ∣ΩS∣/(2π) = 1.09 MHz and 1.79 MHz. The orange and blue curves are measurements with the quadrature angle set to \({\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}\) (ϕ ≈ 180∘) and close to \({\hat{X}}_{{{{{{{{\rm{L}}}}}}}}}\) (ϕ ≈ 80∘), respectively. LO: local oscillator, th: theoretical. b Orange curves show homodyne spectra recorded at ∣ΩS∣/(2π) = 5 MHz and at different quadratures ϕ. The trace with the largest squeezing is highlighted by the blue curve. The black curve is the theoretical prediction based on the global fit including all quadratures (see the SI). c The spectra taken at the \({\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}\) quadrature when the probe beam is Gaussian (blue curve) and tophat (orange curve). The gray curve is the shot noise. The inset shows the beam intensity distributions over the 1 mm × 1 mm channel cross section recorded without the cell.

Full size image

The backaction-imprecision product for the measurements in Fig. 3a is below 1.2 × (ℏ/2) at frequencies higher than 100 kHz. This value is closer to saturating the Heisenberg uncertainty relation than the values in the slow-measurement experiments, because the fast-decaying modes are in the backaction-dominated regime, and do not contribute excess thermal noise. The limiting factors for the product in this case are the dynamical backaction and detection inefficiency.

Fast-decaying spin modes

In addition to the collective oscillators described by the annihilation operators from Eq. (3), in which all atoms contribute equally, there are other modes of the spin in our system40,49. The resonance frequencies of these modes coincide with ΩS, but their decay rates are limited by the rate of atoms flying through the probe field (γ0,flight/(2π) ≈ 300 kHz) rather than collisions with the walls and other atoms (γ0,coll/(2π) ≈ 200 Hz). The annihilation operators of these modes are

$${\hat{b}}_{m}^{{\prime} }=\frac{1}{\sqrt{{{\Delta }}{N}_{m}\,{\left\langle {{\Delta }}g{(t)}^{2}\right\rangle }_{{{{{{{{\rm{c}}}}}}}}}}}\sum\limits_{j=1}^{N}{{\Delta }}{g}_{j}(t)\,{\left\vert m\right\rangle }_{j}{\left\langle m+1\right\vert }_{j},$$
(7)

where gj(t) are the coupling rates between the optical probe and the individual atoms (SI Sec. B) and \({\langle {{\Delta }}{g}^{2}\rangle }_{{{{{{{{\rm{c}}}}}}}}}\) is the squared deviation of the coupling from the mean averaged over classical trajectories, assumed to be the same for all atoms. The measurement rate of the fast-decaying modes is \(\propto {\langle {{\Delta }}{g}^{2}\rangle }_{{{{{{{{\rm{c}}}}}}}}}\), while the measurement rate of the slow-decaying modes is \(\propto {\langle g\rangle }_{{{{{{{{\rm{c}}}}}}}}}^{2}\).

An enabling feature of our experiment is the high 3D uniformity of the optical probe field, achieved using a tophat beam configuration, which reduces \({\langle {{\Delta }}{g}^{2}\rangle }_{{{{{{{{\rm{c}}}}}}}}}\) and thus the readout of the fast-decaying modes. In Fig. 3c, we compare the spectra recorded at the \({\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}\)-quadrature using a tophat and a wide Gaussian probe beam (the exact beam profiles are given in SI Sec. A) with equal optical powers in the slow-measurement regime. The thermal noise contributed by the fast-decaying modes is reduced from 1 to 0.3 shot-noise units on resonance upon switching from the Gaussian to the tophat probe. The absolute non-uniformity of the coupling50,51 for the tophat beam is estimated to be \({\langle {{\Delta }}{g}^{2}\rangle }_{{{{{{{{\rm{c}}}}}}}}}/{\langle g\rangle }_{{{{{{{{\rm{c}}}}}}}}}^{2}=0.6\) based on the camera imaging.

Discussion

Continuous measurements that combine high measurement rate, quantum cooperativity, and detection efficiency can be used for single-shot generation of spin-squeezed states and quantum state tomography52. The entanglement link between the material spin and traveling light entailed by the squeezing enables quantum-coherent coupling of spins with other material systems16,35. While the backaction-imprecision product in all our measurements is already within a factor of two from the Heisenberg bound, it can be further improved by optimizing the probe power for measurements of the \({\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}\)-quadrature. Our measurements were optimized for quadratures intermediate between \({\hat{X}}_{{{{{{{{\rm{L}}}}}}}}}\) and \({\hat{P}}_{{{{{{{{\rm{L}}}}}}}}}\) (i.e., for “variational" readout25), which can yield superior results53 in quantum sensing and control.

This work also establishes room-temperature atomic spin oscillators as a practical platform for engineering quantum light with high levels of squeezing, which is a basic resource for interferometric sensing and optical quantum information processing22. The highest demonstrated squeezing, reaching 8.5 dB at the detection, is narrowband, but its frequency can be tuned by the magnetic field without degrading the level within the range of ~0.8–5 MHz in our experiments.

Methods

Extended details of the measurement setups, additional experimental data, and the theoretical description of the spin–light interaction are presented in Supplementary Information.