To achieve quantum entanglement between strongly interacting particles, I designed and built a new experimental setup together with my team members. To define the parameter space in our experiment, we developed a quantum Langevin model featuring optimal control and active feedback based on continuous measurement. To achieve the high net charge necessary for strong interaction, we implemented several active and passive charging methods, including charging particles by creating plasma inside the vacuum chamber near optical traps or through directed ion flux onto the trapped particles and by modifying the particle surface chemically to achieve a high charge-to-mass ratio. By combining both active and passive charging methods, we successfully brought the experimentally accessible parameter space to the level necessary for quantum entanglement in the conditional state. To bring the normal modes of strongly interacting particles to the ground state of motion, we developed split homodyne detection with a high information collection efficiency of approximately 50%. This efficiency was achieved by using a custom-built trapping lens with an NA of 0.8 optimizing the design of the particle imaging system and mode-matching, and employing a custom-built balanced photodetector that provides up to 94% detection efficiency with an input power of 15mW. This was determined experimentally; the theoretical limit stands at 99%. We then developed an optimal protocol to cool both normal modes to the quantum ground state based on cold damping through a single pair of electrodes and laser parametric cooling addressing each particle individually. The protocol enables ground state preparation for both differential and common normal modes. Towards entanglement generation, we have made significant progress in developing an original protocol that brings our system into the entangled state within the experimental parameter set. Our approach is two-fold; it involves advanced quantum control based on a specifically optimized cost function of our time-dependent Kalman filter combined with modulation of the long-range interaction between particles. Numerical simulations demonstrate the entanglement of both conditional and unconditional states under realistic experimental conditions. Some of our findings related to motion control and Heisenberg-limited detection have been recently published in the manuscript: Reisenbauer et.al "Non-Hermitian dynamics and nonreciprocity of optically coupled nanoparticles," available at arXiv:2310.02610 (2023),
https://doi.org/10.48550/arXiv.2310.02610.
In addition to the main course of the project, I developed some independent ideas towards the experimental realization of the vibrational Bose-Einstein condensate (BEC) and generation of optomechanical-type entanglement between collective vibrational and electronic light-matter states at room temperature. In particular, I propose a novel macro-coherent regime in molecular systems with strong vibronic coupling. In a recent manuscript: Shishkov et.al "Mapping polariton Bose--Einstein condensate onto vibrational degrees of freedom," arXiv:2309.08498 (2023),
https://doi.org/10.48550/arXiv.2309.08498 we introduce an innovative methodology for vibrational amplification in a resonant blue-detuned configuration consistent with state-of-the-art experiments. Our research takes a step forward in the direction of vibrational BEC, offering potential advancements in cavity-controlled chemistry, as well as nonlinear and quantum optics.