DeepLens is a differentiable optical lens simulator for computational imaging, supporting multiple optical models (e.g., ray tracing, wave propagation, ray-wave model, surrogate PSF network). DeepLens can be used for (1) end-to-end optics-algorithm co-design, (2) automated optical design, and (3) image simulation for synthetic datasets.
DeepLens helps researchers and engineers rapidly prototype and design custom optical systems. DeepLens is also AI-native: ask your agent to call DeepLens directly, so that its optical reasoning is grounded in real simulation, for example, use DeepLens to compute the MTF of this lens.
- 08/14/2025: DeepLens joins OpenAI's "Codex for Open Source". We will build AI-native optical simulation and design tools.
- 08/09/2025: We released DeepO, an AI optical design platform.
Commercial optical design software (Zemax, Code V, etc.) is built for a human designer clicking through a GUI. DeepLens is built for optics that lives inside a PyTorch training loop, where the lens is a set of learnable parameters optimized jointly with a neural network, and gradients flow through the optics.
| DeepLens | Zemax / Code V | |
|---|---|---|
| Gradients | Analytic, via autograd | Finite differences |
| Python | Native (tensors in, tensors out) | External API bridge |
| Hardware | GPU (CUDA / ROCm), batched | Mostly CPU-bound |
| End-to-end co-design | Optics + ISP + network in one graph | Not supported |
| Agent-scriptable | Plain Python, no GUI in the loop | GUI-centric |
| License | Free, Apache-2.0 | Commercial |
DeepLens supports comprehensive lens analysis (spot diagram, PSF, MTF, distortion, etc.) and photorealistic image simulation with spatially varying, depth-dependent aberrations.
Fully automated lens design from scratch with gradient-based optimization and advanced optimization algorithms.
Note: Automated lens design is now actively maintained in the AutoLens project. If your focus is automated lens design, we recommend using the AutoLens repo instead, as it receives dedicated updates and improvements for this use case.
A surrogate network for efficient lens PSF representation, supporting fast and accurate image simulation with spatially varying aberrations and defocus.
Differentiable ray-wave optical model for accurate lens aberration and diffraction element simulation, supporting end-to-end refractive-diffractive lens design.
Non-sequential polarization tracing to accurately simulate the polarization state of light passing through a geometric waveguide AR display. End-to-end optimization for coating film inverse design targeting the out-coupling eyebox response.
DeepLens serves as the differentiable optics engine in End2endImaging, an end-to-end differentiable computational imaging framework. End2endImaging integrates optics, sensor/ISP simulation, and neural reconstruction networks into a single PyTorch computation graph, enabling joint optimization of the entire camera pipeline.
Clone this repo:
git clone https://github.com/singer-yang/DeepLens
cd DeepLens
Create a conda environment:
conda create -n deeplens_env python=3.12
conda activate deeplens_env
# Linux and Mac
pip install torch torchvision
# Windows
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt
or
conda env create -f environment.yml -n deeplens_env
Run the demo code:
python 0_hello_geolens.py
The modules below are implemented and working in house, but are not part of the public release yet. If your project needs one, get in touch and we are happy to grant access or collaborate.
- GPU Kernel Acceleration. Custom kernels for NVIDIA and AMD GPUs: >10x speedup and >90% less GPU memory, enough to run full designs on a laptop.
- Polarization Ray Tracing. Polarization-aware tracing with thin-film inverse design via DiffTMM.
- Non-Sequential Ray Tracing. Differentiable non-sequential tracing for stray light analysis and optimization.
- Distributed Optimization. Multi-GPU simulation for billion-scale ray tracing and >100k x 100k diffractive propagation.
Join our Slack workspace and WeChat Group (singeryang1999) to connect with our core contributors, receive the latest industry updates, and be part of our community. For any inquiries, contact Xinge Yang (xinge.yang@kaust.edu.sa).
We welcome all contributions. To get started, please read our Contributing Guide or check out open questions. All project participants are expected to adhere to our Code of Conduct. A list of contributors can be viewed in Contributors and below:
If you use DeepLens in your research, please cite the paper. See more in History of DeepLens.
@article{yang2024curriculum,
title={Curriculum learning for ab initio deep learned refractive optics},
author={Yang, Xinge and Fu, Qiang and Heidrich, Wolfgang},
journal={Nature communications},
volume={15},
number={1},
pages={6572},
year={2024},
publisher={Nature Publishing Group UK London}
}