Jinxiu (Sherry) Liang

1. EventUPS: Uncalibrated Photometric Stereo Using an Event Camera

   Jinxiu Liang^#, Bohan Yu^#, Siqi Yang, Haotian Zhuang, Jieji Ren, Peiqi Duan, and Boxin Shi

   IEEE International Conference on Computer Vision (ICCV), 2025 (Highlight, top 3% of 11,239 submissions)

   video |ICCP 2026 Poster

   The first uncalibrated photometric stereo with an event camera: beyond frame-based accuracy at 5% of the data bandwidth.

   We present EventUPS, the first uncalibrated photometric stereo method using event cameras—neuromorphic sensors that asynchronously detect brightness changes with microsecond resolution. Frame-based uncalibrated photometric stereo methods imposed high bandwidth demands and limiting applicability in dynamic scenes. They require dense image correspondence under varying illumination, cannot be directly applicable due to event data due to their fundamentally different sensing paradigm. Our approach introduces three key innovations: i) an augmented null space formulation that directly relates each event to constraints on surface normals and lighting, naturally handling ambient illumination; ii) a continuous parameterization of time-varying illumination that bridges asynchronous events to synchronized lighting estimation; iii) a structured lighting approach with known relative geometry that resolves the ambiguity to merely convex-concave uncertainty. We validate EventUPS using a custom-built LED-based lighting system implementing dual-ring and trefoil curve patterns. Extensive experiments on synthetic, semi-real, and real data demonstrate that our method achieves accuracy surpassing frame-based counterpart while requiring only 5% of the data bandwidth.

   @inproceedings{liang2025eventups,
     title = {{{EventUPS}}: {{Uncalibrated Photometric Stereo Using}} an {{Event Camera}}},
     author = {Liang#, Jinxiu and Yu#, Bohan and Yang, Siqi and Zhuang, Haotian and Ren, Jieji and Duan, Peiqi and Shi, Boxin},
     booktitle = {{{IEEE International Conference}} on {{Computer Vision}} ({{ICCV}})},
     year = {2025},
   }

2. SpikeDiff: Zero-shot High-Quality Video Reconstruction from Chromatic Spike Camera and Sub-millisecond Spike Streams

   Siqi Yang^†, Jinxiu Liang^*, Zhaojun Huang, Yeliduosi Xiaokaiti, Yakun Chang, Zhaofei Yu, and Boxin Shi^*

   IEEE International Conference on Computer Vision (ICCV), 2025

   Physics-guided diffusion turns sub-millisecond chromatic spike streams into high-quality color video, zero-shot.

   High-speed video reconstruction from neuromorphic spike cameras offers a promising alternative to traditional frame-based imaging, providing superior temporal resolution and dynamic range with reduced power consumption. Nevertheless, reconstructing high-quality colored videos from spikes captured in ultra-short time interval remains challenging due to the noisy nature of spikes. While some existing methods extend temporal capture window to improve reconstruction quality, they compromise the temporal resolution advantages of spike cameras. In this paper, we introduce SpikeDiff, the first zero-shot framework that leverages pretrained diffusion models to reconstruct high-quality colored videos from sub-millisecond chromatic spikes. By incorporating physics-based guidance into the diffusion sampling process, SpikeDiff bridges the domain gap between chromatic spikes and conventional images, enabling high-fidelity reconstruction without requiring domain-specific training data. Extensive experiments demonstrate that SpikeDiff achieves impressive reconstruction quality while maintaining ultra-high temporal resolution, outperforming existing methods across diverse challenging scenarios.

   @inproceedings{yang2025spikediff,
     title = {{{SpikeDiff}}: {{Zero-shot High-Quality Video Reconstruction}} from {{Chromatic Spike Camera}} and {{Sub-millisecond Spike Streams}}},
     author = {Yang, Siqi and Liang, Jinxiu and Huang, Zhaojun and Xiaokaiti, Yeliduosi and Chang, Yakun and Yu, Zhaofei and Shi, Boxin},
     booktitle = {{{IEEE International Conference}} on {{Computer Vision}} ({{ICCV}})},
     year = {2025},
   }

3. Active Hyperspectral Imaging Using an Event Camera

   Bohan Yu^†, Jinxiu Liang, Zhuofeng Wang, Bin Fan, Art Subpa-asa, Boxin Shi, and Imari Sato

   IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025 (Highlight, top 3% of 13,008 submissions)

   Event-based active hyperspectral imaging: real-time spectral capture of dynamic scenes at low bandwidth.

   Hyperspectral imaging plays a critical role in numerous scientific and industrial fields. Conventional hyperspectral imaging systems often struggle with the trade-off between capture speed, spectral resolution, and bandwidth, particularly in dynamic environments. In this work, we present a novel event-based active hyperspectral imaging system designed for real-time capture with low bandwidth in dynamic scenes. By combining an event camera with a dynamic illumination strategy, our system achieves unprecedented temporal resolution while maintaining high spectral fidelity, all at a fraction of the bandwidth requirements of traditional systems. Unlike basis-based methods that sacrifice spectral resolution for efficiency, our approach enables continuous spectral sampling through an innovative “sweeping rainbow" illumination pattern synchronized with a rotating mirror array. The key insight is leveraging the sparse, asynchronous nature of event cameras to encode spectral variations as temporal contrasts, effectively transforming the spectral reconstruction problem into a series of geometric constraints. Extensive evaluations on both synthetic and real data demonstrate that our system outperforms state-of-the-art methods in temporal resolution while maintaining competitive spectral reconstruction quality.

   @inproceedings{yu2024hyperspectral,
     title = {Active {{Hyperspectral Imaging Using}} an {{Event Camera}}},
     author = {Yu, Bohan and Liang, Jinxiu and Wang, Zhuofeng and Fan, Bin and Subpa-asa, Art and Shi, Boxin and Sato, Imari},
     booktitle = {{{IEEE Conference}} on {{Computer Vision}} and {{Pattern Recognition}} ({{CVPR}})},
     year = {2025},
   }

4. Zero-Shot Low-Light Image Enhancement via Latent Diffusion Models

   Yan Huang, Xiaoshan Liao, Jinxiu Liang^*, Yuhui Quan, Boxin Shi, and Yong Xu

   AAAI Conference on Artificial Intelligence (AAAI), 2025

   The first zero-shot low-light enhancement built on pre-trained latent diffusion models; no training data needed.

   Low-light image enhancement (LLIE) aims to improve visibility and signal-to-noise ratio in images captured under poor lighting conditions. Despite impressive improvement, deep learning-based LLIE approaches require extensive training data, which is often difficult and costly to obtain. In this paper, we propose a zero-shot LLIE framework leveraging pre-trained latent diffusion models for the first time, which act as powerful priors to recover latent images from low-light inputs. Our approach introduces several components to alleviate the inherent challenges in utilizing pre-trained latent diffusion models, modeling the degradation process in an image-adaptive manner, penalizing the latent outside the manifold of natural images, and balancing the strengths of the guidance from the given low-light image during the denoising process. Experimental results demonstrate that our framework outperforms existing methods, achieving superior performance across various datasets.

   @inproceedings{huang2024zeroshot,
     title = {Zero-{{Shot Low-Light Image Enhancement}} via {{Latent Diffusion Models}}},
     author = {Huang, Yan and Liao, Xiaoshan and Liang, Jinxiu and Quan, Yuhui and Shi, Boxin and Xu, Yong},
     booktitle = {{{AAAI Conference}} on {{Artificial Intelligence}} ({{AAAI}})},
     year = {2025},
     doi = {10.1609/aaai.v39i4.32398},
   }

5. Zero-Shot Event-Intensity Asymmetric Stereo via Visual Prompting from Image Domain

   Hanyue Lou^#, Jinxiu Liang^#, Minggui Teng, Bin Fan, Yong Xu, and Boxin Shi

   Advances in Neural Information Processing Systems (NeurIPS), 2024

   code

   Visual prompting adapts pre-trained image-domain stereo models to event-intensity camera pairs, zero-shot.

   Event-intensity asymmetric stereo systems have emerged as a promising approach for robust 3D perception in dynamic and challenging environments by integrating event cameras with traditional frame-based sensors in different views. However, existing methods often suffer from overfitting and poor generalization due to limited dataset sizes and lack of scene diversity in the event domain. To address these issues, we propose a novel zero-shot framework that utilizes off-the-shelf monocular depth estimation and stereo matching models trained on diverse image datasets. Our approach introduces a visual prompting technique to align the representations of frames and events, allowing the use of off-the-shelf stereo models without additional training. Furthermore, we introduce a monocular cue-guided disparity refinement module to improve robustness across static and dynamic regions by incorporating monocular depth information from foundation models. Extensive experiments on real-world datasets demonstrate the superior zero-shot evaluation performance and enhanced generalization ability of our method compared to existing approaches.

   @inproceedings{lou2024zeroshot,
     title = {Zero-{{Shot Event-Intensity Asymmetric Stereo}} via {{Visual Prompting}} from {{Image Domain}}},
     author = {Lou#, Hanyue and Liang#, Jinxiu and Teng, Minggui and Fan, Bin and Xu, Yong and Shi, Boxin},
     booktitle = {Advances in {{Neural Information Processing Systems}} ({{NeurIPS}})},
     year = {2024},
     doi = {10.52202/079017-0423},
   }

6. Latency Correction for Event-guided Deblurring and Frame Interpolation

   Yixin Yang^†, Jinxiu Liang, Bohan Yu, Yan Chen, Jimmy S. Ren, and Boxin Shi

   IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

   video

   Modeling and correcting per-pixel event latency sharpens event-guided deblurring and frame interpolation.

   Event cameras with their high temporal resolution dynamic range and low power consumption are particularly good at time-sensitive applications like deblurring and frame interpolation. However their performance is hindered by latency variability especially under low-light conditions and with fast-moving objects. This paper addresses the challenge of latency in event cameras – the temporal discrepancy between the actual occurrence of changes in the corresponding timestamp assigned by the sensor. Focusing on event-guided deblurring and frame interpolation tasks we propose a latency correction method based on a parameterized latency model. To enable data-driven learning we develop an event-based temporal fidelity to describe the sharpness of latent images reconstructed from events and the corresponding blurry images and reformulate the event-based double integral model differentiable to latency. The proposed method is validated using synthetic and real-world datasets demonstrating the benefits of latency correction for deblurring and interpolation across different lighting conditions.

   @inproceedings{yang2024latency,
     title = {Latency {{Correction}} for {{Event-guided Deblurring}} and {{Frame Interpolation}}},
     author = {Yang, Yixin and Liang, Jinxiu and Yu, Bohan and Chen, Yan and Ren, Jimmy S. and Shi, Boxin},
     booktitle = {{{IEEE Conference}} on {{Computer Vision}} and {{Pattern Recognition}} ({{CVPR}})},
     year = {2024},
   }

7. EventPS: Real-Time Photometric Stereo Using an Event Camera

   Bohan Yu^†, Jieji Ren, Jin Han, Feishi Wang, Jinxiu Liang, and Boxin Shi

   IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024 (Best Paper Runner-Up, top 4 of 11,532 submissions)

   code |project page |video

   Real-time photometric stereo powered by an event camera’s microsecond temporal resolution.

   Photometric stereo is a well-established technique to estimate the surface normal of an object. However the requirement of capturing multiple high dynamic range images under different illumination conditions limits the speed and real-time applications. This paper introduces EventPS a novel approach to real-time photometric stereo using an event camera. Capitalizing on the exceptional temporal resolution dynamic range and low bandwidth characteristics of event cameras EventPS estimates surface normal only from the radiance changes significantly enhancing data efficiency. EventPS seamlessly integrates with both optimization-based and deep-learning-based photometric stereo techniques to offer a robust solution for non-Lambertian surfaces. Extensive experiments validate the effectiveness and efficiency of EventPS compared to frame-based counterparts. Our algorithm runs at over 30 fps in real-world scenarios unleashing the potential of EventPS in time-sensitive and high-speed downstream applications.

   @inproceedings{yu2024eventps,
     title = {{{EventPS}}: {{Real-Time Photometric Stereo Using}} an {{Event Camera}}},
     author = {Yu, Bohan and Ren, Jieji and Han, Jin and Wang, Feishi and Liang, Jinxiu and Shi, Boxin},
     booktitle = {{{IEEE Conference}} on {{Computer Vision}} and {{Pattern Recognition}} ({{CVPR}})},
     year = {2024},
   }

8. Coherent Event Guided Low-Light Video Enhancement

   Jinxiu Liang, Yixin Yang, Boyu Li, Peiqi Duan, Yong Xu, and Boxin Shi

   IEEE International Conference on Computer Vision (ICCV), 2023

   code |project page |video

   Event data guide temporally coherent enhancement of fast, low-light videos.

   With frame-based cameras, capturing fast-moving scenes without suffering from blur often comes at the cost of low SNR and low contrast. Worse still, the photometric constancy that enhancement techniques heavily relied on is fragile for frames with short exposure. Event cameras can record brightness changes at an extremely high temporal resolution. For low-light videos, event data are not only suitable to help capture temporal correspondences but also provide alternative observations in the form of intensity ratios between consecutive frames and exposure-invariant information. Motivated by this, we propose a low-light video enhancement method with hybrid inputs of events and frames. Specifically, a neural network is trained to establish spatiotemporal coherence between visual signals with different modalities and resolutions by constructing correlation volume across space and time. Experimental results on synthetic and real data demonstrate the superiority of the proposed method compared to the state-of-the-art methods.

   @inproceedings{liang2023coherent,
     title = {Coherent {{Event Guided Low-Light Video Enhancement}}},
     author = {Liang, Jinxiu and Yang, Yixin and Li, Boyu and Duan, Peiqi and Xu, Yong and Shi, Boxin},
     booktitle = {{{IEEE International Conference}} on {{Computer Vision}} ({{ICCV}})},
     year = {2023},
     langid = {english},
   }

9. Self-Supervised Low-Light Image Enhancement Using Discrepant Untrained Network Priors

   Jinxiu Liang, Yong Xu, Yuhui Quan, Boxin Shi, and Hui Ji

   IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2022

   code

   Low-light enhancement trained on nothing but the input image, via untrained network priors on a Retinex decomposition.

   This paper proposes a deep learning method for low-light image enhancement, which exploits the generation capability of Neural Networks (NNs) while requiring no training samples except the input image itself. Based on the Retinex decomposition model, the reflectance and illumination of a low-light image are parameterized by two untrained NNs. The ambiguity between the two layers is resolved by the discrepancy between the two NNs in terms of architecture and capacity, while the complex noise with spatially-varying characteristics is handled by an illumination-adaptive self-supervised denoising module. The enhancement is done by jointly optimizing the Retinex decomposition and the illumination adjustment. Extensive experiments show that the proposed method not only outperforms existing non-learning-based and unsupervised-learning-based methods, but also competes favorably with some supervised-learning-based methods in extreme low-light conditions.

   @article{liang2022selfsupervised,
     title = {Self-{{Supervised Low-Light Image Enhancement Using Discrepant Untrained Network Priors}}},
     author = {Liang, Jinxiu and Xu, Yong and Quan, Yuhui and Shi, Boxin and Ji, Hui},
     journal = {IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)},
     year = {2022},
     doi = {10.1109/TCSVT.2022.3181781},
   }

10. Recurrent Exposure Generation for Low-Light Face Detection

    Jinxiu Liang, Jingwen Wang, Yuhui Quan, Tianyi Chen, Jiaying Liu, Haibin Ling, and Yong Xu

    IEEE Transactions on Multimedia (TMM), 2021

    code

    A recurrent network generates multi-exposure sequences from a single dark image, making faces detectable at night.

    Face detection from low-light images is challenging due to limited photons and inevitable noise, which, to make the task even harder, are often spatially unevenly distributed. A natural solution is to borrow the idea from multi-exposure, which captures multiple shots to obtain well-exposed images under challenging conditions. High-quality implementation/approximation of multi-exposure from a single image is however nontrivial. Fortunately, as shown in this paper, neither is such high-quality necessary since our task is face detection rather than image enhancement. Specifically, we propose a novel Recurrent Exposure Generation (REG) module and couple it seamlessly with a Multi-Exposure Detection (MED) module, and thus significantly improve face detection performance by effectively inhibiting non-uniform illumination and noise issues. REG produces progressively and efficiently intermediate images corresponding to various exposure settings, and such pseudo-exposures are then fused by MED to detect faces across different lighting conditions. The proposed method, named REGDet, is the first ‘detection-with-enhancement’ framework for low-light face detection. It not only encourages rich interaction and feature fusion across different illumination levels, but also enables effective end-to-end learning of the REG component to be better tailored for face detection. Moreover, as clearly shown in our experiments, REG can be flexibly coupled with different face detectors without extra low/normal-light image pairs for training. We tested REGDet on the DARK FACE low-light face benchmark with thorough ablation study, where REGDet outperforms previous state-of-the-arts by a significant margin, with only negligible extra parameters.

    @article{liang2021recurrent,
      title = {Recurrent {{Exposure Generation}} for {{Low-Light Face Detection}}},
      author = {Liang, Jinxiu and Wang, Jingwen and Quan, Yuhui and Chen, Tianyi and Liu, Jiaying and Ling, Haibin and Xu, Yong},
      journal = {IEEE Transactions on Multimedia (TMM)},
      year = {2021},
      doi = {10.1109/TMM.2021.3068840},
    }

2026

1. COIL-PS: continuous and online illumination planning for photometric stereo

   Jun Hoong Chan^†, Bohan Yu^*, Jieji Ren, Jinxiu Liang^*, and Boxin Shi

   Optics Express, 2026

   Photometric stereo (PS) aims to recover high-fidelity surface normals by observing pixel-wise radiometric variations under different light directions. However, traditional PS methods require dense sampling of the incident light to mitigate non-Lambertian effects, such as cast shadows and specular highlights, creating a significant efficiency bottleneck for practical optical metrology. To address this efficiency bottleneck, illumination planning methods seek to identify an optimal set of light directions to maximize information gain with minimal measurements. A critical limitation of existing illumination planning paradigms is their reliance on selecting from a predefined, discrete set of candidate light directions. This discretization of the light space introduces an artificial bottleneck, severely limiting precision and adaptability. In this paper, we address this limitation by introducing a Continuous and Online ILlumination Planning framework for Photometric Stereo (COIL-PS). Instead of selecting from a fixed grid, our method formulates illumination planning as a continuous regression problem, adaptively steering light positioning in the continuous hemispherical domain. By coupling online illumination planning with feedback from intermediate normal estimates, COIL-PS adaptively navigates non-Lambertian effects such as shadows and specularities, which allow for the precise angular placement of illumination required to resolve geometric ambiguities that fall between fixed grid points. Extensive experiments on a synthetic dataset, semi-real benchmarks, and our custom-built real-world robotic validation system demonstrate that COIL-PS achieves superior normal reconstruction accuracy compared to state-of-the-art discrete planning methods, even with a budget of only ten lights, significantly outperforming discrete planning paradigms.

   @article{chan2026coilps,
     title = {{COIL}-{PS}: continuous and online illumination planning for photometric stereo},
     author = {Chan, Jun Hoong and Yu, Bohan and Ren, Jieji and Liang, Jinxiu and Shi, Boxin},
     journal = {Optics Express},
     year = {2026},
     doi = {10.1364/OE.587541},
     number = {6},
     pages = {9906--9924},
     volume = {34}
   }

2025

1. EventUPS: Uncalibrated Photometric Stereo Using an Event Camera

   Jinxiu Liang^#, Bohan Yu^#, Siqi Yang, Haotian Zhuang, Jieji Ren, Peiqi Duan, and Boxin Shi

   IEEE International Conference on Computer Vision (ICCV), 2025 (Highlight, top 3% of 11,239 submissions)

   video |ICCP 2026 Poster

   The first uncalibrated photometric stereo with an event camera: beyond frame-based accuracy at 5% of the data bandwidth.

   We present EventUPS, the first uncalibrated photometric stereo method using event cameras—neuromorphic sensors that asynchronously detect brightness changes with microsecond resolution. Frame-based uncalibrated photometric stereo methods imposed high bandwidth demands and limiting applicability in dynamic scenes. They require dense image correspondence under varying illumination, cannot be directly applicable due to event data due to their fundamentally different sensing paradigm. Our approach introduces three key innovations: i) an augmented null space formulation that directly relates each event to constraints on surface normals and lighting, naturally handling ambient illumination; ii) a continuous parameterization of time-varying illumination that bridges asynchronous events to synchronized lighting estimation; iii) a structured lighting approach with known relative geometry that resolves the ambiguity to merely convex-concave uncertainty. We validate EventUPS using a custom-built LED-based lighting system implementing dual-ring and trefoil curve patterns. Extensive experiments on synthetic, semi-real, and real data demonstrate that our method achieves accuracy surpassing frame-based counterpart while requiring only 5% of the data bandwidth.

   @inproceedings{liang2025eventups,
     title = {{{EventUPS}}: {{Uncalibrated Photometric Stereo Using}} an {{Event Camera}}},
     author = {Liang#, Jinxiu and Yu#, Bohan and Yang, Siqi and Zhuang, Haotian and Ren, Jieji and Duan, Peiqi and Shi, Boxin},
     booktitle = {{{IEEE International Conference}} on {{Computer Vision}} ({{ICCV}})},
     year = {2025},
   }

2. SpikeDiff: Zero-shot High-Quality Video Reconstruction from Chromatic Spike Camera and Sub-millisecond Spike Streams

   Siqi Yang^†, Jinxiu Liang^*, Zhaojun Huang, Yeliduosi Xiaokaiti, Yakun Chang, Zhaofei Yu, and Boxin Shi^*

   IEEE International Conference on Computer Vision (ICCV), 2025

   Physics-guided diffusion turns sub-millisecond chromatic spike streams into high-quality color video, zero-shot.

   High-speed video reconstruction from neuromorphic spike cameras offers a promising alternative to traditional frame-based imaging, providing superior temporal resolution and dynamic range with reduced power consumption. Nevertheless, reconstructing high-quality colored videos from spikes captured in ultra-short time interval remains challenging due to the noisy nature of spikes. While some existing methods extend temporal capture window to improve reconstruction quality, they compromise the temporal resolution advantages of spike cameras. In this paper, we introduce SpikeDiff, the first zero-shot framework that leverages pretrained diffusion models to reconstruct high-quality colored videos from sub-millisecond chromatic spikes. By incorporating physics-based guidance into the diffusion sampling process, SpikeDiff bridges the domain gap between chromatic spikes and conventional images, enabling high-fidelity reconstruction without requiring domain-specific training data. Extensive experiments demonstrate that SpikeDiff achieves impressive reconstruction quality while maintaining ultra-high temporal resolution, outperforming existing methods across diverse challenging scenarios.

   @inproceedings{yang2025spikediff,
     title = {{{SpikeDiff}}: {{Zero-shot High-Quality Video Reconstruction}} from {{Chromatic Spike Camera}} and {{Sub-millisecond Spike Streams}}},
     author = {Yang, Siqi and Liang, Jinxiu and Huang, Zhaojun and Xiaokaiti, Yeliduosi and Chang, Yakun and Yu, Zhaofei and Shi, Boxin},
     booktitle = {{{IEEE International Conference}} on {{Computer Vision}} ({{ICCV}})},
     year = {2025},
   }

3. Multi-Focus Image Fusion via Explicit Defocus Blur Modelling

   Yuhui Quan, Xi Wan, Zitao Tang, Jinxiu Liang^*, and Hui Ji

   AAAI Conference on Artificial Intelligence (AAAI), 2025

   Multi-focus image fusion (MFIF) is a critical technique for enhancing depth of field in photography, producing an all-in-focus image from multiple images captured at different focal lengths. While deep learning has shown promise in MFIF, most existing methods ignore the physical model of defocus blurring in their neural architecture design, limiting their interoperability and generalization. This paper presents a novel framework that integrates explicit defocus blur modeling into the MFIF process, leading to enhanced interpretability and performance. Leveraging an atom-based spatially-varying parameterized defocus blurring model, our approach first calculates pixel-wise defocus descriptors and initial focused images from multi-focus source images through a scale-recurrent fashion, based on which soft decision maps are estimated. Afterward, image fusion is performed using masks constructed from the decision maps, with a separate treatment on pixels that are probably defocused in all source images or near boundaries of defocused/focused regions. Model training is done with a fusion loss and a cross-scale defocus estimation loss. Extensive experiments on benchmark datasets have demonstrated the effectiveness of our approach.

   @inproceedings{quan2024multifocus,
     title = {Multi-{{Focus Image Fusion}} via {{Explicit Defocus Blur Modelling}}},
     author = {Quan, Yuhui and Wan, Xi and Tang, Zitao and Liang, Jinxiu and Ji, Hui},
     booktitle = {{{AAAI Conference}} on {{Artificial Intelligence}} ({{AAAI}})},
     year = {2025},
     doi = {10.1609/aaai.v39i6.32714},
   }

4. Active Hyperspectral Imaging Using an Event Camera

   Bohan Yu^†, Jinxiu Liang, Zhuofeng Wang, Bin Fan, Art Subpa-asa, Boxin Shi, and Imari Sato

   IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025 (Highlight, top 3% of 13,008 submissions)

   Event-based active hyperspectral imaging: real-time spectral capture of dynamic scenes at low bandwidth.

   Hyperspectral imaging plays a critical role in numerous scientific and industrial fields. Conventional hyperspectral imaging systems often struggle with the trade-off between capture speed, spectral resolution, and bandwidth, particularly in dynamic environments. In this work, we present a novel event-based active hyperspectral imaging system designed for real-time capture with low bandwidth in dynamic scenes. By combining an event camera with a dynamic illumination strategy, our system achieves unprecedented temporal resolution while maintaining high spectral fidelity, all at a fraction of the bandwidth requirements of traditional systems. Unlike basis-based methods that sacrifice spectral resolution for efficiency, our approach enables continuous spectral sampling through an innovative “sweeping rainbow" illumination pattern synchronized with a rotating mirror array. The key insight is leveraging the sparse, asynchronous nature of event cameras to encode spectral variations as temporal contrasts, effectively transforming the spectral reconstruction problem into a series of geometric constraints. Extensive evaluations on both synthetic and real data demonstrate that our system outperforms state-of-the-art methods in temporal resolution while maintaining competitive spectral reconstruction quality.

   @inproceedings{yu2024hyperspectral,
     title = {Active {{Hyperspectral Imaging Using}} an {{Event Camera}}},
     author = {Yu, Bohan and Liang, Jinxiu and Wang, Zhuofeng and Fan, Bin and Subpa-asa, Art and Shi, Boxin and Sato, Imari},
     booktitle = {{{IEEE Conference}} on {{Computer Vision}} and {{Pattern Recognition}} ({{CVPR}})},
     year = {2025},
   }

5. Zero-Shot Low-Light Image Enhancement via Latent Diffusion Models

   Yan Huang, Xiaoshan Liao, Jinxiu Liang^*, Yuhui Quan, Boxin Shi, and Yong Xu

   AAAI Conference on Artificial Intelligence (AAAI), 2025

   The first zero-shot low-light enhancement built on pre-trained latent diffusion models; no training data needed.

   Low-light image enhancement (LLIE) aims to improve visibility and signal-to-noise ratio in images captured under poor lighting conditions. Despite impressive improvement, deep learning-based LLIE approaches require extensive training data, which is often difficult and costly to obtain. In this paper, we propose a zero-shot LLIE framework leveraging pre-trained latent diffusion models for the first time, which act as powerful priors to recover latent images from low-light inputs. Our approach introduces several components to alleviate the inherent challenges in utilizing pre-trained latent diffusion models, modeling the degradation process in an image-adaptive manner, penalizing the latent outside the manifold of natural images, and balancing the strengths of the guidance from the given low-light image during the denoising process. Experimental results demonstrate that our framework outperforms existing methods, achieving superior performance across various datasets.

   @inproceedings{huang2024zeroshot,
     title = {Zero-{{Shot Low-Light Image Enhancement}} via {{Latent Diffusion Models}}},
     author = {Huang, Yan and Liao, Xiaoshan and Liang, Jinxiu and Quan, Yuhui and Shi, Boxin and Xu, Yong},
     booktitle = {{{AAAI Conference}} on {{Artificial Intelligence}} ({{AAAI}})},
     year = {2025},
     doi = {10.1609/aaai.v39i4.32398},
   }

6. V2V: Scaling Event-Based Vision through Efficient Video-to-Voxel Simulation

   Hanyue Lou^†, Jinxiu Liang, Minggui Teng, Yi Wang, and Boxin Shi

   Advances in Neural Information Processing Systems (NeurIPS), 2025

   Event-based cameras offer unique advantages such as high temporal resolution, high dynamic range, and low power consumption. However, the massive storage requirements and I/O burdens of existing synthetic data generation pipelines and the scarcity of real data prevent event-based training datasets from scaling up, limiting the development and generalization capabilities of event vision models. To address this challenge, we introduce Video-to-Voxel (V2V), an approach that directly converts conventional video frames into event-based voxel grid representations, bypassing the storage-intensive event stream generation entirely. V2V enables a 150× reduction in storage requirements while supporting on-the-fly parameter randomization for enhanced model robustness. Leveraging this efficiency, we train several video reconstruction and optical flow estimation model architectures on 10,000 diverse videos totaling 52 hours—an order of magnitude larger than existing event datasets, yielding substantial improvements.

   @inproceedings{lou2025v2v,
     title = {{{V2V}}: {{Scaling Event-Based Vision}} through {{Efficient Video-to-Voxel Simulation}}},
     author = {Lou, Hanyue and Liang, Jinxiu and Teng, Minggui and Wang, Yi and Shi, Boxin},
     booktitle = {Advances in {{Neural Information Processing Systems}} ({{NeurIPS}})},
     year = {2025},
   }

2024

1. Detail-Preserving Diffusion Models for Low-Light Image Enhancement

   Yan Huang, Xiaoshan Liao, Jinxiu Liang^*, Boxin Shi, Yong Xu, and Patrick Le Callet

   IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2024

   code

   Existing diffusion models for low-light image enhancement typically focus on incrementally removing noise introduced during the forward diffusion process using a denoising loss, with the process being conditioned on input low-light images. While these models demonstrate remarkable abilities in generating realistic high-frequency details, they often struggle to accurately restore fine details that are faithful to the input. To address this, we present a novel detail-preserving diffusion model for realistic and faithful low-light image enhancement. Our approach integrates a size-agnostic diffusion process with a reverse process reconstruction loss, significantly enhancing the fidelity of enhanced images to their low-light counterparts and enabling more accurate recovery of fine details. To ensure the preservation of region- and content-aware details, we employ an efficient noise estimation network with a simplified channel-spatial attention mechanism. Additionally, we propose a multiscale ensemble scheme to maintain detail fidelity across diverse illumination regions. Comprehensive experiments on eight benchmark low-light image enhancement datasets demonstrate that our method achieves state-of-the-art results compared to 20 existing methods in terms of both perceptual quality (LPIPS) and distortion metrics (PSNR and SSIM).

   @article{huang2024detailpreserving,
     title = {Detail-{{Preserving Diffusion Models}} for {{Low-Light Image Enhancement}}},
     author = {Huang, Yan and Liao, Xiaoshan and Liang, Jinxiu and Shi, Boxin and Xu, Yong and Callet, Patrick Le},
     journal = {IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)},
     year = {2024},
     doi = {10.1109/TCSVT.2024.3502801},
   }

2. Zero-Shot Event-Intensity Asymmetric Stereo via Visual Prompting from Image Domain

   Hanyue Lou^#, Jinxiu Liang^#, Minggui Teng, Bin Fan, Yong Xu, and Boxin Shi

   Advances in Neural Information Processing Systems (NeurIPS), 2024

   code

   Visual prompting adapts pre-trained image-domain stereo models to event-intensity camera pairs, zero-shot.

   Event-intensity asymmetric stereo systems have emerged as a promising approach for robust 3D perception in dynamic and challenging environments by integrating event cameras with traditional frame-based sensors in different views. However, existing methods often suffer from overfitting and poor generalization due to limited dataset sizes and lack of scene diversity in the event domain. To address these issues, we propose a novel zero-shot framework that utilizes off-the-shelf monocular depth estimation and stereo matching models trained on diverse image datasets. Our approach introduces a visual prompting technique to align the representations of frames and events, allowing the use of off-the-shelf stereo models without additional training. Furthermore, we introduce a monocular cue-guided disparity refinement module to improve robustness across static and dynamic regions by incorporating monocular depth information from foundation models. Extensive experiments on real-world datasets demonstrate the superior zero-shot evaluation performance and enhanced generalization ability of our method compared to existing approaches.

   @inproceedings{lou2024zeroshot,
     title = {Zero-{{Shot Event-Intensity Asymmetric Stereo}} via {{Visual Prompting}} from {{Image Domain}}},
     author = {Lou#, Hanyue and Liang#, Jinxiu and Teng, Minggui and Fan, Bin and Xu, Yong and Shi, Boxin},
     booktitle = {Advances in {{Neural Information Processing Systems}} ({{NeurIPS}})},
     year = {2024},
     doi = {10.52202/079017-0423},
   }

3. Latency Correction for Event-guided Deblurring and Frame Interpolation

   Yixin Yang^†, Jinxiu Liang, Bohan Yu, Yan Chen, Jimmy S. Ren, and Boxin Shi

   IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

   video

   Modeling and correcting per-pixel event latency sharpens event-guided deblurring and frame interpolation.

   Event cameras with their high temporal resolution dynamic range and low power consumption are particularly good at time-sensitive applications like deblurring and frame interpolation. However their performance is hindered by latency variability especially under low-light conditions and with fast-moving objects. This paper addresses the challenge of latency in event cameras – the temporal discrepancy between the actual occurrence of changes in the corresponding timestamp assigned by the sensor. Focusing on event-guided deblurring and frame interpolation tasks we propose a latency correction method based on a parameterized latency model. To enable data-driven learning we develop an event-based temporal fidelity to describe the sharpness of latent images reconstructed from events and the corresponding blurry images and reformulate the event-based double integral model differentiable to latency. The proposed method is validated using synthetic and real-world datasets demonstrating the benefits of latency correction for deblurring and interpolation across different lighting conditions.

   @inproceedings{yang2024latency,
     title = {Latency {{Correction}} for {{Event-guided Deblurring}} and {{Frame Interpolation}}},
     author = {Yang, Yixin and Liang, Jinxiu and Yu, Bohan and Chen, Yan and Ren, Jimmy S. and Shi, Boxin},
     booktitle = {{{IEEE Conference}} on {{Computer Vision}} and {{Pattern Recognition}} ({{CVPR}})},
     year = {2024},
   }

4. EventPS: Real-Time Photometric Stereo Using an Event Camera

   Bohan Yu^†, Jieji Ren, Jin Han, Feishi Wang, Jinxiu Liang, and Boxin Shi

   IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024 (Best Paper Runner-Up, top 4 of 11,532 submissions)

   code |project page |video

   Real-time photometric stereo powered by an event camera’s microsecond temporal resolution.

   Photometric stereo is a well-established technique to estimate the surface normal of an object. However the requirement of capturing multiple high dynamic range images under different illumination conditions limits the speed and real-time applications. This paper introduces EventPS a novel approach to real-time photometric stereo using an event camera. Capitalizing on the exceptional temporal resolution dynamic range and low bandwidth characteristics of event cameras EventPS estimates surface normal only from the radiance changes significantly enhancing data efficiency. EventPS seamlessly integrates with both optimization-based and deep-learning-based photometric stereo techniques to offer a robust solution for non-Lambertian surfaces. Extensive experiments validate the effectiveness and efficiency of EventPS compared to frame-based counterparts. Our algorithm runs at over 30 fps in real-world scenarios unleashing the potential of EventPS in time-sensitive and high-speed downstream applications.

   @inproceedings{yu2024eventps,
     title = {{{EventPS}}: {{Real-Time Photometric Stereo Using}} an {{Event Camera}}},
     author = {Yu, Bohan and Ren, Jieji and Han, Jin and Wang, Feishi and Liang, Jinxiu and Shi, Boxin},
     booktitle = {{{IEEE Conference}} on {{Computer Vision}} and {{Pattern Recognition}} ({{CVPR}})},
     year = {2024},
   }

5. L-DiffER: Single Image Reflection Removal with Language-based Diffusion Model

   Yuchen Hong, Haofeng Zhong, Shuchen Weng, Jinxiu Liang, and Boxin Shi

   European Conference on Computer Vision (ECCV), 2024

   In this paper, we introduce L-DiffER, a language-based diffusion model designed for the ill-posed single image reflection removal task. Although having shown impressive performance for image generation, existing language-based diffusion models struggle with precise control and faithfulness in image restoration. To overcome these limitations, we propose an iterative condition refinement strategy to resolve the problem of inaccurate control conditions. A multi-condition constraint mechanism is employed to ensure the recovery faithfulness of image color and structure while retaining the generation capability to handle low-transmitted reflections. We demonstrate the superiority of the proposed method through extensive experiments, showcasing both quantitative and qualitative improvements over existing methods.

   @inproceedings{hong2024ldiffer,
     title = {L-{{DiffER}}: {{Single Image Reflection Removal}} with {{Language-based Diffusion Model}}},
     author = {Hong, Yuchen and Zhong, Haofeng and Weng, Shuchen and Liang, Jinxiu and Shi, Boxin},
     booktitle = {{{European Conference}} on {{Computer Vision}} ({{ECCV}})},
     year = {2024},
     doi = {10.1007/978-3-031-72661-3_4},
   }

6. Light Flickering Guided Reflection Removal

   Yuchen Hong, Yakun Chang, Jinxiu Liang, Lei Ma, Tiejun Huang, and Boxin Shi

   International Journal of Computer Vision (IJCV), 2024

   When photographing through a piece of glass, reflections usually degrade the quality of captured images or videos. In this paper, by exploiting periodically varying light flickering, we investigate the problem of removing strong reflections from contaminated image sequences or videos with a unified capturing setup. We propose a learning-based method that utilizes short-term and long-term observations of mixture videos to exploit one-side contextual clues in fluctuant components and brightness-consistent clues in consistent components for achieving layer separation and flickering removal, respectively. A dataset containing synthetic and real mixture videos with light flickering is built for network training and testing. The effectiveness of the proposed method is demonstrated by the comprehensive evaluation on synthetic and real data, the application for video flickering removal, and the exploratory experiment on high-speed scenes.

   @article{hong2024light,
     title = {Light {{Flickering Guided Reflection Removal}}},
     author = {Hong, Yuchen and Chang, Yakun and Liang, Jinxiu and Ma, Lei and Huang, Tiejun and Shi, Boxin},
     journal = {International Journal of Computer Vision (IJCV)},
     year = {2024},
     doi = {10.1007/s11263-024-02073-z},
   }

7. Language-Guided Image Reflection Separation

   Haofeng Zhong, Yuchen Hong, Shuchen Weng, Jinxiu Liang, and Boxin Shi

   IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024

   This paper studies the problem of language-guided reflection separation which aims at addressing the ill-posed reflection separation problem by introducing language descriptions to provide layer content. We propose a unified framework to solve this problem which leverages the cross-attention mechanism with contrastive learning strategies to construct the correspondence between language descriptions and image layers. A gated network design and a randomized training strategy are employed to tackle the recognizable layer ambiguity. The effectiveness of the proposed method is validated by the significant performance advantage over existing reflection separation methods on both quantitative and qualitative comparisons.

   @inproceedings{zhong2024languageguided,
     title = {Language-Guided {{Image Reflection Separation}}},
     author = {Zhong, Haofeng and Hong, Yuchen and Weng, Shuchen and Liang, Jinxiu and Shi, Boxin},
     booktitle = {{{IEEE Conference}} on {{Computer Vision}} and {{Pattern Recognition}} ({{CVPR}})},
     year = {2024},
   }

2023

1. Coherent Event Guided Low-Light Video Enhancement

   Jinxiu Liang, Yixin Yang, Boyu Li, Peiqi Duan, Yong Xu, and Boxin Shi

   IEEE International Conference on Computer Vision (ICCV), 2023

   code |project page |video

   Event data guide temporally coherent enhancement of fast, low-light videos.

   With frame-based cameras, capturing fast-moving scenes without suffering from blur often comes at the cost of low SNR and low contrast. Worse still, the photometric constancy that enhancement techniques heavily relied on is fragile for frames with short exposure. Event cameras can record brightness changes at an extremely high temporal resolution. For low-light videos, event data are not only suitable to help capture temporal correspondences but also provide alternative observations in the form of intensity ratios between consecutive frames and exposure-invariant information. Motivated by this, we propose a low-light video enhancement method with hybrid inputs of events and frames. Specifically, a neural network is trained to establish spatiotemporal coherence between visual signals with different modalities and resolutions by constructing correlation volume across space and time. Experimental results on synthetic and real data demonstrate the superiority of the proposed method compared to the state-of-the-art methods.

   @inproceedings{liang2023coherent,
     title = {Coherent {{Event Guided Low-Light Video Enhancement}}},
     author = {Liang, Jinxiu and Yang, Yixin and Li, Boyu and Duan, Peiqi and Xu, Yong and Shi, Boxin},
     booktitle = {{{IEEE International Conference}} on {{Computer Vision}} ({{ICCV}})},
     year = {2023},
     langid = {english},
   }

2. Learning Event Guided High Dynamic Range Video Reconstruction

   Yixin Yang^†, Jin Han, Jinxiu Liang, Imari Sato, and Boxin Shi

   IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023

   code |project page |video

   Limited by the trade-off between frame rate and exposure time when capturing moving scenes with conventional cameras, frame based HDR video reconstruction suffers from scene-dependent exposure ratio balancing and ghosting artifacts. Event cameras provide an alternative visual representation with a much higher dynamic range and temporal resolution free from the above issues, which could be an effective guidance for HDR imaging from LDR videos. In this paper, we propose a multimodal learning framework for event guided HDR video reconstruction. In order to better leverage the knowledge of the same scene from the two modalities of visual signals, a multimodal representation alignment strategy to learn a shared latent space and a fusion module tailored to complementing two types of signals for different dynamic ranges in different regions are proposed. Temporal correlations are utilized recurrently to suppress the flickering effects in the reconstructed HDR video. The proposed HDRev-Net demonstrates state-of-the-art performance quantitatively and qualitatively for both synthetic and real-world data.

   @inproceedings{yang2023learninga,
     title = {Learning {{Event Guided High Dynamic Range Video Reconstruction}}},
     author = {Yang, Yixin and Han, Jin and Liang, Jinxiu and Sato, Imari and Shi, Boxin},
     booktitle = {{{IEEE Conference}} on {{Computer Vision}} and {{Pattern Recognition}} ({{CVPR}})},
     year = {2023},
     doi = {10.1109/CVPR52729.2023.01338},
   }

3. Deblurring Low-Light Images with Events

   Chu Zhou, Minggui Teng, Jin Han, Jinxiu Liang, Chao Xu, Gang Cao, and Boxin Shi

   International Journal of Computer Vision (IJCV), 2023

   code

   Modern image-based deblurring methods usually show degenerate performance in low-light conditions since the images often contain most of the poorly visible dark regions and a few saturated bright regions, making the amount of effective features that can be extracted for deblurring limited. In contrast, event cameras can trigger events with a very high dynamic range and low latency, which hardly suffer from saturation and naturally encode dense temporal information about motion. However, in low-light conditions existing event-based deblurring methods would become less robust since the events triggered in dark regions are often severely contaminated by noise, leading to inaccurate reconstruction of the corresponding intensity values. Besides, since they directly adopt the event-based double integral model to perform pixel-wise reconstruction, they can only handle low-resolution grayscale active pixel sensor images provided by the DAVIS camera, which cannot meet the requirement of daily photography. In this paper, to apply events to deblurring low-light images robustly, we propose a unified two-stage framework along with a motion-aware neural network tailored to it, reconstructing the sharp image under the guidance of high-fidelity motion clues extracted from events. Besides, we build an RGB-DAVIS hybrid camera system to demonstrate that our method has the ability to deblur high-resolution RGB images due to the natural advantages of our two-stage framework. Experimental results show our method achieves state-of-the-art performance on both synthetic and real-world images.

   @article{zhou2023deblurring,
     title = {Deblurring {{Low-Light Images}} with {{Events}}},
     author = {Zhou, Chu and Teng, Minggui and Han, Jin and Liang, Jinxiu and Xu, Chao and Cao, Gang and Shi, Boxin},
     journal = {International Journal of Computer Vision (IJCV)},
     year = {2023},
     doi = {10.1007/s11263-023-01754-5},
   }

4. Non-Lambertian Multispectral Photometric Stereo via Spectral Reflectance Decomposition

   Jipeng Lv, Heng Guo, Guanying Chen, Jinxiu Liang, and Boxin Shi

   Thirty-Second International Joint Conference on Artificial Intelligence (IJCAI), 2023

   code

   Multispectral photometric stereo (MPS) aims at recovering the surface normal of a scene from a single-shot multispectral image captured under multispectral illuminations. Existing MPS methods adopt the Lambertian reflectance model to make the problem tractable, but it greatly limits their application to real-world surfaces. In this paper, we propose a deep neural network named NeuralMPS to solve the MPS problem under non-Lambertian spectral reflectances. Specifically, we present a spectral reflectance decomposition model to disentangle the spectral reflectance into a geometric component and a spectral component. With this decomposition, we show that the MPS problem for surfaces with a uniform material is equivalent to the conventional photometric stereo (CPS) with unknown light intensities. In this way, NeuralMPS reduces the difficulty of the non-Lambertian MPS problem by leveraging the well-studied non-Lambertian CPS methods. Experiments on both synthetic and real-world scenes demonstrate the effectiveness of our method.

   @inproceedings{lv2023nonlambertian,
     title = {Non-{{Lambertian Multispectral Photometric Stereo}} via {{Spectral Reflectance Decomposition}}},
     author = {Lv, Jipeng and Guo, Heng and Chen, Guanying and Liang, Jinxiu and Shi, Boxin},
     booktitle = {{{Thirty-Second International Joint Conference}} on {{Artificial Intelligence}} ({{IJCAI}})},
     year = {2023},
     doi = {10.24963/ijcai.2023/139},
   }

2022

1. Self-Supervised Low-Light Image Enhancement Using Discrepant Untrained Network Priors

   Jinxiu Liang, Yong Xu, Yuhui Quan, Boxin Shi, and Hui Ji

   IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), 2022

   code

   Low-light enhancement trained on nothing but the input image, via untrained network priors on a Retinex decomposition.

   This paper proposes a deep learning method for low-light image enhancement, which exploits the generation capability of Neural Networks (NNs) while requiring no training samples except the input image itself. Based on the Retinex decomposition model, the reflectance and illumination of a low-light image are parameterized by two untrained NNs. The ambiguity between the two layers is resolved by the discrepancy between the two NNs in terms of architecture and capacity, while the complex noise with spatially-varying characteristics is handled by an illumination-adaptive self-supervised denoising module. The enhancement is done by jointly optimizing the Retinex decomposition and the illumination adjustment. Extensive experiments show that the proposed method not only outperforms existing non-learning-based and unsupervised-learning-based methods, but also competes favorably with some supervised-learning-based methods in extreme low-light conditions.

   @article{liang2022selfsupervised,
     title = {Self-{{Supervised Low-Light Image Enhancement Using Discrepant Untrained Network Priors}}},
     author = {Liang, Jinxiu and Xu, Yong and Quan, Yuhui and Shi, Boxin and Ji, Hui},
     journal = {IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)},
     year = {2022},
     doi = {10.1109/TCSVT.2022.3181781},
   }

2. MARN: Multi-level Attentional Reconstruction Networks for Weakly Supervised Video Temporal Grounding

   Yijun Song, Jingwen Wang, Lin Ma, Jun Yu, Jinxiu Liang, Liu Yuan, and Zhou Yu

   Neurocomputing, 2022

   @article{song2022marn,
     title = {{{MARN}}: {{Multi-level Attentional Reconstruction Networks}} for {{Weakly Supervised Video Temporal Grounding}}},
     author = {Song, Yijun and Wang, Jingwen and Ma, Lin and Yu, Jun and Liang, Jinxiu and Yuan, Liu and Yu, Zhou},
     journal = {Neurocomputing},
     year = {2022},
     doi = {10.1016/j.neucom.2023.126625},
   }

2021

1. Recurrent Exposure Generation for Low-Light Face Detection

   Jinxiu Liang, Jingwen Wang, Yuhui Quan, Tianyi Chen, Jiaying Liu, Haibin Ling, and Yong Xu

   IEEE Transactions on Multimedia (TMM), 2021

   code

   A recurrent network generates multi-exposure sequences from a single dark image, making faces detectable at night.

   Face detection from low-light images is challenging due to limited photons and inevitable noise, which, to make the task even harder, are often spatially unevenly distributed. A natural solution is to borrow the idea from multi-exposure, which captures multiple shots to obtain well-exposed images under challenging conditions. High-quality implementation/approximation of multi-exposure from a single image is however nontrivial. Fortunately, as shown in this paper, neither is such high-quality necessary since our task is face detection rather than image enhancement. Specifically, we propose a novel Recurrent Exposure Generation (REG) module and couple it seamlessly with a Multi-Exposure Detection (MED) module, and thus significantly improve face detection performance by effectively inhibiting non-uniform illumination and noise issues. REG produces progressively and efficiently intermediate images corresponding to various exposure settings, and such pseudo-exposures are then fused by MED to detect faces across different lighting conditions. The proposed method, named REGDet, is the first ‘detection-with-enhancement’ framework for low-light face detection. It not only encourages rich interaction and feature fusion across different illumination levels, but also enables effective end-to-end learning of the REG component to be better tailored for face detection. Moreover, as clearly shown in our experiments, REG can be flexibly coupled with different face detectors without extra low/normal-light image pairs for training. We tested REGDet on the DARK FACE low-light face benchmark with thorough ablation study, where REGDet outperforms previous state-of-the-arts by a significant margin, with only negligible extra parameters.

   @article{liang2021recurrent,
     title = {Recurrent {{Exposure Generation}} for {{Low-Light Face Detection}}},
     author = {Liang, Jinxiu and Wang, Jingwen and Quan, Yuhui and Chen, Tianyi and Liu, Jiaying and Ling, Haibin and Xu, Yong},
     journal = {IEEE Transactions on Multimedia (TMM)},
     year = {2021},
     doi = {10.1109/TMM.2021.3068840},
   }

2. Encoding Spatial Distribution of Convolutional Features for Texture Representation

   Yong Xu, Feng Li, Zhile Chen, Jinxiu Liang, and Yuhui Quan

   Advances in Neural Information Processing Systems (NeurIPS), 2021

   @inproceedings{xu2021encoding,
     title = {Encoding {{Spatial Distribution}} of {{Convolutional Features}} for {{Texture Representation}}},
     author = {Xu, Yong and Li, Feng and Chen, Zhile and Liang, Jinxiu and Quan, Yuhui},
     booktitle = {Advances in {{Neural Information Processing Systems}} ({{NeurIPS}})},
     year = {2021},
     url = {https://openreview.net/forum?id=KnN6mh23cSX}
   }

2020

1. Deep Bilateral Retinex for Low-Light Image Enhancement

   Jinxiu Liang, Yong Xu, Yuhui Quan, Jingwen Wang, Haibin Ling, and Hui Ji

   2020

   arXiv

   Low-light images, i.e. the images captured in low-light conditions, suffer from very poor visibility caused by low contrast, color distortion and significant measurement noise. Low-light image enhancement is about improving the visibility of low-light images. As the measurement noise in low-light images is usually significant yet complex with spatially-varying characteristic, how to handle the noise effectively is an important yet challenging problem in low-light image enhancement. Based on the Retinex decomposition of natural images, this paper proposes a deep learning method for low-light image enhancement with a particular focus on handling the measurement noise. The basic idea is to train a neural network to generate a set of pixel-wise operators for simultaneously predicting the noise and the illumination layer, where the operators are defined in the bilateral space. Such an integrated approach allows us to have an accurate prediction of the reflectance layer in the presence of significant spatially-varying measurement noise. Extensive experiments on several benchmark datasets have shown that the proposed method is very competitive to the state-of-the-art methods, and has significant advantage over others when processing images captured in extremely low lighting conditions.

   @misc{liang2020deep,
     title = {Deep {{Bilateral Retinex}} for {{Low-Light Image Enhancement}}},
     author = {Liang, Jinxiu and Xu, Yong and Quan, Yuhui and Wang, Jingwen and Ling, Haibin and Ji, Hui},
     year = {2020},
     archiveprefix = {arXiv},
   }

2. Advancing Image Understanding in Poor Visibility Environments: A Collective Benchmark Study

   Wenhan Yang, Ye Yuan, Wenqi Ren, Jiaying Liu, Walter J. Scheirer, Zhangyang Wang, Taiheng Zhang, Qiaoyong Zhong, Di Xie, Shiliang Pu, Yuqiang Zheng, Yanyun Qu, Yuhong Xie, Liang Chen, Zhonghao Li, Chen Hong, Hao Jiang, Siyuan Yang, Yan Liu, Xiaochao Qu, Pengfei Wan, Shuai Zheng, Minhui Zhong, Taiyi Su, Lingzhi He, Yandong Guo, Yao Zhao, Zhenfeng Zhu, Jinxiu Liang, Jingwen Wang, Tianyi Chen, Yuhui Quan, Yong Xu, Bo Liu, Xin Liu, Qi Sun, Tingyu Lin, Xiaochuan Li, Feng Lu, Lin Gu, Shengdi Zhou, Cong Cao, Shifeng Zhang, Cheng Chi, Chubing Zhuang, Zhen Lei, Stan Z. Li, Shizheng Wang, Ruizhe Liu, Dong Yi, Zheming Zuo, Jianning Chi, Huan Wang, Kai Wang, Yixiu Liu, Xingyu Gao, Zhenyu Chen, Chang Guo, Yongzhou Li, Huicai Zhong, Jing Huang, Heng Guo, Jianfei Yang, Wenjuan Liao, Jiangang Yang, Liguo Zhou, Mingyue Feng, and Likun Qin

   IEEE Transactions on Image Processing (TIP), 2020

   @article{yang2020advancing,
     title = {Advancing {{Image Understanding}} in {{Poor Visibility Environments}}: {{A Collective Benchmark Study}}},
     author = {Yang, Wenhan and Yuan, Ye and Ren, Wenqi and Liu, Jiaying and Scheirer, Walter J. and Wang, Zhangyang and Zhang, Taiheng and Zhong, Qiaoyong and Xie, Di and Pu, Shiliang and Zheng, Yuqiang and Qu, Yanyun and Xie, Yuhong and Chen, Liang and Li, Zhonghao and Hong, Chen and Jiang, Hao and Yang, Siyuan and Liu, Yan and Qu, Xiaochao and Wan, Pengfei and Zheng, Shuai and Zhong, Minhui and Su, Taiyi and He, Lingzhi and Guo, Yandong and Zhao, Yao and Zhu, Zhenfeng and Liang, Jinxiu and Wang, Jingwen and Chen, Tianyi and Quan, Yuhui and Xu, Yong and Liu, Bo and Liu, Xin and Sun, Qi and Lin, Tingyu and Li, Xiaochuan and Lu, Feng and Gu, Lin and Zhou, Shengdi and Cao, Cong and Zhang, Shifeng and Chi, Cheng and Zhuang, Chubing and Lei, Zhen and Li, Stan Z. and Wang, Shizheng and Liu, Ruizhe and Yi, Dong and Zuo, Zheming and Chi, Jianning and Wang, Huan and Wang, Kai and Liu, Yixiu and Gao, Xingyu and Chen, Zhenyu and Guo, Chang and Li, Yongzhou and Zhong, Huicai and Huang, Jing and Guo, Heng and Yang, Jianfei and Liao, Wenjuan and Yang, Jiangang and Zhou, Liguo and Feng, Mingyue and Qin, Likun},
     journal = {IEEE Transactions on Image Processing (TIP)},
     year = {2020},
     doi = {10.1109/TIP.2020.2981922},
   }

2019

1. Barzilai–Borwein-based Adaptive Learning Rate for Deep Learning

   Jinxiu Liang, Yong Xu, Chenglong Bao, Yuhui Quan, and Hui Ji

   Pattern Recognition Letters (PRL), 2019

   code

   Learning rate is arguably the most important hyper-parameter to tune when training a neural network. As manually setting right learning rate remains a cumbersome process, adaptive learning rate algorithms aim at automating such a process. Motivated by the success of the Barzilai–Borwein (BB) step-size method in many gradient descent methods for solving convex problems, this paper aims at investigating the potential of the BB method for training neural networks. With strong motivation from related convergence analysis, the BB method is generalized to adaptive learning rate of mini-batch gradient descent. The experiments showed that, in contrast to many existing methods, the proposed BB method is highly insensitive to initial learning rate, especially in terms of generalization performance. Also, the BB method showed its advantages on both learning speed and generalization performance over other available methods.

   @article{liang2019barzilai,
     title = {Barzilai--{{Borwein-based}} Adaptive Learning Rate for Deep Learning},
     author = {Liang, Jinxiu and Xu, Yong and Bao, Chenglong and Quan, Yuhui and Ji, Hui},
     journal = {Pattern Recognition Letters (PRL)},
     year = {2019},
     doi = {10.1016/j.patrec.2019.08.029},
     langid = {english},
     pages = {197--203},
     volume = {128}
   }