I am a Tenure-Track Assistant Professor in the CSE department at University of Louisville (UofL), and am also affiliated with the Louisville Automation & Robotics Research Institute (LARRI). I received my PhD in Information Sciences and Technology from the Pennsylvania State University, and my Bachelor’s and Master’s degrees from Tsinghua University. During my PhD, I also worked as a research intern at Snap Research, Kitware, and Toyota Research Institute.

I lead the Intelligent Vision and Interaction (InVision) Lab. Our research vision is to advance artificial intelligence to better understand our world and augment human capabilities. To achieve this, our work bridges foundational research in visual representation and synthesis with human-centered applications, focusing on creating trustworthy AI for society and pioneering new tools for medicine and healthcare.

  • Visual Representation and Synthesis: We build computational models that learn rich representations of the visual world. Our lab develops novel methods for machines to perceive their environment and synthesize new visual content, from reconstructing scenes to generating novel imagery. By creating powerful and flexible visual representations, we are laying the essential groundwork for more capable robotics and interactive AI.
  • Human-Centered and Trustworthy AI: We design and evaluate intelligent systems that augment human capabilities and foster equitable human-AI collaboration. This research spans from creating novel assistive technologies for people with disabilities to developing fundamentally fair and privacy-preserving machine learning models for society.
  • AI in Medicine and Healthcare: We develop robust and reliable AI to tackle high-stakes challenges in healthcare. Our goal is to enhance the quality of medical imaging and create specialized machine learning systems for tasks like segmentation and anomaly detection, providing powerful tools to support clinical decision-making.

📣 Call for Papers

I am serving as a guest editor for the special issue “Human-Centered Artificial Intelligence” in the journal Future Internet (ISSN 1999-5903). Submission Deadline: February 28, 2026

📰 Recent News

  • 07/2025: Honored with the Best Student Paper Award at ECML-PKDD 2025 🏆
  • 06/2025: Two papers accepted to ICCV 2025: Top2Pano and DAC
  • 06/2025: Appointed Area Chair for WACV 2026
  • 05/2025: One paper accepted to MICCAI 2025
  • 05/2025: One paper accepted to ECML-PKDD 2025
  • 03/2025: Review paper on layout generation accepted to Computational Visual Media (CVMJ); to be presented at CVM 2025
  • 02/2025: Two papers accepted to CVPR 2025: ZeroPlane and PlanarSplatting See you in Nashville!
  • 01/2025: Paper on interactions of visually impaired users with LMMs accepted to CHI 2025
  • 11/2024: Awarded NSF Campus Cyberinfrastructure (CC*) grant as co-PI. Grateful for the support!
  • 05/2024: Three papers accepted to MICCAI 2024
  • 01/2024: Paper on privacy-preserving remote assistance accepted to CHI 2024
  • 01/2024: One paper accepted to ICRA 2024

📚 Selected Recent Publications [Full List]

🤖 Visual Representation and Synthesis

Top2Pano: Learning to Generate Indoor Panoramas from Top-Down View
Zitong Zhang, Suranjan Gautam, Rui Yu
IEEE/CVF International Conference on Computer Vision (ICCV), 2025.
[paper][project][code]

Describe, Adapt and Combine: Empowering CLIP Encoders for Open-set 3D Object Retrieval
Zhichuan Wang, Yang Zhou, Zhe Liu, Rui Yu, Song Bai, Yulong Wang, Xinwei He, Xiang Bai
IEEE/CVF International Conference on Computer Vision (ICCV), 2025.
[paper][code]

Towards In-the-wild 3D Plane Reconstruction from a Single Image
Jiachen Liu*, Rui Yu*, Sili Chen, Sharon X. Huang, Hengkai Guo
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
✨ Highlight Paper (Top 2.98% of 13,008 submissions)
[paper][code] (*equal contribution)

PlanarSplatting: Accurate Planar Surface Reconstruction in 3 Minutes
Bin Tan, Rui Yu, Yujun Shen, Nan Xue
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025.
✨ Highlight Paper (Top 2.98% of 13,008 submissions)
[paper][project][code]

NeRF-Enhanced Outpainting for Faithful Field-of-View Extrapolation
Rui Yu*, Jiachen Liu*, Zihan Zhou, Sharon X. Huang
IEEE International Conference on Robotics and Automation (ICRA), 2024.
[paper] (*equal contribution)

Be Real in Scale: Swing for True Scale in Dual Camera Mode
Rui Yu, Jian Wang, Sizhuo Ma, Sharon X. Huang, Guru Krishnan, Yicheng Wu
IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2023.
[paper][code]

Cascade Transformers for End-to-End Person Search
Rui Yu, Dawei Du, Rodney LaLonde, Daniel Davila, Christopher Funk, Anthony Hoogs, Brian Clipp
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[paper][code]

🤝 Human-Centered and Trustworthy AI

Beyond Visual Perception: Insights from Smartphone Interaction of Visually Impaired Users with Large Multimodal Models
Jingyi Xie, Rui Yu, He Zhang, Syed Masum Billah, Sooyeon Lee, John M. Carroll
ACM Conference on Human Factors in Computing Systems (CHI), 2025.
[paper][data]

Fairness-Aware Graph Representation Learning with Limited Demographic Information
Zichong Wang, Zhipeng Yin, Liping Yang, Jun Zhuang, Rui Yu, Qingzhao Kong, Wenbin Zhang
European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), 2025.
🏆 Best Student Paper Award
[paper to appear]

Enhancing the Travel Experience for People with Visual Impairments through Multimodal Interaction: NaviGPT, A Real-Time AI-Driven Mobile Navigation System
He Zhang, Nicholas J. Falletta, Jingyi Xie, Rui Yu, Sooyeon Lee, Syed Masum Billah, John M. Carroll
ACM International Conference on Supporting Group Work (GROUP), 2025.
[extended abstract]

BubbleCam: Engaging Privacy in Remote Sighted Assistance
Jingyi Xie*, Rui Yu*, He Zhang, Sooyeon Lee, Syed Masum Billah, John M. Carroll
ACM Conference on Human Factors in Computing Systems (CHI), 2024.
[paper] (*equal contribution)

Are Two Heads Better than One? Investigating Remote Sighted Assistance with Paired Volunteers
Jingyi Xie, Rui Yu, Kaiming Cui, Sooyeon Lee, John M. Carroll, Syed Masum Billah
ACM SIGCHI Conference on Designing Interactive Systems (DIS), 2023.
[paper]

Helping Helpers: Supporting Volunteers in Remote Sighted Assistance with Augmented Reality Maps
Jingyi Xie*, Rui Yu*, Sooyeon Lee, Yao Lyu, Syed Masum Billah, John M. Carroll
ACM SIGCHI Conference on Designing Interactive Systems (DIS), 2022.
[paper][video] (*equal contribution)

Opportunities for Human-AI Collaboration in Remote Sighted Assistance
Sooyeon Lee*, Rui Yu*, Jingyi Xie, Syed Masum Billah, John M. Carroll
ACM International Conference on Intelligent User Interfaces (IUI), 2022.
[paper][video][PSU News] (*equal contribution)

🩺 AI in Medicine and Healthcare

Noise-Robust Tuning of SAM for Domain Generalized Ultrasound Image Segmentation
Zhikai Wei, Chao Wu, Hanyu Du, Rui Yu, Bo Du, Yongchao Xu
International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2025.
[paper to appear]

WIA-LD2ND: Wavelet-based Image Alignment for Self-supervised Low-Dose CT Denoising
Haoyu Zhao, Guyu Liang, Zhou Zhao, Bo Du, Yongchao Xu, Rui Yu
International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2024.
[paper]

MoreStyle: Relax Low-frequency Constraint of Fourier-based Image Reconstruction in Generalizable Medical Image Segmentation
Haoyu Zhao, Wenhui Dong, Rui Yu, Zhou Zhao, Du Bo, Yongchao Xu
International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2024.
[paper]

Spatial-aware Attention Generative Adversarial Network for Semi-supervised Anomaly Detection in Medical Image
Zerui Zhang, Zhichao Sun, Zelong Liu, Bo Du, Rui Yu, Zhou Zhao, Yongchao Xu
International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2024.
[paper][code]