EgoSound: Benchmarking Sound Understanding in Egocentric Videos
Zhu, B., Fu, Y., Dong, Q., Sun, G., Qian, T., Wu, Y., Paudel, D. P., Fu, Y., & Xue, X. (2026). EgoSound: Benchmarking Sound Understanding in Egocentric Videos. In CVPR.
CV
Education
- Ph.D in computer vision & AI, ETH Zurich
- M.S. in computer science, KAUST
- B.S. in Science, Huazhong University of Science & Technology
Work experience
- Fall 2023: Research Scientist Intern
- Meta AI
- Projects: Multi-modal large language models (MLLMs)
- Summer 2023: Research Scientist Intern
- Adobe Research
- Projects: Video matting & segmentation
- 2018-2019: Research Engineer
- Inception Institute of Artificial Intelligence (IIAI)
- Duties: develop CV/AI algorithms on scene understanding, segmentation, and object counting
- Supervisor: Prof. Fahad Kan, Prof. Hisham Cholakkal, and Prof. Salman Khan.
Skills
- Deal with large-scale datasets
- Deep learning frameworks: Pytorch and Tensorflow
- Various computer vision algorithms: object detection, image/video segmentation, object counting, denoising, super resolution.
Publications
EgoNight: Towards Egocentric Vision Understanding at Night with a Challenging Benchmark
Zhang, D., Fu, Y., Yang, R., Miao, Y., Qian, T., Zheng, X., Sun, G., Chhatkuli, A., Huang, X., Jiang, Y.-G., Van Gool, L., & Paudel, D. P. (2026). EgoNight: Towards Egocentric Vision Understanding at Night with a Challenging Benchmark. In ICLR.
CamSAM2: Segment Anything Accurately in Camouflaged Videos
Zhou, Y., Li, Y., Fu, Y., Benini, L., Konukoglu, E., & Sun, G. (2025). CamSAM2: Segment Anything Accurately in Camouflaged Videos. In NeurIPS.
XTrack: Multimodal Training Boosts RGB-X Video Object Trackers
Tan, Y., Wu, Z., Fu, Y., Zhou, Z., Sun, G., Zamfir, E., Ma, C., Paudel, D. P., Van Gool, L., & Timofte, R. (2025). XTrack: Multimodal Training Boosts RGB-X Video Object Trackers. In ICCV.
ObjectRelator: Enabling Cross-View Object Relation Understanding Across Ego-Centric and Exo-Centric Perspectives
Fu, Y., Wang, R., Ren, B., Sun, G., Gong, B., Fu, Y., Paudel, D. P., Huang, X., & Van Gool, L. (2025). ObjectRelator: Enabling Cross-View Object Relation Understanding Across Ego-Centric and Exo-Centric Perspectives. In ICCV.
Generalized Few-shot 3D Point Cloud Segmentation with Vision-Language Model
An, Z., Sun, G., Liu, Y., Li, R., Han, J., Konukoglu, E., & Belongie, S. (2025). Generalized Few-shot 3D Point Cloud Segmentation with Vision-Language Model. In CVPR.
Multimodality Helps Few-shot 3D Point Cloud Semantic Segmentation
An, Z., Sun, G., Liu, Y., Li, R., Wu, M., Cheng, M.-M., Konukoglu, E., & Belongie, S. (2025). Multimodality Helps Few-shot 3D Point Cloud Semantic Segmentation. In ICLR.
Towards Open-Vocabulary Video Semantic Segmentation
Li, X., Liu, Y., Sun, G., Wu, M., Zhang, L., & Zhu, C. (2025). Towards Open-Vocabulary Video Semantic Segmentation. IEEE Transactions on Multimedia, 27, 2924–2934.
Indiscernible Object Counting in Underwater Scenes
Sun, G., An, Z., Liu, Y., Liu, C., Sakaridis, C., Fan, D. P., & Van Gool, L. (2023). Indiscernible Object Counting in Underwater Scenes. In CVPR.
Edge Guided GANs with Multi-Scale Contrastive Learning for Semantic Image Synthesis
Tang, H., Sun, G., Sebe, N., & Van Gool, L. (2023). Edge Guided GANs With Multi-Scale Contrastive Learning for Semantic Image Synthesis. TPAMI.
Mining Relations among Cross-Frame Affinities for Video Semantic Segmentation
Sun, G., Liu, Y., Tang, H., Chhatkuli, A., Zhang, L., & Van Gool, L. (2022). Mining relations among cross-frame affinities for video semantic segmentation. In ECCV.
Coarse-to-Fine Feature Mining for Video Semantic Segmentation
Sun, G., Liu, Y., Ding, H., Probst, T., & Van Gool, L. (2022). Coarse-to-fine feature mining for video semantic segmentation. In CVPR.
Looking Beyond Single Images for Weakly Supervised Semantic Segmentation Learning
Wang, W., Sun, G., & Van Gool, L. (2022). Looking beyond single images for weakly supervised semantic segmentation learning. TPAMI.
SwinIR: Image Restoration using Swin Transformer
Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., & Timofte, R. (2021). Swinir: Image restoration using swin transformer. In ICCVW.
Task Switching Network for Multi-task Learning
Sun, G., Probst, T., Paudel, D. P., Popović, N., Kanakis, M., Patel, J., ... & Van Gool, L. (2021). Task switching network for multi-task learning. In ICCV.
Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-resolution
Liang, J., Sun, G., Zhang, K., Van Gool, L., & Timofte, R. (2021). Mutual affine network for spatially variant kernel estimation in blind image super-resolution. In ICCV.
CompositeTasking: Understanding Images by Spatial Composition of Tasks
Popovic, N., Paudel, D. P., Probst, T., Sun, G., & Van Gool, L. (2021). Compositetasking: Understanding images by spatial composition of tasks. In CVPR.
Towards Partial Supervision for Generic Object Counting in Natural Scenes
Cholakkal, H., Sun, G., Khan, S., Khan, F. S., Shao, L., & Van Gool, L. (2020). Towards partial supervision for generic object counting in natural scenes. TPAMI, 44(3), 1604-1622.
Mining Cross-image Semantics for Weakly Supervised Semantic Segmentation
Sun, G., Wang, W., Dai, J., & Van Gool, L. (2020). Mining cross-image semantics for weakly supervised semantic segmentation. In ECCV.
Fixing Localization Errors to Improve Image Classification
Sun, G., Khan, S., Li, W., Cholakkal, H., Khan, F. S., & Van Gool, L. (2020). Fixing localization errors to improve image classification. In ECCV.
Camouflaged Object Detection
Fan, D. P., Ji, G. P., Sun, G., Cheng, M. M., Shen, J., & Shao, L. (2020). Camouflaged object detection. In CVPR.
Fine-grained Recognition: Accounting for Subtle Differences between Similar Classes
Sun, G., Cholakkal, H., Khan, S., Khan, F., & Shao, L. (2020, April). Fine-grained recognition: Accounting for subtle differences between similar classes. In AAAI.
Object Counting and Instance Segmentation with Image-level Supervision
Cholakkal, H., Sun, G., Khan, F. S., & Shao, L. (2019). Object counting and instance segmentation with image-level supervision. In CVPR.
isaid: A Large-scale Dataset for Instance Segmentation in Aerial Images
Waqas Zamir, S., Arora, A., Gupta, A., Khan, S., Sun, G., Shahbaz Khan, F., ... & Bai, X. (2019). isaid: A large-scale dataset for instance segmentation in aerial images. In CVPRW.
Patents
- US Patent App. 16/278,392, 2019, Object counting and instance segmentation using neural network architectures with image-level supervision.