Portrait of Guanxiong Sun

Assistant Professor · Queen’s University Belfast

Guanxiong Sun

I build memory for machines that see, and next, for machines that act.

I’m an Assistant Professor at Queen’s University Belfast, working where computer vision meets machine learning. My work covers video object detection, spatio-temporal representation learning, data-efficient generative models and 3D human motion, and has appeared at CVPR, ICCV, ECCV, ICML and AAAI. I did my PhD at Queen’s with Dr. Yang Hua, Prof. Hui Wang and Prof. Neil Robertson.

01 — News

Recent

  1. vfe.pytorch 2.0 released.

  2. Outstanding Reviewer of CVPR 2026 (top 6.1% of reviewers).

  3. Joined Queen’s University Belfast as an Assistant Professor.

  4. TACTFL in BMVC 2025 (oral).

  5. Training diffusion-based generative models with limited data in ICML 2025.

  6. Quality-aware dynamic discriminator rejection sampling in CVPR 2025.

Earlier news
  1. Neural Sign Actors in CVPR 2024.

02 — Throughline

Perception with memory, generation with structure, action with both.

Each dot is a paper: memory for video perception, structure for generation, and next, memory for agents that act.

  1. 2021 MAMBA AAAI 2021 · Perceive
  2. 2022 EOVOD ECCV 2022 · Perceive
  3. 2022 TDViT ECCV 2022 · Perceive
  4. 2023 STPN ICCV 2023 · Perceive
  5. 2024 Negative augmentation WACV 2024 · Generate
  6. 2024 Neural Sign Actors CVPR 2024 · Generate
  7. 2024 Noise injection ACM MM 2024 · Generate
  8. 2025 Rejection sampling CVPR 2025 · Generate
  9. 2025 Limited-data diffusion ICML 2025 · Generate
  10. 2025 TACTFL BMVC 2025 · Learn
  11. 2026 Physical AI Direction · Act
  12. Next Your project? See open problems

Hover, tap or tab to a dot. Accent dots have project pages.

  • Perceive

    Memory banks, temporal transformers and prompts that make video features accurate, robust and fast.

  • Generate

    Generative models that learn from little data, and diffusion models of human motion.

  • Learn

    Learning from multimodal, distributed data without centralising it.

  • Act

    Memory for embodied agents and vision-language-action models. In progress.

03 — Experience

Education and experience

Education

  1. Harbin Institute of TechnologyB.Sc. and M.Sc. in Computer Science2012–2018
  2. Queen’s University BelfastPh.D. in Computer Vision2019–2023

Work

  1. Baidu Institute of Deep Learning (IDL)Intern2017
  2. SenseTimeIntern2019
  3. AnyVision now OostoResearch Intern2019–2022
  4. Huawei Noah’s Ark LabResearch Scientist2023–2026
  5. University of BristolSenior Research Associate2025–2026
  6. Queen’s University BelfastAssistant Professor2026–now

04 — Selected work

Three projects, opened up.

Interactive explainers, real results and code for the work I’m asked about most.

Plate IFrom the paper
MAMBA’s memory bank as the video advances: features are sampled for each read and replaced one by one. Method animation by the paper’s authors.

Plate I · AAAI 2021 · Video object detection

MAMBA

Multi-level aggregation via memory bank

Video detectors improve the current frame by attending to other frames. Sliding windows and queues attend to everything they store and forget whole frames at once. MAMBA keeps a memory bank that is updated feature by feature, so it can hold knowledge from the entire video, and it reads the bank through a small sampled key set to stay fast.

Not the 2023 state-space model: this MAMBA dates from 2021.

ImageNet VID val, ResNet-101
83.8AP50
at 83.7 mAP, ResNet-101
12.6FPS
Plate IIFrom the paper
Show panel of the STPN figure
Panel (a), previous pipeline: support frames and the blurred current frame each pass through a backbone, then tangled connections feed three separate integration modules, A for detection, B for segmentation and C for tracking. Panel (b), STPN: a DVP predictor turns the support frames into video prompts, which join the current frame at the input of one shared backbone feeding detection, segmentation and tracking heads.
(a) Prior pipelines: a backbone per frame plus a separate integration module for each task (A: detection, B: segmentation, C: tracking). (b) STPN: a DVP predictor turns support frames into video prompts for a single backbone shared by all tasks. Paper Fig. 1.

Plate II · ICCV 2023 · Video understanding

STPN

Spatio-temporal prompting network

Blur, defocus and occlusion degrade single frames. Instead of bolting a heavy, task-specific integration module onto the backbone, STPN predicts a handful of video prompts from neighbouring frames and prepends them to the current frame’s patch embeddings. One backbone then serves detection, instance segmentation and tracking.

ImageNet VID val, Swin-T
85.2AP50
task-specific modules (VOD · VIS · VOT)
3 → 0
prompt tokens (released config)
5

Source: vfe.pytorch README and released config; paper Fig. 1.

Plate IIIFrom the paper
Enlarge figure: For the sentence in the speech bubble, four frames of a human signer on a green screen appear above the matching poses of a 3D signing avatar.
“You want to just kinda take it easy and then use more paint as you start filling in your details.” Reference signer (top) and 3D SMPL-X avatar (bottom). Paper Fig. 1.

Plate III · CVPR 2024 · 3D humans

Neural Sign Actors

A diffusion model for 3D sign language production from text

A diffusion model that turns English text directly into 3D signing avatars (body, hands and face). It is trained on a 3D reconstruction of How2Sign with SMPL-X, and denoises motion with an anatomically informed graph network on the skeleton.

Led by V. Baltatzis and R. A. Potamias (Imperial College London), with E. Ververas, J. Deng and S. Zafeiriou.

best prior 60.32
35.87MPJPE ↓
best prior 3.98
1.56FID ↓
of 3D ASL (How2Sign, SMPL-X)
~79hours

Source: CVPR’24 paper / poster, SLP evaluation and dataset table. ↓ lower is better.

Also notable

Code, models & data

  • Open source

    vfe.pytorch 2.0

    MAMBA and STPN re-implemented on plain PyTorch 2.x, checked tensor by tensor against the original. Released checkpoints reproduce to within 0.02 AP50.

    github.com/guanxiongsun/vfe.pytorch

  • Open source

    Checkpoints

    The released MAMBA (ResNet-101) and STPN (Swin-T) models, one folder per config, ready for the evaluation command.

    huggingface.co/guanxiongsun/vfe.pytorch

  • Open source

    ImageNet VID mirror

    The official links are gone; the data and COCO-style annotations live here.

    huggingface.co/datasets/guanxiongsun/imagenetvid

05 — Publications

Publications

Full list on Google Scholar

Showing 10 of 10 publications

From the paper Thumbnails are figures from each paper; the MAMBA and EOVOD clips are the authors’ own animation and demo. Select a figure to enlarge.

2025

Enlarge figure: TACTFL overview: each client trains its encoder on unlabelled multimodal sensor data with temporal contrastive learning; the server aggregates client models by similarity in embedding space.

BMVC 2025 Oral

TACTFL: Temporal Contrastive Training for Multi-modal Federated Learning with Similarity-guided Model Aggregation

Guanxiong Sun, Majid Mirmehdi, Zahraa Abdallah, Raul Santos-Rodriguez, Ian Craddock, Telmo de Menezes e Silva Filho

Temporal contrastive training for multi-modal federated learning, with similarity-guided model aggregation.

Enlarge figure: Framework for training latent diffusion models with limited data: augmented images are encoded by a pretrained autoencoder and denoised in the latent space, with a negative-augmentation loss.

ICML 2025

Training Diffusion-based Generative Models with Limited Data

Zhaoyu Zhang, Yang Hua, Guanxiong Sun, Hui Wang, Seán McLoone

Diffusion models trained with limited data, by constraining the denoiser’s hypothesis space and adding mixed augmentation.

Enlarge figure: Method diagram: when the discriminator is updated, the highest-scoring real samples and lowest-scoring generated samples are rejected, with the number set dynamically.

CVPR 2025

Improving the Training of Data-Efficient GANs via Quality Aware Dynamic Discriminator Rejection Sampling

Zhaoyu Zhang, Yang Hua, Guanxiong Sun, Hui Wang, Seán McLoone

A quality-aware, dynamic discriminator rejection-sampling scheme for data-efficient GANs.

2024

Enlarge figure: Two 2 by 2 grids of 100-shot Obama face images, labelled (a) and (b), comparing image quality for GAN training with limited data.

ACM MM 2024

Improving the Training of the GANs with Limited Data via Dual Adaptive Noise Injection

Zhaoyu Zhang, Yang Hua, Guanxiong Sun, Hui Wang, Seán McLoone

Dual adaptive noise injection to improve GAN training with limited data.

Enlarge figure: Four frames of a human signer on a green screen above the matching poses of a 3D signing avatar.

CVPR 2024 Project page

Neural Sign Actors: A Diffusion Model for 3D Sign Language Production from Text

Vasileios Baltatzis, Rolandos Alexandros Potamias, Evangelos Ververas, Guanxiong Sun, Jiankang Deng, Stefanos Zafeiriou

Text to 3D signing avatars with a diffusion model on an anatomically informed SMPL-X skeleton graph.

Enlarge figure: 100-shot Obama, Panda and Grumpy Cat samples: Diffusion-GAN baseline (left, FID 28.55, 8.69, 21.87) versus Diffusion-GAN with ANDA (right, FID 26.40, 7.88, 19.01).

WACV 2024

Improving the Leaking of Augmentations in Data-Efficient GANs via Adaptive Negative Data Augmentation

Zhaoyu Zhang, Yang Hua, Guanxiong Sun, Hui Wang, Seán McLoone

Adaptive negative data augmentation to reduce augmentation leaking in data-efficient GANs.

2023

Enlarge figure: Task-specific integration modules versus STPN’s prompt predictor feeding one shared backbone.

ICCV 2023 Project page

Spatio-temporal Prompting Network for Robust Video Feature Extraction

Guanxiong Sun, Chi Wang, Zhaoyu Zhang, Jiankang Deng, Stefanos Zafeiriou, Yang Hua

Video prompts predicted from neighbouring frames make one backbone robust for detection, segmentation and tracking.

2022

ECCV 2022

Efficient One-stage Video Object Detection by Exploiting Temporal Consistency

Guanxiong Sun, Yang Hua, Guosheng Hu, Neil Robertson

A one-stage video detector exploiting temporal consistency; 5.5× faster than SELSA in the authors’ demo video.

Enlarge figure: TDViT architecture: four hierarchical stages of temporal dilated transformer blocks with temporal dilation growing from 3 to 24 over a memory of past spatio-temporal features.

ECCV 2022

TDViT: Temporal Dilated Video Transformer for Dense Video Tasks

Guanxiong Sun, Yang Hua, Guosheng Hu, Neil Robertson

A temporal dilated video transformer with a hierarchical memory structure for dense video tasks.

2021

AAAI 2021 Project page

MAMBA: Multi-level Aggregation via Memory Bank for Video Object Detection

Guanxiong Sun, Yang Hua, Guosheng Hu, Neil Robertson

A memory bank with sampled key sets and feature-wise updates for fast, accurate video detection.

06 — Vision

Next: memory for machines that act.

Where the thread leads: from memory banks for video to memory for agents that act in the physical world, and the work in progress.

Direction · 2026

Physical AI: perception and generation, grounded in interaction

Grounding perception and generative models in physical interaction, for robots that share space with people.

Read more: Physical AI

Read the vision

07 — Collaborate

Open problems. Let’s work on them together.

I’m looking for collaborators (faculty, industry researchers and students) who care about memory in perception and action. If one of these is your problem too, write to me.

  1. Efficient memory for long-horizon VLA policies

    • Robot learning
    • VLA
    • Long context
  2. Memory as architecture

    • Efficient attention
    • KV cache
    • Memory layers
  3. Persistent world models for shared spaces

    • Video generation
    • World models
    • Human–robot interaction

See all open problems

School of EEECS, Queen’s University Belfast, Belfast, UK

08 — Service & background

Service and background

Reviewing

  • CVPR
  • ICCV
  • ECCV
  • NeurIPS
  • ICLR
  • ICML
  • AAAI
  • WACV

Outstanding Reviewer, CVPR 2026 (top 6.1%).

Education

  • Ph.D. in Computer Vision, Queen’s University Belfast (2019–2023). Video object detection, segmentation and tracking, advised by Dr. Yang Hua, Prof. Hui Wang and Prof. Neil Robertson.
  • B.Sc. and M.Sc. in Computer Science, Harbin Institute of Technology (2012–2018). M.Sc. research in medical image processing with Prof. Kuanquan Wang.

For hosts

Guanxiong Sun is an Assistant Professor at Queen’s University Belfast working on computer vision and machine learning, with a focus on memory for perception and action. He received his PhD in computer vision from Queen’s University Belfast (advised by Dr Yang Hua, Prof. Hui Wang and Prof. Neil Robertson) and his B.Sc. and M.Sc. in computer science from Harbin Institute of Technology. His work appears at CVPR, ICCV, ECCV, ICML and AAAI.