Francis
≡
profile photo

Francis Xiatian Zhang

I am a Research Associate at the University of Edinburgh working on robot vision, with interests in 3D perception, motion estimation, and visual navigation. My research combines geometric modelling and deep learning, with recent work on depth estimation, visual odometry, and segmentation for medical robotics and bronchoscopic navigation. I received my PhD from Durham University, where I studied geometric representations for clinical video analysis. More broadly, I am interested in how robots use visual information to understand their surroundings and navigate reliably, and I am open to research and engineering opportunities in robotic perception and applied computer vision.

CV (PDF)  /  Email  /  Academic Email  /  University Profile  /  GitHub  /  Google Scholar  /  Linkedin  /  Twitter

News

  1. 6/2026: Our paper Geometry-Aware Visual Odometry for Bronchoscopic Navigation via High-Gain Observer Fusion received the ICRA 2026 Best Paper Award in Medical Robotics.
  2. 6/2026: Cross-view Multimodal Vision-Based Assessment Framework for Traditional Chinese Medicine Rehabilitation Training (IEEE Transactions on Neural Systems and Rehabilitation Engineering) — accepted.
  3. 6/2026: FDRAS: Failure Diagnosis and Repair for Airway Segmentation — accepted at MICCAI 2026.
  4. 5/2026: Our paper Geometry-Aware Visual Odometry for Bronchoscopic Navigation via High-Gain Observer Fusion was selected as a finalist for the ICRA 2026 Best Paper Award in Medical Robotics.
  5. 2/2026: Three papers at ICRA — ROOM: A Physics-Based Continuum Robot Simulator for Photorealistic Medical Datasets Generation, A Model-Based Framework for Assessing Operator Performance in Navigational Bronchoscopy, and Geometry-Aware Visual Odometry for Bronchoscopic Navigation via High-Gain Observer Fusion — accepted.
  6. 1/2026: Understanding psychiatrist readiness for AI: a study of access, self-efficacy, trust, and design expectations (BMC Health Services Research) — accepted.
  7. 6/2025: Two papers at MICCAI 2025 — Navigational Bronchoscopy in Critical Care via End-to-End Pose Regression (accepted) and BREA-Depth: Bronchoscopy Realistic Airway-geometric Depth Estimation (early accepted).
  8. 5/2025: I officially received approval for my PhD thesis—finally a formal PhD now!
  9. 1/2025: I passed my PhD viva with minor corrections. Examiners: Dr. Hansung Kim and Dr. Stamos Katsigiannis.
  10. 12/2024: Adaptive Graph Learning from Spatial Information for Surgical Workflow Anticipation (IEEE Transactions on Medical Robotics and Bionics) — accepted.
  11. 11/2024: I began serving as a Research Associate at the University of Edinburgh (Research supervisor: Dr. Mohsen Khadem).
  12. 9/2024: Unraveling the brain dynamics of Depersonalization-Derealization Disorder: a dynamic functional network connectivity analysis (BMC Psychiatry) — accepted.
  13. 8/2024: Advancing healthcare practice and education via data sharing: demonstrating the utility of open data by training an artificial intelligence model to assess cardiopulmonary resuscitation skills (Advances in Health Sciences Education) — accepted. Dataset.
  14. 6/2024: Depth-Aware Endoscopic Video Inpainting (MICCAI 2024) — accepted.
  15. 12/2023: Pose-based Tremor Type and Level Analysis for Parkinson's Disease from Video (International Journal of Computer Assisted Radiology and Surgery) — accepted.
  16. 11/2023: I began working as a Research Assistant on the project Course Development in Edge Computing and Analytics 2.0 (PI: Dr. Anish Jindal).
  17. 8/2023: Correlation-Distance Graph Learning for Treatment Response Prediction from rs-fMRI (ICONIP 2023) — accepted.
  18. 4/2023: The Treatment of Depersonalization-Derealization Disorder: A Systematic Review (Journal of Trauma & Dissociation) — accepted.

Recent Projects

I design and build applied machine learning systems for safety-critical and data-constrained domains, with an emphasis on end-to-end pipelines: data curation, model training, evaluation, and deployment-ready inference. My work spans geometry-aware perception, temporal and graph-based modeling, and reproducible tooling that produces integration-friendly outputs (e.g., depth, graphs, structured predictions) for downstream systems. If you’d like to collaborate, please reach out at francis.xiatian.zhang@outlook.com (primary) or francis.zhang@ed.ac.uk (academic).

Geometry-Aware VO

ICRA 2026 · Best Paper Award in Medical Robotics

Geometry-Aware Visual Odometry for Bronchoscopic Navigation via High-Gain Observer Fusion

RGB Depth

Combines depth-based airway geometry with high-gain observer fusion to improve bronchoscopic tracking and reduce trajectory drift.

  • Uses BREA-Depth as the depth-perception foundation for lumen localization and geometric heading estimation.
  • Reduced absolute trajectory error by over 50% compared with baseline visual odometry pipelines in ex-vivo lung experiments.

Paper | arXiv | Award

FDRAS

MICCAI 2026

Failure Diagnosis and Repair for Airway Segmentation

Airway segmentation before and after FDRAS: nnUNet input and refined output on AeroPath

Diagnoses and repairs structural failures in chest CT airway segmentations, using predicted error maps to guide targeted corrections.

  • Localizes boundary discrepancies and disconnected branches with geometric and tree-level connectivity error maps.
  • Provides a trained checkpoint, training and inference scripts, and error-map visualization tools.

GitHub | Paper

CME-AQA

IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2026

Cross-view Multimodal Vision-Based Assessment Framework for Traditional Chinese Medicine Rehabilitation Training

TCM-AQA61 dataset examples showing acupuncture and Tuina training from first-person and third-person views

Assesses acupuncture and Tuina training skills by combining visual and pose features, learning from first-person and third-person videos while requiring only a third-person view at inference.

  • Introduces two synchronized dual-view datasets with expert annotations from 61 participants for each task.
  • Provides training and inference code, sample videos, and demonstrations of skill assessment.

GitHub | Paper | arXiv

BREA-Depth

MICCAI 2025

Bronchoscopy Realistic Airway-geometric Depth Estimation

BREA-Depth airway depth illustration

Publishes ready-to-run inference code for bronchoscopic navigation research stacks, adapting foundation models to bronchoscopy footage so downstream perception modules inherit reliable depth.

  • Produces stable depth estimates that downstream planners can use for CT-to-video alignment and collision checks.
  • Pipeline: synthetic airway renderer → domain adaptation → depth inference package → evaluation harness, all versioned so surgical robotics teams can reproduce deployments.

GitHub | Paper | arXiv

Adaptive Graph Learning

IEEE Transactions on Medical Robotics and Bionics, 2024

Adaptive Graph Learning from Spatial Information for Surgical Workflow Anticipation

Adaptive graph learning visualization

Turns live tool detections into actionable workflow forecasts so scrub nurses and displays know what to prepare next.

  • Continuously re-learns instrument relationships from spatial cues to surface impending steps minutes ahead.
  • Prototype pipeline: detector feed → spatial graph builder → anticipation head → dashboard concept, with warning throttling heuristics and fallbacks when detections drop.

GitHub | Paper | arXiv

DAEVI

MICCAI 2024

Depth-Aware Endoscopic Video Inpainting

DAEVI depth-aware inpainting visualization

Provides a depth-aware inpainting toolkit that teams can run locally to clean surgeon video archives prior to research sharing or dataset release.

  • Accepts raw video plus masks and returns temporally consistent fills, ready for downstream training or review.
  • Bundled training/inference scripts with deterministic preprocessing and checkpoints so biomedical researchers can rerun inpainting and compare outputs to their own reference sets.

GitHub | Paper | arXiv

Research

My research focuses on Computer-assisted Intervention, Medical Robotics, and Vision-based Skill Assessment. Below are selected publications, including recent collaborations in biomedical engineering and medical robotics. A complete list of my publications can be found on my Google Scholar page.

Conference Papers:

  1. FDRAS: Failure Diagnosis and Repair for Airway Segmentation
    Francis Xiatian Zhang, Kevin Dhaliwal and Mohsen Khadem
    MICCAI 2026 | paper | code

    Contribution: Predicts geometric and connectivity error maps to guide targeted repair of airway segmentations under domain shift.

    Medical Imaging
  2. Geometry-Aware Visual Odometry for Bronchoscopic Navigation via High-Gain Observer Fusion
    Mohammadreza Kasaei, Francis Xiatian Zhang, Feng Li, Farshid Alambeigi, Kevin Dhaliwal and Mohsen Khadem
    IEEE International Conference on Robotics and Automation (ICRA) 2026 | paper | arXiv

    Contribution: Combines BREA-Depth-based depth perception with high-gain observer fusion to improve bronchoscopic tracking and reduce trajectory drift.

    Robotics
  3. A Model-Based Framework for Assessing Operator Performance in Navigational Bronchoscopy
    Zhaoxing Deng, David Hanley, Francis Xiatian Zhang, Kevin Dhaliwal and Mohsen Khadem
    IEEE International Conference on Robotics and Automation (ICRA) 2026 | paper

    Contribution: Uses tracked bronchoscope poses, kinematic constraints, and model-predictive reference trajectories to assess operator performance.

    Robotics
  4. ROOM: A Physics-Based Continuum Robot Simulator for Photorealistic Medical Datasets Generation
    Salvatore Esposito, Matías Mattamala, Daniel Rebain, Francis Xiatian Zhang, Kevin Dhaliwal, Mohsen Khadem and Subramanian Ramamoorthy
    IEEE International Conference on Robotics and Automation (ICRA) 2026 | paper | arXiv

    Contribution: Combines continuum robot simulation with photorealistic rendering to generate datasets for robotic bronchoscopy.

    Robotics
  5. Navigational Bronchoscopy in Critical Care via End-to-End Pose Regression
    Emile Mackute, Francis Xiatian Zhang, Kevin Dhaliwal and Mohsen Khadem
    MICCAI 2025 | paper

    Contribution: Estimates bronchoscope pose directly from video to support navigation in critical care.

    Robotics
  6. BREA-Depth: Bronchoscopy Realistic Airway-geometric Depth Estimation
    Francis Xiatian Zhang, Emile Mackute, Mohammadreza Kasaei, Kevin Dhaliwal, Robert Thomson and Mohsen Khadem
    MICCAI 2025 | paper | arXiv | code

    Contribution: Adapts foundation models for bronchoscopic depth estimation using depth-aware domain translation, airway structure losses, and airway-specific evaluation metrics.

    Medical Imaging
  7. Depth-Aware Endoscopic Video Inpainting
    Francis Xiatian Zhang, Shuang Chen, Xianghua Xie and Hubert P. H. Shum
    MICCAI 2024 | paper | arXiv | code

    Contribution: Combines spatial-temporal depth estimation, bi-modal fusion, and a depth-enhanced discriminator to recover occluded anatomy in endoscopic video.

    Medical Imaging
  8. Pose-Based Tremor Classification for Parkinson's Disease Diagnosis from Video
    Haozheng Zhang, Edmond S. L. Ho, Francis Xiatian Zhang and Hubert P. H. Shum
    MICCAI 2022 | paper | arXiv | code

    Contribution: Supports pose-based Parkinsonian tremor classification through dataset curation, evaluation design, and clinical interpretation.

    Human Motion Analysis

Journal Papers:

  1. Cross-view Multimodal Vision-Based Assessment Framework for Traditional Chinese Medicine Rehabilitation Training
    Francis Xiatian Zhang, Hao Yao, Shengxuan Chen, Hong Zhu, Hongxiao Jia, Sisi Zheng and Hubert P. H. Shum
    IEEE Transactions on Neural Systems and Rehabilitation Engineering 2026 | paper | arXiv | code

    Contribution: Combines visual and pose features with cross-view training to assess acupuncture and Tuina skills using expert-annotated dual-view datasets.

    Human Motion Analysis
  2. Adaptive Anticipation: Adaptive Graph Learning From Spatial Information for Surgical Workflow Anticipation
    Francis Xiatian Zhang, Jingjing Deng, Robert Lieck and Hubert P. H. Shum
    IEEE Transactions on Medical Robotics and Bionics 2024 | paper | arXiv | code

    Contribution: Learns spatial relationships between surgical instruments with an adaptive graph to anticipate surgical workflow.

    Robotics
  3. Advancing healthcare practice and education via data sharing: demonstrating the utility of open data by training an artificial intelligence model to assess cardiopulmonary resuscitation skills
    Merryn D. Constable, Francis Xiatian Zhang*, Tony Conner, Daniel Monk, Jason Rajsic, Claire Ford, Laura Jillian Park, Alan Platt, Debra Porteous, Lawrence Grierson and Hubert P. H. Shum
    Advances in Health Sciences Education 2024 | paper | code | dataset

    Contribution: Combines multi-view pose estimation and spatial-temporal graph modelling to assess CPR skills.

    Human Motion Analysis

* = co-first author.

Awards

  1. 2026, ICRA Best Paper Award in Medical Robotics
    Awarded to Geometry-Aware Visual Odometry for Bronchoscopic Navigation via High-Gain Observer Fusion, co-authored with Mohammadreza Kasaei, Feng Li, Farshid Alambeigi, Kevin Dhaliwal and Mohsen Khadem.
  2. 2020, Dean's List Award for Outstanding Achievement, Mathematics Science, Faculty of Social Sciences, University of Southampton.

Experience

  1. Research Associate at The University of Edinburgh, 11/2024-.
    Develops deep learning visual navigation prototypes for robotic bronchoscopy—airway segmentation, geometric graph construction, and failure-aware inference evaluated on recorded clinical video streams.
    Research supervisor: Dr Mohsen Khadem
  2. Research Assistant at Durham University, 10/2023-11/2023;7/2024-9/2024.
    Delivered the Edge Computing and Analytics 2.0 course by teaching end-to-end model workflows—from training to ONNX export and deployment through Python APIs with lightweight GUIs.
    PI: Dr Anish Jindal
  3. Demonstrator at Durham University, 11/2021-6/2024.
    Led weekly lab support for Computational Thinking, Data Science, Programming for Data Science, and Text Mining modules—debugging starter code, clarifying marking rubrics, and routing stubborn tooling issues to instructors.
  4. Student Helper at Durham University, 08/2022-09/2022.
    Handled visiting speaker transport, couch bookings, and leisure tours around Durham Castle for the 21st ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA 2022).
  5. Research Assistant at Northumbria University, 04/2022-07/2022.
    Set up multi-camera capture, collected nursing simulation footage, and led training of the automatic rating system that turned pose-estimation outputs into competency scores.
    PI: Dr Merryn Constable

Grants Involved

  1. U-care: Deep Ultraviolet Light Therapies
    Research Associate; EPSRC programme grant: EP/T020903/1; Total award: £6,132,366
    Programme PI: Prof Robert Thomson
    Research supervisor: Dr Mohsen Khadem
    Funded by UKRI, 2021–2026 (I joined in 2024)
  2. Course Development in Edge Computing and Analytics 2.0
    Research Assistant, UK-Egypt Trans-National Education (TNE) Grant: £30,000 (PI: Dr. Anish Jindal)
    Funded by the British Council, UK, 2022–2024
  3. Pose Estimation for Health Professional Education: Development of an Objective Computerized Approach for Measuring and Assessing Technical Competencies in Nursing
    Research Assistant, Northumbria University Application Seed Funding Scheme: £16,428 (PI: Dr. Merryn Constable)
    Funded by Northumbria University, UK, 2022
Thanks to Yunhe Wang, who kindly shared the open-source code for the website template I used ✩.

Powered by w3.css