StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modeling

2025 ArXiv preprint

[Project Page] [Paper] [Code] [Zhihu]

We propose StreamVLN, a streaming VLN framework that employs a hybrid slow-fast context modeling strategy to support multi-modal reasoning over interleaved vision, language and action inputs. StreamVLN can understand complex human instructions and finish VLN task in various indoor/outdoor scenarios.

NavDP: Learning Sim-to-Real Navigation Diffusion Policy with Privileged Information Guidance

2025 ArXiv preprint

[Project Page] [Paper] [Code] [Zhihu]

We propose NavDP, an end-to-end framework trained solely in simulation that combines diffusion-based trajectory generation and a critic function for trajectory selection (conditioned on local observation tokens from a shared policy transformer); it can zero-shot transfer to different robot embodiments in diverse real-world environments.

RAMPAGE: Towards Whole-body, Real-Time and Agile Motion Planning in Dynamic Cluttered Environments for Mobile Manipulators

[Video] [PDF]

2024 IEEE Transaction on Industrial Electronics

We proposed a hierarchical topology-guided search and AL-DDP-based optimization to solve whole-body kinodynamic planning in dynamic environments, and achieved real-time planning (≈30ms) and ≈80% success rate with accurate collision detection via ESDF map and sphere decomposition.

Learn to Coordinate: a Whole-Body Learning from Demonstration Framework for Differential Drive Mobile Manipulators

[Video1, Video2] [PDF]

2023 IEEE Conference on Systems, Man, and Cybernetics

We developed a Gaussian Process-based learning framework with WLN inverse kinematics for few-shot whole-body skill learning, which enabled coordinated door opening with disturbance rejection and simplified human guidance via admittance control.

Locomotion in complex terrain through reinforcement learning in Isaac lab

[Video]

We used PPO algorithm with a fine-tuned reward function to train Unitree H1’s locomotion via curriculum learning, and realized robust Sim-to-Sim transfer of RL policies to diverse photo-realistic environments in Nvidia Isaac Sim.

Risk-aware contingency motion planning under uncertainties for Automated Valet Parking(AVP)

DJI Automotive

We proposed Voronoi-based safe corridors and SplineGrid optimization for lateral bypass plus iLQR for risk-aware longitudinal speed, and handled prediction multimodality via tree-branch iLQR to ensure safe trajectory under perception/prediction uncertainties.

Low-cost and efficient location, mapping, planning and control for multicopter in embedded system

[Video1, Video2, Video3]

Application Innovate Laboratory, Huawei; FastLab, Zhejiang University | 2023.2 - 2024.3

We enhanced VINS-Fusion with learning-based features and QR optimization for better accuracy/robustness on low-performance chips, and implemented fast OGM updating (incremental inflation) and robot-centric ESDF for efficient MPCC-based collision-aware control.

SE3 planning and control for multicopter to cross narrow gap

[Video]

FastLab, Zhejiang University | 2023.10 - 2023.12

We built safe flight corridors (SFC) and used MINCO for spatial-temporal trajectory optimization via L-BFGS, and achieved accurate narrow gap crossing by calibrating thrust mapping and tuning controllers for large-attitude tracking.

Pedestrian following and collision avoidance with spatial-temporal optimization for differential car

[Video]

Application Innovate Laboratory, Huawei; FastLab, Zhejiang University | 2023.2 - 2023.11

We developed multi-level hybrid A* for EKF-predicted pedestrian following plus MINCO optimization for smooth trajectories, and ensured collision avoidance via lidar filtering and MPC control to handle kinematic constraints and communication delay.

Self-balanced racing car with wireless charging capability

[Video]

School of Automation Science and Engineering, SCUT | 2020.01 - 2020.08

We designed adaptive PD wireless charging (≈30W) for super-capacitors and tuned cascade controllers for stable track navigation, which helped win 5th national place with a 23.8s race time and successful passage through circles, slopes, and crossroads.

Wholebody pick-and-place for mobile manipulator

[Video]

Application Innovate Laboratory, Huawei | 2022.10 - 2022.11

We trained GGCNN for 6-D object perception and used OSQP to solve dynamic-weighted QP for whole-body coordination, which enabled smooth pick-and-place by balancing manipulability, energy, and trajectory tracking performance.