X³-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment Dongjie Fu, Di Cao, Xize Cheng, Zihan Zhang, Wenxu Jia, Yifu Chen, Shengpeng Ji, Yu Zhang, Tao Jin EMNLP 2026 Submission (First Author)

Paper

We propose X³-OPD, a cross-modal on-policy distillation framework that transfers reasoning from a text teacher to an audio-language student by scoring the student’s own audio-conditioned trajectories with matched textual inputs. Built on a 71.8K three-tier symmetric corpus spanning logical, audio-event, and prosody-aware dialogue reasoning, X³-OPD improves BIG Bench Audio from 87.9 to 93.6 (+5.7) while largely preserving out-of-domain capabilities.

Character Beyond Speech: Leveraging Role-Playing Evaluation in Audio Large Language Models via Reinforcement Learning Dongjie Fu, Fangming Feng, Xize Cheng*, Linjun Li, Zhou Zhao, Tao Jin ACM MM 2026 (First Author)

Paper

We propose RoleJudge, a multimodal and multidimensional evaluator for voice role-playing agents, and build RoleChat, the first reasoning-enhanced voice role-playing evaluation dataset. A multi-stage SFT-RL framework with Standard Alignment improves character-alignment evaluation while mitigating reward misalignment.

PACHAT: Persona-Aware Speech Assistant for Multi-party Dialogue Dongjie Fu, Xize Cheng, Linjun Li, Xiaoda Yang, Lujia Yang, Tao Jin EMNLP 2025 (First Author)

Demo | Paper

We build Persona-Dialogue, the first large-scale multi-party spoken dialogue dataset with user profiles, and propose PAChat, which jointly models semantic and speaker representations to enable persona-aware personalized responses.

X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs Di Cao*, Dongjie Fu*, Hai Yu, Siqi Zheng, Xu Tan, Tao Jin Interspeech 2026 (Co-first Author)

Paper

We propose X-OPD, a cross-modal on-policy distillation framework that aligns the capabilities of Speech LLMs to their text-based counterparts via token-level feedback from a text teacher, closing the performance gap while preserving speech abilities.

Boosting Speech Recognition Robustness to Modality-Distortion with Contrast-Augmented Prompts Dongjie Fu, Xize Cheng, Xiaoda Yang, Hanting Wang, Zhou Zhao, Tao Jin ACM MM 2024 (Oral, First Author)

Paper

We propose PCD, a contrast-augmented prompt framework that enhances the robustness of audio-visual speech recognition to modality-distortion (asynchrony, visual noise and audio noise), achieving SOTA under distorted resources on LRS2.

Emphasizing Domain Differences through Interactive-Augmented Prompts in Continual Audio-Visual Speech Recognition Dongjie Fu, Xize Cheng, Jingyuan Chen, Tao Jin, Zhongfei Zhang IEEE TIP (First Author)

Paper

We introduce the Continual Audio-Visual Speech Recognition (CL-AVSR) problem and propose IMP, an interaction-enhanced multimodal prompt learning framework that transfers cross-domain knowledge with minimal parameter overhead, mitigating forgetting via cross-modal alignment and cross-task contrast.