MedGRPO: Multi-Task Reinforcement Learning for Heterogeneous Medical Video UnderstandingMedGRPO: 异构医学视频理解里的多任务强化学习

This paper does two things: it builds MedVidBench, a large medical video instruction benchmark, and proposes MedGRPO, a reinforcement learning framework for heterogeneous medical video tasks. The target is not everyday video understanding. The model must handle surgery, nursing, endoscopy, and robotic procedures while solving localization, summarization, region captioning, safety scoring, and next-action prediction.

The most important observation is that standard GRPO collapses in this multi-dataset setting. The issue is not that RL is unusable, but that raw rewards from different dataset-task pairs are not comparable. Easy datasets naturally produce higher rewards and hard datasets lower rewards, so the optimizer over-prioritizes easy sources. MedGRPO fixes this by aligning reward distributions across dataset-task pairs and by using a medical LLM judge for clinical caption quality.

Background

• Medical video understanding is fine-grained. Models must identify instruments, anatomy, procedure phase, safety criteria, and clinically meaningful actions.
• Existing datasets are not instruction-ready. CholecT50, EgoSurgery, AVOS, NurViD, and others contain useful annotations, but not unified VideoLLM-style QA.
• Single-dataset adaptation generalizes poorly. Fine-tuning on one procedure or source does not cover nursing, open surgery, laparoscopic surgery, and robotic surgery together.
• Multi-task RL is sensitive to reward scale. If reward magnitudes reflect dataset difficulty rather than model quality, policy gradients optimize the wrong distribution.

Problem

• No unified medical video instruction benchmark. Existing resources are fragmented across formats, procedures, tasks, and annotation granularities.
• SFT helps but is not enough. Qwen2.5-VL-7B fine-tuned on MedVidBench beats GPT-4.1 and Gemini-2.5-Flash, yet grounding and captioning still leave room for improvement.
• Standard GRPO collapses. CoPESD has much higher median STG mIoU than EgoSurgery, so raw mIoU rewards bias training toward the easier source.
• Generic caption metrics miss clinical errors. “Tool grasps tissue” and “grasper dissects the cystic duct” may look similar to embeddings, but are clinically different.

Core Idea

• First unify data into instruction format. MedVidBench converts 8 medical video sources into 531,850 video-instruction pairs across 8 tasks.
• Use SFT before RL. Supervised fine-tuning adapts the base VLM to the medical video domain and provides baseline reward statistics.
• Normalize rewards per dataset-task pair. Each median baseline performance is mapped to the same normalized reward, preventing easy datasets from dominating.
• Use a medical LLM judge for captions. The reward evaluates terminology, instrument/anatomy identification, specificity, clinical context, and action/state accuracy.

MedVidBench

• Scale. 531,850 QA samples: 461,413 train and 70,437 test, from 626 videos.
• Sources. CholecT50, CholecTrack20, Cholec80-CVS, CoPESD, AVOS, EgoSurgery, JIGSAWS, and NurViD.
• Tasks. Video-level tasks include video summarization, critical view of safety, next action prediction, and skill assessment; segment-level tasks include temporal action grounding, dense video captioning, and region captioning; frame-level tasks include spatiotemporal grounding.
• Quality control. Frame-annotated datasets use visual prompts with bounding boxes and labels; web-sourced videos use Whisper-X transcripts and metadata; GPT-4.1 and Gemini-2.5-Flash are used for dual-model generation and similarity filtering.
• Human validation. Twelve medical data analysis experts prefer expert-prompt captions over frame-only captions by 82.0% vs. 18.0%.

MedGRPO Method

1. Two-stage training. Qwen2.5-VL-7B is first SFT-trained on MedVidBench, then optimized with GRPO on representative grounding and captioning tasks.

2. Group-relative advantage. For each prompt, the model samples G = 8 responses and computes advantages from within-group normalized rewards. The paper follows DAPO-style asymmetric clipping and removes the KL penalty.

3. Cross-dataset reward normalization. For every dataset-task pair, SFT baseline p25, p50, and p75 statistics define a logistic normalization. The median p50 maps to 0.5 and the interquartile range controls scale.

4. Grounding reward. TAG and STG use normalized mIoU as content reward and a format penalty for parse failures. The two are multiplied rather than added.

5. Caption reward. VS and RC combine normalized semantic similarity with the medical LLM judge score using an equal-weighted average.

Medical LLM Judge

• Medical terminology precision. Whether the caption uses correct clinical terms instead of generic language.
• Instrument and anatomy identification. Whether tools and structures such as grasper, hook, cystic duct, and gallbladder are named correctly.
• Specificity versus vagueness. Whether the caption provides precise spatial and procedural details.
• Clinical procedure context. Whether the caption understands the procedural workflow and purpose.
• Action and state accuracy. Whether verbs such as grasps, retracts, dissects, and coagulates match the visual evidence.

The supplementary study reports strong agreement with 10 board-certified clinicians: Pearson r = 0.977 and Cohen's Kappa = 0.817.

Results

• SFT already beats closed-source baselines. Qwen2.5-VL-7B SFT reaches 0.894 on CVS, while GPT-4.1 is 0.018 and Gemini-2.5-Flash is 0.101.
• MedGRPO improves grounding. On Qwen2.5-VL-7B, STG mIoU improves from 0.177 to 0.202, TAG@0.3 from 0.142 to 0.216, and TAG@0.5 from 0.091 to 0.156.
• Caption gains are larger. VS LLM judge improves from 3.596 to 4.184, RC from 2.757 to 3.442, and DVC from 3.665 to 3.797.
• The recipe transfers to Qwen3-VL-4B. MedGRPO further improves the SFT model across STG, TAG, DVC, VS, and RC.
• Off-the-shelf models still need domain adaptation. Newer 2026 models are better, but remain weak on medical grounding compared with MedVidBench-adapted models.

Ablation

• Removing reward normalization breaks training. CVS drops to 0.020, STG to 0.010, and TAG@0.3 to 0.004.
• Multi-task RL has positive transfer. Adding caption rewards to grounding rewards improves STG and TAG.
• The LLM judge matters. VS and RC are both better when the medical judge is included instead of relying only on semantic similarity.
• CVS remains difficult. The model struggles to calibrate 0/1/2 safety scores, especially partial versus complete satisfaction.

Limitations

• Data generation depends on strong VLMs. MedVidBench captions inherit some biases from GPT-4.1 and Gemini-2.5-Flash.
• The judge needs broader validation. It correlates well with clinicians in this setting, but other specialties and languages may behave differently.
• RL optimizes only selected tasks. NAP is not directly rewarded and slightly drops in the main Qwen2.5-VL experiment.
• Safety scoring may require specialized calibration. CVS is ordinal and clinically nuanced, not just a recognition task.

My Take

The main value of MedGRPO is not simply applying GRPO to medical videos. It identifies a real failure mode in multi-dataset medical RL: rewards cannot be mixed raw. Otherwise, the policy learns dataset difficulty bias rather than medical capability.

The evaluation design is also worth reusing. Medical captions are not generic captions; instrument names, anatomy, action verbs, and workflow context can change the clinical meaning. Combining task metrics, a clinical LLM judge, and reward normalization may be more important than simply scaling SFT data.