Best Presentation Award
Learning to Match 2D Keypoints Across Preoperative MR and Intraoperative Ultrasound

- The problem
- During brain surgery the brain shifts, so the MRI taken beforehand stops lining up with reality. Ultrasound shows the current state, but it looks nothing like MRI: different texture, heavy speckle noise. Classic keypoint matchers find almost nothing in common between the two.
- The idea
- Matching by synthesis. From each patient’s MRI we generate many synthetic ultrasound images covering several MR contrasts and realistic ultrasound variation, so every keypoint has known correspondences. A Siamese network is then trained per patient with supervised contrastive learning to describe each keypoint by its structure, not its texture.
- The result
- On real clinical cases with ground truth from the ReMIND dataset, the descriptors reached 80.35% average matching precision, outperforming the state-of-the-art methods it was compared against.
My part Designed the descriptor and the synthesis pipeline, ran the experiments and wrote the paper. It became my TUM master’s thesis, done at Harvard Medical School / Brigham and Women’s Hospital.
