Fédération Française de Tennis
Efrei Research LabParis, FranceWhereSports field registration
CourtAlign
- 0.87 pxmean reprojection error on tennis, best of all compared methods
- 0false registrations on frames with no usable court: CourtAlign abstains instead, in tennis and badminton
- 8.62Mtrainable parameters, the smallest budget compared (PnLCalib: 131.90M)
Overview
The problem
Quantitative analysis of broadcast sport happens in court coordinates, not in pixels, so every frame has to be mapped to the court through a homography. In racket-sport broadcasts that mapping rests on thin painted lines that players occlude, that are brushed over on clay, compressed by perspective, or absent altogether in replays, close-ups and crowd shots. A method that always returns a homography will project a court onto those frames too.
What was missing
Sports-field registration has been studied mainly on soccer and basketball. Across line-fitting, keypoint and segmentation approaches, the intermediate representation and the geometric estimate are optimised against different objectives, so a representation can score well on its own metric while still inducing an unstable court-to-image mapping. No public racket-sport dataset supplied court-zone masks, per-frame homographies and frames without a court.
Architecture
Swipe to see the whole diagram →
Technical details
- CourtAlign-E2E: frozen SAM 3 ViT encoder; fine-tuned FPN neck (7.80M) and convolutional heads (0.82M) predict 14 (tennis) or 30 (badminton) line-axis landmarks, their visibility and the court zones.
- The homography comes from a confidence-weighted, Hartley-normalised DLT with a differentiable SVD backward pass, which replaces RANSAC and Levenberg–Marquardt refinement.
- Objective: Dice segmentation + heatmap cross-entropy + visibility BCE + Huber coordinate term + Huber reprojection in pixels (δ = 1 px) + template consistency. Geometric terms warm up only once the keypoints are non-degenerate.
- Validity: a frame is accepted only if the predicted court covers ≥ 0.02 of the frame, the keypoints spread ≥ 0.05 of the image diagonal and mean visibility ≥ 0.5. The thresholds are fixed before training and identical for both sports.
- CourtAlign-2S: DeepLabV3+ with a ResNet-34 encoder (22.44M); per-zone convex hulls and four-side fitting give corner correspondences; exact four-point DLT for tennis, RANSAC (8 px) + normalised DLT + Levenberg–Marquardt for badminton.
- Multi-seed evaluation: IoU, projection error (m), reprojection error (px), PCK-H at 5 px, Line-IoU at 0, 3 and 5 px, and false registrations on non-registrable frames (NC-FP).
What I built
- 01
A registration framework for racket sports
Two variants that place the geometric objective at different points of the pipeline: segmentation then geometry (2S), and keypoints trained directly in registration space (E2E).
- 02
Three-phase class-focused training
Isolates the far-side badminton zones that perspective compresses, recovers them and restores the correspondences they contribute.
- 03
A differentiable homography solver
A confidence-weighted DLT lets reprojection and template losses reach the visual representation; supervision in image pixels gave the strongest precision.
- 04
Learned visibility and an explicit validity test
The model decides whether a frame shows a court before reporting any homography.
- 05
A racket-sport registration benchmark
Court-zone masks, per-frame homographies, line masks and non-registrable frames for tennis and badminton broadcasts.
Main results
| Method | IoU (%) ↑ | Proj. (m) ↓ | Reproj. (px) ↓ | PCK-H @5 px (%) ↑ | Line-IoU (%) ↑ | NC-FP ↓ | ||
|---|---|---|---|---|---|---|---|---|
| @0 | @3 | @5 | ||||||
| KpSFR | 96.43±0.31 | 0.11±0.01 | 3.71±0.33 | 48.65±5.32 | 7.34±0.67 | 42.85±2.82 | 57.18±2.58 | 0/20 |
| No-Bells-Just-Whistles | 98.99±0.11 | 0.06±0.00 | 2.51±0.15 | 84.70±0.96 | 11.56±0.52 | 57.71±0.63 | 69.45±0.55 | 18.5/20 |
| KaliCalib | 95.53±0.22 | 0.08±0.01 | 3.60±1.33 | 56.20±7.20 | 9.94±0.72 | 48.08±2.60 | 61.95±2.78 | 20/20 |
| TVCalib | 98.11±0.31 | 0.11±0.00 | 4.61±0.36 | 52.09±2.66 | 7.36±0.22 | 47.06±0.70 | 60.90±1.05 | 0/20 |
| PnLCalib | 98.93±0.15 | 0.05±0.00 | 2.20±0.32 | 89.39±8.22 | 13.78±2.03 | 61.52±1.68 | 72.40±1.35 | 2/20 |
| CourtAlign-2S (Ours) | 99.68±0.05 | 0.05±0.00 | 2.20±0.29 | 83.41±1.12 | 13.72±0.54 | 60.44±0.78 | 71.41±0.58 | 0/20 |
| CourtAlign-E2E (Ours) | 99.60±0.08 | 0.02±0.01 | 0.87±0.12 | 97.33±0.73 | 18.54±0.17 | 65.07±0.15 | 75.11±0.11 | 0/20 |
| Method | IoU (%) ↑ | Proj. (m) ↓ | Reproj. (px) ↓ | PCK-H @5 px (%) ↑ | Line-IoU (%) ↑ | NC-FP ↓ | ||
|---|---|---|---|---|---|---|---|---|
| @0 | @3 | @5 | ||||||
| KpSFR | 99.27±0.20 | 0.03±0.00 | 1.67±0.08 | 97.95±0.66 | 12.51±0.53 | 55.93±0.65 | 67.25±0.51 | 0.75/20 |
| No-Bells-Just-Whistles | 99.33±0.12 | 0.05±0.00 | 2.90±0.17 | 97.19±1.20 | 11.13±0.63 | 53.26±0.72 | 65.12±0.57 | 0/20 |
| KaliCalib | 97.53±0.25 | 0.10±0.03 | 6.18±2.19 | 65.00±9.48 | 7.91±0.89 | 45.43±3.11 | 57.70±3.00 | 20/20 |
| TVCalib | 98.70±0.36 | 0.15±0.01 | 9.06±0.47 | 19.76±3.88 | 5.64±0.38 | 32.27±1.52 | 45.40±1.12 | 5.25/20 |
| PnLCalib | 99.06±0.42 | 0.06±0.02 | 3.22±0.73 | 90.97±10.43 | 11.37±1.16 | 52.60±3.31 | 64.42±2.93 | 4.00/20 |
| CourtAlign-2S (Ours) | 99.45±0.19 | 0.05±0.01 | 2.28±0.36 | 97.27±2.25 | 11.85±0.85 | 54.79±1.32 | 66.29±1.05 | 0/20 |
| CourtAlign-E2E (Ours) | 99.47±0.01 | 0.03±0.00 | 1.69±0.05 | 94.86±0.69 | 13.20±0.16 | 56.84±0.12 | 67.97±0.09 | 0/20 |
Why soccer: to test whether the end-to-end formulation still holds on a much larger field, usually seen only in part, after SoccerNet pre-training and WC14 fine-tuning.
| Method | Proj. (m) ↓ | Reproj. ↓ | ||
|---|---|---|---|---|
| Mean | Median | Mean | Median | |
| Nie et al. | 0.84 | 0.65 | 0.019 | 0.014 |
| KpSFR | 0.81 | 0.63 | 0.019 | 0.014 |
| No-Bells-Just-Whistles* | 1.23 | 0.58 | 0.026 | 0.014 |
| PnLCalib*† | 0.60 | 0.42 | 0.014 | 0.010 |
| CourtAlign-E2E (Ours)*† | 0.83 | 0.46 | 0.020 | 0.012 |
| CourtAlign-E2E w/ Rob-Solv (Ours)*† | 0.78 | 0.48 | 0.020 | 0.013 |
* SoccerNet pre-training · † WC14 fine-tuning · reprojection in normalised image coordinates
Qualitative results
Demo
Limits and next steps
CourtAlign estimates a planar homography without recovering physical camera parameters or lens distortion. Evaluation is frame by frame, so continuity across adjacent frames is not yet used, and the set of non-registrable frames should grow with more independent matches.
Beyond this project
Camera-to-world geometry for any planar scene: field and venue calibration, metric tracking, broadcast graphics and augmented reality. After SoccerNet pre-training and WC14 fine-tuning, CourtAlign-E2E is competitive with soccer-specific methods.
Research output
Paper
A Deep Learning-Based Framework for Racket Sports Court Registration
Ahmed Jouini, Melek Elloumi, Faten Chaieb
AIAI 2024, IFIP AICT vol. 713, Springer, pp. 17–29
Peer-reviewed core idea (CourtAlign-2S). CourtAlign-E2E and the final results are thesis work.
Dataset
Court-registration annotations
Badminton frames, masks and homographies; tennis masks and homographies (no broadcast imagery). Research use.
DatasetContact
Interested in the work?
I'm looking for AI research engineer roles in computer vision and video understanding, ideally in sports analytics, though not only. Ready to start a new journey.
Paris, France