Home

15 min read

Tennis Analysis

Sports and informatics. Where my dream fullfills it self. Know how fast you were running or how fast your serve was, this is what I want to know when I play tennis. And that's why I made this project. There is simply no cheap option that tracks your stats or your game. In tennis, there's one competitor named "swing vision" who does a great job in what they do, but it's only focused on tennis; other sports are not really cared about. For example, in handball, camera systems cost thousands of euros. So why not have a cost-effective solution that just uses a GoPro and a Python script to do the rest of the magic?

In this project, I aimed to analyze sports data, choosing tennis for its simple setup: mount a camera on the fence, film a game with two players, and process the video data. Here’s a deeper look into the approach and technologies used.

What Do We Have to Analyze?

  • Ball
  • Court
  • Players
  • Rackets

Let's start with the Ball.

Ball Detection

My idea was to:

  • Capture every movement (i.e., the absolute difference between two images),
  • Cast them into Regions of Interest (ROIs)
  • Analyze these ROIs with a small image classifier that outputs "Ball" or "No ball" for each ROI (as shown in the diagram below).
Process of Ball Detection

I experimented with two models:

  1. MobileNetV1 – Very fast in processing time, but prone to false positives (e.g., confusing a white shoe for a ball).
  2. DLA Simple – More accurate, yet about 5 times slower.

To balance speed and accuracy, I combined these models by:

  • Making a pre-prediction with MobileNetV1,
  • Validating those predictions with DLA Simple.

Then, to further optimize performance, I added an algorithm that only scans the area around the already detected tennis ball. If the system loses track, it scans the whole image every 40 frames.

Speedup of Optimization

Court Detection

For court detection, I used YoloV11-Keypoint. The data annotation process involved:

  • Annotating court data manually,
  • Applying data augmentation (e.g., rotations, changes in the HSV color space, cropping/zooming) to add variety and prevent overfitting.

This process resulted in:

  • Astonishing results after many training runs,
  • A training period of about 16 hours on an RTX 4060,
  • Annotation of around 1200 tennis fields.
Labeling Court Data Augmentation for Court Detection

Result of the models corner detection:

Corner Points Loose Corner Points

I even thought about making some merch out of it

Racket and Person Detection

Next, I focussed on racket and person detection:

  • I trained a YoloV11 BoundingBox model using an already annotated dataset.
  • The results are showcased in the animated GIF at the top of the page.

Pose Prediction

For pose prediction, I used the smallest standard pre-trained YoloV11-Pose model (trained on the COCO dataset). Although:

  • The smallest model does not offer the best accuracy (since the image is resized to 640x640),
  • Enlarging the model input would slow down processing too much for real-time analysis.

Thus, I opted to work with 640x640 images and only analyze the bounding boxes from the person detection model. The outcome was impressive—the pose predictions were highly accurate since the person nearly filled the frame.

Player Pose Prediction

Minimap

In the final step, I added a minimap to display the players' positions on the court:

  • I applied a homographic transformation to map the pixels of the court corners to real-world coordinates.
  • Since every tennis field has the same dimensions, this transformation accurately provides a bird’s-eye view of the game.
  • I then multiplied the players' bottom-middle points by the transformation matrix to determine their positions.
Tennis Simulation

Conclusion

This project was a fun and challenging exploration into sports analytics using AI and computer vision. By:

  • Combining models for fast yet reliable ball detection,
  • Employing advanced data augmentation for robust court detection,
  • Integrating person, racket, and pose detection,

I demonstrated how a cost-effective system can deliver comprehensive game analysis. Although real-time performance remains challenging (especially in ball detection), the project lays the groundwork for a generalized solution applicable to various sports. The future of sports analytics is exciting, and there’s always room for innovation and improvement.

I hope you enjoy this deep dive into tennis analysis as much as I enjoyed building it!