Immersive Match Analysis Platform (IMAP)

What is IMAP?

The Immersive Match Analysis Platform, or IMAP, was created as my thesis project for my M.S. in Immersive Media Communications from the University of Oregon.

IMAP is a prototype video analysis program that combines real match tracking data with virtual reality (VR) to allow coaches, players, and analysts to view matches from the perspective of players on the field. I designed it to be used by coaches, players, and analysts to support training, scouting, and video analysis, as well as to serve as a supplement to team, group, and individual film sessions.

IMAP creates an accurate 3D recreation of football matches where players move exactly as they did in the real match, so users can better analyze match events by allowing them to move to find the optimal angle or select a player to watch the situation from their point of view.

IMAP Features

Since I designed IMAP to be used for video analysis, I included general features that bring the program in-line with existing video analysis programs. These include:

Clipping
Tagging
Annotating

In addition, IMAP has features that cannot be recreated by existing video analysis programs. These include:

Allowing users to immerse themselves in virtual reality
Walking and 'flying' around the field to change the viewing angle
Locking view to a player to move as they did in the match and see exactly what they were able to see

These exclusive features allow IMAP to overcome the limitation of limited viewing angles that current video analysis programs have, and can provide new and additional insights for players, coaches, and teams.

Why Use VR?

One purpose of video analysis is to help improve players off-the-field through film sessions, where they are able to watch match situations to see what was done well and what could have been better.

VR was found to be more appropriate for visual perceptual analysis in sports situations than video clips were (Vignais et al. 2015)
Watching a video in VR as opposed to in 2D correlated with the viewer experiencing a higher degree of presence and having an easier time recollecting memories (Kisker et al. 2020)
Watching a video in VR as opposed to in 2D correlated with the viewer rating the experience as more realistic and performing twice as well on a memory task (Schöne et al. 2017)

As found in Schöne et al. 2017, VR leads to a more realistic experience. This matters because coaches, coaching courses, and coaching licenses stress the importance of creating realistic scenarios for players. Realistic scenarios in trainings better prepare players for matches, and the same should be true for video analysis. Although current video analysis is very useful and being able to see the field from a tactical camera can help coaches, players, and analysts see more broad tactical concepts, the angles provided aren't realistic to what players see on the field.

In addition, using VR and recreating the match in a 3D engine allows for more user control and overcomes some limitations that come with current video analysis software.

Software Used

Python

Python was used to:

Import the multiple files provided by PFF FC
Remove the unnecessary data in PFF FC's files
Create a .csv with just the necessary data for each frame: the frame number, time elapsed since the half began, the 3D location of the ball, and the names and 2D locations of each player on the field

Unity

Unity was used to:

Import the .csv into Unity
Create the 3D recreation of the match by moving the players between their locations in each frame as time progressed
Create VR functionality, such as head rotation, hand tracking, and movement controls
Create match controls and video analysis features
- Clipping/Tagging
- Visuals
- Default camera angles
- etc.
Create the User Interface (UI)
- Create controls to pause/play the recreation, skip forward and backward in time, and scrub through the match time
- Create controls for clips, selecting players, and changing camera angles

IMAP Next Steps

Currently, IMAP is just a prototype, and work still needs to be done to turn it into a useful tool for coaches and analysts. In addition to improving the already existing features of IMAP like the clipping and tagging system, as well as the controls and UI, I plan on adding more features that will help improve functionality and impact.

Multiplayer

I plan to add multiplayer functionality, a feature that would elevate IMAP to the next level. Currently, multiple users cannot connect to the same match recreation. When multiplayer functionality is implemented, entire teams could connect and immerse themselves in the same match simultaneously. Doing so would greatly improve IMAP, allowing coaches and analysts to more easily transfer information to their players akin to how it is done in film sessions currently, just more immersive.

Desktop Support

Since VR headsets aren't always intuitive to use, I plan on creating controls for IMAP to be used on traditional computers. Since coaches and analysts are used to using computers to do things such as analyzing video and creating clips, I plan on creating a way for them to continue using a computer to do everything they're used to doing on a computer with the option to go into VR. This can be useful for clipping for film sessions, where then they and their players can use VR to view the clips for better viewing angles and a more realistic experience.

Additional Features

In addition to improving the already existing features, I plan on adding additional features. I want to focus on adding tools that can help coaches and analysts show their colleagues and players their analysis and reasoning using visuals such as dynamic convex hull graphics and lines between players (both shown right) as well as drawings or shapes placed on the field. I also plan on adding more data functionality, such as the ability to use event data to show graphics like pass maps and shot maps, as well as being able to show both team and player statistics to users.

Additional Uses

While IMAP has been designed as a tool for coaches and analysts to use for video analysis, it can also be repurposed for other uses. One such use is to allow broadcasters and pundits to show analysis of certain plays in VR to their viewers on shows like Sky Sports Match of the Day or during Champions League pre-or-post-game shows.

IMAP could also be used for fan experience by allowing fans to watch live or recorded matches in VR, being able to watch as though they were in the stadium.

Skeletal Tracking

As previously mentioned, one of the primary benefits of VR is realism. While IMAP significantly improves realism in many ways as compared to 2D video analysis, it is limited in the ways it does so by the 2D location data it uses, which captures the player's location on the field, but cannot capture and model realistic player movement. This could be solved by using skeletal tracking.

Skeletal tracking is a motion capture technique that is already being used in the sport, most notably for semi-automated offsides calls. Using this technique would allow IMAP to show exactly how each player moves, their body position, when and where they scan, and many more intricate motions. This would add much more context that is important for players to read, which would greatly elevate the analysis that is able to be done in the program, the realism, and the visuals.