Teaching machine’s perception, reasoning, and language has been a recent trend in the tech industry. But teaching AI to be creative and generate aesthetic attributes is still a challenge because of how subjective these areas are.
Facebook AI releases a system that analyzes musical tracks from nearly any genre and, just a moment later, gives us some synchronized dance moves.
Diving into the technical aspect of how the system works, the users have to upload a song. Then the system analyses the music and identifies the similarities in the piece (crescendo, a drumbeat, etc.). Lastly, the system searches for similar dance routines matching to the music pattern throughout the song.
Next, the machine looks into the dataset of ‘Dancing Agents’, which comprises different actions and states. Then it turns these movements and steps into a particular sequence that is synchronized with the music’s tune.
The system’s algorithm represents the track’s sound waves in a matrix defined as heatmaps with ascending and descending audio frequencies clusters. After the complete processing of input music, exceptionally original and creative moves are returned in humanoid sticks, deforming geometric shapes, and pulsating discs.
The system is currently only a research project, but it can provide inspiration and create insights for the choreographers and dancers.