The Making of Sophia: Sophia’s Singing Voice

The Making of Sophia:  Sophia’s Singing Stammer

On November 21st, 2018, Sophia the Robotic of Hanson Robotics Restricted made history when she sang Christina Aguilera’s ‘Hiss One thing’ on ‘The Tonight Demonstrate’ with host Jimmy Fallon, in what is believed to be the first robotic-human duet ever performed on live tv.

The fulfillment became the end results of a protracted lasting partnership between Hanson Robotics and Cereproc, a textual bid material to speech (TTS) skills firm that creates synthetic voices with personality and personality.

Cereproc first created the speaking explain of Sophia in 2016 by coaching a machine finding out model on many hours of explain recordings by one of Hanson Robotics’ personality pattern team members. The Cereproc model became ready to capture the ordinary personality and personality traits of the recordings, giving Sophia her straight recognizable explain.

This previous one year, CereProc partnered with Hanson over again, giving Sophia the ability to teach for the first time.

Studying Advise

The goal of Hanson Robotics and CereProc became to translate the identical personality and personality of Sophia’s speaking explain actual into a singing explain.

The necessary step became for the girl who made the everyday recordings for Sophia’s explain to epic a chain of musical passages at a recording studio. CereProc then constructed a brand contemporary database for singing synthesis and knowledgeable a Deep Neural Network (DNN) to reproduce the expression, contours, and timbre of the recorded passages.

The following model became then ready to assemble a chain of musical notes blended with vocal phonemes, which manufacture the sounds predominant to create human speech. To add an extra layer of richness, they oldschool frequency and amplitude modulation to manipulate the pitch and volume of Sophia’s explain, growing a pure-sounding vibrato that brings her efficiency to existence.

Applications

This contemporary skills has the probably for many purposes within the musical industry and within the world of social robotics. Musical AI explain synthesis shall be oldschool as a musical instrument, an automatic vocal accompaniment for musical performances, or as a teaching instrument for contemporary musicians. In social robotics, singing may give a raise to robotic efficiency in a chain of areas including healthcare, elder care, and training. As an illustration, robots can teach to present a raise to the temper or stimulate the mind of sufferers or create songs to encourage bellow young college students in a creative and noteworthy come.

While Sophia may per chance per chance wish been the first humanoid robotic to teach a human-robotic duet, she absolutely acquired’t be the last!

Carolyn Ayers, Hanson Robotics