Building out Sophia 2020: An Integrative Platform for Embodied Cognition

Initially posted on February 1, 2021 on Medium by Alishba Imran.

I’ve been working with Hanson Robotics and in collaboration with the Worldwide Iberian Nanotechnology Laboratory (INL) & SingularityNet to possess the unique Sophia 2020 platform.

Sophia is a humanoid that acts as a platform for evolved robotics and AI analysis, in particular for figuring out human-robot interactions.

The Sophia 2020 platform is a framework for human-fancy embodied cognition that we’ve developed to abet come Sophia’s expression, manipulation, perception, and possess. Our work with Sophia will act as a broken-down for figuring out human-robot interactions and be extinct in the future of autism treatment and scientific depression reports.

On this article, I’ll be highlighting just among the key areas that we’ve developed however in case you’d opt to head deeper, here are two sources that you may well be ready to survey at to study more:

• The pre-print paper we printed outlining this complete platform in additional element. Link.
• AAAS Poster that we offered in the future of the AAAS Annual Assembly. Link.

Sophia 2020 makes expend of expressive human-fancy robotic faces, arms, locomotion, and makes expend of ML neuro-symbolic AI dialog ensembles, NLP, and NLG instruments, inside of an initiating artistic toolset.

Sensors and Touch Perception

In human emulation robotics, contact perception & stimulation are identified as crucial for the come of contemporary human-impressed cognitive machine studying neural architectures.

Our framework makes expend of a bioinspired reverse micelle self-assembling porous polysiloxane emulsion synthetic pores and skin for entertaining facial mechanisms, growing naturalism while lowering energy requirements by 23x vs. prior provides.

We integrate a extremely flexible Frubber pores and skin for terribly-realistic facial expressions however assuredly the utilization of extinct rigid tactile sensors hinders the Frubber’s flexibility. To fix this, we devised a novel polymeric tender stress and stress sensor neatly matched with the Frubber pores and skin, with flexible polymeric rectangular microfluidic channels in polysiloxane substrate, crammed with a liquid metallic to create a resistive sensor.

As the sensor is pressed or stretched, the deformation of the microfluidic channels alters the resistance proportionally. Microfabricated silicon mildew masters produced the sensors microfluidic channels by hasten coating the polymer over the grasp mildew, and channels are injected with EGaln liquid metallic thru the reservoir-aligned vias.

Rigidity sensitivities of 0.23%/kPa in the O-40kPa differ had been measured the expend of a flat 1cm³ probe and incremental weights, agreeing with simulations and that is comparable with identical old human interplay forces. These Frubher-neatly matched sensors pave the vogue for robots with contact-pretty flexible skins.

Fingers and Arms

For the platform, we developed unique 14 DOF robotic arms with humanlike proportions, put, and pressure feedback in every joint, collection elastic actuators in all hand DOF, with relatively low-charge manufacturing.

Just among the options we’ve developed on the arms and arms are:

• PID management (finding out the sensor and computing the desired actuator output).
• Servo motors with 360 levels of put management
• URDF models in numerous circulation management frameworks (Roodle, Gazebo, Moveit).
• Pressure feedback controls, IK solvers, and PID loops, combining classic circulation management with laptop animation, wrapped in ROS API.

Greedy Retain watch over

We examined visible-servoing for grasping detection the expend of an iteration on the Generative Greedy Convolutional Neural Network (GG-CNN). This algorithm takes in-depth images of objects and predicts the pose of grasps at every pixel for diverse grasping projects and objects.

It used to be ready to possess an 83% intention shut success rate on beforehand unseen objects and 88% on family objects that are moved in the future of the intention shut strive and 81% accuracy when grasping in dynamic clutter. You may perchance well well study more about how this works in this article I wrote.

We extinct classic IK solvers which demonstrated over 98% success in projects of opt and put, establish drawing, handshaking, and games of baccarat.

Hanson AI SDK

The Hanson-AI SDK entails varied perception and management commands. Listed below are just among the key parts:
Perception:
– Face tracking, recognition, expressions, saliency, gestures, STT, SLAM, and rather a lot others
– Procedural animation responses to perception: tracking, imitation, saccades
– Gazebo, Blender, and Team spirit simulations
Robotic controls:
– ROS, IK solver, PID loops, perceptual fusion, logging & debugging instruments
– Film-quality animation, with authoring instruments for interactive performances
– Arms & arms: social gestures, rock paper scissors, establish drawing, baccarat
– Dancing, wheeled mobility, strolling (KAIST with /UNLV DRC Hubo).

To position this all together, that is an overview of the total key arrangement of Sophia:

Consciousness Experiments

Consciousness is one thing that is refined to picture in particular inside of robots. Though there’s mute quite just a few debate spherical if/how we are in a position to measure consciousness we attempted to conduct reports to measure this.

On the Sophia integrated platform, we investigated simplistic indicators of consciousness in Sophia’s blueprint, by Giulio Tononi’s Phi coefficient of the integrated files theory. This used to be created by Univ. Of Wisconsin psychiatrist and neuroscientist Giulio Tononi in 2004. It is an evolving blueprint and calculus for finding out and quantifying consciousness. You may perchance well well study this paper to make a deeper figuring out.

The tips extinct to calculate Phi comprises time-collection of Short Term Significance (STI) values akin to Atoms (nodes and hyperlinks) in OpenCog’s Attentional Focal point. To possess these computations possible, we preprocessed our files the expend of Self ample Aspect Evaluation and fed the diminished express of time collection into blueprint that applies identified solutions for approximating Phi.

All the device thru a meditation-Guiding Dialogue, we tracked spreading consideration activations as Hanson-Al and OpenCog pursued dialogue desires as Sophia guided human participants thru guided meditation sessions.

A comparability of blueprint log files with the Phi time collection indicated that Phi used to be elevated soon after the initiating of more intense verbal interplay and lower while Sophia used to be staring at her area meditate or breathe deeply, and rather a lot others.

Searching Forward

As we switch forward, our intention is to test our management blueprint (akin to manipulation commands) in real environments and options. We’ll most definitely be taking share in additional reports with humans to realize how robotic interactions influence the mood, behaviour, and actions of humans.

Alongside this, we’re engaged on growing a corporation as governance steering to Sophia’s evolution. This may perchance well well be a collaboration of interdisciplinary experts and orderly-robot followers that can approve and select key parts of Sophia’s vogue. Extra on this soon!