Emotional tracking

What do we mean by emotional tracking?

At first, the concept might be obtuse and strange. Can you track the emotions of people in a meaningful way? How can you accomplish this and what data will this analysis provide?

The first question is how?, modern solutions integrate a mixture of computer vision and deep learning algorithms. These are both very resource-intensive operations.

Facial tracking is one of the algorithms used. It works by dividing the input, usually a video stream, in frames. From the first frame, the position the subjects' face is established. This is accomplished by locating the position of several known facial landmarks, such as the tip of the nose, the eyebrows and the tips of the mouth. This position is passed to the next iteration to speed up recognition from frame to frame, the algorithm only needs to refine the position of the face in each frame.

Once the position of the face is determined, the subtle changes in the facial expression are analyzed. Several metrics are analyzed in tandem to determine the emotion conveyed by the subject. This is a rather complex task, given that the signals humans interpret easily can be very hard to read for a computer. Changes of state may not occur instantly and they may be too subtle for the algorithm to detect.

There are companies, such as Affectiva and Realeye that offer emotional tracking solutions as SDKs for mobile app development or even Web development.

But how do we convert the data collected by these SDKs into usable data streams that will improve, personalize and react to every customers' experience?

How to capitalize on emotional tracking

In my current agency, there is a heavy focus on experience, and how experiences transform brand perception and provoke customer conversion. Experiences that are tailored to a client are meant to have a more profound emotional impact, but there is no hard data to support these claims.

Emotional tracking can be used, however, to measure the emotional response of a customer to a given stimulus. The data collected can then be used to generate a correlation between the emotional transformation of a customer and the transformation of brand perception and conversion rates.

This sounds great, but an obstacle remains regarding data collection and protection. How can a customer be convinced to willingly submit themselves to participate? Should an incentive be provided? and, how does that incentive pollutes the reading collected?

This could be easily implemented in mobile applications that implement any core functionality that requires the usage of the phones' camera. The incentive is already provided and expected, so no contamination and from that point the experience can be personalized for each client by reading their current emotional state and readily reacting to any change.

Conclusions

Emotional tracking is a new and exciting possibility. The technical challenges are complex, but the companies that manage to implement them in their user journies will be in the bleeding edge of innovation.

It will be one of those things that once it's implemented, everyone will want to have, clients will expect it and it will only grow bigger every year, with faster, more complex implementations that will be able to even predict the emotional responses that the customer will have and provide personalized solutions based off this.