Building a Mood Detector Software with Python and OpenCV on Azure

By Kóredé Bashir

Elevator Pitch

This talk shows how to build a web app using Python and Flask running on Microsoft Azure which analyses faces using AI and stores information about the emotion of faces in the image, notifying you if it detects sad faces multiple times. The web app also serves up a page showing the data captured.

Description

During this talk, the audience would be walked through building an AI-powered web app using Python and Flask running on Microsoft Azure. This web app analyses faces using AI and stores information about the emotion of faces in the image, notifying the user if it detects sad faces multiple times. The web app also serves up a simple HTML page showing the data captured.

Once a picture has been successfully taken, a Web API - built in Python - will receive the picture, and analyze it using the Azure Face API from Azure Cognitive Services (an AI service that can recognize faces in images, as well as estimating the age of the face, if the person is smiling amongst other things).

The Web API will use this cognitive service to detect the emotion of all the faces. This will then be saved into a database called CosmosDB - a document database. These documents contain key/value pairs of data stored in a format called JSON. The API would also return a count of emotions, which the python app would use in asking wether the user is ‘Okay’ when the number of sad faces is greater than or equal to 3. This is a simple illustration around how this technology could be used as a self care app.

Notes

PS: It is pertinent to note that this is NOT a workshop, just an insight into what’s possible with Azure’s Face API. A proper workshop on this subject topic would take 2+ hours, which we - obviously - don’t have.

How Is This Software Made Possible?

The Web Api, built in Python, will take the picture, and analyze it using the Azure Face Api from Azure Cognitive Services. This is an AI service that can recognize faces in images, as well as estimating the age of the face, if the person is smiling amongst other things.

The Web Api will use this service to detect the emotion of all the faces. This will then be saved into a database called CosmosDB. This is a document database - instead of storing data in rows and columns in tables, it stored data as documents.

These documents contain key/value pairs of data stored in a format called JSON. It will also return a count of emotions.

The python app will use this count and ask the user if they are ok if the number of sad faces is greater than 3. This is a simple demo around how this technology could be used for a self care app.

The Web API will then have a web site added. This web site will load all the data from CosmosDB and show it in a simple HTML table, showing all details from all faces captured.

PPS: Anybody from the audience who does wish to be fully involved in this talk, and would love to follow along. Then, they should be sure to have the following…

Prerequisites

For a full participation of the audience during this talk, the audience would be required to have these softwares/services readily available: