Add more Smarts to your bot:Detecting emotions from giphy posts
On msteams
In this post I'll talk about understanding user emotion using the Giphy posts embedded within Microsoft teams. This can be generalized to any image communication between you and the bot.
First Let's get the Image Content
Using Microsoft teams insert Giphy functionality we can see that it adds an attachment to the communication message between the user and the bot, this attachment is a mere link to the chosen Giphy.
Let's add the image understanding capability to our bot
We will use emotion detection service which is a part of cognitive service to detect the giphy emotion which is basically a POST request to emotion detection service with the subscription key added as a "Ocp-Apim-Subscription-Key" header
request(
{
url: 'https://westus.api.cognitive.microsoft.com/emotion/v1.0/recognize',
method: 'POST',
headers: {
'Ocp-Apim-Subscription-Key': '*Add Your Subscription key here*',
'Content-Type': 'application/json'
},
body: {
'url': 'your Giphy image URL'
},
json: true
}, function (err, response, body) {
if (err) {
callback(err, null);
}
//successful call
callback(null, getHighScoreEmotion(body));
});
the successful response will contain an object with score of each possible emotion, I've created a function that returns the highest emotion and then send a message to the user to reflect the emotion detection.
function getHighScoreEmotion(body) {
var val=0;;
var emotion;
if (body.length > 0) {
for (score in body[0].scores) {
if(body[0].scores[score]>val){
val=body[0].scores[score];
emotion=score;
}
}
return emotion;
}
return null;
}
and now how it's look like when you send your bot a giphy :)
Now our bot can understand and response to the Giphys shared by the user