A study from Penn Medicine used computer vision and artificial intelligence to determine the association between a person's Twitter posts and state of emotion.
Profiles and images posted on the social media platform may help point if a person has symptoms of anxiety and depression. As of Q1 2019, there are 330 million monthly active users on Twitter, 68 million of which are from the United States.
Algorithms And Aesthetic Measures
The research aimed to use social media images to study how depression and anxiety are related to the content of images that people post or choose as profile pictures on social media.
The Penn research built a language prediction model based on survey-reported anxiety and depression of 28,749 Facebook users. The same metrics were used on nearly 4,000 Twitter users to interpret the features of their posts. The last 3,200 Twitter posts of the participants were screened and analyzed, particularly images and profile photos. Some 887 participants also answered a survey to determine their depression and anxiety scores.
Features such as colors and facial expressions, and various aesthetic measures such as depth of field, symmetry, and lighting were extracted and interpreted using algorithms. As a result, the study found that Twitter users with depression and anxiety are more inclined to post images with less vivid colors, and lower aesthetic values, like images in grayscale or what experts describe as the 'flat affect' or reduced expression and displays of emotion.
The research noted the tendency of users who are depressed to show only their single face rather than their photos with a group of friends or family, a trait that is regarded as an increased focus on self. Dominant image features also show grayscale and low aesthetic cohesion. The posts of anxious users also showed similar features, but less dominant compared to the posts of depressed individuals.
The content analysis applied to the images shows that users high in both traits are posting images containing texts and images of animals. On the other hand, users low in depression and anxiety post images of sports, nature, everyday things, meals, motor vehicles, and outdoor activities.
Mental Health And Social Media
There is an increased interest in studying mental health through social media and recent research have investigated the association of language and social media use with several mental illnesses, including stress, depression, and suicidality.
"It is challenging to transform pixels that form the images to interpretable features, but with the advances in computer vision algorithms, we are now attempting to uncover another dimension of the condition as it manifests online," said Sharath Guntuku, Ph.D., a research scientist with Penn Medicine's Center for Digital Health and the study's lead author.
According to the researchers, this technique using algorithms could also be applied to Instagram and text messaging. The experts are also looking at several other conditions ranging from loneliness to ADHD.
The research will be presented at the International AAAI Conference on Web and Social Media on June 11 to 14 in Munich, Germany.
