Is AI better at Quantifying Emotion?

May 2024 | Reading time: 7 minutes

Quantifying emotions is a difficult task for humans and a fundamentally flawed method for assessing the well-being of young people. Despite this, it’s still a prominent method of assessment within the mental health sphere.

We are exploring whether advances in artificial intelligence can supplement or replace self-quantification as a more effective tool for assessing well-being on an ongoing basis.

The problems with quantifying emotions

How happy are you right now, on a scale of 1-10? This is a common question used to gauge happiness within mental health programs. The idea is that 10 is the happiest you’ve ever been and 1 is the saddest.

There are several significant problems with this system:

1. Identifying and assigning a number to an abstract construct is difficult and inaccurate. [1]
2. The scale is relative to the subject’s experience. A 5 for one person won’t necessarily be the same as a 5 for another.
3. The scale is defined by the person’s happiest and saddest points. If the range of the scale increases (where the subject experiences a new high or low point) then all previous answers become inaccurate. This point is particularly problematic as this scale is usually used to track changes in emotion.
4. Lastly, subjects, particularly young people may be reluctant to indicate that they are feeling sad. [2]

So why is the system still in use? Despite the flaws of self-assessment, it is still effective, particularly at scale. For example, it would be entirely impractical to regularly assess an entire school via one-on-one assessments with a well-being or mental health professional.

However, we may be on the cusp of being able to use AI to provide more effective ongoing assessments at scale.

The advent of AI

In 1967 Joseph Weizenbaum began developing the ELIZA natural language computer program. Today, we would call it a chatbot. Relatively simple by modern standards, ELIZA was able to provide simple non-committal responses to text using pattern matching. The most famous of which was called DOCTOR which you can try out below:

In more recent years, artificial intelligence has advanced significantly, particularly in the development of large language models (LLMs). These sophisticated AI systems, trained on vast amounts of text data, have demonstrated remarkable capabilities in ‘understanding’ and generating human-like language. Alongside LLMs, vector databases have emerged as a necessary tool for efficient data storage and retrieval.

How do Vector databases work?

Vector databases, such as Pinecone are used as an efficient means of storing and retrieving high-dimensional data [3]. Unlike traditional databases which are typically stored in tables, vector databases represent data as vectors in a high-dimensional space. This representation allows for fast and accurate similarity searches, enabling the retrieval of relevant information based on the semantic similarity between data points.

Detecting Patterns in Writing for Mental Wellbeing Assessment

The combination of LLMs and vector databases opens up exciting possibilities for analyzing writing patterns to assess a child’s mental well-being. By training an LLM on a large corpus of text data, including writing samples from individuals who have contended with common mental health issues such as anxiety and depression, the model can learn to identify subtle patterns and linguistic cues that may indicate underlying concerns which are not specifically disclosed.

Here’s how such a system could work:

1. Data Collection: A diverse dataset of writing samples, along with corresponding mental health labels, would be collected from consenting individuals.
2. Vector Representation: Each writing sample would be converted into a high-dimensional vector representation using techniques like word embeddings or sentence embeddings.
3. Vector Database Storage: The vector representations of the writing samples would be stored in a vector database, along with their associated mental health labels.
4. Pattern Detection: An LLM would be trained on the vector database, learning to recognize patterns and correlations between writing styles and mental health conditions.
5. Analysis and Assessment: When presented with a new writing sample, the LLM would convert it into a vector representation and compare it against the patterns learned from the training data. Based on the similarity scores and learned associations, the model could provide an assessment of the young person’s potential mental well-being.
6. Effectively, the system would still be quantifying emotions.

Privacy and Safeguarding

Given the sensitive nature of the data, it would be preferable for the AI to run locally. There are several promising open models such as Llama 3 which can perform at a similar level to GPT3.5 and Claud Sonnet [4] without the need to send data away to OpenAI, Google, etc.

Running an LLM and vector database locally would still require significant computational resources, including powerful GPUs and ample storage capacity. However, with the increasing availability of high-performance hardware and optimised software frameworks, it is becoming more feasible to train and deploy these systems on local machines.

Challenges and Risks

While the potential of using LLMs and vector databases for mental well-being assessment through writing patterns is promising, there are several challenges and risks to consider:

1. Data Privacy and Consent: Collecting and storing sensitive mental health data raises concerns about privacy and informed consent. Strict protocols and security measures would need to be in place to protect individuals’ information.
2. Bias and Fairness: The training data used to develop the AI system may contain biases, leading to potentially skewed or discriminatory assessments. Ensuring fairness and mitigating biases is crucial to prevent harm.
3. Interpretability and Explainability: Understanding how the AI system arrives at its assessments is important for transparency and trust. Developing interpretable models and providing clear explanations of the decision-making process is essential.
4. Ethical Considerations: The use of AI in mental health assessment raises ethical questions about the role of technology in sensitive domains. Careful consideration must be given to the potential impact on individuals and society as a whole.
5. Safeguarding and Support: If the AI system identifies potential mental health concerns, appropriate safeguarding measures and support mechanisms must be in place to ensure the well-being of the individuals involved.

The combination of large language models and vector databases presents an intriguing opportunity for detecting patterns in writing that may indicate a young person’s mental well-being. By training AI systems on vast amounts of text data and leveraging the power of vector similarity searches, it may be possible to develop tools that can assist in early identification and intervention for mental health concerns.

However, the development and deployment of such systems must be approached with caution, considering the challenges and risks involved. Ensuring data privacy, fairness, interpretability, and ethical considerations are paramount. Running AI systems locally can provide an additional layer of security, but it requires significant computational resources and expertise.

It is likely that this system would initially be used in tandem with the existing scale for quantifying emotion.

As research in this area progresses at Eliza, it is crucial to engage in open dialogue among researchers, mental health professionals, and the public to navigate the complexities and potential implications of using AI for mental well-being assessment. By working together and prioritizing the well-being of individuals, we can harness the power of technology to support mental health while mitigating the risks and challenges that may arise.

Further reading

How teaching well-being in schools can improve educational outcomes.

The Draft RSHE Guidance – The Introduction of Age Restrictions.