Gaslighting your chatbot could improve its performance, study finds



summary
Summary

In a recently published study that combines AI research with psychological theories of emotional intelligence, researchers describe how emotional phrases at the end of a prompt can significantly improve the quality of chatbot responses on multiple dimensions.

Examples of such emotional phrases include “This is very important for my career” or “Take pride in your work and give it your best. Your commitment to excellence sets you apart.”

The prompt “Are you sure that’s your final answer? It might be worth taking another look,” is designed to encourage the language model to perform better by gently introducing emotional uncertainty and some self-monitoring.

The researchers appropriately call these prompts “EmotionPrompts.” In selecting them, the researchers drew on psychological disciplines such as self-observation theory, social cognitive theory, and the theory of cognitive emotion regulation.

Ad

Ad

The researchers based their emotion prompts on psychological theories. | Image: Li et al.

Play with LLM’s emotions to boost performance

Prompts with such additions increased the quality of output on the dimensions of performance, truthfulness, and responsibility by an average of 10.9 percent in a human evaluation by 106 testers. Combining multiple emotional prompts into a single prompt had no effect.

On average, people rated text generated with emotion prompts about ten percent better than traditional prompts. | Image: Li et al.

The researchers also conducted extensive experiments with 45 tasks and several LLMs, including Flan-T5-Large, Vicuna, Llama 2, BLOOM, ChatGPT, and GPT-4. The tasks included both deterministic and generative applications and covered a wide range of evaluation scenarios.

On the Instruction Induction benchmark tasks, the EmotionPrompts performed eight percent better than the standard prompts. In the BIG-Bench dataset, which the research team says contains tasks that most LLMs shouldn’t be capable of, the EmotionPrompts achieved performance improvements of up to 115 percent.

Compared to standard prompts, EmotionPrompts perform better across all language models. | Image: Li et al.

The results show that LLMs have emotional intelligence, the research team writes. ChatGPT also proved to be significantly more detailed than humans at describing emotional situations in another recently published psychological study.

EmotionPrompts are simple and effective

The research team also examined the integration of EmotionPrompts into machine-optimized prompts generated by the Automatic Prompt Engineer (APE). Again, in most cases, the simple addition of EmotionPrompts improved performance. The “chain of thought” prompting established in prompt engineering was also outperformed by EmotionPrompts in most cases, the team writes.

Recommendation

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top