Beyond Right and Wrong

A Critical Look at Social Norm Understanding in Large Language Models

Large Language Models (LLMs) have made tremendous progress in recent years, surpassing human performance in various tasks such as language translation, question-answering, and text generation. However, despite their impressive capabilities, LLMs often struggle to understand the nuances of human social behavior. This is particularly evident in their tendency to overpredict negative sanctions in response to social norm violations.

The Problem with First-Order Social Norms

Previous AI alignment efforts have focused primarily on teaching models what is socially acceptable or unacceptable. This approach is often referred to as first-order social norms, where the model learns to recognize and respond to specific social rules. For instance, a model might be trained to recognize that "stealing is wrong" and respond accordingly.

However, social intelligence depends not only on norm recognition but also on anticipating who will enforce it and how. These second-order expectations, known as metanorms, govern how people respond when social rules are broken. For example, in the case of stealing, a metanorm might dictate that the person who stole will face public shame or even imprisonment.

Evaluating Metanorm Reasoning in LLMs

To evaluate metanorm reasoning in LLMs, researchers have proposed a novel framework along two dimensions: emotional appraisal and behavioral response. The framework involves predicting how an individual will respond to a social norm violation, taking into account their emotional state and behavioral intentions.

The NormReact Dataset

To help this evaluation, researchers have released a multi-perspective dataset called NormReact. The dataset consists of 450 norm violation scenarios, hand-annotated for emotions and behavioral responses across norm violators' gender and observers' social closeness.

Current State of LLMs

Current LLMs portray a harsher social world, overpredicting negative sanctions where humans would expect inaction. This is particularly evident in scenarios where social distance increases, and alignment with human judgments deteriorates.

Implications for AI Systems

These findings suggest that AI systems in norm-sensitive domains, such as conflict mediation and policy simulation, may risk producing a distorted picture of social regulation. One that over-represents punishment and under-represents the tolerance, restraint, and relational calibration that characterize actual norm enforcement in the real world.

The Need for Improved Social Reasoning in AI

To address this issue, researchers and developers must focus on improving the social reasoning capabilities of LLMs. This requires a deeper understanding of human social behavior, including the complexities of metanorms and the nuances of emotional appraisal and behavioral response.

Future Directions

Future research should focus on developing more sophisticated models that can accurately predict human behavior in response to social norm violations. This may involve incorporating more advanced cognitive architectures, such as theory of mind or mentalizing, to enable LLMs to better understand the mental states and intentions of others.

Evaluating second-order social reasoning in LLMs is a critical step towards developing more effective and socially intelligent AI systems. By improving the social reasoning capabilities of LLMs, we can create AI systems that better understand and respond to human social behavior, leading to more accurate and nuanced predictions in norm-sensitive domains.