Large Language Models for Morality Classification in Texts

Abstract

Moral values have an important role in shaping the human behavior, influencing how we should act or react in any situation. However, we cannot identify someone’s moral sentiment only by observing them; we have to infer from their behavior, and these inferences are often unreliable. Human beings use morally relevant language. In other words, we use words or phrases that can be linked to values and through which our conduct or way of acting can be conveyed. These words can be viewed as true or false signals, but, in any case, expressions of moral sentiment serve as or at least as indicators of the values one wishes to display. One way to compensate for the inaccuracy of this information is to express our moral values through language. Based on this observation, this work intends to illustrate how morality may be inferred from text by evaluating six large language models (LLMs) publicly available for moral classification, and comparing accuracy using different machine learning methods (ML) as baselines. We introduce a novel corpus of text in the Spanish language labeled with several moral concerns like Care, Fairness, Loyalty, Authority, and Fairness based on the Moral Foundation Theory (MFT). We collected opinions about dead penalty in México from native spanish speakers along with their responses on the Moral Foundations Questionnaire (MFQ).