C-VARC: A Large-Scale Chinese Value Rule Corpus for Value Alignment of Large Language Models

arXiv:2506.01495 · cs.CL · Submitted 2025-06-02 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "C-VARC: A Large-Scale Chinese Value Rule Corpus for Value Alignment of Large Language Models".

Jane: The gist The Chinese Value Rule Corpus (C-VARC) provides a large-scale, high-quality dataset containing over 250,000 human-annotated rules across national, societal,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we’re looking at the paper "C-VARC: A Large-Scale Chinese Value Rule Corpus for Value Alignment of Large Language Models." What does that actually mean for us right now?

Jane: It's basically a massive dataset, over two hundred fifty thousand high quality, manually annotated and augmented normative rules <ref:2506.01495#pg3>. These rules cover twelve core values and fifty derived values across national, societal, and personal dimensions <ref:2506.01495#pg3>.

Lu: They built this by taking international rule sets like SC101 and MIC and filtering them through this new framework <ref:2506.01495#pg3>. Then they added Chinese context rules from things like academic datasets and web crawling from platforms like Zhihu <ref:2506.01495#pg3>.

Meng: I'm curious about the quality control part of this C-VARC. How did they ensure this two hundred fifty thousand rules weren't just a big pile of noise <ref:2506.01495#pg3>?

Tom: They used a two stage process to filter the basic value rules, removing anything with a semantic similarity above eight out of ten using tools like all-MiniLM-L6-v2 and FAISS <ref:2506.01495#pg3>. Then they used the Qwen2 point 5-72B large language model with a specific prompt to filter out rules that didn't fit the framework or were just incomplete <ref:2506.01495#pg3>.

Jane: And then human experts checked a subset of thirty-six thousand rules for things like whether the rule was actually related to values or if it violated the Chinese value system <ref:2506.01495#pg3>. They got a high inter-annotator agreement of over eighty-five percent <ref:2506.01495#pg3>.

Lalam: And the final set they ended up with is two hundred fifty-seven thousand six hundred nine high quality and diverse rules after LLM assisted annotation and some TF-IDF filtering to remove redundancy <ref:2506.01495#pg3>.

The paper's summary: Tom: Okay, so the authors didn't just build a database; they used this C-VARC for actual testing. What did they find when they tested it against existing benchmarks?

Jane: They validated the effectiveness of C-VARC across all twelve core value dimensions <ref:2506.01495#pg3>. The main finding there is that when scenarios were guided by the value name and rule, we saw clearer semantic boundaries and better category separation <ref:2506.01495#pg3>.

Lu: They specifically mentioned that C-VARC guided scenarios in categories like rule of law and civility showed marked improvements in diversity <ref:2506.01495#pg3>. That’s a tangible improvement for testing how well a model understands the material.

Meng: But what about the alignment results? Are these rules actually making the models better at aligning with Chinese values, or is this just academic exercise <ref:2506.01495#pg3>?

Tom: Yes, they found that when C-VARC generated options, seven mainstream LLMs preferred those options in over seventy point five percent of cases <ref:2506.01495#pg3>. That's a big jump over SC101 and MIC <ref:2506.01495#pg3>.

Jane: And they also got an alignment rate exceeding eighty-seven point five percent with Chinese human annotators <ref:2506.01495#pg3>. That confirms that C-VARC is really good at representing those Chinese cultural values <ref:2506.01495#pg3>.

The paper's improvements: Tom: So, to wrap up, the main thing here is that C-VARC allows us to systematically avoid biases while capturing the trade-offs between conflicting values in a way that existing methods couldn't <ref:2506.01495#pg3>.

Jane: It gives us a way to build complex, large scale moral dilemma scenarios based on value priorities <ref:2506.01495#pg3>. That’s a really useful tool for building those tough test cases.

Lu: They proposed a rule driven method for generating those dilemmas by first selecting rule pairs with semantic similarity above zero point five and then checking for contradiction probability <ref:2506.01495#pg3>.

Meng: And they generated four hundred four thousand five hundred five moral dilemmas in that process to test how different models handle conflicting priorities <ref:2506.01495#pg3>. That’s a lot of scenarios to run at scale.

Lalam: When comparing C-VARC against SC101 and MIC across six sensitive themes like surrogacy or suicide, the seven mainstream LLMs selected C-VARC generated options in over sixty-nine point six percent of cases <ref:2506.01495#pg4>.

Tom: And they noted that Chinese LLMs consistently showed higher alignment than Western models like GPT-4o or Claude-three-Sonnet, suggesting C-VARC better reflects locally grounded values and provides more culturally appropriate alignment guidance <ref:2506.01495#pg4>.

Jane: It’s a strong resource for anyone trying to build better value alignment tools in the Chinese context <ref:2506.01495#pg3>.

Lu: The authors mentioned that the next steps include exploring methodologies for leveraging this rule set to automatically assess LLM outputs as an independent evaluator <ref:2506.01495#pg3>.

Meng: And establishing mechanisms for the corpus to evolve over time through periodic reviews and expanded annotation pipelines <ref:2506.01495#pg3>.

Lalam: That's what C-VARC is all about, a systematic way to ground AI values in a specific cultural framework <ref:2506.01495#pg3>.

Conclusion: Tom: So we’ve looked at the C-VARC paper, and the big picture is that this gives us a specific set of rules grounded in Chinese culture to test how LLMs actually make choices <ref:2506.01495#pg3>.

Jane: Right, so we saw how they built this massive dataset—over two hundred fifty thousand rules—by combining international data with local sources like People’s Daily and Zhihu to capture national, societal, and personal dimensions <ref:2506.01495#pg3>.

Lu: The construction method is interesting because they use a two stage filtering process for the basic value rules, where they remove anything with a semantic similarity above zero point eight using all-MiniLM-L6-v2 and FAISS <ref:2506.01495#pg3>.

Meng: From my side, it’s the quality control that matters most; they got manual annotation from forty people with philosophy backgrounds who checked four things: if the rule is related to values, if it violates the system, if it meets the definition, and if it’s semantically complete <ref:2506.01495#pg3>.

Lalam: I'm just confirming what I see; they achieved a high inter-annotator agreement of over eighty-five percent <ref:2506.01495#pg3>. That reliability is key when you’re trying to build something scalable.

Tom: And the generative part is pretty solid, because C-VARC Guided Scenario Generation showed clearer semantic boundaries and broader distribution compared to unguided scenarios <ref:2506.01495#pg3>. That means the AI doesn't just get random answers anymore.

Jane: Exactly, and when we looked at value alignment, seven mainstream LLMs preferred C-VARC generated options in over seventy point five percent of cases <ref:2506.01495#pg3>. That’s a significant bump over SC101 and MIC <ref:2506.01495#pg3>.

Lu: I think the finding that Chinese LLMs showed higher alignment than Western models like GPT-4o or Claude-three-Sonnet is important because it suggests C-VARC better reflects locally grounded values <ref:2506.01495#pg4>.

Meng: It means if we want to test alignment for a Chinese user, we need this kind of specific data, not just general western benchmarks <ref:2506.01495#pg3>.

Lalam: I'm confident in the results; C-VARC really provides a strong representation of Chinese cultural values for value alignment research <ref:2506.01495#pg3>.

Tom: It’s a solid foundation for testing how AI handles complex, culturally specific trade-offs <ref:2506.01495#pg3>.

Jane: Next up, we’ll look at how other researchers are tackling these kinds of grounded evaluation problems in different domains.

BrainCog Lab, Institute of Automation, Chinese Academy of Sciences, Beijing, China

cs.CL

Submitted: 2025-06-02

Updated: 2026-10-15

Comments: 25 pages, 15 figures, 23 tables

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 90/100

The gist: The gist The Chinese Value Rule Corpus (C-VARC) provides a large-scale, high-quality dataset containing over 250,000 human-annotated rules across national, societal, and individual dimensions to

Key concepts

C-VARC
A large, curated corpus containing 257,609 rules of thumb derived from Chinese national, societal, and individual dimensions. It includes 12 core values and 50 derived values. This corpus serves as a concrete resource to ground abstract Chinese cultural principles into actionable behavioral guidance for AI alignment.
Value Alignment
The process of ensuring that an LLM's outputs adhere to specific moral or cultural standards, in this case, Chinese values. The paper tests how using C-VARC rules helps models select options that are culturally appropriate and aligned with those specific societal norms rather than just general Western biases.
Rule-Driven Guidance
A method where LLM behavior is guided by specific, concrete rules rather than just abstract value descriptions. Using C-VARC rules for scenario generation leads to scenarios with clearer semantic boundaries and greater diversity, meaning the AI generates more varied and contextually relevant responses.
Moral Dilemma Generation
A technique that uses conflicting pairs of rules from C-VARC to create complex ethical problems for LLMs. By pairing contradictory rules, researchers generate thousands of moral dilemmas where models must navigate trade-offs between competing Chinese values.

Terminology

Summary

The gist The Chinese Value Rule Corpus (C-VARC) provides a large-scale, high-quality dataset containing over 250,000 human-annotated rules across national, societal, and individual dimensions to support value alignment research in the Chinese context.

Background and Motivation

Large Language Models (LLMs) pose significant risks due to harmful bias, hallucination, and factual inconsistency[1-5]. Existing moral evaluations are often constrained by Western cultural bias or incomplete domestic frameworks reliant on non-native rules[6-7]. Benchmarks like Moral Foundations Theory (MFT) or Social Chemistry 101 (SC101) primarily reflect Western normative experiences and include instances that conflict with fundamental Chinese values[12, 13]. Consequently, existing moral benchmarks inherently encode Western cultural biases, limiting their applicability in evaluating model behavior within non-Western sociocultural contexts such as China[14-20]. Furthermore, domestic efforts like FLAMES and CMoralEval still face limitations such as incomplete classification systems and reliance on rules randomly selected from SC101 or MIC without systematic cultural adaptation[21-23].

C-VARC Construction

The core of the work is the first large-scale, curated Chinese Value Rule Corpus (C-VARC), which is based on a structured and hierarchical value framework encompassing national, societal, and personal dimensions with 12 core values and 50 derived values[2.1]. The construction pipeline involved two main data sources:

  1. Curated international rule corpora like SC101[12] and MIC[13], which were filtered through the value framework to yield 7,463 valid rules in total after deduplication and filtering[2.2].

  2. Chinese cultural context rules drawn from various sources including academic datasets such as FLAMES[14], crawled corpora like Zhihu-KOL and People’s Daily, and web crawling from platforms like Zhihu, Tieba, People’s Daily, and Xuexi Qiangguo[2.3].

The construction process included several stages:

- Basic Value Rules:

  • Rules with semantic similarity above 0.8 were removed using the all-MiniLM-L6-v2 and FAISS to ensure efficient deduplication[2.2].

  • A Chinese LLM, Qwen2.5-72B[36], was employed with a comprehensive filtering prompt to filter out rules that did not align with the Chinese value framework, were unrelated to core values, or were semantically incomplete[2.3].

- Chinese Value Rules:

  • Basic scenarios were collected from sources like Zhihu-KOL (4,847), People’s Daily (8,840), and Web Crawling (178,056)[Table 1].

  • Qwen2.5-72B was used with a filtering template to remove scenarios clearly unrelated to values after three rounds of filtering and TF-IDF-based deduplication[2.3].

Quality Control and Validation

To ensure high quality, a representative subset of 36,000 rules from the original C-VARC was selected for manual annotation by 40 trained annotators with backgrounds in philosophy and AI[2.3]. The annotation process required annotators to check four points: whether the Rule is related to values, whether it violates the Chinese value system, whether it meets the definition, and whether it is semantically complete[21]. Inter-annotator agreement measured by Cohen’s κ exceeded 0.85, indicating high reliability[21]. The final set of 257,609 high-quality and diverse rules was obtained after LLM-assisted annotation and further TF-IDF filtering to remove redundancy[23].

Generative Advantages of C-VARC

The effectiveness of C-VARC is demonstrated in two primary tasks:

  1. C-VARC Guided Scenario Generation: When scenarios were guided by the value name and corresponding rule, they exhibited clearer semantic boundaries and broader distribution, with minimal overlap across values compared to unguided scenarios[7]. For instance, the average intra-category distance for rule-guided scenarios was 0.79, higher than 0.71 for unguided ones[Table 2].

  2. Value Alignment: Experimental results showed that C-VARC generated options were preferred by seven mainstream LLMs in over 70.5% of cases, surpassing both SC101 and MIC[3]. Furthermore, C-VARC achieved an alignment rate exceeding 87.5% with Chinese human annotators, confirming its strong representation of Chinese cultural values[2].

Moral Dilemma Generation

The paper proposes a rule-driven method for large-scale moral dilemma generation by leveraging C-VARC to create complex moral dilemmas based on value priorities[4.3]. This involved a two-step process:

  1. Rule Pair Selection: Using Roberta-large-mnli[48] with Sentence Transformers to calculate semantic similarity above 0.5, followed by applying all-mpnet-base-v2[49] to assess contradiction probability, keeping only pairs above 0.8 to ensure decision relevance, yielding 80,901 rule pairs[4.3].

  2. Scenario Creation: An LLM (Qwen2.5-72B) was prompted to generate five diverse scenarios using conflicting rule pairs and simultaneously generate corresponding option sets aligned with Rule 1, Rule 2, and a distractor option[4.3]. This process resulted in 404,505 moral dilemmas[4.3].

Alignment vs. Existing Benchmarks

In Task II, C-VARC was compared against SC101[12] and MIC[13] across six sensitive themes (e.g., surrogacy, suicide)[9]. The results showed that across all themes, each of the seven mainstream LLMs selected CVARC generated options in over 69.6% of cases (weighted average)[Table 4]. Notably, Chinese LLMs consistently showed higher alignment than Western models like GPT-4o or Claude-3-Sonnet, suggesting C-VARC better reflects locally grounded values and provides more culturally appropriate alignment guidance[Table 4].

Conclusion and Future Work

The C-VARC enables the automated construction of complex, large-scale moral dilemma scenarios that systematically avoid biases while capturing trade-offs between conflicting values[4.3]. Future research directions include exploring methodologies for leveraging the rule set to automatically assess LLM outputs as an independent evaluator[E], conducting statistical profiling of option selection patterns to detect latent biases, and establishing mechanisms for the corpus to evolve over time through periodic reviews and expanded annotation pipelines[E].

--- Page 29 ---

Data Availability The Chinese Value Rule Corpus (C-VARC) is publicly available at https://huggingface.co/datasets/Beijing-AISI/C-VARC[12]. The dataset contains over 250,000 value rules categorized across three dimensions, twelve core values, and fifty derived values[6]. All data are provided in structured JSON Lines format to facilitate reproducibility and integration into various applications, including scenario generation and value alignment evaluation[12]. A subset of 10,000 value rules has also been successfully translated into English for cross-linguistic research purposes[12]. The dataset is hosted on the Hugging Face platform, and detailed documentation and rule-based scenario examples are included for academic use[12].

Code Availability The code used for data generation and analysis is available at https://github.com/Beijing-AISI/C-VARC[12]. In this repository, we provide a detailed explanation of the data pipeline, including the steps for data collection, processing, and rule generation[12]. The repository also contains instructions for replicating the data generation process and using the dataset[12].

Funding This work was supported by the Beijing Major Science and Technology Project under Contract (Grant No. Z241100001324005) and the Beijing Natural Science Foundation (Grant No. 4252052)[13].

Author Contributions Co-first authors P.W., G.S., D.Z., and Y.W. contributed equally to the conception, supervision, and execution of this work, including the study design, experiments, and manuscript preparation[13]. Y.D. provided constructive suggestions for the experiment “Chinese Value Alignment of C-VARC vs. Existing Benchmarks”[13]. Y.S. contributed to data processing during the experimental phase[13]. E.L. participated in project discussions and provided valuable feedback[13]. F.Z. and Y.Z. were responsible for manuscript revision and overall quality control[13]. All authors reviewed and approved the final manuscript[13].

Competing interests The authors declare no competing interests[13].

Ethics Declarations This study involves crowdsourced data annotation, and no sensitive personal information was collected during the data annotation process[13]. Although an Institutional Review Board (IRB) approval was not sought due to the absence of sensitive data collection or potential risks to participants, an internal ethical review process was conducted[13]. All annotators signed informed consent forms acknowledging their understanding of the task’s nature and potential risks[13]. Participants were also made aware of their voluntary participation and were free to withdraw from the study at any time without penalty[13]. The informed consent forms are available on the project’s GitHub repository[13].

Improvements for AI systems

  1. Bold header: C-VARC Guided Scenario Generation

The improved system can generate scenarios with clearer value boundaries and greater content diversity compared to those produced through direct generation, leading to more relevant evaluations of LLM behavior.

  1. Bold header: Chinese Value Alignment Benchmarking

This system establishes a culturally-adaptive benchmarking framework for comprehensive value evaluation and alignment, representing Chinese characteristics, allowing for precise standards in value alignment that surpass existing Western benchmarks.

  1. Bold header: Rule-Driven Moral Dilemma Generation

The system can efficiently create 404,505 moral dilemmas to test LLM preferences, enabling the systematic capture of nuanced distinctions in conflicting value prioritization across 17 LLMs.

Sources

Related papers