Background Traditional Chinese Medicine (TCM) rheumatology presents unique challenges for AI-assisted clinical decision support, as the diagnostic process relies heavily on tacit knowledge and individualized reasoning. While Large Language Models (LLMs) have shown promise in medical applications, they remain limited by hallucination risks and inability to replicate expert TCM reasoning. Retrieval-Augmented Generation (RAG) offers a potential solution, yet its application to complex TCM dialectical reasoning remains underexplored. Methods We developed TCM-CoT-RAG, a hybrid framework combining RAG with Chain-of-Thought (CoT) prompting, grounded in 1,700 expert-curated clinical cases (1,600 for RAG retrieval; 100 for evaluation, including 50 for blinded expert review by three senior TCM rheumatologists). Deployed on Alibaba Cloud, the five system leverages state-of-the-art LLMs (DeepSeek-V3, Qwen3-235B) under a human-in-the-loop paradigm. We designed a dual-tier evaluation: (1) Objective extraction tasks (Task 1–2) quantified using F1-scores; (2) Generative tasks (Task 3–5) assessed using BERTScore. Two senior TCM rheumatologists (≥15 years clinical experience) blindly assessed model outputs, and a senior chief expert quantified consistency between model predictions and ground truth (GT). Comprehensive ablation studies (S1-S4, S-Skip) isolated the contributions of each CoT module. Results TCM-CoT-RAG substantially improved diagnostic accuracy across five LLMs. DeepSeek-V3 with full-chain CoT-RAG achieved Entity F1 of 44.89% (+16.45% over baseline) and Formula F1 of 32.13% (+8.74% over baseline), with BERTScore of 0.81 indicating strong semantic alignment with expert reasoning. Ablation confirmed that the complete CoT pipeline was essential—removing any reasoning module caused performance collapse below the zero-shot baseline. Two independent experts validated clinical utility (Cohen’s κ > 0.7). DeepSeek-V3 achieved the highest ground-truth consistency at 81.6%, and consistency metrics were quantified by the third expert holding the most senior professional title. Conclusion This proof-of-concept framework demonstrates the potential of RAG-enhanced CoT reasoning to improve diagnostic consistency in TCM, objectifying the Symptom-Diagnosis-Prescription pipeline. It is important to note that this system is designed as an AI-assisted clinical decision-support tool. All recommendations require validation by qualified TCM practitioners before clinical application.
Abstract Software requirement documents—natural-language specifications that define a system’s entities, rules, processes, and interfaces—remain the core knowledge artifact in enterprise software development. Yet they remain inaccessible to automated tooling: downstream tasks like test generation, code scaffolding, compliance checking, and AI-assisted development cannot directly process unstructured prose. We present DKSE (Domain Knowledge Structuring Engine), a tool that automatically converts requirement documents into machine-readable structured ontologies organized around six core asset types: entities, relations, rules, processes, APIs, and dictionaries. DKSE uses an LLM-guided extraction pipeline that accepts multi-format inputs (DOCX, PDF, HTML), outputs YAML-encoded ontologies with full provenance tracing back to source sections, and includes built-in quality assurance tooling for validation, version diffing, and probe generation. In a case study across four banking sub-domains, we ran DKSE on six requirement documents totaling roughly 800,000 Chinese characters. It extracted 3,439 structured assets: 215 entities, 1,227 rules, 739 relations, 182 processes, 482 dictionaries, and 594 APIs. Expert review confirmed full functional-module coverage, with 96% of a stratified sample rated fully accurate and zero hallucinated assets. We validate DKSE’s practical value through three downstream use cases: automated benchmark generation (1,214 machine-graded probes), domain-specific LLM training corpus construction, and knowledge graph ingestion for retrieval-augmented generation. DKSE is built in Rust (~8,000 lines of code) and shipped as a CLI tool with an accompanying web interface. We position this work as a proof-of-concept within a single industry (Chinese banking), not a general-purpose validation. Quantitative evaluation across additional domains, languages, and against baseline extraction methods is left for future work.