Evaluating Large Language Models for Converting Traditional Construction Contracts into Smart Contracts

Abstract

This study evaluates open, local large language models (LLMs) for converting construction contract clauses into smart-contract code without commercial APIs. We curate a balanced test set of 100 clauses across three high-impact categories—payment, time/delay, and termination—and assess (1) clause classification using domain encoders (LegalBERT and BERT); and (2) Solidity code generation using local instruction-tuned LLMs, followed by compilation, static security analysis, and an LLM-assisted consistency review. LegalBERT achieved a macro/micro F1 of 0.89, demonstrating reliable clause identification. For code generation, 78.0% of snippets compile successfully; however, static analysis flags critical (n = 12) and high-severity (n = 23) issues, and the mean clause-to-code consistency score is 0.674 ± 0.202, revealing frequent partial mappings of amounts, deadlines, and roles. Results suggest that a hybrid approach—domain encoders for clause detection, structured parameter extraction, and template-based code emission—offers a safer, more controllable path to contract automation in construction. All experiments were executed with free/open models, enabling full reproducibility.