Trusted LLM Inference on the Edge with Smart Contracts

Abstract

In this era, significant transformations in industries and tool utilization are driven by AI/Large Language Models (LLMs) and advancements in Machine Learning. There’s a growing emphasis on MLOps for managing and deploying these AI models, along with a focus on distributed inferences. Concurrently, the imperative for secure on-chain computation is escalating. Our paper introduces an innovative framework that integrates blockchain technology, particularly the Cosmos SDK, to facilitate distributed AI inferences on edge devices. This system, built on WebAssembly (WASM), enables interchain communication and deployment of WASM modules executing AI inferences across multiple blockchain nodes. We critically assess this system’s safety, scalability, and model security, with a special focus on its portability and engine-model agnostic deployment on edge devices.