Vision-language-model (VLM)-guided reinforcement learning (RL) has recently attracted significant attention for it, replacing brittle hand-crafted rewards with semantically grounded signals; however, deploying such simulation-trained policies on real vehicles remains a fundamental challenge, because they rely on simulator-native observations and simulator-coupled action semantics with no counterpart on physical hardware. We identify a general principle: the simulation-to-reality gap decomposes into two largely orthogonal axes, a sensing-and-dynamics domain gap and a task-and-geometry gap, the former closable without real-world policy training by re-projecting real perception and control onto the policy's training manifold. We formalize this as a transfer guarantee that bounds the deployment gap by three independently controllable error terms, and instantiate it as Sim2Real-AD, which combines a Geometric Observation Bridge, a Physics-Aware Action Mapping, a Two-Phase Progressive Training curriculum, and a Real-time Deployment Pipeline. As a proof of concept, a CARLA-trained VLM-guided RL policy is transferred zero-shot to a full-scale battery-electric Ford E-Transit van in Madison, WI, USA, and drives across car-following, obstacle-avoidance, and stop-sign scenarios using no real-world training data. To our knowledge, this is among the first zero-shot closed-loop deployments of a CARLA-trained VLM-guided RL policy on a full-scale real vehicle, and the decomposition offers a principled, broadly applicable route for moving simulation-trained, foundation-model-guided policies into the physical world, supporting energy-efficient intelligent driving on electrified transportation platforms. The demo video, code, and model checkpoint are available at: https://zilin-huang.github.io/Sim2Real-AD-website/.
When individual robots have limited sensing capabilities or insufficient fault tolerance, it becomes necessary for multiple robots to form teams during exploration, thereby increasing the collective observation range and reliability. Traditionally, swarm formation has often been managed by a central controller; however, from the perspectives of robustness and flexibility, it is preferable for the swarm to operate autonomously even in the absence of centralized control. In addition, the determination of exploration targets for each team is crucial for efficient exploration in such multi-team exploration scenarios. This study therefore proposes an exploration method that combines (1) an algorithm for self-organization, enabling the autonomous and dynamic formation of multiple teams, and (2) an algorithm that allows each team to autonomously determine its next exploration target (destination). In particular, for (2), this study explores a novel strategy based on large language models (LLMs), while classical frontier-based methods and deep reinforcement learning approaches have been widely studied. The effectiveness of the proposed method was validated through simulations involving tens to hundreds of robots.
In this work, we present a novel machine learning framework called Self-Directed Task Identification (SDTI), which enables models to autonomously identify the correct target variable for each dataset in a zero-shot setting without pre-training. SDTI is a minimal, interpretable framework demonstrating the feasibility of repurposing core machine learning concepts for a novel task structure. To our knowledge, no existing architectures have demonstrated this ability. Traditional approaches lack this capability, leaving data annotation as a time-consuming process that relies heavily on human effort. Using only standard neural network components, we show that SDTI can be achieved through appropriate problem formulation and architectural design. We evaluate the proposed framework on a range of benchmark tasks and demonstrate its effectiveness in reliably identifying the ground truth out of a set of potential target variables. SDTI outperformed baseline architectures by 14% in F1 score on synthetic task identification benchmarks. These proof-of-concept experiments highlight the future potential of SDTI to reduce dependence on manual annotation and to enhance the scalability of autonomous learning systems in real-world applications.
In this paper, I prove that sublinear regret across the environment Class C requires six functional properties, that these properties are mutually independent, and that they compose into a directed informational chain closing back on itself — a six-link cycle whose final link is grounded in an explicit Doob martingale construction over cycles of play. All six properties are defined functionally — as conditions on the distributions a decision-maker induces over actions and canonical summaries — so the results are invariant under implementation and apply to any decision-making system that can be modelled within the class: a person, an institution, or a machine. Every theorem in this paper, without exception, is checked line by line in the Lean 4 proof assistant against Mathlib: the formalization (~12,700 lines) contains zero `sorry`, zero custom axioms, and zero opaque definitions. Class C is the union of all POMDPs satisfying at least one of six structural properties covering the fundamental qualitative dimensions of adaptive hardness: reward ambiguity (P1), absorbing traps (P2), local optima (P3), deterministic optimality (P4), constrained feasibility (P5), and nonstationarity (P6). * Part I (Necessity). I define six primitives X1–X6 as purely functional properties of decision rules: Objective Tracking, Cross-Context Safety Transfer, Global Attractor Exploration, Policy Simplification, Feasibility Projection, and Feedback Adaptation. For each, I construct an explicit environment in C and prove an unconditional Ω(T) regret lower bound for any decision-maker lacking that primitive.* Part II (Independence). For every ordered pair (i,j) with i≠j, I exhibit an explicit decision rule possessing Xj but lacking Xi that suffers Ω(T) regret on the matching environment. All thirty directed-pair results are shown to follow from one master theorem, verified on a single compound environment with full non-interference analysis.* Part III (Sequential Dependence). Necessity is domain-invariant — a structural failure is a structural failure no matter what "success" means to the decision-maker — which is why Parts I and II hold unconditionally. Sufficiency is not: what counts as success is supplied by the domain, not by the theorem, so a single closed-form sufficiency result covering every domain at once would have to either fix one arbitrary notion of success and stop being general, or say nothing of substance. Part III proves exactly what generalizes. I prove six Information Enhancement Theorems establishing that the six primitives compose into a directed information chain: possessing Xi strictly increases the mutual information available toward any goal variable at Xi+1's task. Each of the six links is established outright — a forward theorem, a reverse theorem, and a non-reversibility result — with the exact point where a domain's own definition of success enters the chain named explicitly, as an Implementation Obligation, rather than assumed away. The closing link, X6→X1, is grounded in an actual Doob martingale construction: given that the cycle-indexed posterior is a martingale, it converges almost surely to the truth across cycles — the precise sense in which the chain accumulates rather than resets. To this paper's knowledge, no prior formalization unifies this many independently-proven-necessary structural properties into a single machine-checked class with proven mutual independence across all of them. All mathematical work is provided in full transparency and independent verification is highly encouraged: the complete Lean formalization, with a passing build and every theorem cross-referenced to its exact identifier, is at github.com/M-Ismail-ZA/IsmailsPrimitives. For any feedback or collaboration, please contact me via the email address listed on the paper. Updated: 3 July 2026 (V6.1).
Abstract: The Financial Technology (Fintech) sector is changing at a swift pace, as artificial intelligence (AI) is extending its influence. Greater complexity and global linkages are going to demand from fintech the power to rethink the integrity of its cybersecurity mechanisms and fraud tactics that have gotten intense up to a growing extent. The paper argues for the necessity of an "Algorithmic Fortress," an AI-driven cybernetic system incorporating all possible technologies targeted at securing digital financial networks against cyber-attacks and acts of financial fraud. The article delves into AI/ML, deep learning, anomaly detection through generative adversarial networks, etc., scope to predict battle, detect and fight problems. It does address adverse effects of AI risk, threatened system independence through synthetic identity fraud, application of AI for fraud detection in decentralized finance, DeFi, as well as the threat-hunting models that need to become autonomous. Supervised learning, unsupervised learning, and reinforcement learning are examination methodologies that are being applied in taking high recourse to the preservation of cybersecurity amongst their uncertainties. Our analysis will involve different experimentations of Python-based simulated attack scenarios to compare the two forms of cybersecurity. Also brought in are SmartArt visual representations revealed in multi-tier defensive architectures, combined with some strategic recommendations destined to protect future-facing fintech infrastructures from doing illicit deeds of algorithms. This study sketches possible solutions for securing the future-ready, trustworthy, and resilient fintech ecosystems once assisted by AI-enhanced, digital fortresses.
The emergence of AI agents and agentic systems represents a significant milestone in artificial intelligence, enabling autonomous systems to operate, learn, and collaborate in complex environments with minimal human intervention. This paper, drawing on multi-expert perspectives, examines the potential of AI agents and agentic systems to reshape industries by decentralizing decision-making, redefining organizational structures, and enhancing cross-functional collaboration. Specific applications include healthcare systems capable of creating adaptive treatment plans, supply chain agents that predict and address disruptions in real-time, and business process automation that reallocates tasks from humans to AI, improving efficiency and innovation. However, the integration of these systems raises critical challenges, including issues of attribution and shared accountability in decision-making, compatibility with legacy systems, and addressing biases in AI-driven processes. The paper concludes that while agentic systems hold immense promise, robust governance frameworks, cross-industry collaboration, and interdisciplinary research into ethical design are essential. Future research should explore adaptive workforce reskilling strategies, transparent accountability mechanisms, and energy-efficient deployment models to ensure ethical and scalable implementation.
The rapid adoption of artificial intelligence (AI) systems, such as predictive AI, generative AI, and explainable AI, is in contrast to the slower development and uptake of robotic AI systems. Dynamic environments, sensory processing, mechanical movements, power management, and safety are inherent complexities of robotic intelligence capabilities that can be addressed using novel AI approaches. The current AI landscape is dominated by machine learning techniques, specifically deep learning algorithms, that have been effective in addressing some of these challenges. However, these algorithms are subject to computationally complex processing and operational needs such as high data dependency. In this paper, we propose a computation-efficient and data-efficient framework for robotic motion intelligence (RMI) based on vector symbolic architectures (VSAs) and blockchain-based smart contracts. The capabilities of VSAs are leveraged for computationally efficient learning and noise suppression during perception, motion, movement, and decision-making tasks. As a distributed ledger technology, smart contracts address data dependency through a decentralized, distributed, and secure transactions ledger that satisfies contractual conditions. An empirical evaluation of the framework confirms its value and contribution towards addressing the practical challenges of robotic motion intelligence by significantly reducing the learnable parameters by 10 times while preserving sufficient accuracy compared to existing deep learning solutions.
ABSTRACT This paper deals with the two‐player zero‐sum game problem, which is a bounded ‐gain robust control problem. Finding an analytical solution to the complex Hamilton‐Jacobi‐Issacs (HJI) equation is a challenging task. Hence, a novel Q‐learning algorithm for unknown continuous‐time (CT) affine‐in‐inputs nonlinear systems is proposed for generating an approximate solution to the HJI equation, which is valid in a local domain due to the use of a local approximator, that is, a Neural Network (NN) structure. The approach is model‐free and does not require the knowledge of system drift dynamics, and input and disturbance gains. The algorithm learns online from measurements of state variables in real time. To generate the local approximate solution of the HJI equation for the two‐player zero‐sum game problem for nonlinear systems, the proposed non‐iterative algorithm requires only a single critic NN instead of the commonly used triple NN approximator structure. A persistence of excitation condition is required to guarantee Uniformly Ultimately Boundedness (UUB) and convergence to the optimal solution. The effectiveness of the proposed Q‐learning approach for the two‐player zero‐sum game problem is demonstrated via simulations of a linear F‐16 aircraft plant and a highly complex nonlinear system. Proof of closed‐loop system stability is provided using Lyapunov Analysis, and convergence of the approximate solution to the true saddle‐point solution is guaranteed in a UUB‐sense.
Optimistic Rollup is a blockchain scalability solution. However, it assumes that transactions are correct unless challenged by a challenger, which leads to the concept of a challenge period. If there is no challenge during this time, the user needs to wait until the end of the challenge period, which is very inconvenient for the user. This is because challengers and task publishers are self-interested and aim to maximize their own benefits, leading to internal incentive issues within Optimistic Rollup. Although zero-knowledge proofs can effectively address this issue, ZK rollups face challenges such as high computational complexity and the need for transaction-specific proofs. Optimistic rollups have low computational requirements and simple design, making the design of an effective incentive mechanism a good choice. This paper proposes a reward-based incentive model to encourage rational workers to actively verify transactions based on game theory. The rewards in this incentive model are related to a time factor, where earlier verification yields greater rewards, motivating verifiers to verify transactions more quickly. We simulate under various proportions of false declarations and the results show that our method could incentivize the workers effectively to exert their best efforts.
Noor Al-Gburi, András Földvári, Kristóf Marussy, Oszkár Semeráth · 5 authors
Cross-organizational, blockchain-based distributed ledger networks in general, and those based on Hyperledger Fabric in particular, have an architecture which can be adapted to specific application requirements. However, network design can be a particularly challenging task, as the connection between architectural and deployment decisions and extra-functional properties can be subtle and the requirements may contradict each other, requiring trade-offs.
Abhimanyu Nag, Samrat Gupta, Sudipan Sinha, Arka Datta
Decentralized Finance (DeFi) governance models have become increasingly complex due to the involvement of numerous independent agents, each with their own incentives and strategies. To effectively analyze these systems, we propose using Multi Agent Influence Diagrams (MAIDs) as a powerful tool for modeling and studying the strategic interactions within DeFi governance. MAIDs allow for a comprehensive representation of the decision-making processes of various agents, capturing the influence of their actions on one another and on the overall governance outcomes. In this paper, we study a simple governance game that approximates real governance protocols and compute the Nash equilibria using MAIDs. We further outline the structure of a MAID in MakerDAO.
The emergence of blockchain and smart contracts has revolutionized various industries by enabling automated code execution. However, the development of smart contracts, rooted in programming languages, inherits common challenges in traditional software development, notably concerning efficiency, reliability, and security. The application of deep learning techniques holds promise for addressing these challenges. However, a critical issue in applying deep learning to smart contracts is the lack of extensive datasets, as smart contracts have only recently emerged compared to traditional programming languages. To address this problem, we propose a novel approach called Compiler-Guided Generation Networks (CGGNet) for augmenting smart contract datasets. In contrast to existing methods, CGGNet utilizes a compiler as an oracle in generative networks, ensuring the augmentation of a valid smart contract. By incorporating the Montecarlo tree search, CGGNet significantly enhances the diversity and validity of the generated contracts, overcoming the limitations posed by GAN-based models in code augmentations. To the best of our knowledge, this is the first study on code augmentation targeting smart contracts. Our experiments show that millions of unique and valid smart contracts can be augmented from thousands of valid smart contracts, and augmented datasets can mitigate the underfitting problem in practical deep learning applications.
Blockchain is a distributed ledger that was introduced to decentralize monetary systems. However, with time, the applications of blockchain in different realms have been identified. Swarm robotics is a field that combines swarm intelligence and robotics to solve real-world problems that cannot be solved by monolithic robots. Collective decision-making is one of the major behaviors implemented by swarm robotics. This study analyzes existing literature on the applications of blockchain in the collective decision-making scenarios in swarm robotics. Consequently, this study introduces a novel taxonomy to study the different applications effectively. The taxonomy categorizes existing literature into (i) application of blockchain in other areas of swarm robotics, (ii) application of blockchain in continuous collective decision-making scenarios, (iii) application of blockchain in discrete collective decision-making scenarios, (iv) application of blockchain in other discrete collective decision-making scenarios, and (v) application of blockchain in the collective perception scenario. Finally, the limitations of existing work such as excessive resource consumption and violation of swarm robotics principles are discussed.
The paper introduces an advanced Decentralized Energy Marketplace (DEM) integrating blockchain technology and artificial intelligence to manage energy exchanges among smart homes with energy storage systems. The proposed framework uses Non-Fungible Tokens (NFTs) to represent unique energy profiles in a transparent and secure trading environment. Leveraging Federated Deep Reinforcement Learning (FDRL), the system promotes collaborative and adaptive energy management strategies, maintaining user privacy. A notable innovation is the use of smart contracts, ensuring high efficiency and integrity in energy transactions. Extensive evaluations demonstrate the system's scalability and the effectiveness of the FDRL method in optimizing energy distribution. This research significantly contributes to developing sophisticated decentralized smart grid infrastructures. Our approach broadens potential blockchain and AI applications in sustainable energy systems and addresses incentive alignment and transparency challenges in traditional energy trading mechanisms. The implementation of this paper is publicly accessible at \url{https://github.com/RasoulNik/DEM}.
Most blockchain systems currently adopt resource-consuming protocols to achieve consensus between miners; for example, the Proof-of-Work (PoW) and Practical Byzantine Fault Tolerant (PBFT) schemes, which have a high consumption of computing/communication resources and usually require reliable communications with bounded delay. However, these protocols may be unsuitable for Internet of Things (IoT) networks because the IoT devices are usually lightweight, battery-operated, and deployed in an unreliable wireless environment. Therefore, this paper studies an efficient consensus protocol for blockchain in IoT networks via reinforcement learning. Specifically, the consensus protocol in this work is designed on the basis of the Proof-of-Communication (PoC) scheme directly in a single-hop wireless network with unreliable communications. A distributed MultiAgent Reinforcement Learning (MARL) algorithm is proposed to improve the efficiency and fairness of consensus for miners in the blockchain system. In this algorithm, each agent uses a matrix to depict the efficiency and fairness of the recent consensus and tunes its actions and rewards carefully in an actor-critic framework to seek effective performance. Empirical results from the simulation show that the fairness of consensus in the proposed algorithm is guaranteed, and the efficiency nearly reaches a centralized optimal solution.
In this paper, we formulate inverse reinforcement learning (IRL) as an expert-learner interaction whereby the optimal performance intent of an expert or target agent is unknown to a learner agent. The learner observes the states and controls of the expert and hence seeks to reconstruct the expert's cost function intent and thus mimics the expert's optimal response. Next, we add non-cooperative disturbances that seek to disrupt the learning and stability of the learner agent. This leads to the formulation of a new interaction we call zero-sum game IRL. We develop a framework to solve the zero-sum game IRL problem that is a modified extension of RL policy iteration (PI) to allow unknown expert performance intentions to be computed and non-cooperative disturbances to be rejected. The framework has two parts: a value function and control action update based on an extension of PI, and a cost function update based on standard inverse optimal control. Then, we eventually develop an off-policy IRL algorithm that does not require knowledge of the expert and learner agent dynamics and performs single-loop learning. Rigorous proofs and analyses are given. Finally, simulation experiments are presented to show the effectiveness of the new approach.
Blockchain is an essentially distributed database recording all transactions or digital events among participating parties. Each transaction in the records is approved and verified by consensus of the participants in the system that requires solving a hard mathematical puzzle, which is known as proof-of-work. To make the approved records immutable, the mathematical puzzle is not trivial to solve and therefore consumes substantial computing resources. However, it is energy-wasteful to have many computational nodes installed in the blockchain competing to approve the records by just solving a meaningless puzzle. Here, we pose proof-of-work as a reinforcement-learning problem by modeling the blockchain growing as a Markov decision process, in which a learning agent makes an optimal decision over the environment's state, whereas a new block is added and verified. Specifically, we design the block verification and consensus mechanism as a deep reinforcement-learning iteration process. As a result, our method utilizes the determination of state transition and the randomness of action selection of a Markov decision process, as well as the computational complexity of a deep neural network, collectively to make the blocks not easy to recompute and to preserve the order of transactions, while the blockchain nodes are exploited to train the same deep neural network with different data samples (state-action pairs) in parallel, allowing the model to experience multiple episodes across computing nodes but at one time. Our method is used to design the next generation of public blockchain networks, which has the potential not only to spare computational resources for industrial applications but also to encourage data sharing and AI model design for common problems.
Blockchains can be either permissioned or permissionless. Existing blockchain consensus algorithms used to create permissioned blockchains to not allow new validators to be dynamically added to the blockchain during runtime. This can be a problem when blockchain is used in swarm robotics, where redundancy is required. Besides, popular algorithms such as proof of work (PoW) that offer redundancy consume a lot of resources, making them unsuitable to run on the powerless robots used in swarm robotics. In addition, such algorithms are also vulnerable to threats such as the 51% attack. This research proposes a novel proof-of-identity (PoI) blockchain consensus algorithm to create a dynamically permissioned blockchain to be used in swarm robotics. This algorithm uses a swarm controller to generate a private-public key pair that is used to authorise validators. Tests carried out showed that the PoI algorithm is immune to the 51% attack threat and can run on low-powered devices such as Raspberry Pi 3. Moreover, the consensus time was also shown to be shorter in comparison to PoW.
The incorporation of robots in the social fabric of our society has taken giant leaps, enabled by advances in artificial intelligence and big data. As these robots become increasingly adept at parsing through enormous datasets and making decisions where humans fall short, a significant challenge lies in the analysis of robot behavior. Capturing interactions between robots, humans and IoT devices in traditional structures such as graphs poses challenges in the storage and analysis of large data sets in dense graphs generated by frequent activities. This paper proposes a framework that uses the blockchain for the storage of robotic interactions, and the use of sheaf theory for analysis of these interactions. Applications of our framework for social robots and swarm robots incorporating imperfect information and irrationality on the blockchain sheaf are proposed. This work shows the application of such a framework for various blockchain applications on the spectrum of human-robot interaction, and identifies key challenges that arise as a result of using the blockchain for robotic applications.
Jianbo Du, Wenjie Cheng, Guangyue Lu, Haotong Cao · 7 authors
When using blockchain in mobile systems, computation intensive mining tasks pose great challenges to the processing capabilities of mobile miner equipment. Mobile edge computing (MEC) is an effective solution to alleviating the problem via task offloading. In the mining process, miners compete for rewards through puzzle solving, where only the miner that first completes the process will be rewarded. Thus, miners may wish to pay higher price and use more communication resources in task offloading and more computation resources in task processing for latency reduction. However, there are risks for the miners not profiting from consuming more resources or paying a higher price, so miners are rational in blockchain systems. In order to maximize the rational total profit of all miners, we use an asynchronous advantage actor-critic (A3C) deep reinforcement learning algorithm to obtain the resource pricing and allocation, considering the stochastic properties of wireless channels, and the prospect theory is employed to strike a good balance between risks and rewards. Numerical results show that our proposed A3C based joint optimization algorithm converges fast and outperforms the baseline algorithms in terms of the total reward.