The Topological Governor: A Deterministic Solution to Catastrophic Forgetting Full Summary The Problem Catastrophic forgetting is a fundamental limitation in artificial intelligence where neural networks overwrite previously learned knowledge when trained on new sequential tasks. Since its formal characterization by McCloskey and Cohen in 1989, this has hindered the development of lifelong learning systems in robotics, autonomous systems, and personalized assistants. The Solution: Topological Governor The paper presents a deterministic mechanism that definitively solves catastrophic forgetting through mathematical invariance, unlike probabilistic approaches (EWC, replay-based methods, parameter isolation) that provide only statistical guarantees with growing memory requirements. Key Technical Contributions 1. Mathematical Foundation: Arithmetic Spectral Theory Leverages the Sieve of Eratosthenes (a deterministic algorithm proven for over two millennia) to select the first six prime numbers: [2, 3, 5, 7, 11, 13] The Safety Constant ($\Lambda = 0.9785142874$) is derived from Euler's attenuation product and provides mathematical proof of protection: $\Lambda = 1 - \prod_{p \in \{2,3,5,7,11,13\}} (1 - p^{-0.5})$ Never hardcoded; recomputed at initialization for auditability 2. Three-Step Mechanism Step 1: Snapshot Capture (Memory Consolidation) When the first task reaches 100% accuracy, the Governor captures the state of prime-indexed embedding rows as an immutable reference frame Step 2: Gradient Enforcement (Memory Protection) During backpropagation on subsequent tasks, the Governor blocks gradient updates to anchored rows All gradients at prime indices are set to zero Step 3: Anchor Restoration (Memory Integration) After optimizer steps, performs final verification and restoration of anchored positions as a fail-safe against numerical drift 3. Implementation Architecture Core class: TopologicalGovernor with O(1) memory complexity Multi-layer support: Can protect embedding and attention layers simultaneously Hybrid architecture support: Works on SSM + Transformer hybrids (StripedHyena) Universal: Works across vision transformers, language models, and genomic models Experimental Results 5-Task Sequential Learning (Synthetic) Metric Result Tasks Learned 5 Average Accuracy 99.96% Average Forgetting 0.00% Anchor Preservation 6/6 â Production Models on Hugging Face 1. Vision Domain: TOPO-Gemma-4-E4B-Vision-13Tasks Architecture: Gemma-4-E4B Vision Transformer (4B parameters) 13 visual classification tasks (STL-10) 100% accuracy on all tasks, 0% forgetting 2. Language Domain: Topological-AI-Muse-Glimmer-30B-Final Architecture: Muse-Glimmer Multimodal (30B parameters) AG News Classification 96.48% accuracy, 6.21% forgetting 3. Genomic Domain: Evo2-TOPO-Governed Architecture: Evo2-7B (StripedHyena + Transformer, 7B parameters) 13 genomic prediction tasks 100% final task accuracy, 1.32% global forgetting, 5/5 successful runs Complexity Analysis Memory Complexity: O(1) Method Memory Usage Scaling EWC 4.4 GB Grows with tasks Replay-based Variable Grows with tasks Topological Governor 184 KB Constant (O(1)) Minimal storage: 6 anchors Ă embedding_dim (32) Ă 4 bytes = < 1 KB for anchor storage Computational Overhead Operation Time Gradient Enforcement 0.11 ms/step Anchor Restoration 0.08 ms/step Snapshot Capture 0.04 ms (once) Total Overhead ~0.23 ms/step Represents a 75.7Ă improvement over Google's Full HOPE architecture Theoretical Implications Paradigm Shift: Probabilistic â Deterministic Aspect Probabilistic Methods Topological Governor Protection Statistical Deterministic Guarantee Probabilistic Mathematical Auditability Limited Full (SHA-256) Reproducibility Variable 100% Trustworthiness Moderate High Cognitive Analogy Hippocampus: Forms new memories (Task 2 learning) Cortex: Consolidates stable knowledge (Prime anchors) Result: Continued learning without forgetting Key Achievements Summary Metric Result Tasks Learned 5 Average Accuracy 99.96% Average Forgetting 0.00% Anchor Preservation 6/6 â Topological Integrity PASSED â Safety Constant 0.9785142874 Broader Implications Theoretical: Shifts AI from probabilistic regularization to deterministic cognitive engineering Practical: Enables deployment of lifelong learning systems in real-world applications Economic: Reduces computational costs through O(1) memory and 75.7Ă performance improvement Ethical: Provides auditability and mathematical guarantees for safety-critical applications Availability GitHub (Full Code) : https://github.com/frank-morales2020/AST/blob/main/TG_DEMO.ipynb Hugging Face Models: TOPO-Gemma-4-E4B-Vision-13Tasks Topological-AI-Muse-Glimmer-30B-Final Evo2-TOPO-Governed Final Conclusion The Topological Governor definitively solves Catastrophic Forgetting with mathematical guarantees, achieving 0.00% forgetting across sequential tasks while maintaining O(1) memory complexity and demonstrating universal applicability across vision, language, and genomic domains. This represents a fundamental breakthrough in continual learning and a paradigm shift from probabilistic to deterministic approaches in artificial intelligence.
Swarm intelligence (SI) is a collective behavior exhibited by groups of simple agents, such as ants, bees, and birds, which can achieve complex tasks that would be difficult or impossible for a single individual (Dorigo et al., 2019). The collective behavior of these organisms is characterized by decentralized decision-making, self-organization, adaptive responses to environmental changes, and emergent properties that are not present in individual organisms ( Beni & Wang, 2004 ). SI algorithms emulate these features to solve complex optimization, control, classification, clustering, routing, and prediction problems in diverse domains, such as engineering, robotics, biology, economics, social sciences, and humanities. There are two main categories of SI algorithms: swarm-based algorithms and swarm-inspired algorithms ( Dorigo & Gambardella, 1996 ). Swarm-based algorithms involve the simulation of a population of individuals (agents) that interact with each other and their environment to achieve a collective goal. Examples of swarm-based algorithms include ant colony optimization (ACO), particle swarm optimization (PSO), artificial bee colony (ABC), and firefly algorithm (FA) ( Kennedy & Eberhart, 1995 ; Karaboga & Basturk, 2007 ; Yang, 2010). Swarm-inspired algorithms, on the other hand, extract specific mechanisms or principles from natural swarms and incorporate them into conventional optimization or machine learning algorithms. Examples of swarm-inspired algorithms include artificial immune systems (AIS), bacterial foraging optimization (BFO), and grey wolf optimizer (GWO) ( Dasgupta & GonzĂĄlez, 2002 ; Passino, 2002 ; Mirjalili et al., 2014 ).
Standard probabilistic models face fundamental challenges such as data scarcity, a large hypothesis space, and poor data transparency. To address these challenges, we propose a novel probabilistic model of data-driven temporal propositional reasoning. Unlike conventional probabilistic models where data is a product of domain knowledge encoded in the probabilistic model, we explore the reverse direction where domain knowledge is a product of data encoded in the probabilistic model. This more data-driven perspective suggests no distinction between maximum likelihood parameter learning and temporal propositional reasoning. We show that our probabilistic model is equivalent to a highest-order, i.e., full-memory, Markov chain, and it can also be viewed as a hidden Markov model requiring no distinction between hidden and observable variables. We discuss that limits provide a natural and mathematically rigorous way to handle data scarcity, including the zero-frequency problem. We also discuss that a probability distribution over data generated by our probabilistic model helps data transparency by revealing influential data used in predictions. The reproducibility of this theoretical work is fully demonstrated by the included proofs.
Feature selection reduces the dimension of input variables by eliminating irrelevant features. We propose feature selection techniques based on a genetic algorithm, which is a metaheuristic inspired by a natural selection process. We compare two types of feature selection for predicting a stock market index and cryptocurrency price. The first method is a newly devised genetic filter involving a fitness function designed to increase the relevance between the target and the selected features and decrease the redundancy between the selected features. The second method is a genetic wrapper, whereby we can find the better feature subsets related to KOPSI by exploring the solution space more thoroughly. Both genetic feature selection methods improved the predictive performance of various regression functions. Our best model was applied to predict the KOSPI, cryptocurrency price, and their respective trends after COVID-19.
Information theory is a powerful tool to express principles to drive autonomous systems because it is domain invariant and allows for an intuitive interpretation. This paper studies the use of the predictive information (PI), also called excess entropy or effective measure complexity, of the sensorimotor process as a driving force to generate behavior. We study nonlinear and nonstationary systems and introduce the time-local predicting information (TiPI) which allows us to derive exact results together with explicit update rules for the parameters of the controller in the dynamical systems framework. In this way the information principle, formulated at the level of behavior, is translated to the dynamics of the synapses. We underpin our results with a number of case studies with high-dimensional robotic systems. We show the spontaneous cooperativity in a complex physical system with decentralized control. Moreover, a jointly controlled humanoid robot develops a high behavioral variety depending on its physics and the environment it is dynamically embedded into. The behavior can be decomposed into a succession of low-dimensional modes that increasingly explore the behavior space. This is a promising way to avoid the curse of dimensionality which hinders learning systems to scale well.
Due to the explosive development of global network structure, electronic commerce is increasingly playing an important role in many organizations and individual consumerâs daily life. It offers opportunities to significantly improve the way for businesses interactions between both customers and suppliers. More and more large scale and decentralized ecommerce mechanisms have emerged in industrial and commercial domains in a wide range. In particular, among all these applications, online auctions, which are flexible pricing mechanisms over internet, make the physical limitations of traditional auctions disappear. They gain their extra popularity in the daily life and attract globally dispersed users due to having the characteristics that âbargainingâ and ânegotiationâ besides all of the convenience. Thus, online auctions become one of the most widely studied and employed negotiation mechanisms today. Traditionally, in most current online auction applications, the traders are generally humans who operate all the behaviors to make transactions. These behaviors may involve observing the auctions, analyzing the auction information, and bidding the suitable price for the items. However, facing the increasingly demanding requirements and complexity of online trading, this kind of manual operation does not reveal the full potential of this new mode of commerce. Thus, in order to relieve the users and be more effective, exploring possible types and automating the behaviors in the online auction attract high interest. Now, in many studies, the agent-oriented auction mechanism, with its emphasis on autonomous actions and flexible interactions, arises as an effective and robust model for the dynamic and sensitive commerce environment. In such systems, the agent acts flexibly on behalf of its owner and is capable of local decision-making based on the environment information and pre-knowledge about the system. Among many different types of online auction, two of the most popular and studied types are Multiple Round English Auctions (MREA), which is single side auction, and Continuous Double Auction (CDA), which is double side auction. These auctions are newly emerged in e-commerce era based on the traditional auction types. They allow multiple agents to participate and one agent can deal with several auctions continuously or simultaneously, which are effective auction types to save time and relieve the users. Towards to these types, because there is no centralized system-wide control, the major challenge for automatic bidding strategies is to improve the degree of automation and optimize the agentâs bidding behavior in order to maximize the ownerâs profit. Most of the related researches have been conducted by using heuristic methods and fixed mathematical functions to compute the final optimal bidding price for the items or to compute how much should bid at each time step. Nevertheless, because auction environments are complicated and highly dynamic due to have many factors affecting each other, these approaches are not flexible enough for the dynamic environment, and there is no dominant strategy. Against this background, this thesis is concerned with developing the intelligence of autonomous agentâs bidding strategy in order to make the agent to be more efficient and competitive for agent-based online auction mechanisms, especially in MREA and CDA. In order to be more flexible and better exploit the market information, Genetic Network Programming (GNP) is firstly employed to the agentâs bidding strategy since its applicability and efficiency have been clarified in complex and dynamic problems in many other fields. GNP is one of the evolutionary optimization techniques developed as an extension of Genetic Algorithm (GA) and Genetic Programming (GP), which uses compact directed graph structures as solutions. Basically speaking, in the proposed method, the GNP population represents the group of potential bidding strategies, and each individual uses the as-if/then decision-making functions to judge the auction information and guides the agent to take the suitable actions under different situations. Thus, it could be flexible and capable to adaptive to various auction situations. During the evolution, the GNP structure will be systematically organized, and finally, the individual which can obtain the highest profit is selected as the optimal bidding strategy at the end of training phase. In chapter 2, we introduced the conception of MREA and CDA in detail, which are the study environments in this thesis. The related researches are also introduced. In chapter 3, focusing on MREA, the bidding strategy for the auction agents in MREA is proposed using GNP. The performance of GNP-based agents is evaluated and studied in two situations: MREA is no time limit (NTL), and MREA is time limit (TL). Furthermore, according to the amount of the money each agent has, each situation is divided into 2 cases: general case and poorest case. All the participating agents in the simulations use GNP strategy. This chapter aims to study and analyze the capability and effectiveness of GNP for guiding bidding actions through the phenomenon of the simulations. The simulation results reveal that the agents using GNP strategy can understand various environments well through experiences and become smarter through evolution. In chapter 4, as an extension of the bidding strategy in chapter 3, in order to improving the agentâs intelligence and sensitivity, an enhanced bidding strategy for MREA is developed using GNP. Firstly, the GNP structure is modified to be able to judge more kinds of information and more situations at a time. Secondly, the strategy is improved to be able to consider the bidderâs attitude towards to each good, which makes the strategy to be more personalized for each bidder and could make the bidder more satisfied with the auction result and profit. The proposed strategy is compared with the previous GNP strategy and the other conventional strategies in the simulations. The simulation results demonstrated that the proposed method can outperform the previous one and is more competitive than the agents based on mathematical functions. In chapter 5, focusing on CDA, GNP with rectify nodes (GNP-RN) has been applied for CDA bidding strategy combined with proposed heuristic rules, which are derived based on the common believes for assisting agentâs bidding behavior. GNP-RN is developed aiming to guide the agent to be competitive under different CDA environments, and maximize the agentâs profit without losing chances for trading. Rectify Node (RN) is a newly proposed kind of nodes, which is used for bringing more flexible and various options for bidding action choices. 4 groups of simulations are designed to compare GNP-RN with conventional GNP and other strategies in CDA. In each simulation, the kinds of opponent agents are different in order to fully analyze the agentsâ performance. The simulation results show that the proposed method can outperform all the other strategies and achieve high success rate as well as high profit even when the situation is highly competitive. In chapter 6, as an extension of GNP-RN, GNP with adjusting parameters (GNP-AP) for developing bidding strategy in large-scale CDAs is proposed and studied. In large-scale CDAs, much more history information can be obtained than small-scale CDAs. In order to enhance the sensitivity for large-scale CDAs and the capability of judging abundant information, the parameters used by GNP-AP decision-making functions are adjusted during the evolution instead of being fixed in GNP-RN. Moreover, the structure of GNP-AP is designed to be more comprehensive that the number of branches of some kinds of nodes is increased to adapt to the complicated environment situations. The simulation results show that GNP-AP can obtain a good guidance for the large-scale CDAs and could be very efficient for the markets. In chapter 7, after giving the objectives and motivation of each research in this thesis, some conclusions about the proposed algorithms are described based on the simulation results.
Practical knowledge on the design and application of multi-objective evolutionary algorithms (MOEAs) is available but well-founded theoretical analyses of the runtime are rare. Laumanns, Thiele, Zitzler, Welzel and Deb (2002) have started such an analysis for two simple mutation-based algorithms including SEMO. These algorithms search locally in the neighborhood of their current population by selecting an individual and flipping one randomly chosen bit. Due to its local search operator, SEMO cannot escape from local optima, and, therefore, has no finite expected runtime in general. In this talk, we investigate the runtime of a variant of SEMO whose mutation operator flips each bit independently. It is proven that its expected runtime is O(n^n) for all objective functions f: {0,1}^n -> R^m, and that there are bicriteria problems among the hardest problem for this algorithm. Moreover, for each d between 2 and n, a bicriteria problem with expected runtime Theta(n^d) is presented. This shows that bicriteria problems cover the full range of potential runtimes of this variant of SEMO. For the problem LOTZ (Leading-Ones-Trailing Zeroes), the runtime does not increase substantially if we use the global search operator. Finally, we consider the problem MOCO (Multi-Objective-Counting-Ones). We show that the conjectured bound O((n^2)log n) on the expected runtime is wrong for both variants of SEMO. In fact, MOCO is almost a worst case example for SEMO if we consider the expected runtime; however, the runtime is O((n^2)log n) with high probability. Some ideas from the proof will be presented.