Omer Aziz, Muhammad Shoaib Farooq, Junaid Nasir Qureshi, Muhammad Faraz Manzoor ¡ 5 authors
(1) Background: A blockchain-based framework for distributed agile Open-Source Software for Archaeological Photogrammetry (OSSAP) testing life cycle is an innovative approach that uses blockchain technology to optimize the Open-Source Software for Archaeological Photogrammetry process. Previously, various methods have been employed to address communication and collaboration challenges in Open-Source Software for Archaeological Photogrammetry, but they were inadequate in aspects such as trust, traceability, and security. Additionally, a significant cause of project failure was the non-completion of unit testing by developers, leading to delayed testing. (2) Methods: This article discusses the integration of blockchain technology in Open-Source Software for Archaeological Photogrammetry and resolves critical concerns related to transparency, trust, coordination, testing and communication. A novel approach is proposed based on a blockchain framework named Open-Source Software for Archaeological Photogrammetry Testing-Plus. (3) Results: The Open-Source Software for Archaeological Photogrammetry Testing-Plus framework utilizes blockchain technology to provide a secure and transparent platform for acceptance testing and payment verification. Moreover, by leveraging smart contracts on a private Ethereum blockchain, Open-Source Software for Archaeological Photogrammetry Testing-Plus ensures that both the testing team and the development team are working towards a common goal and are compensated fairly for their contributions. (4) Conclusions: The experimental results conclusively show that this innovative approach substantially improves transparency, trust, coordination, testing and communication and provides security for both the testing team and the development team engaged in the distributed agile Open-Source Software for Archaeological Photogrammetry (Open-Source Software for Archaeological Photogrammetry) testing life cycle.
Giovanni De Gasperis, Sante Dino Facchini, Asif Saeed
In recent years, numerous regions worldwide have experienced devastating natural disasters, leading to significant structural damage to buildings and loss of human lives. The reconstruction process highlights the need for a reliable method to document and track the maintenance history of buildings. This paper introduces a novel approach for managing and monitoring restoring interventions using a secure and transparent digital framework. We will also present an application aimed at improving building structures with respect to earthquake resistance. The proposed system, referred as the âBuilding Ledger Dossierâ, leverages a Digital Twin approach applied to blockchain to establish an immutable record of all structural interventions. The framework models buildings using OpenSees, while all maintenance, repair activities, and documents are registered as Non-Fungible Tokens on a blockchain network, ensuring timestamping, transparency, and accountability. A Decentralized Autonomous Organization oversees identity management and work validation, enhancing security and efficiency in building restoration efforts. This approach provides a scalable and globally applicable solution for improving both ante-disaster monitoring and post-disaster reconstruction, ensuring a comprehensive, verifiable history of structural interventions and fostering trust among stakeholders. The proposed method is also applicable to other types of processes that require the aforementioned properties for document monitoring, such as the life-cycle management of tax credits and operations in the financial or banking sectors.
Timely approvals and payments to the project participants are crucial for successful completion of construction projects. However, the construction industry faces persistent delays and non-payments to contractors. Despite the desirable benefits of automated payments and enhanced access to digitized data progress, most payment applications rely on centralized control mechanisms; inefficient procedures; and documentation that takes time to prepare, review, and approve. As such, there is a need for a reliable payment automation system that guarantees timely execution of payments upon the detection of completed works. Therefore, this study used a cutting-edge approach to automate construction payments by integrating blockchain-enabled smart contracts and scan-to-Building Information Modeling (BIM). In this approach, scan-to-BIM provides accurate, real-time building progress data, which serve as the source of verifiable off-chain data. A chain-link is then used to securely relay these data to the blockchain system. Blockchain-enabled smart contracts automate the execution of payments upon meeting contract conditions. The proposed approach was implemented on a real case study project. The actual site scan was captured using a photogrammetry 360° camera, which uses a combination of structured light and infrared depth sensing technology to capture 3D data and create detailed 3D models of spaces. This study leveraged accurate, real-time building progress data to automate payments using blockchain-enabled smart contracts upon work completion, thus reducing payment disputes by tying payments to verifiable construction progress, leading to faster release of payments. The findings show that this approach provides a transparent basis for payment, enhancing trust and allowing precise project progress tracking.
NFTs (Non-Fungible Tokens) and blockchain are revolutionizing the art market by offering advantages in terms of traceability and authenticity of works. Through blockchain, every transaction is recorded transparently and immutably, ensuring provenance and control over the circulation of the artwork. NFTs certify the uniqueness of a digital work, acting as a guarantee of authenticity and tracking each ownership transfer. Additionally, NFTs enable the tokenization of art, breaking ownership into shares, making it accessible to a broader audience and creating new investment and engagement opportunities.
Ămer Ăzeren, Bahar Sultan Qurraie, Mustafa Haki Eraslan
This study explores the integration of Non-Fungible Tokens (NFTs) into architectural education to equip students with the knowledge and skills necessary to thrive in the digital age. By incorporating NFT technology into the curriculum, the study aimed to foster a comprehensive understanding of digital design, intellectual property protection, and the potential of NFTs in preserving and promoting architectural heritage. The research found integrating NFTs empowered students to explore the intersection of art, technology, and commerce within the architectural realm. By transforming their designs into unique digital assets, students developed a deeper appreciation for their workâs commercial potential and the importance of intellectual property protection. The study demonstrated the role of NFTs in fostering networking opportunities, enabling students to connect with a wider audience and potential collaborators. To fully harness the potential of NFTs in architectural education, the study emphasizes the need for a holistic approach that addresses ethical, legal, and environmental considerations. Educators must instil in students a strong sense of responsibility regarding the creation and utilization of digital assets, including copyright, licensing, and the environmental impact of blockchain technology. Fostering interdisciplinary collaboration is also crucial for equipping students with the diverse skill set required to navigate the complexities of the digital landscape. By addressing these critical dimensions, architectural education can effectively prepare students to become proficient digital designers and informed participants in the evolving NFT ecosystem. This research contributes to the ongoing discourse on the role of technology in shaping the future of architecture and preserving cultural heritage.
This research paper thoroughly addresses the processes of digitizing ceramic art and offering it as Non-fungible Tokens (NFTs) for sale. It examines the effects of transitioning from traditional ceramic molding methods to modern 3D printing and Stereolithography (SLA) technologies on the digital transfer of ceramic works. A case study on the use of porcelain highlights thanks to white color the impact of material choice on aesthetics and functionality. The rise of digitalization and NFTs introduces new mechanisms for preserving the originality and ownership rights of works, while also discussing the technical challenges and ethical issues this process entails. The article evaluates the place and future of ceramic art in the digital age from both technical and cultural perspectives. Our research posits that the digitization of ceramic works opens new avenues for the preservation, dissemination, and commerce of art, and that this process could have profound effects on the future of ceramic art. This process attempts to determine how ceramic works gain presence in the digital realm and their position in the NFT market. However, this study is one of the first to cover the entire process of ceramic art digitization and to address stage with academic rigor. The process from the creation of the work to its digitization, NFT registration, and sale, is detailed in this study. Our work demonstrates that the process of digitizing ceramic art and offering it as NFTs can have significant impacts on the future of art. It sheds light on the future of ceramic art by presenting both the opportunities brought by digitalization and the challenges encountered, as well as potential solutions. Furthermore, it emphasizes the importance and potential of ceramic art in the digital age and aims to fill the gaps in this field.
Open access
3D Surveying and Cultural Heritage
3D Shape Modeling and Analysis
Additive Manufacturing and 3D Printing Technologies
David F. Bucher, Jens Hunhevicz, Ranjith K. Soman, Pieter Pauwels ¡ 5 authors
The field of construction informatics is fragmented and lacks clarity in understanding the interconnection of different data management strategies. This makes it challenging to address industry-specific data management issues. Using a critical interpretive synthesis, this study reviews and integrates both present and emerging data management approaches in construction informatics. The review is meant to be comprehensive, encompassing technologies and concepts such as Open Schema, Information Container, Common Data Environments, Linked Data, as well as cutting-edge Web3 technologies such as blockchain and decentralized data protocols. The different approaches are identified and classified into five categories and mapped into a two-dimensional framework that considers data storage and data processing modes. The systematic categorization provides a simple, but comprehensive understanding of data management strategies in construction informatics. Moreover, the framework allows to identify the state of the art and trends of data management approaches, providing guidance for future research perspectives, especially in the intersection with Web3 technologies.
Omer Aziz, Muhammad Shoaib Farooq, Adel Khelifi, Mahdia Shoaib
Abstract The rapid evolution of the digital landscape has catalyzed the integration of blockchain technology within the domain of cultural heritage, particularly in virtual museums within the Metaverse. This study introduces ArchaeoMeta, a novel framework designed to leverage blockchain technology to enhance security, authenticity, and visitor interaction in a virtual museum environment. Utilizing smart contracts deployed on the Ethereum Sepolia testnet, the framework manages visitor interactions and secures digital artifacts, addressing challenges associated with scalability and user experience under varying loads. The performance evaluation involved simulating user interactions, scaling up to ten thousand concurrent users, to assess the impact on transaction latency, gas usage, and blockchain size. Findings reveal significant scalability challenges, as transaction latency and blockchain size increased with the number of users, highlighting areas for optimization in managing high user traffic within the blockchain infrastructure. This study contributes to the understanding of blockchain applications in cultural heritage, suggesting that while ArchaeoMeta offers a robust platform for virtual museums, enhancements in scalability through layer-2 solutions or alternative blockchain platforms are essential for its practical implementation. The framework sets a precedent for future research in the convergence of blockchain technology and cultural heritage preservation, promising a transformative impact on how digital cultural experiences are curated and consumed.
Increasing expectancy for efficiency in the delivery of building projects and the adoption of lean production processes for construction has made the necessity for the development of an integrated system for cost estimating, cost monitoring, cost control, and payments in the construction lifecycle important. Existing 5D BIM tools are used to estimate the cost of projects during the preconstruction period. There is a lack of integration between the 5D BIM models, existing progress monitoring tools, and payment systems used in construction. Lack of standardization in the use of model elements through the project lifecycle has also been identified as one of the factors limiting automation in 5D BIM. Construction project monitoring can be automated by combining modern technologies that allow for visualization of building progress (Laser scanners, computer vision) with 5D BIM cost estimation tools. These project monitoring tools can be combined with Artificial Intelligence (AI), and Smart contracts to develop an integrated lifecycle system for cost management in construction. This paper examines existing systems used in 5D BIM to develop integrated practices and systems that will streamline the process of cost estimating, cost monitoring, cost control, and cash flow in the construction supply chain. This will reduce the inefficiency that exists today with traditional contracts and payment applications that do not interact with the 5D BIM application. By leveraging a standardized classification ID system throughout a project life cycle and applying AI and smart contract, features like cost estimation cost control, and payments can be fully streamlined, integrated, and automated. A case study of an existing construction project utilizing 5D BIM was examined. According to the study, 5D BIM is used in the pre-construction stage of a cost estimation project. It was also revealed that 5D BIM improves project cost visualization and budget control.
Feb 14, 2024¡ÂThe Âinternational archives of the photogrammetry, remote sensing and spatial information sciences/International archives of the photogrammetry, remote sensing and spatial information sciences
Nikolai Abramov, Havana Lankegowda, Suo Liu, Luigi Barazzetti ¡ 6 authors
Abstract. This research employs a digital strategy to counter rural population decline and urban migration by integrating drone photogrammetry, metaverse, and blockchain technology. The focus is creating detailed 3D models of the settlements, which are subsequently optimized for metaverse platforms such as Voxels and Mona. Voxels facilitates participatory design, utilizing voxel parcels and Rhino- Grasshopper to involve all the stakeholders for renovation design work. Mona transforms parameterized projects into an immersive virtual experience, fuelled by blockchain technology, providing non-fungible token (NFT) marketplaces and economic transitions. These virtual and real realms serve purposes such as decision-making, cultural preservation, economy generation, and virtual tourism, fostering rural revival, virtual economy, and digital representation of real-world environments.
Digitization marks the initial phase of digital transformation, encompassing a spectrum of socio-technical developments that unfold thereafter. These processes are pivotal components within a broader domain that: 1) encompasses physical inspection methodologies, business models geared towards enhancing inspection efficiency, integrity engineering, and decision-making; and 2) furnishes invaluable data to enrich the design, production, and maintenance over the lifespan of cultural heritage artifacts. Vrana and Sing (2020) advocate for designating this domain as Non-Destructive Evaluation (NDE) 4.0, underscoring its characterization as "a suite of cyber-physical technologies." These technologies encompass Digital Twin (DT), Industrial Internet of Things (IIoT), Semantic Interoperability and Ontologies, Industry 4.0 Data Processing (Big Data Analysis), Blockchains, Non-Fungible Tokens (NFTs), and Artificial Intelligence. The concept of Digital Twin, introduced in 2003 at the University of Michigan during the Executive Course on Product Lifecycle Management (PLM), as articulated by Grieves (2014), holds particular significance. Glaessgen and Stargel (2012) subsequently advocated for Digital Twins to support the production of NASA and U.S. Air Force vehicles, envisioning them as enabling "integrated multiphysics, multiscale, probabilistic simulation of an as-built vehicle or system that utilizes the best available physical models, sensor updates, fleet history, etc." Within this framework, a methodology will be elucidated for devising and implementing innovative tools and techniques for digitizing and digitalizing attributes of cultural heritage items.
For over a decade, digital modelling has transcended geometric representations to more advanced object-based modelling using real-life attirbutes. Digital technologies, especially Building Information Modelling (BIM) have been widely adopted in the Architecture, Engineering, Construction, and Operations (AECO) industry, while a newer niche - City Information Modelling (CIM) has emerged as an extension of BIM for urban informatics. This research proposes a framework that integrates heterogeneous CIM using a multi-level, nested data environment. The CIM is developed through a network of BIM models synchronized into a Geographic Information Systems (GIS) interface. The individual BIM models are blockchain-enabled by connecting them to a distributed ledger and shared across a network of project collaborators using a Common Data Environment (CDE) in a Building Level Framework.. The resulting CIM from the network of BIM models is shared between the asset owners in a City Level Framework. Use case scenarios are presented to illustrate the application of the research in real life, and research limitations are discussed. The study aims to improve management of buildings and urban assets, providing a more efficient and secure platform for collaboration and data sharing through a blockchain-CIM integration, providing opportunities for further research in digital modelling and smart technologies.
For over a decade, digital modelling has transcended geometric representations to more advanced object-based modelling using real-life attirbutes. Digital technologies, especially Building Information Modelling (BIM) have been widely adopted in the Architecture, Engineering, Construction, and Operations (AECO) industry, while a newer niche - City Information Modelling (CIM) has emerged as an extension of BIM for urban informatics. This research proposes a framework that integrates heterogeneous CIM using a multi-level, nested data environment. The CIM is developed through a network of BIM models synchronized into a Geographic Information Systems (GIS) interface. The individual BIM models are blockchain-enabled by connecting them to a distributed ledger and shared across a network of project collaborators using a Common Data Environment (CDE) in a Building Level Framework.. The resulting CIM from the network of BIM models is shared between the asset owners in a City Level Framework. Use case scenarios are presented to illustrate the application of the research in real life, and research limitations are discussed. The study aims to improve management of buildings and urban assets, providing a more efficient and secure platform for collaboration and data sharing through a blockchain-CIM integration, providing opportunities for further research in digital modelling and smart technologies.
Digital Twins (DTs), enriched with Artificial Intelligence (AI) and Blockchain technology, promise a revolutionary breakthrough in smart asset management and predictive maintenance in the built environment. This study aims to portray a conceptual framework of Blockchain and AI-based DTs and outline its key characteristics, requirements, and system architecture by composing a functional model using IDEF0. Such an approach is expected to enhance predictive maintenance in building facilities, simplify the management and operation of smart built environments, and ultimately deliver valuable outcomes for facility operators, real estate practitioners, and end-users.
Zongliang Zhang, Sherong Zhang, Zhiyong Zhao, Lei Yan ¡ 6 authors
Improving the design efficiency of hydropower hub buildings and promoting the application of intelligence in engineering construction and operation management have been critical issues in hydropower engineering construction. The major challenges are mainly reflected in the difficulties of multidisciplinary collaboration and the inefficient coordination of work involving multiple parties. This paper proposes a building information modeling (BIM)-based technology system architecture for digital design, intelligent construction, and intelligent operation, which combines BIM technology with geographic information system (GIS), computer aided engineering (CAE), internet of things (IoT), artificial intelligence (AI), and other technologies. The proposed system includes a BIM-based multiprofessional forward collaborative design method, a BIM-based engineering construction management model, and a real-time safety analysis and evaluation technology system based on actual measured safety information and construction. The hydroelectrical engineering BIM (HydroBIM) comprehensive control platform is developed. In the planning and design stage, the platform enables the whole process and full-professional collaborative digital design. In the engineering construction management stage, it supports the whole process and all-round information management and control of contract, schedule, quality, safety, and investment. In the operation management stage, the platform facilitates the integrated management of engineering safety evaluation, early warning, and emergency plan based on monitoring data and the consistency criterion of positive and negative evaluation. The application of this technology in more than 20 engineering, has proven to improve the efficiency of design and analysis of hydropower hub buildings and the intelligence level of engineering construction and operation management, and overcome the difficulties of multi-professional collaboration in the design stage and low efficiency of multi-participant coordination in the construction stage.
Non-Fungible Tokens (NFTs) are a type of digital asset that represents a proof of ownership over a particular digital item such as art, music, or real estate. Due to the non-fungible nature of NFTs, duplicate tokens should not possess the same value. However, with the surge of new blockchains and a massive influx of NFTs being created, a wealth of NFT data is being generated without a method of tracking similarity. This enables people to create almost identical NFTs by changing one pixel or one byte of data. Despite the similarity among NFTs, each NFT is assigned a completely different token ID. To address the NFT duplication issue, we developed a modular, easily-extendable, hardware-agnostic, cloud-centered NFT processing system that represents NFTs as vectors. We established a database containing a vector representation of the NFTs in accordance with the Ethereum Request for Comment 721 (ERC-721) token standards to initiate the process of aggregating NFT data from various blockchains. Finally, we developed an NFT visualization dashboard application with a user-friendly graphical user interface (GUI) to provide non-technical users access to the aggregated NFT data. The Universal NFT Vector Database is an off-chain framework for NFT data aggregation based on similarity, which provides an organized way to query and analyze NFT data that was previously unavailable through on-chain solutions.
Jan-Iwo Jäkel, Sophie Kalisch, Peter GĂślzhäuser, Katharina KlemtâAlbert
To ensure intact infrastructure systems and a long service life of bridge structures in the operational phase, the establishment of a predictive maintenance management based on digital 3D models is essential.These 3D models are often not provided, especially for existing bridge structures.In addition, their creation is very complex, resource-intensive and involves many stakeholders.Through the use of digital methods and technologies, important efficiency steps have been taken in the creation of digital models in recent years.Although semi-automated modeling of digital bridge models is possible, the quality agreed upon is not always achieved at the end of each development step.In this article, an approach is developed to create transparency, safety, consistency and traceability for the generation process of digital models of existing bridge structures.The approach involves all stakeholders participating in the process and creates a decentralized control and documentation mechanism using distributed ledger technology (DLT) and blockchain.First, the current status on the use of DLT in the construction industry and bridge engineering is presented.Then, the system concept is presented and the basic algorithm is described.The general system architecture and the workflow are presented and described in models.Then the basic feasibility of the approach is presented and the previously shown concept is implemented.The basic functions of the algorithm are described and the results are critically reviewed.The result of the article is showing the possibilities to use DLT and blockchain to improve the accuracy, transparency and security in the creation of digital models of existing bridge structures.
The present study uses a bibliometric and systematic literature review (SLR) to examine the use of Building Information Modeling (BIM), the Internet of Things (IoT), and Digital Twins (DT) in the construction industry. The network visualization and other approaches based on the Web of Science (WOS) database and the patterns of research interactions were explored in 1879 academic publications using co-occurrence and co-citation investigations. Significant publications, conferences, influential authors, countries, organizations, and funding agencies have been recognized. Our study demonstrates that BIM, IoT, and DT in construction, Heritage BIM (HBIM), Smart Contracts, BIM, and Ontology, and VR and AR in BIM and DT are the main study themes. Finally, several prospective areas for future study are identified, including BIM and Metaverse technology, BIM and Artificial Intelligence (AI), Metaheuristic algorithms for optimization purposes in BIM, and the Circular Economy with BIM and IoT.
The management of construction projects requires adequate techniques to support the continual exchange of information across disciplines. Recent advances in Building Information Modelling (BIM) have exposed new ways for process and data integration with open data formats, process mapping, and terminology. In construction projects where multiple disciplines produce and share BIM data, mechanisms for defining information priority, provenance and suitability become necessary in order to have consistent and traceable use of data. This includes objects or collection of attributes for data objects that are associated with a discipline or organisation including clear identification of the transaction that has introduced the information. Blockchain can be used to record metadata of BIM objects such as the issuing discipline, object version and responsibility/ liability associated with the data. Blockchain can offer the capability to apply levels of âtrustâ to individual BIM objects and a more secured framework of collaboration across stakeholders. This paper proposes a Blockchain-based BIM data provenance model to support information exchange in construction projects. By testing the solution in a real-world bridge construction scenario, it has been shown that the approach can recognise the levels of competence and can improve the process of BIM implementation. The proposed approach gives stakeholders more confidence when sharing their BIM data, reduces costs, and improves risk contingencies in construction projects. The paper provides a cost analysis to evidence the implications of using Blockchain for BIM data provenance through an experimental framework supported by an Ethereum public test network. A front-end web page has also been created to facilitate interaction with smart contracts and to monitor the BIM data provenance process.
Shenghui Cheng, Yue Zhang, Xiaofei Li, Lin Yang ¡ 6 authors
Truth must be told, but not muchâa hybrid society of real and virtual is coming. Metaverse,1Cheng S.H. Metaverse: Concepts, Technologies and Ecology. China Machine Press, 2022Google Scholar rise recently, is attracting significant attention from academia to industry. A metaverse is a network of three-dimensional (3D) virtual worlds focused on social connection. Bearing the outbreak of the coronavirus 2019 pandemic, people are physically isolated, which triggered the growth of the metaverse. Different from existing work, this commentary targets the roadmap of the metaverse from an artificial intelligence (AI) perspective. First, we blueprint a roadmap for this digital cyberspace transformation, including immersion creation, hardware support, text interpretation, audio processing, connection construction, economy operation, and security protection. Second, at each phase of the roadmap, we address the status as well as the advanced technologies to provide in-depth views accordingly. The whole pipeline is illustrated from an AI2Xu Y.J. Liu X. Cao X. et al.Artificial intelligence: a powerful paradigm for scientific research.Innovation. 2021; 2: 100179Scopus (114) Google Scholar perspective, as shown in Figure 1, and we believe that AI is playing an increasingly important role in core techniques toward this technological singularity. Immersion creation refers to the technique that provides human beings with immersive feelings by constructing a 3D virtual world. Inspired by computer vision, graphics, and visualization techniques, immersion creation includes generating virtual scenes in the metaverse and displaying them to end users. AI has revolutionized these techniques recently. For instance, the scene generation process has been significantly speeded up to nearly real time. However, the scenes generated by these computer-vision-based methodologies are limited by the pre-defined elements used in AI algorithms, which would preclude the immersive feeling of human interaction with a digital human in the metaverse since these elements are different from the real ones, but we desire both parties to share the same scene and thus the same feeling of the environment. This brings challenges in sensing, sampling, and scene generation in the metaverse. One solution is to capture real scenes and use those images or videos to generate new ones and update the scenes in the metaverse with a short period of time, possibly in real time eventually. To this end, computational imaging has provided a promising solution to capture scenes efficiently in a low-cost, low-bandwidth manner;3Altmann Y. McLaughlin S. Padgett M.J. Goyal V.K. Hero A.O. Faccio D. Quantum-inspired computational imaging.Science. 2018; 361: 6403https://doi.org/10.1126/science.aat2298Crossref Scopus (119) Google Scholar along with AI, computational imaging may develop rapidly in metaverse-related applications in the future to provide a higher sense of immersion. Hardware denotes the end-user devices, such as the brain-computer interface, robotics, and virtual reality or augmented reality headsets as well as glass-free 3D devices, which determine the quality of the userâs immersive experience. These devices have been revolutionized many times in history;4Lee L.K. Braud T. Zhou P.Y. et al.All one needs to know about metaverse: a complete survey on technological singularity, virtual ecosystem, and research agenda.arXiv. 2021; (Preprint at)https://doi.org/10.48550/arXiv.2110.05352Crossref Google Scholar taking the AR headset as an example, it was designed from macro- to micro-optics and now is heading toward nano-optics, and the size has been reduced dramatically. However, one main challenge is the design of large-scale diffractive devices used in glasses, and AI is now helping with and speeding up the design in a number of ways, from accelerating the iterations to proposing new solutions. The AI accelerator, such as neural-inspired AI chips, designed to accelerate AI and its applications, is growing tremendously now and will evolve to task-specific chips to be used in the metaverse. We believe this is a trend in optical or other devicesâ design and is not limited to the metaverse. Text interpretation regards to text generation and text for communication purpose. The metaverse brings convenience for recording activities in business and social life, which potentially facilitates the role of natural language processing,5Zhang Y. Teng Z.Y. Natural Language Processing: A Machine Learning Perspective. Cambridge University Press, 2021Crossref Google Scholar a research domain widely supported by AI. Most natural language processing applications in the metaverse focus on personal assistance and business meeting analysis: from the personal side, new dialogue systems could benefit book-keeping of social activities, enabling a personal assistant to give clear instructions, and from a business perspective, it keeps track of meeting minutes, where AI algorithms generate summaries and answer questions concerning specific details. However, in a hybrid society, how to communicate with people of different backgrounds even living in different eras could be an interesting AI-powered application. Audio processing aims for the rendering of auditory immersion. Voice is considered one important interface for humans to enter the metaverse, and it is also the main interaction model between entities (avatars, digital humans, or even non-human objects) in the metaverse. Voice processing consists of two major tasks: automatic speech recognition and text to speech (or speech synthesis), which transforms voice signals to text, or vice versa. Together with the language-understanding techniques, automatic speech recognition and text to speech enable entities in the metaverse to understand the message and intention of others and to speak as if they were in the real world. In addition, audio signals could be rendered as binaural signals, from which humans can sense the location of the sound source and the presence of an enclosed space; in other words, to have the sense of auditory immersion. For auditory immersion in the metaverse, many challenges need to be further overcome, such as separating metaverse sound from real-world sound, customized speech synthesis, complex 3D soundscape generation, etc., which highly rely on advanced signal-processing and machine-learning techniques. Sound is ubiquitous in the metaverse, and we believe that AI is playing a leading role in audio management in the metaverse. Connection construction in the metaverse includes network connection and social construction to get connected with others. Network connections require high-speed network to transfer massive amounts of data, accounting for the obstacles in latency, bandwidth, and consistency. AI overcomes the obstacles in various aspects including traffic planning, routing and classification, congestion control, quality of service, and quality of experience management. After being networked, the society emerges but faces challenges like behavior communication or community management. AI is capable of optimizing social group management by matching people with their desired communities so that the whole society will be divided spiritually rather than physically. As the core purpose of the connection, social activity is a requisite of some recognition techniques, such as face recognition for meeting people and gesture recognition for behavior interaction, as well as pose tracking, texture restoration, and blur correctionâall of which are common cases where AI excels. Economy operation is for exchanging virtual goods through a blockchain digital system. With the exponential growth of virtual economy, some challenges have arisen, like large-scale asset management and fraudulent-transaction detection. AI empowers the virtual economy by generating and managing digital assets, monitoring transactions among huge databases, and more. New classifiers and mechanisms supported by AI verify the authenticity of transactions via a decentralized blockchain infrastructure to improve security posture. AI-generated non-fungible tokens and AI-assisted asset tagging increase the efficiency of asset generation and management. In the future, AI can build a more reliable technology-based virtual economy system. Security protection corresponding to the security framework and privacy protection becomes extremely challenging when tremendous virtual devices are connected. This connection is not that reliable and gives space to get attacked, e.g., identity theft and spying, which are common problems that security faces. To be protected, the security system has to require frequent authentications when accessing the service related to virtual devices, which results in time-consuming and other problems. New mechanisms are demanding for authentication with alternative modalities, such as biometric authentication driven by muscle movements or eye gazes, as well as seamless authentication. None of these would happen without AI. Moreover, countless activities and interactions are recorded, creating a crisis of personal privacy. AI is capable of privacy protection through algorithms that automatically and dynamically detect user privacy preferences from diverse contexts in the metaverse. An AI-powered security system will emerge in the metaverse. Besides, other techniques like robotics, cloud computing, the Internet of Things, and decision-making are also important components for the metaverse. Likewise, AI also contributes significantly to themâfor example, with the help of reinforcement learning, decision-making demonstrates great potential since it enables AI to make decisions through huge amounts of data with various factors and has now become a new trend. AI not only advances the above technologies but is itself evolving rapidly. Current components such as immersion creation and hardware techniques are not really âintelligentâ by themselves but are composed âsensing/sampling + AI algorithmsâ to claim to be intelligent. Looking forward, the next step of AI in the metaverse should move from âmake the hardware intelligent by AI algorithmsâ to âbuild intelligent hardware by AI algorithms,â where the latter is a hybrid mode and focuses on end-to-end solutions. In this solution, other important techniques in AI such as reinforcement learning, federated learning, and few-shot learning will probably lead to new revolutions. A new era driven by AI is emerging. The digital human produced by AI will increase dramatically, and a digital second life is appearingâa society combined with real and digital humans is launching. It is likely that other species or organisms will be reborn and that a âcreator-verseâ will turn up. Immortal technologyâliving foreverâwill eventually become true. At the current stage, it is more from the inorganic side where everything is generated with digital computers; however, the society will continue, and people will eventually reach the third-verse or even the multi-verse, where the organic world and the inorganic world may merge. AI will speed up this revolution. We would like to thank the support from the Research Center for Industries of the Future (RCIF) at Westlake University, Westlake Fundation (2021B1501-2) and the funding from Lochn Optics. The authors declare no competing interests.
User Interaction for NFTs (Non-fungible Tokens) is gaining increasing attention. Although NFTs have been traditionally single-use and monolithic, recent applications aim to connect multimodal interaction with human behavior. This paper reviews the related technological approaches and business practices in NFT art. We highlight that multimodal interaction is a currently under-studied issue in mainstream NFT art, and conjecture that multimodal interaction is a crucial enabler for decentralization in the NFT community. We present a continuum theory and propose a framework combining a bottom-up approach with AI multimodal process. Through this framework, we put forward integrating human behavior data into generative NFT units, as "multimodal interactive NFT." Our work displays the possibilities of NFTs in the art world, beyond the traditional 2D and 3D static content.