The proliferation of deep fake technology has raised significant concerns regarding the authenticity and integrity of digital media. This paper proposes a novel approach that leverages Non-Fungible Token (NFT) technology within blockchain combined with generative AI to detect and authenticate digital media, addressing the growing issue of deep fakes. Using NFTs, unique digital certificates of authenticity are created for each media piece and securely stored on a blockchain to ensure immutability and traceability. Generative AI models, specifically designed for anomaly detection, are employed to analyze media files and identify potential manipulations by comparing them against the original authenticated versions. This dual-layered system not only enhances the reliability of media verification but also provides a robust framework for ensuring the security and provenance of digital content. The implementation of this system demonstrates a significant improvement in detecting deep fakes, offering a practical solution to mitigate the risks associated with synthetic media. Intermediate results highlight the potential of integrating blockchain and AI technologies to establish a more secure and trustworthy digital ecosystem.
Digital Media Forensic Detection
Generative Adversarial Networks and Image Synthesis
Given a JPEG pipeline (compression or decompression), this paper demonstrates how to find the antecedent of an 8x8 block. If it exists, the block is considered compatible with the pipeline. For unaltered images, all blocks remain compatible with the original pipeline; however, for manipulated images, this is not necessarily true. This article provides a first demonstration of the potential of compatibility-based approaches for JPEG image forensics. It introduces a method to address the key challenge of finding a block antecedent in a high-dimensional space, relying on a local search algorithm with restrictions on the search space. We show that inpainting, copy-move, and splicing, when applied after JPEG compression, result in three distinct mismatch problems that can be detected. In particular, if the image is re-compressed after modification, the manipulation can be detected when the quality factor of the second compression is higher than that of the first. Through extensive experiments, we highlight the potential of this compatibility attack under varying degrees of assumptions. While our approach shows promising results-outperforming three state-of-the-art deep learning models in an idealized setting-it remains a proof of concept rather than an off-the-shelf forensic tool. Notably, with a perfect knowledge of the JPEG pipeline, our method guarantees zero false alarms in block-by-block localization, given sufficient computational power.
Open access
Digital Media Forensic Detection
Generative Adversarial Networks and Image Synthesis
This research paper proposes a novel approach of integrating traditional steganography techniques with modern deep learning methodology as a solution against prevailing cyber fraud in the blockchain space. Despite standard authentication measures in place, Non-Fungible Tokens (NFTs) are prone to data theft, replication, and misrepresentation of ownership. Hence, our work leverages deep learning infused with advanced crypto-steganography to add multi-layered authentication to these digital tokens. The research elaborates on multiple utilities relevant to NFTs such as their generation, verification, and price forecasting, while emphasizing mainly on authentication to provide a secure ecosystem. The research aims to delve into multiple methodologies that are integrated into a robust, comprehensive and a user-friendly NFT minting platform that works cohesively with the blockchain network. Hence, an extensive study is undertaken on the applications of deep learning in countering data theft in the NFT marketplaces.
Advanced Steganography and Watermarking Techniques
Digital Media Forensic Detection
Generative Adversarial Networks and Image Synthesis
Himanshu Tiwari, Ayush Raj, Ujjwal Kr. Singh, Hoor Fatima
Incorporating generative artificial intelligence (AI) into design and art has upended established creative paradigms, sparking discussions on the validity of AI-generated art and the development of non-fungible token (NFT) marketplaces. The US Copyright Office rendered a significant decision in February 2023 that highlights the contentious nature of AI work and the need of human intervention in its commercialization. This paper traces the development of artificial intelligence in neural networks and examines how it has affected visual arts. We investigate the idea of autonomously creating digital art in the NFT style utilizing generative adversarial networks (GANs), with striking results. Our work links deep learning and blockchain, enabling AI to find a place in the digital art market.
Generative Adversarial Networks and Image Synthesis
Artists are digitally disenfranchised, rampant online sharing and ease of copying make protecting their work from unauthorized use an uphill battle. Digital art is effortlessly duplicated, shared, and manipulated without credit or compensation, even leading to individuals profiting off stolen work. This widespread issue, particularly plaguing social media, demands a solution. We propose a blockchain-based platform utilizing Non-Fungible Tokens (NFTs) to empower artists. By creating NFTs for their art, they claim irrefutable ownership, authenticity, and copyright, enabling secure sales via cryptocurrencies and combating unauthorized use. This innovative approach empowers artists to thrive in the digital age
Open access
Visual Attention and Saliency Detection
Generative Adversarial Networks and Image Synthesis
Arash Heidari, Nima Jafari Navimipour, Hasan Dağ, Samira Talebi · 5 authors
Abstract In recent years, the proliferation of deep learning (DL) techniques has given rise to a significant challenge in the form of deepfake videos, posing a grave threat to the authenticity of media content. With the rapid advancement of DL technology, the creation of convincingly realistic deepfake videos has become increasingly prevalent, raising serious concerns about the potential misuse of such content. Deepfakes have the potential to undermine trust in visual media, with implications for fields as diverse as journalism, entertainment, and security. This study presents an innovative solution by harnessing blockchain-based federated learning (FL) to address this issue, focusing on preserving data source anonymity. The approach combines the strengths of SegCaps and convolutional neural network (CNN) methods for improved image feature extraction, followed by capsule network (CN) training to enhance generalization. A novel data normalization technique is introduced to tackle data heterogeneity stemming from diverse global data sources. Moreover, transfer learning (TL) and preprocessing methods are deployed to elevate DL performance. These efforts culminate in collaborative global model training zfacilitated by blockchain and FL while maintaining the utmost confidentiality of data sources. The effectiveness of our methodology is rigorously tested and validated through extensive experiments. These experiments reveal a substantial improvement in accuracy, with an impressive average increase of 6.6% compared to six benchmark models. Furthermore, our approach demonstrates a 5.1% enhancement in the area under the curve (AUC) metric, underscoring its ability to outperform existing detection methods. These results substantiate the effectiveness of our proposed solution in countering the proliferation of deepfake content. In conclusion, our innovative approach represents a promising avenue for advancing deepfake detection. By leveraging existing data resources and the power of FL and blockchain technology, we address a critical need for media authenticity and security. As the threat of deepfake videos continues to grow, our comprehensive solution provides an effective means to protect the integrity and trustworthiness of visual media, with far-reaching implications for both industry and society. This work stands as a significant step toward countering the deepfake menace and preserving the authenticity of visual content in a rapidly evolving digital landscape.
Open access
Digital Media Forensic Detection
Generative Adversarial Networks and Image Synthesis
Advanced Steganography and Watermarking Techniques
Non Fungible Tokens (NFTs) are blockchain-based unique digital assets defining ownership deeds. They can characterize various different objects such as collectible, art, and in-game items. In general, NFTs are encoded by blockchains smart contracts, and they are traded via cryptocurrencies. Their price and investors attention on them has remarkably increased especially in 2021, making them promising alternative class of investment. Surprisingly, predicting their prices has only recently started to be analyzed systematically.
Open access
Blockchain Technology Applications and Security
Art History and Market Analysis
Generative Adversarial Networks and Image Synthesis
Kar Balan, Andrew Gilbert, Alexander Black, Simon Jenni · 6 authors
We present DECORAIT; a decentralized registry through which content creators may assert their right to opt in or out of AI training as well as receive reward for their contributions. Generative AI (GenAI) enables images to be synthesized using AI models trained on vast amounts of data scraped from public sources. Model and content creators who may wish to share their work openly without sanctioning its use for training are thus presented with a data governance challenge. Further, establishing the provenance of GenAI training data is important to creatives to ensure fair recognition and reward for their such use. We report a prototype of DECORAIT, which explores hierarchical clustering and a combination of on/off-chain storage to create a scalable decentralized registry to trace the provenance of GenAI training data in order to determine training consent and reward creatives who contribute that data. DECORAIT combines distributed ledger technology (DLT) with visual fingerprinting, leveraging the emerging C2PA (Coalition for Content Provenance and Authenticity) standard to create a secure, open registry through which creatives may express consent and data ownership for GenAI.
Recently, the intellectual property protection methods based on deep learning have achieved great success, but there are still serious infringement issues that the network topology or hyper parameters of the trained model are stolen by third parties. In this paper, we construct a deep learning model based on the autoencoder to remove the bone from the medical images containing chest x-ray, and the specific trigger set is trained and predicted to get the effect of the backdoor watermark. The scheme of zero-knowledge proof is applied to transform the backdoor watermark of the model into the fixed-length string, which is published in the block chain to verify the ownership of the model. Through the non-interactive verification between the model owner and the third party, the ownership of the model can be confirmed by the third party and the verification process will not disclose any information of the model itself. The method proposed in this paper can support infinite times of verification and does not reveal any information about the model, so as to achieve the protection of intellectual property rights of the model.
Adversarial Robustness in Machine Learning
Generative Adversarial Networks and Image Synthesis
We study the task of generating profitable Non-Fungible Token (NFT) images from user-input texts. Recent advances in diffusion models have shown great potential for image generation. However, existing works can fall short in generating visually-pleasing and highly-profitable NFT images, mainly due to the lack of 1) plentiful and fine-grained visual attribute prompts for an NFT image, and 2) effective optimization metrics for generating high-quality NFT images. To solve these challenges, we propose a Diffusion based generation framework with Multiple Visual-Policies as rewards (i.e., Diffusion-MVP) for NFT images. The proposed framework consists of a large language model (LLM), a diffusion-based image generator, and a series of visual rewards by design. First, the LLM enhances a basic human input (such as "panda") by generating more comprehensive NFT-style prompts that include specific visual attributes, such as "panda with Ninja style and green background." Second, the diffusion-based image generator is fine-tuned using a large-scale NFT dataset to capture fine-grained image styles and accessory compositions of popular NFT elements. Third, we further propose to utilize multiple visual-policies as optimization goals, including visual rarity levels, visual aesthetic scores, and CLIP-based text-image relevances. This design ensures that our proposed Diffusion-MVP is capable of minting NFT images with high visual quality and market value. To facilitate this research, we have collected the largest publicly available NFT image dataset to date, consisting of 1.5 million high-quality images with corresponding texts and market values. Extensive experiments including objective evaluations and user studies demonstrate that our framework can generate NFT images showing more visually engaging elements and higher market value, compared with state-of-the-art approaches.
Open access
3 source records
Generative Adversarial Networks and Image Synthesis
Web3 (also known as Web 3.0) metaverse is a blockchain-driven networked, decentralized, and open virtual world. The key feature of the Web3 metaverse is that the ownership of digital assets is recorded by non-fungible token (NFT) protocol on the blockchain. Thus, users are better encouraged to construct Web3 metaverse due to the ownership of their user-generated content (UGC). However, the existing UGC editors mainly face two challenges: they cannot guarantee the uniqueness of UGC; and they are hard-pressed to find a trade-off between model granularity and 3D modeling difficulty. In this article, we design a novel UGC editor for the Web3 metaverse, named MetaCube, to address these challenges. MetaCube applies an artificial intelligence (AI) method to assist the UGC creation for decreasing the 3D modeling difficulty while maintaining the model granularity. To guarantee the uniqueness of UGC, this article proposes 3D Crypto-dropout, a specially designed dropout that can utilize user information to control the UGC creation process and generate unique fine-grained 3D models. Our experimental results demonstrate that the proposed 3D Crypto-dropout can effectively guarantee the uniqueness of UGC from both numerical and human-centered evaluation. Moreover, the existing challenges and open research topics for the uniqueness of UGC are also profoundly discussed.
Generative Adversarial Networks and Image Synthesis
Yifan Chen, Lei Li, Xinyu Hu, Jiahao Li · 6 authors
The use of Artificial Intelligence (AI) generators to create digital artwork as the content of Non-Fungible Tokens (NFTs) is prevalent. Typically, when minting AI-generated digital artwork into NFTs, the data of digital artwork is stored in the cloud or decentralized storage system, and a Uniform Resource Identifier (URI) or Content Identifier (CID) of the data is stored in the smart contract of NFTs to access the data. This makes AI art NFTs suffer from potential asset loss as conventional NFTs. Can AI be utilized to enhance the availability of AI-generated digital assets as NFT content? In this paper, we propose a new method for minting AI-generated digital assets into NFTs. The key idea of our approach is to store the latent codes of the generated assets on the blockchain instead of URI or CID in conventional NFTs. Here, the latent codes are intermediate variables in the process of generating digital assets by the generator and could restore the assets through the generator. Meanwhile, to be able to restore assets, the universal generator is stored on a distributed system, and its high popularity guarantees its availability. Experiments demonstrate the feasibility of our method. In addition, the integrity and the availability of assets minted by the proposed method and the existing ones are discussed, concluding that our approach has better availability while safeguarding integrity.
Generative Adversarial Networks and Image Synthesis
Kar Balan, Shruti Agarwal, Simon Jenni, Andy Parsons · 6 authors
We present EKILA; a decentralized framework that enables creatives to receive recognition and reward for their contributions to generative AI (GenAI). EKILA proposes a robust visual attribution technique and combines this with an emerging content provenance standard (C2PA) to address the problem of synthetic image provenance -- determining the generative model and training data responsible for an AI-generated image. Furthermore, EKILA extends the non-fungible token (NFT) ecosystem to introduce a tokenized representation for rights, enabling a triangular relationship between the asset's Ownership, Rights, and Attribution (ORA). Leveraging the ORA relationship enables creators to express agency over training consent and, through our attribution model, to receive apportioned credit, including royalty payments for the use of their assets in GenAI.
Open access
3 source records
Generative Adversarial Networks and Image Synthesis
A generative adversarial network is a deep learning model, an unsupervised learning method. In computer vision, the generative adversarial network is a research direction with rapid development in recent years; Similarly, the rise of cryptocurrency Non-Fungible Tokens (NFT) in recent years has also attracted much attention to the field of art. As an "irreplaceable currency" NFT provides a more novel and convenient way for content creators and artists to create and increases the continuous income of original creators. At the same time, it has also attracted widespread attention to the financial field. Therefore, this paper is determined to combine the generative adversarial network of the production of NFT and discuss and analyze the autonomous computer generation of artworks. Firstly, this paper starts with the model's structure, the design of the objective function, Block chain technology, and Irreplaceable tokens encrypted using blockchain technology. Then, the image generated by the whole generative adversarial network and transformed into NFT works are described in detail. In addition, this paper briefly discusses the development ethics of human art and machine art and the prospects for its development trend.
Open access
Generative Adversarial Networks and Image Synthesis
Mar 22, 2023·2023 Joint International Conference on Digital Arts, Media and Technology with ECTI Northern Section Conference on Electrical, Electronics, Computer and Telecommunications Engineering (ECTI DAMT & NCON)
The purpose of this study is to create Digital images in the experimental creation with the support from Artificial Intelligence (AI) that is used to assist in the creation of artwork with frameworks newly designed to reduce time of the creation of digital art works without the decrease of quality of the works but will lever the works without destroying the personalities of the artists and the styles of works. Concerning the contents of the works, Thai arts and culture are used as the contents of contemporary works so that people nowadays still recognize applied arts in new forms. Therefore, there is the cooperation between AI and the artist in the new form of Digital Image, which has been publicized in NFT11Non-fungible token (NFT) is a unique digital identify in a blockchain used to certify authenticity and ownership. In this case is digital art image type. [7](Non-fungible Token) art market in order to get feedback which will be applied for further development. From the creation of the work, it has been discovered that AI generated Art is faster and fits standards. However, there still are many arguments in several issues in the art field. Thus, I, as the Artist, have designed the Frameworks to correct such arguments until this work is accepted and auctioned in NFT market eventually.
Generative Adversarial Networks and Image Synthesis
Blockchain technology is used to support digital assets such as cryptocurrencies and tokens. Commonly, smart contracts are used to generate tokens on top of the blockchain network. There are two fundamental types of tokens: fungible and non-fungible (NFTs). This paper focuses on NFTs and offers a technique to spot plagiarism in NFT images. NFTs are information that is appended to files to produce distinctive signatures. It can be found in image files, real artifacts, literature published online, and various other digital media. Plagiarism and fraudulent NFT images are becoming a big concern for artists and customers. This paper proposes an efficient deep learning-based approach for NFT image plagiarism detection using the EfficientNet-B0 architecture and the Triplet Semi-Hard Loss function. We trained our model using a dataset of NFT images and evaluated its performance using several metrics, including loss and accuracy. The results showed that the EfficientNet-B0-based deep neural network with triplet semi-hard loss outperformed other models such as Resnet50, DenseNet, and MobileNetV2 in detecting plagiarized NFTs. The experimental results demonstrate sufficient to be implemented in various NFT marketplaces.
Open access
Digital Media Forensic Detection
Generative Adversarial Networks and Image Synthesis
Advanced Steganography and Watermarking Techniques
Abstract Advances in Deep Learning (DL), Big Data and image processing have facilitated online disinformation spreading through Deepfakes. This entails severe threats including public opinion manipulation, geopolitical tensions, chaos in financial markets, scams, defamation and identity theft among others. Therefore, it is imperative to develop techniques to prevent, detect, and stop the spreading of deepfake content. Along these lines, the goal of this paper is to present a big picture perspective of the deepfake paradigm, by reviewing current and future trends. First, a compact summary of DL techniques used for deepfakes is presented. Then, a review of the fight between generation and detection techniques is elaborated. Moreover, we delve into the potential that new technologies, such as distributed ledgers and blockchain, can offer with regard to cybersecurity and the fight against digital deception. Two scenarios of application, including online social networks engineering attacks and Internet of Things, are reviewed where main insights and open challenges are tackled. Finally, future trends and research lines are discussed, pointing out potential key agents and technologies.
Open access
2 source records
Generative Adversarial Networks and Image Synthesis
The evolution of technology and science has brought groundbreaking developments to the current visual arts, and the application of digital technology has brought significant changes to the creation and aesthetic taste of traditional art. This study, therefore, investigates the current state-of-the-art artificial intelligence (AI) technologies and applications in generating visual art while giving a brief history of the intersection of AI and art, including the milestone advancements in neural networks. Fifteen interviews were conducted with technical artists who use text-to-image AI generators to gather data. Based on the findings from the interviews, the state-of-the-art applications were reviewed and analyzed in six categories: Accessibility, Barrier to Entry, Novelty, Ethics and Morality, Control, Non-fungible tokens (NFT) and Monetization which were widely discussed along with their success and limitations. The research concludes with three main findings; (a) monetization of digital media through NFTs that has a direct impact on the advancement of art generating AI applications, (b) there is a significant change in the traditional creative process with the integration of AI applications, and AI is not just a tool but it's a creative agent that artists collaborate with (c) art generating AI applications can generate limitless possibilities within the same aesthetics as a result revolutionize the way humans create and interact with art.
Aesthetic Perception and Analysis
Generative Adversarial Networks and Image Synthesis
Daniel Kang, Tatsunori Hashimoto, Ion Stoica, Yi Sun
Over the past few years, AI methods of generating images have been increasing in capabilities, with recent breakthroughs enabling high-resolution, photorealistic "deepfakes" (artificially generated images with the purpose of misinformation or harm). The rise of deepfakes has potential for social disruption. Recent work has proposed using ZK-SNARKs (zero-knowledge succinct non-interactive argument of knowledge) and attested cameras to verify that images were taken by a camera. ZK-SNARKs allow verification of image transformations non-interactively (i.e., post-hoc) with only standard cryptographic hardness assumptions. Unfortunately, this work does not preserve input privacy, is impractically slow (working only on 128$\times$128 images), and/or requires custom cryptographic arguments. To address these issues, we present zk-img, a library for attesting to image transformations while hiding the pre-transformed image. zk-img allows application developers to specify high level image transformations. Then, zk-img will transparently compile these specifications to ZK-SNARKs. To hide the input or output images, zk-img will compute the hash of the images inside the ZK-SNARK. We further propose methods of chaining image transformations securely and privately, which allows for arbitrarily many transformations. By combining these optimizations, zk-img is the first system to be able to transform HD images on commodity hardware, securely and privately.
Open access
2 source records
Digital Media Forensic Detection
Adversarial Robustness in Machine Learning
Generative Adversarial Networks and Image Synthesis
Art is an artistic method of using digital technologies as a part of the generative or creative process. With the advent of digital currency and NFTs (Non-Fungible Token), the demand for digital art is growing aggressively. In this manuscript, we advocate the concept of using deep generative networks with adversarial training for a stable and variant art generation. The work mainly focuses on using the Deep Convolutional Generative Adversarial Network (DC-GAN) and explores the techniques to address the common pitfalls in GAN training. We compare various architectures and designs of DC-GANs to arrive at a recommendable design choice for a stable and realistic generation. The main focus of the work is to generate realistic images that do not exist in reality but are synthesised from random noise by the proposed model. We provide visual results of generated animal face images (some pieces of evidence showing a blend of species) along with recommendations for training, architecture and design choices. We also show how training image preprocessing plays a massive role in GAN training.
Open access
2 source records
Generative Adversarial Networks and Image Synthesis
The purposes are to recognize and classify different music characteristics and strengthen the copyright protection system for original digital music in the big data era. Deep learning (DL) and blockchain technology are applied and researched herein. Based on CNN (Convolutional Neural Network), a music recognition method combined with hashing learning is proposed. The error generated when outputting the binary hash code is considered, and the semantic similarity of the hash code is ensured. Besides, the application of blockchain technology in the current intellectual property protection in original music is discussed. According to digital music property rights protection needs, the system is divided into modules, and its functions are designed. The system ensures its various functions by applying the application protocol designed in the Algor and network. In the experiments, the MagnaTagATune dataset is selected to verify the performance of the proposed CRNNH (Convolutional Recurrent Neural Network Hashing) algorithm. The algorithm shows the best music recognition performance under different bit numbers. When the number of connections is about 100, the QPS value of the blockchain-based music property rights protection system can be stabilized at about 20,000. At any number of threads, the system pressure will increase dramatically with the increase in the number of analog connections. The music recognition algorithm based on DL and hash method discussed is of great significance in improving the classification accuracy of music recognition. The application of blockchain technology in the copyright protection platform of original music works can protect the copyright of digital music and ensure the operation performance of the system.
Open access
Music and Audio Processing
Diverse Musicological Studies
Generative Adversarial Networks and Image Synthesis
Frederik Temmermans, Deepayan Bhowmik, Fernando Pereira, Touradj Ebrahimi
Advances in deep neural networks (DNN) and distributed ledger technology (DLT) have shown major influence on media security, authenticity and privacy. Current deepfake techniques can produce near realistic media content which can be used in both good and bad intended use cases. At the same time, DLTs are finding their way in the industry as fair, transparent and reliable means for content distribution. In particular non-fungible tokens (NFTs) are emerging in the digital art market. However, such new developments also introduce new challenges, including the need for robust and reliable metadata, a mechanism to secure the media and associated metadata, means to verify authenticity and interoperability between various stakeholders. This paper identifies emerging challenges in fake media and NFT, and proposes a novel framework to effectively cope with secure media applications allowing for a structured, systematic, and interoperable solution. The framework relies on an architecture that is modular, flexible, extensible, and scalable in the sense that it can be implemented in both lighter as well as more feature-rich and more complex configurations depending on the underlying application, needed features and available resources, while enabling products and services in various ecosystems with desired trust and security capabilities. The framework is inspired by activities and developments within JPEG standardisation related to security, authenticity and privacy.
Open access
Advanced Steganography and Watermarking Techniques
Generative Adversarial Networks and Image Synthesis