This paper asks what must be added to three-dimensional semantic segmentation before a distributed biological system can be said to organize differentiated, object-specific content. It begins with ordinary object perception and separates class labels, instance identity, border ownership, recurrent completion, multisensory registration, receiver state, Phase Wave Differentials, action, and returned sensory correction. The paper introduces a fifty-equation formal specification and an Object-Boundary Registration Benchmark. A deterministic reference application, fitted synthetic pilots, distribution-shift tests, latent and global alternatives, targeted ablations, calibration analysis, six evidence figures, and a machine-checked finite contract kernel make the proposal auditable. The synthetic results are mixed: typed receiver structure outperforms a global summary, while stronger latent alternatives match or exceed it under some noise and missingness conditions. Those adverse results remain central to the paper. The work therefore presents a testable research program, not completed biological or consciousness validation. A staged biological protocol is frozen, but it has not been run and the final test remains sealed. The public companion archive contains the complete cumulative manuscripts, source and claim ledgers, executable application, tests, structured results, negative-result record, figure provenance, formal proof receipts, and reproducibility instructions.
Open access
2 source records
Cell Image Analysis Techniques
Face Recognition and Perception
Generative Adversarial Networks and Image Synthesis
Suman Bijapur, Shilpa Patil, Parimala, Shantala P H
With unparalleled threats to the integrity of digital information, democratic practices, and public confidence in media, deepfake technology comprises a new class of harm. Deep generative models can be used to generate realistic looking (and sounding) fake human faces and voices, which is great news for bad actors who seek to spread misinformation, commit crimes and ruin journalism. CyberLink Fights Deepfakes with New AI Model That Uses Neural Network Traditional methods used to identify deepfakes have depended on centralised AI systems that can't be trusted at face value and there is little or no way of proving a piece of content's authenticity. In this work, we have presented a solution that involves multi-modal deepfake detection and has utilized learning-based forgery detection framework to be deployed on blockchain for evidence tamper resistance. The proposed framework employs a hybrid CNN-RNN architecture that computes facial, audio and metadata feature in parallel to detect unseen deepfakes with accuracy of 94.2%, compared to the single-modal baselines (CNN only: 81.3%, and audio only: 67.4%). Novelty: Blockchain timestamping with cryptographically secured certificates of authenticity for third-party verification while protected proprietary detection logic is not revealed. At the computational efficiency and bandwidth threshold required for edge deployment, video processing at 30 FPS and only 2.1 MBs makes this applicable on any average mobile device. It holds for 12k synthetic videos (celebrities, politicians, newscasters) and diverse deepfake generation methods (FaceSwap, DeepFaceLab, StyleGAN). Societal impact: framework mitigates $1.2T annual disinformation damage and champions digital rights through decentralized verification. Via mashable.com Framework addresses the convergence of deepfake detection, blockchain authentication and the case for sustainable cybersecurity: As a global community grapples with synthetic media in ways we've never seen before, support online safety experts to respond.
Open access
2 source records
Generative Adversarial Networks and Image Synthesis
Abstract Non-fungible tokens (NFTs) have revolutionized digital ownership, offering unique provenance and value to digital assets. Existing text-to-image models do not have the incentive mechanisms to generate statistically rare features, even when they optimize for visual fidelity. This paper introduces DiffNFTGen, a new generative framework that is the first to combine a customized RarityReward measure derived from a Vision Transformer (ViT) with reinforcement learning. The suggested method ensures fidelity to NFT styles while explicitly maximizing the generation of rare features by fine-tuning Stable Diffusion using Proximal Policy Optimization (PPO) and Kullback-Leibler (KL) divergence regularization. DiNFTGen achieves a 2.4x greater rarity score than baseline models while keeping competitive visual quality, according to quantitative evaluation utilizing Frédechet Inception Distance (FID) and Rarity Score. In order to examine the trade-off between fidelity and rarity, we also perform ablation studies regarding reward weighting. The model’s capacity to generalize NFT styles to new domains is confirmed by qualitative evaluations. The datasets, analysis code, and suggested approach are accessible on https://github.com/seferlab/diffnftgen .
Open access
Cell Image Analysis Techniques
Generative Adversarial Networks and Image Synthesis
Gustav Olaf Yunus Laitinen-Fredriksson Lundström-Imanov, N I Abdullayeva
Generative artificial intelligence now synthesizes photorealistic imagery, audio, and video at a cost that defeats traditional forensic intuition. The legal consequences span three regimes studied so far in isolation: international operational law, domestic procedure, and product regulation. This article presents a unified evidentiary framework that maps cryptographic content provenance, robust statistical watermarking, and zero knowledge attestation to the proof requirements of each regime. We define a five tier threat model spanning naive regeneration, adversarial laundering, cross model regeneration, active watermark removal, and insider provenance forgery. We release a public benchmark of 12000 generated items across image, audio, and video modalities under six laundering pipelines for 72000 evaluation samples. We evaluate four representative schemes and report true positive rate at fixed false positive rate, robustness area under the curve, computational overhead, and a regime conditioned legal sufficiency score. We translate empirical detection bounds into legal sufficiency thresholds for command decisions under the law of armed conflict, for criminal and civil admissibility under domestic procedure, and for persistence audits under the European Union Artificial Intelligence Act and analogous regimes. The result is a reproducible reference pipeline, a public benchmark, and model annexes that lawyers, engineers, and operators can deploy together.
Open access
2 source records
Adversarial Robustness in Machine Learning
Digital Media Forensic Detection
Generative Adversarial Networks and Image Synthesis
Deepfake technology poses a growing threat to digital trust across journalism, law, and politics. Current CNN-based detectors capture local artifacts but struggle with high-quality fakes and offer no way to prove their predictions are genuine. This paper presents DeepTrust, a framework combining a hybrid CNN–Transformer detector with Zero-Knowledge Proof (ZKP) verification and blockchain-based record-keeping. The detection model fuses spatial features from an attention-enhanced Xception network, global context from ViT-B/16, and spectral cues from a Frequency Encoder through a cross-attention mechanism. Predictions are cryptographically committed using a Pedersen scheme with the Fiat-Shamir heuristic, then stored on a proof-of-work blockchain. Evaluated on FaceForensics++, Celeb-DF, DFD, and 140K Real vs Fake, DeepTrust achieves 97.00% accuracy and 0.999 AUC on FaceForensics++, with balanced per-class accuracy despite imbalance ratios up to 1:8.5. ZKP overhead remains below one millisecond per prediction.
Open access
Adversarial Robustness in Machine Learning
Generative Adversarial Networks and Image Synthesis
As neural language models are deployed in regulated domains, verifiable model provenance becomes a critical security requirement. We construct an Inference-Time Physical Unclonable Function (IT-PUF) that provides a challenge-response authentication protocol for neural networks, achieving zero false acceptances across 1,012 comparisons spanning 23 models and 16 vendor families. The IT-PUF derives its entropy from a geometrically intrinsic behavioral fingerprint—the delta-gene (the third pre-softmax logit gap)—which we prove is invariant to inference temperature and empirically validate as invariant across six distinct neural architectures. We provide a formal impossibility result for fingerprint spoofing: an interval-splitting theorem proves that no adversarial Kullback-Leibler (KL) budget can simultaneously close the fingerprint gap and avoid detection via accumulated noise. To establish that this security does not degrade at scale, we validate an Equation of State across three independent model families spanning a 147x parameter range (0.5B to 72B). We falsify the assumption of unbounded stiffness but discover a strict positive empirical floor (S_min = 1.1797), from which the Cramér-Rao bound guarantees a computable minimum spoofing cost. The theoretical foundation is formally verified in the Coq proof assistant: 311 theorems across 16 files, with zero uses of "Admitted" and zero vacuous definitions. The Neural Network Identity Series — Mathematical foundations, empirical validation, and governance frameworks for verifying which model is running Newest addition: Technical Note: The Disappearing Window — AI Logprob Access Withdrawal and the Structural Verifiability of Frontier Model Contracts (DOI: 10.5281/zenodo.20362098) Paper 1: The δ-Gene: Inference-Time Physical Unclonable Functions from Architecture-Invariant Output Geometry (DOI: 10.5281/zenodo.18704275) Paper 2: Template-Based Endpoint Verification via Logprob Order-Statistic Geometry (DOI: 10.5281/zenodo.18776711) Paper 3: The Geometry of Model Theft: Distillation Forensics, Adversarial Erasure, and the Illusion of Spoofing (DOI: 10.5281/zenodo.18818608) Paper 4: Provenance Generalization and Verification Scaling for Neural Network Forensics (DOI: 10.5281/zenodo.18872071) Paper 5: Beneath the Character: The Structural Identity of Neural Networks — Mathematical Evidence for a Non-Narrative Layer of AI Identity (DOI: 10.5281/zenodo.18907292) Paper 6: Which Model Is Running?: Structural Identity as a Prerequisite for Trustworthy Zero-Knowledge Machine Learning (DOI: 10.5281/zenodo.19008116) Paper 7: The Deformation Laws of Neural Identity (DOI: 10.5281/zenodo.19055966) Paper 8: What Counts as Proof? — Admissible Evidence for Neural Network Identity Claims (DOI: 10.5281/zenodo.19058540) Paper 9: Composable Model Identity — Formal Hardening of Structural Attestations in the Enterprise Identity Stack (DOI: 10.5281/zenodo.19099911) Paper 10:Where Identity Comes From: Path Sensitivity and Endpoint Underdetermination in Neural Network Training (DOI: 10.5281/zenodo.19118807) Paper 11: Post-Hoc Disclosure Is Not Runtime Proof: Model Identity at Frontier Scale (DOI: 10.5281/zenodo.19216634) Paper 12: Family-Dependent Response to Reasoning Distillation Across Structural and Functional Identity Layers (DOI: 10.5281/zenodo.19298857) Paper 13: Safety-Alignment Removal as a Model-Identity Failure — Structural Evidence from Published Weight-Level Mutation Checkpoints (DOI: 10.5281/zenodo.19383019) Technical Note: Agent Identity Is Not Model Identity (DOI: 10.5281/zenodo.19240883) Technical Note: Gap Invariance: Why PPP Measurements Are Domain-Independent by Construction (DOI: 10.5281/zenodo.19275524) Technical Note: Measured Model Substitution Under Valid Agent Credentials (DOI: 10.5281/zenodo.19342848) Technical Note: Artifact Identity Is Not Runtime Identity — Trustfall Lite and the Boundary of File-Level Model Verification (DOI: 10.5281/zenodo.20019127) Formal Verification Stack for Neural Network Structural Identity (IT-PUF Coq Proofs) (DOI: 10.5281/zenodo.18930621) Copyright (c) 2026 Anthony Ray Coslett / Fall Risk AI, LLC. All Rights Reserved. Confidential and Proprietary. Patent Pending (Applications 63/982,893, 63/990,487, 63/996,680, 64/003,244).
Open access
2 source records
Physical Unclonable Functions (PUFs) and Hardware Security
Adversarial Robustness in Machine Learning
Generative Adversarial Networks and Image Synthesis
[Depreciated and replaced by V3] The application-specific clean rebuild has not yet been published; its authoritative theoretical boundary is now the governing V3 branch: After Turing: The Fold Machine - An Exact, Parameter-Free and Machine-Closed Derivation of Classical Computational Science from Smithian Fold Theory; From Fold to Consciousness: An Exact, Zero-Parameter and Machine-Closed Foundational Reconstruction of Consciousness and Cognitive Science from Smithian Fold Theory. The V3 source platform is https://github.com/MettaMazza/ernos-labs-sft-platform. The original DOI, concept DOI, version number and files are preserved for transparent historical provenance; this record must not be presented or cited as current V3 work. v4.0 — the word-scale gap closes within the fold. Rung 5e (pre-registered): the fold-factor mixing law — every context level that holds contributes, weighted 2^level, the engine's own forced halving constant — carries the pure counted engine, with no twin, no prose flood, zero training and zero parameters, past the gradient-trained transformer at word scale: cross-entropy 3.1907 vs the same-day twin's 3.4292 (replicated across two independent anchorings; stacked with the Rung 5d extraction: 3.1344). Both scales of the task gate now belong to the counted engine. Rung 5d's transfer-in verdict is SUPPORTED across three independent arena anchorings in one day. New in the architecture: tool graduation (acts held, values never — a question territory that a tool answered once runs the tool itself thereafter, fresh), recall as regeneration across every memory tier, and judge-independent graduation scoring. End-to-end verification: 36/36. v3.4: Rung 5d, the transfer-in — pre-registered verdict SUPPORTED: the trained twin's dyadically-loud fold content is extracted and installed INTO the counted engine as a counted prior with zero new parameters, closing 55.6/87.9/101.4% of the available gap at k=16/32/64 while the random-truncated null closes 10.1/24.5/56.5%; at half budget the loud shape beats the full twin's own. The word-scale rematch is recorded in full (twin retrained on today's text; decomposition included). Also: judge-independent graduation scoring (boot-discovered pool, cycle-parity alternation), multi-orbit binding (XI-4 in full), recall-is-regeneration (a held experience re-walks its own orbit, never reprinted), the public SOTA table beside the local giants with cited published figures, and one-command replication kits (GPT-2 weights auto-fetch; 13/13, 39/39 proven on a fresh clone). End-to-end verification: 36/36. Full paper v1.1 — supersedes the pre-paper (From One Axiom to Master-Level Chess — and the Law Inside Neural Networks). Built from scratch by one woman, working alone, in under twenty-four accumulated hours: where a score falls short it marks an implementation gap at measurement time, never a limit of the mathematics — the gains between releases are the finding. v1.4 adds the fold eye (vision as exact integer Walsh spectra, self-certified by integer Parseval per image, recognition of seen images with no image model in the loop) and the graduation score (blind head-to-head vs the teacher, tallied per question-territory; the teacher retires as wins cross the majority lock) -- and documents the 2026 convergence: DeepSeek Engram arrives at deterministically-addressed exact memory from the gradient side, and two independent results place the optimal curriculum at p = 1/2, the fold lock. v1.6: the full omnimodal engine (the voice via Kokoro, the fold ear -- sound as Parseval-certified integer Walsh spectra, video composed from frames + sound), speaker-transparent reasoning threads, and 32/32 end-to-end empirical verification of the entire architecture including persistence across process death. v1.7: removal-proof omnimodality, measured -- every supporting model is a teacher with an exit: a sound taught once by the synthesis teacher is re-spoken from the engine's own exact counted record in 0.00s with no model; a sound heard once is recognized natively with no transcriber; 34/34 end-to-end verification. v1.9: zero-model perceptual learning (the human observer -- a novel image learned and re-recognized at share 1.00 with no model in the loop); agentic self-knowledge (the observer reads the engine's own source, measured); the hourly progress instrument with a committed pre-boot birth line; one-tap y/n closure. v2.0 (flight-ready): the full modern-agent toolkit (live web search/fetch, paginated reading, in-file grep -- every call held as a training trace), the 43-domain everything-curriculum under the fold-only law, SOTA 1-1 benching on the public MMLU test split with the newborn baseline committed, generation closure (the Learning Law reaches generate() itself), and 36/36 end-to-end verification. v2.1: the ReAct law (reason-act-observe enforced in-turn; narrated intent without an act is detected and forced), reasoning trained on the observer's NATIVE thinking tokens (STaR-gated) with both minds' full thinking streamed to the user, and document intake (a sent file is reading -- inboxed, counted, persistent). v2.2: the identity stated correctly -- UnisonAI is an OMNI MODEL (language, sight, hearing, speech, and video on one held memory), not a language model; LLMs remain the contrast class only. v3.0: the full-altitude rewrite -- the complete omni model documented at the same depth as the spectral science: thirteen sections, the architecture organ by organ with every measurement, Rung 5c as its own section, the empirical record and its committed birth line, 36/36 end-to-end verification, and the 2026 convergence. This paper is a PROOF of The Smithian Fold Theory of Everything, not the main event: the theory (one axiom, zero free parameters, 1,844 machine-verified forced checks) is at DOI 10.5281/zenodo.21182469 and github.com/MettaMazza/Smithian-Fold-Theory-Of-Everything -- run the prover yourself. The engine: github.com/MettaMazza/UnisonAI. v3.3: the LLM-native presence suite -- the exact registered protocol applied to GPT-2's entire knowledge-storage class: 13/13 tensors, 39/39 checks, unanimous (margins 3.4-79.3x); the flagship claim now rests on the flagship objects, with diffusion/speech models recast as cross-domain breadth. Three connected results and the architecture they force. First, a pre-registered, self-certifying spectral instrument shows trained neural-network weights carry placement-law in the dyadic (Walsh) basis: 18/18 unanimous on validated released models; the law concentrated in transformer expansion projections and token embeddings across three unrelated architectures (up to 230x chance in GPT-2), attention at chance; strictly training-caused (He-initialised controls at 1.0x); surviving 4-bit deployment quantization. A recipe map from 124M to one trillion parameters shows the law tracks training recipe, not scale or architecture — strongest carrier DeepSeek-R1-671B at 43–47x — and loud-recipe weights transform under the fold's transformation group exactly as solved game-theoretic value fields do. Second, the "learned similarity space" is a counted object: word kinship as exact co-occurrence shares reproduces semantic family structure (quark → lepton, neutrino, proton) with zero parameters and zero gradients. Third, UnisonAI: a complete language architecture in which every LLM mechanism — memory, attention, similarity, learning, prediction, generation — is replaced by a machine-verified law of the Smithian Fold Theory, zero trained parameters end to end. On identical held-out text the fold-native engine outperformed its trained transformer twin (cross-entropy 1.289 vs 1.888) after reading the corpus once (26 seconds) against 48,000 gradient readings (21 minutes per seed). Deployed as a live, continuously-learning agent whose teaching loop also runs autonomously: a teacher model asks, judges, and closes the learning law itself, and the engine self-plays against its own held lessons. Negative results reported in full with their scopes. Companion to The Smithian Fold Theory of Everything (DOI: 10.5281/zenodo.21182469; 307 suites, 1,844 forced checks, 0 failures). Engine and records: github.com/MettaMazza/UnisonAI and github.com/MettaMazza/Smithian-Fold-Theory-Of-Everything.
Open access
8 source records
Explainable Artificial Intelligence (XAI)
Generative Adversarial Networks and Image Synthesis
Ahmet Ramazan Ağırtaş, Arda Buğra Özer, Zülfükar SAYGI, Oğuz Yayla
Unbiased and unpredictable randomness is a cornerstone of Web3 security, underpinning everything from consensus protocols to DeFi logic. Although Distributed Verifiable Random Functions (DVRFs) eliminate central points of failure, current designs often have to compromise performance. Most existing protocols are hindered by one of three limitations: proofs that scale linearly with the number of participants, high computational cost of bilinear pairings, or latency introduced by mandatory interactive steps during generation. In this work, we present Icy-DVRF, a protocol that improves DVRFwCP by employing a preprocessing scheme similar to FROST to reduce the number of interaction rounds among participants and lowering the additional communication cost from <inline-formula> <tex-math notation="LaTeX">$O(n^{2} t)$ </tex-math></inline-formula> to <inline-formula> <tex-math notation="LaTeX">$O(t)$ </tex-math></inline-formula> while maintaining constant-size proofs. The downside of our construction is that, relative to DDH-DVRF and GLOW-DVRF, this approach incurs an additional off-chain communication round due to the threshold structure of our non-interactive zero-knowledge proof. This architecture ensures that verification costs remain low, regardless of the set of participants. While theoretical estimates suggest verification costs of approximately one quarter of those of standard designs, our empirical benchmarks on the Sepolia testnet, utilizing the EIP-2537: Precompile for BLS12-381 curve operations, confirm that Icy-DVRF requires only 88,803 gas for full execution. This represents a significant 43.02% reduction in total gas consumption compared to existing pairing-based constructions, saving 67,035 gas per on-chain verification. Off-chain, eliminating DVRFwCP’s Augmented Secure-DKG round yields a per-node speedup ranging from a factor of 1.46 at <inline-formula> <tex-math notation="LaTeX">$(n,t)=(5,3)$ </tex-math></inline-formula> to a factor of 4.43 at <inline-formula> <tex-math notation="LaTeX">$(n,t)=(50,34)$ </tex-math></inline-formula>.
Open access
Generative Adversarial Networks and Image Synthesis
The DeepFake tech has had a theatrical impact on the visual arts, not only the provision of creative technology, but also the question of authenticity, copyright and misinformation. The deep learning and generative adversarial networks (GANs) produce deepfakes artificial images, which are extremely harmful to art. The article discusses the DeepFake detection and management within visual art work with emphasis on the practical application of analysis through multiple-layered approaches that would assist in ensuring the presence of the digital authenticity. DeepFake was managed through three core approaches, namely AI-Based Detection Frameworks, Blockchain-Based Authentication System, and Human-AI Collaborative Review Models. The decentralized strategy was based on blockchain technology, which was the Non-Fungible Token (NFT) registration by the cryptographic hashing to authenticate the provenance and ownership of the artworks. The human-AI composite system has integrated the inspection of the specialists on the visual level with the automatic monitoring of the anomalies to increase the readability and reduce the number of false alarms. The experiment revealed that the AI-based systems, blockchain approaches, and the collusion between human beings and AI detected 92.3, 87.6 and 94.1 % of people respectively. These findings suggest that the incorporation of algorithmic intelligence, a safe check, and human knowledge can help in quite a powerful DeepFake verification and management in the field of visual arts.
Open access
2 source records
Aesthetic Perception and Analysis
Digital Media and Visual Art
Generative Adversarial Networks and Image Synthesis
Mr. DEVENDAR, Nandi J. Reddy, B.Sahasra, T.Srileka
Artificial intelligence and the quick development of photograph editing software in latest years have made it very simple to regulate virtual pix covertly. The authenticity and dependability of digital media utilized in social networks, journalism, and criminal proof have come below scrutiny because of manipulations like copy-circulate forgery and deepfake creation. The aim of this work is to perceive photograph forgeries via combining deep gaining knowledge of-based class techniques with traditional feature extraction methods.The cautioned device extracts precise neighborhood functions from input images the usage of the oriented speedy and turned around brief (ORB) algorithm. For powerful feature matching, 2-Nearest Neighbor (2NN) and Hierarchical Agglomerative Clustering (HAC) are then used. A Convolutional Neural community (CNN) model is trained to distinguish among authentic and manipulated photos by means of figuring out pixel-degree irregularities and texture changes if you want to growth type accuracy. examined on the publicly reachable MICC-F220 and MICC-F2000 datasets, the device outperforms baseline SVM strategies with a ninety% detection accuracy and a zero.1 false tremendous charge
Open access
Digital Media Forensic Detection
Generative Adversarial Networks and Image Synthesis
A. G. Ramakrishnan, Shubham Agarwal, Sharmila Kumari Selvanayagam, Kunwar P. Singh
As image generation models grow increasingly powerful and accessible, concerns around authenticity, ownership, and misuse of synthetic media have become critical. The ability to generate lifelike images indistinguishable from real ones introduces risks such as misinformation, deepfakes, and intellectual property violations. Traditional watermarking methods either degrade image quality, are easily removed, or require access to confidential model internals – making them unsuitable for secure and scalable deployment. We are the first to introduce ZK-WAGON, a novel system for watermarking image generation models using the Zero-Knowledge Succinct Non-Interactive Argument of Knowledge (ZK-SNARKs). Our approach enables verifiable proof of origin without exposing model weights, generation prompts, or any sensitive internal information. We propose Selective Layer ZK-Circuit Creation (SL-ZKCC), a method to selectively convert key layers of an image generation model into a circuit, reducing proof generation time significantly. Generated ZK-SNARK proofs are imperceptibly embedded into a generated image via Least Significant Bit (LSB) steganography. We demonstrate this system on both GAN and Diffusion models, providing a secure, model-agnostic pipeline for trustworthy AI image generation.
Open access
2 source records
Physical Unclonable Functions (PUFs) and Hardware Security
Adversarial Robustness in Machine Learning
Generative Adversarial Networks and Image Synthesis
Hasib Ahmed Md Khyrul Islam, Huy T. Vo, Aditya Rane
In the era of synthetic media, deepfake manipulations pose a significant threat to information integrity. To address this challenge, we propose TrustDefender, a two-stage framework comprising (i) a lightweight convolutional neural network (CNN) that detects deepfake imagery in real-time extended reality (XR) streams, and (ii) an integrated succinct zero-knowledge proof (ZKP) protocol that validates detection results without disclosing raw user data. Our design addresses both the computational constraints of XR platforms while adhering to the stringent privacy requirements in sensitive settings. Experimental evaluations on multiple benchmark deepfake datasets demonstrate that TrustDefender achieves 95.3% detection accuracy, coupled with efficient proof generation underpinned by rigorous cryptography, ensuring seamless integration with high-performance artificial intelligence (AI) systems. By fusing advanced computer vision models with provable security mechanisms, our work establishes a foundation for reliable AI in immersive and privacy-sensitive applications.
Open access
2 source records
Adversarial Robustness in Machine Learning
Digital Media Forensic Detection
Generative Adversarial Networks and Image Synthesis
Abdullah Ayub Khan, Asif Ali Laghari, Syed Azeem Inam, Sajid Ullah · 6 authors
Rapid technological breakthroughs in recent years, like Deepfake, have made it feasible to produce synthetic media that is remarkably lifelike, but they also present significant hazards to public trust, privacy, and security. This survey paper reviews the latest techniques for detecting deepfakes, focussing on important components as image and video manipulation, audio spoofing, and multimodal synthesis. It features state-of-the-art methods including machine learning (ML), deep learning (DL), and multimodal architectures that are especially made to address the previously described deepfake criteria. The report provides a critical review of assessment measures used to assess detection model performance, including precision, accuracy, recall, computing effectiveness and efficiency, and fast responses to adversarial attacks. In order to assist direct future research, this highlights recent advancements in the subject, including explainable AI, federated learning, and self-supervised learning hierarchy. In order to examine the problems with adversarial attacks, scalability across different datasets, and the ethical implications of detection techniques, it is also vital to look into the technological and societal challenges surrounding multimedia-enabled deepfake detection. In particular, the usage of Blockchain Distributed Ledger Technology (BDLT) for traceability, lightweight modelling, and resilient systems forms for cross-model deepfake evaluation are discussed in this review study along with potential solutions to these limitations and areas for further research. This paper offers a comprehensive resource for future research, experts, and practitioners looking to combat the growing threat of deepfake, especially in the social media space, using innovative and useful detection tools.
Open access
Digital Media Forensic Detection
Generative Adversarial Networks and Image Synthesis
Given a JPEG pipeline (compression or decompression), this paper demonstrates how to find the antecedent of an 8x8 block. If it exists, the block is considered compatible with the pipeline. For unaltered images, all blocks remain compatible with the original pipeline; however, for manipulated images, this is not necessarily true. This article provides a first demonstration of the potential of compatibility-based approaches for JPEG image forensics. It introduces a method to address the key challenge of finding a block antecedent in a high-dimensional space, relying on a local search algorithm with restrictions on the search space. We show that inpainting, copy-move, and splicing, when applied after JPEG compression, result in three distinct mismatch problems that can be detected. In particular, if the image is re-compressed after modification, the manipulation can be detected when the quality factor of the second compression is higher than that of the first. Through extensive experiments, we highlight the potential of this compatibility attack under varying degrees of assumptions. While our approach shows promising results-outperforming three state-of-the-art deep learning models in an idealized setting-it remains a proof of concept rather than an off-the-shelf forensic tool. Notably, with a perfect knowledge of the JPEG pipeline, our method guarantees zero false alarms in block-by-block localization, given sufficient computational power.
Open access
Digital Media Forensic Detection
Generative Adversarial Networks and Image Synthesis
Artists are digitally disenfranchised, rampant online sharing and ease of copying make protecting their work from unauthorized use an uphill battle. Digital art is effortlessly duplicated, shared, and manipulated without credit or compensation, even leading to individuals profiting off stolen work. This widespread issue, particularly plaguing social media, demands a solution. We propose a blockchain-based platform utilizing Non-Fungible Tokens (NFTs) to empower artists. By creating NFTs for their art, they claim irrefutable ownership, authenticity, and copyright, enabling secure sales via cryptocurrencies and combating unauthorized use. This innovative approach empowers artists to thrive in the digital age
Open access
Visual Attention and Saliency Detection
Generative Adversarial Networks and Image Synthesis
Arash Heidari, Nima Jafari Navimipour, Hasan Dağ, Samira Talebi · 5 authors
Abstract In recent years, the proliferation of deep learning (DL) techniques has given rise to a significant challenge in the form of deepfake videos, posing a grave threat to the authenticity of media content. With the rapid advancement of DL technology, the creation of convincingly realistic deepfake videos has become increasingly prevalent, raising serious concerns about the potential misuse of such content. Deepfakes have the potential to undermine trust in visual media, with implications for fields as diverse as journalism, entertainment, and security. This study presents an innovative solution by harnessing blockchain-based federated learning (FL) to address this issue, focusing on preserving data source anonymity. The approach combines the strengths of SegCaps and convolutional neural network (CNN) methods for improved image feature extraction, followed by capsule network (CN) training to enhance generalization. A novel data normalization technique is introduced to tackle data heterogeneity stemming from diverse global data sources. Moreover, transfer learning (TL) and preprocessing methods are deployed to elevate DL performance. These efforts culminate in collaborative global model training zfacilitated by blockchain and FL while maintaining the utmost confidentiality of data sources. The effectiveness of our methodology is rigorously tested and validated through extensive experiments. These experiments reveal a substantial improvement in accuracy, with an impressive average increase of 6.6% compared to six benchmark models. Furthermore, our approach demonstrates a 5.1% enhancement in the area under the curve (AUC) metric, underscoring its ability to outperform existing detection methods. These results substantiate the effectiveness of our proposed solution in countering the proliferation of deepfake content. In conclusion, our innovative approach represents a promising avenue for advancing deepfake detection. By leveraging existing data resources and the power of FL and blockchain technology, we address a critical need for media authenticity and security. As the threat of deepfake videos continues to grow, our comprehensive solution provides an effective means to protect the integrity and trustworthiness of visual media, with far-reaching implications for both industry and society. This work stands as a significant step toward countering the deepfake menace and preserving the authenticity of visual content in a rapidly evolving digital landscape.
Open access
Digital Media Forensic Detection
Generative Adversarial Networks and Image Synthesis
Advanced Steganography and Watermarking Techniques
Non Fungible Tokens (NFTs) are blockchain-based unique digital assets defining ownership deeds. They can characterize various different objects such as collectible, art, and in-game items. In general, NFTs are encoded by blockchains smart contracts, and they are traded via cryptocurrencies. Their price and investors attention on them has remarkably increased especially in 2021, making them promising alternative class of investment. Surprisingly, predicting their prices has only recently started to be analyzed systematically.
Open access
Blockchain Technology Applications and Security
Art History and Market Analysis
Generative Adversarial Networks and Image Synthesis
Kar Balan, Andrew Gilbert, Alexander Black, Simon Jenni · 6 authors
We present DECORAIT; a decentralized registry through which content creators may assert their right to opt in or out of AI training as well as receive reward for their contributions. Generative AI (GenAI) enables images to be synthesized using AI models trained on vast amounts of data scraped from public sources. Model and content creators who may wish to share their work openly without sanctioning its use for training are thus presented with a data governance challenge. Further, establishing the provenance of GenAI training data is important to creatives to ensure fair recognition and reward for their such use. We report a prototype of DECORAIT, which explores hierarchical clustering and a combination of on/off-chain storage to create a scalable decentralized registry to trace the provenance of GenAI training data in order to determine training consent and reward creatives who contribute that data. DECORAIT combines distributed ledger technology (DLT) with visual fingerprinting, leveraging the emerging C2PA (Coalition for Content Provenance and Authenticity) standard to create a secure, open registry through which creatives may express consent and data ownership for GenAI.
We study the task of generating profitable Non-Fungible Token (NFT) images from user-input texts. Recent advances in diffusion models have shown great potential for image generation. However, existing works can fall short in generating visually-pleasing and highly-profitable NFT images, mainly due to the lack of 1) plentiful and fine-grained visual attribute prompts for an NFT image, and 2) effective optimization metrics for generating high-quality NFT images. To solve these challenges, we propose a Diffusion based generation framework with Multiple Visual-Policies as rewards (i.e., Diffusion-MVP) for NFT images. The proposed framework consists of a large language model (LLM), a diffusion-based image generator, and a series of visual rewards by design. First, the LLM enhances a basic human input (such as "panda") by generating more comprehensive NFT-style prompts that include specific visual attributes, such as "panda with Ninja style and green background." Second, the diffusion-based image generator is fine-tuned using a large-scale NFT dataset to capture fine-grained image styles and accessory compositions of popular NFT elements. Third, we further propose to utilize multiple visual-policies as optimization goals, including visual rarity levels, visual aesthetic scores, and CLIP-based text-image relevances. This design ensures that our proposed Diffusion-MVP is capable of minting NFT images with high visual quality and market value. To facilitate this research, we have collected the largest publicly available NFT image dataset to date, consisting of 1.5 million high-quality images with corresponding texts and market values. Extensive experiments including objective evaluations and user studies demonstrate that our framework can generate NFT images showing more visually engaging elements and higher market value, compared with state-of-the-art approaches.
Open access
3 source records
Generative Adversarial Networks and Image Synthesis
Kar Balan, Shruti Agarwal, Simon Jenni, Andy Parsons · 6 authors
We present EKILA; a decentralized framework that enables creatives to receive recognition and reward for their contributions to generative AI (GenAI). EKILA proposes a robust visual attribution technique and combines this with an emerging content provenance standard (C2PA) to address the problem of synthetic image provenance -- determining the generative model and training data responsible for an AI-generated image. Furthermore, EKILA extends the non-fungible token (NFT) ecosystem to introduce a tokenized representation for rights, enabling a triangular relationship between the asset's Ownership, Rights, and Attribution (ORA). Leveraging the ORA relationship enables creators to express agency over training consent and, through our attribution model, to receive apportioned credit, including royalty payments for the use of their assets in GenAI.
Open access
3 source records
Generative Adversarial Networks and Image Synthesis
A generative adversarial network is a deep learning model, an unsupervised learning method. In computer vision, the generative adversarial network is a research direction with rapid development in recent years; Similarly, the rise of cryptocurrency Non-Fungible Tokens (NFT) in recent years has also attracted much attention to the field of art. As an "irreplaceable currency" NFT provides a more novel and convenient way for content creators and artists to create and increases the continuous income of original creators. At the same time, it has also attracted widespread attention to the financial field. Therefore, this paper is determined to combine the generative adversarial network of the production of NFT and discuss and analyze the autonomous computer generation of artworks. Firstly, this paper starts with the model's structure, the design of the objective function, Block chain technology, and Irreplaceable tokens encrypted using blockchain technology. Then, the image generated by the whole generative adversarial network and transformed into NFT works are described in detail. In addition, this paper briefly discusses the development ethics of human art and machine art and the prospects for its development trend.
Open access
Generative Adversarial Networks and Image Synthesis
Blockchain technology is used to support digital assets such as cryptocurrencies and tokens. Commonly, smart contracts are used to generate tokens on top of the blockchain network. There are two fundamental types of tokens: fungible and non-fungible (NFTs). This paper focuses on NFTs and offers a technique to spot plagiarism in NFT images. NFTs are information that is appended to files to produce distinctive signatures. It can be found in image files, real artifacts, literature published online, and various other digital media. Plagiarism and fraudulent NFT images are becoming a big concern for artists and customers. This paper proposes an efficient deep learning-based approach for NFT image plagiarism detection using the EfficientNet-B0 architecture and the Triplet Semi-Hard Loss function. We trained our model using a dataset of NFT images and evaluated its performance using several metrics, including loss and accuracy. The results showed that the EfficientNet-B0-based deep neural network with triplet semi-hard loss outperformed other models such as Resnet50, DenseNet, and MobileNetV2 in detecting plagiarized NFTs. The experimental results demonstrate sufficient to be implemented in various NFT marketplaces.
Open access
Digital Media Forensic Detection
Generative Adversarial Networks and Image Synthesis
Advanced Steganography and Watermarking Techniques
Abstract Advances in Deep Learning (DL), Big Data and image processing have facilitated online disinformation spreading through Deepfakes. This entails severe threats including public opinion manipulation, geopolitical tensions, chaos in financial markets, scams, defamation and identity theft among others. Therefore, it is imperative to develop techniques to prevent, detect, and stop the spreading of deepfake content. Along these lines, the goal of this paper is to present a big picture perspective of the deepfake paradigm, by reviewing current and future trends. First, a compact summary of DL techniques used for deepfakes is presented. Then, a review of the fight between generation and detection techniques is elaborated. Moreover, we delve into the potential that new technologies, such as distributed ledgers and blockchain, can offer with regard to cybersecurity and the fight against digital deception. Two scenarios of application, including online social networks engineering attacks and Internet of Things, are reviewed where main insights and open challenges are tackled. Finally, future trends and research lines are discussed, pointing out potential key agents and technologies.
Open access
2 source records
Generative Adversarial Networks and Image Synthesis
Daniel Kang, Tatsunori Hashimoto, Ion Stoica, Yi Sun
Over the past few years, AI methods of generating images have been increasing in capabilities, with recent breakthroughs enabling high-resolution, photorealistic "deepfakes" (artificially generated images with the purpose of misinformation or harm). The rise of deepfakes has potential for social disruption. Recent work has proposed using ZK-SNARKs (zero-knowledge succinct non-interactive argument of knowledge) and attested cameras to verify that images were taken by a camera. ZK-SNARKs allow verification of image transformations non-interactively (i.e., post-hoc) with only standard cryptographic hardness assumptions. Unfortunately, this work does not preserve input privacy, is impractically slow (working only on 128$\times$128 images), and/or requires custom cryptographic arguments. To address these issues, we present zk-img, a library for attesting to image transformations while hiding the pre-transformed image. zk-img allows application developers to specify high level image transformations. Then, zk-img will transparently compile these specifications to ZK-SNARKs. To hide the input or output images, zk-img will compute the hash of the images inside the ZK-SNARK. We further propose methods of chaining image transformations securely and privately, which allows for arbitrarily many transformations. By combining these optimizations, zk-img is the first system to be able to transform HD images on commodity hardware, securely and privately.
Open access
2 source records
Digital Media Forensic Detection
Adversarial Robustness in Machine Learning
Generative Adversarial Networks and Image Synthesis