With the growing popularity of cloud services, companies and individuals outsource images to cloud servers to reduce storage and computing burdens. The images are encrypted before outsourcing for privacy protection. It has become urgent to solve the privacy-preserving image retrieval problem on the cloud. There are three main challenges in this area. First, how can we achieve high retrieval accuracy on the encryption domain? Second, how can we improve efficiency in large-scale encrypted image retrieval? Third, how can we ensure the reliability of the retrieval results? The existing schemes only consider some of these characteristics and the retrieval accuracy is insufficient. In this paper, we propose a privacy-preserving large-scale image retrieval framework with vision graph convolutional neural network hashing (ViGH). To the best of our knowledge, this is the first framework that is able to address all the above challenges with more advanced accuracy performance. To be specific, cycle-consistent adversarial networks and vision graph convolutional networks (ViG) are utilized to increase retrieval accuracy. By embedding encrypted images into hash codes, we can obtain high retrieval efficiency by Hamming distances. Cloud servers store the hash codes on the blockchain (Ethereum). The retrieval algorithm on the smart contracts and the consensus mechanism of blockchain ensure reliability of the retrieval results. The experimental results on three common datasets verify the effectiveness and efficiency of the proposed privacy-preserving image retrieval framework. The reliability of the retrieval results is ensured by the consensus mechanism of blockchain with no need for verification.
Yilin Ye, Qian Zhu, Shishi Xiao, Kang Zhang · 5 authors
Image search is an essential and user-friendly method to explore vast galleries of digital images. However, existing image search methods heavily rely on proximity measurements like tag matching or image similarity, requiring precise user inputs for satisfactory results. To meet the growing demand for a contemporary image search engine that enables accurate comprehension of users' search intentions, we introduce an innovative user intent expansion framework. Our framework leverages visual-language models to parse and compose multi-modal user inputs to provide more accurate and satisfying results. It comprises two-stage processes: 1) a parsing stage that incorporates a language parsing module with large language models to enhance the comprehension of textual inputs, along with a visual parsing module that integrates an interactive segmentation module to swiftly identify detailed visual elements within images; and 2) a logic composition stage that combines multiple user search intents into a unified logic expression for more sophisticated operations in complex searching scenarios. Moreover, the intent expansion framework enables users to perform flexible contextualized interactions with the search results to further specify or adjust their detailed search intents iteratively. We implemented the framework into an image search system for NFT (non-fungible token) search and conducted a user study to evaluate its usability and novel properties. The results indicate that the proposed framework significantly improves users' image search experience. Particularly the parsing and contextualized interactions prove useful in allowing users to express their search intents more accurately and engage in a more enjoyable iterative search experience.
In this work, we present SONAR, a web-based tool for multimodal exploration of Non-Fungible Token (NFT) inspiration networks. SONAR is conceived to support both creators and traders in the emerging Web3 by providing an interactive visualization of the inspiration-driven connections between NFTs, at both individual level and collection level. SONAR can hence be useful to identify new investment opportunities as well as anomalous inspirations. To demonstrate SONAR's capabilities, we present an application to the largest and most representative dataset concerning the NFT landscape to date, showing how our proposed tool can scale and ensure high-level user experience up to millions of edges.
We study the task of generating profitable Non-Fungible Token (NFT) images from user-input texts. Recent advances in diffusion models have shown great potential for image generation. However, existing works can fall short in generating visually-pleasing and highly-profitable NFT images, mainly due to the lack of 1) plentiful and fine-grained visual attribute prompts for an NFT image, and 2) effective optimization metrics for generating high-quality NFT images. To solve these challenges, we propose a Diffusion based generation framework with Multiple Visual-Policies as rewards (i.e., Diffusion-MVP) for NFT images. The proposed framework consists of a large language model (LLM), a diffusion-based image generator, and a series of visual rewards by design. First, the LLM enhances a basic human input (such as "panda") by generating more comprehensive NFT-style prompts that include specific visual attributes, such as "panda with Ninja style and green background." Second, the diffusion-based image generator is fine-tuned using a large-scale NFT dataset to capture fine-grained image styles and accessory compositions of popular NFT elements. Third, we further propose to utilize multiple visual-policies as optimization goals, including visual rarity levels, visual aesthetic scores, and CLIP-based text-image relevances. This design ensures that our proposed Diffusion-MVP is capable of minting NFT images with high visual quality and market value. To facilitate this research, we have collected the largest publicly available NFT image dataset to date, consisting of 1.5 million high-quality images with corresponding texts and market values. Extensive experiments including objective evaluations and user studies demonstrate that our framework can generate NFT images showing more visually engaging elements and higher market value, compared with state-of-the-art approaches.
Open access
3 source records
Generative Adversarial Networks and Image Synthesis
Portfolio management is essential for any investment decision. Yet, traditional methods in the literature are ill-suited for the characteristics and dynamics of cryptocurrencies. This work presents a method to build an investment portfolio consisting of more than 1500 cryptocurrencies covering 6 years of market data. It is centred around Topological Data Analysis (TDA), a recent approach to analyze data sets from the perspective of their topological structure. This publication proposes a system combining persistence landscapes to identify suitable investment opportunities in cryptocurrencies. Using a novel and comprehensive data set of cryptocurrency prices, this research shows that the proposed system enables analysts to outperform a classic method from the literature without requiring any feature engineering or domain knowledge in TDA. This work thus introduces TDA-based portfolio management of cryptocurrencies as a viable tool for the practitioner.
The Netlab project is part of an ongoing effort in image informatics that began in early 1985. This early work resulted in the development of the first digital histology atlas system, which was presented at a NASA conference in 1987. That system used a technology called MetaMAP to make every pixel of the image an independently‐addressable hotspot, linkable to histology knowledge data relating to the cell or structure at that location in the image. The MetaMAP system was then applied to larger anatomy imaging projects, first in the Visible Embryo Project, at UCSF, in 1994, then in the Muritech Atlas of Mouse Development, in 1996, and the Eolas AnatLab project in 1998 and later. In the AnatLab system, MetaMAP allowed the creation of over 800,000 3D anatomical annotations to be made on the image data from the Visible Human Project. The scale of that effort exposed the extreme difficulty involved in validating this kind of data, even when the annotators were trained anatomy students supervised by a professor. The Netlab system is an attempt to lay the groundwork for both building virtual microscopy resources of unlimited scale, and leveraging the power of crowdsourcing to facilitate detailed and validated annotation of the microscopic image data accessible through the system. The Netlab client software allows users to access Aperio‐scanned slide image data streamed in real time over the cloud from the Netlab servers to either a Web browser or a mobile app. Users can pan and zoom in and out across the magnification ranges typical of microscopes in the lab, and can place textual annotations at any locations on any slides in the collection. Each annotation can be classified as private, group or public. Any user with access permission to view another user's annotation can validate that annotation. Each validation is assigned a weight based upon the role of the validator (peer or professor, for example). Group moderators (e.g., professors) can set thresholds for validation. Annotations above the threshold have a “verified valid” icon displayed when viewed. Future work will focus on both expanding the image collection and the development of a more automated validation system, using an approach inspired by the proof‐of‐stake distributed consensus algorithms currently popular in the blockchain technology field. This abstract is from the Experimental Biology 2018 Meeting. There is no full text article associated with this abstract published in The FASEB Journal .
In this article, we introduce a prototype of an innovative technology for proving the origins of captured digital media. In an era of fake news, when someone shows us a video or picture of some event, how can we trust its authenticity? It seems that the public no longer believe that traditional media is a reliable reference of fact, perhaps due, in part, to the onset of many diverse sources of conflicting information, via social media. Indeed, the issue of "fake" reached a crescendo during the 2016 U.S. Presidential Election, when the winner, Donald Trump, claimed that The New York Times was trying to discredit him by pushing disinformation. Current research into overcoming the problem of fake news does not focus on establishing the ownership of media resources used in such stories-the blockchain-based application introduced in this article is technology that is capable of indicating the authenticity of digital media. Put simply, using the trust mechanisms of blockchain technology, the tool can show, beyond doubt, the provenance of any source of digital media, including images used out of context in attempts to mislead. Although the application is an early prototype and its capability to find fake resources is somewhat limited, we outline future improvements that would overcome such limitations. Furthermore, we believe that our application (and its use of blockchain technology and standardized metadata) introduces a novel approach to overcoming falsities in news reporting and the provenance of media resources used therein. However, while our application has the potential to be able to verify the originality of media resources, we believe that technology is only capable of providing a partial solution to fake news. That is because it is incapable of proving the authenticity of a news story as a whole. We believe that takes human skills.
We prove that the scale map of the zero-crossings of almost all signals filtered by a gaussian of variable size determines the signal uniquely, up to a constant scaling. Exceptions are signals that are antisymmetric about all their zeros (for instance infinitely periodic gratings). Our proof provides a method for reconstructing almost all signals from knowledge of how the zero-crossing contours of the signal, filtered by a gaussian filter, change with the size of the filter. The proof assumes that the filtered signal can be represented as a polynomial of finite, albeit possibly very high, order. The result applies to zero- and level-crossings of signals filtered by gaussian filters. The theorem is extended to two dimensions, that is to images. These results imply that extrema (for instance of derivatives) at different scales are a complete representation of a signal.