In this report, we compare the performance of our UltraGroth-based zero-knowledge machine learning framework Bionetta to other tools of similar purpose such as EZKL, Lagrange's deep-prove, or zkml. The results show a significant boost in the proving time for custom-crafted neural networks: they can be proven even on mobile devices, enabling numerous client-side proving applications. While our scheme increases the cost of one-time preprocessing steps, such as circuit compilation and generating trusted setup, our approach is, to the best of our knowledge, the only one that is deployable on the native EVM smart contracts without overwhelming proof size and verification overheads.
Abdullah Alsaleh, Ghanshyam G. Tejani, Shailendra Mishra, Sunil Kumar Sharma · 5 authors
The detection of brain tumors is crucial in medical imaging, because accurate and early diagnosis can have a positive effect on patients. Because traditional deep learning models store all their data together, they raise questions about privacy, complying with regulations and the different types of data used by various institutions. We introduce the anisotropic-residual capsule hybrid Gorilla Badger optimized network (Aniso-ResCapHGBO-Net) framework for detecting brain tumors in a privacy-preserving, decentralized system used by many healthcare institutions. ResNet-50 and capsule networks are incorporated to achieve better feature extraction and maintain the structure of images' spatial data. To get the best results, the hybrid Gorilla Badger optimization algorithm (HGBOA) is applied for selecting the key features. Preprocessing techniques include anisotropic diffusion filtering, morphological operations, and mutual information-based image registration. Updates to the model are made secure and tamper-evident on the Ethereum network with its private blockchain and SHA-256 hashing scheme. The project is built using Python, TensorFlow and PyTorch. The model displays 99.07% accuracy, 98.54% precision and 99.82% sensitivity on assessments from benchmark CT imaging of brain tumors. This approach also helps to reduce the number of cases where no disease is found when there is one and vice versa. The framework ensures that patients' data is protected and does not decrease the accuracy of brain tumor detection.
Yanming Zhu, Xuefei Yin, Alan WeeâChung Liew, Hui Tian
With the rapid advancement of artificial intelligence and deep learning, medical image analysis has become a critical tool in modern healthcare, significantly improving diagnostic accuracy and efficiency. However, AI-based methods also raise serious privacy concerns, as medical images often contain highly sensitive patient information. This review offers a comprehensive overview of privacy-preserving techniques in medical image analysis, including encryption, differential privacy, homomorphic encryption, federated learning, and generative adversarial networks. We explore the application of these techniques across various medical image analysis tasks, such as diagnosis, pathology, and telemedicine. Notably, we organizes the review based on specific challenges and their corresponding solutions in different medical image analysis applications, so that technical applications are directly aligned with practical issues, addressing gaps in the current research landscape. Additionally, we discuss emerging trends, such as zero-knowledge proofs and secure multi-party computation, offering insights for future research. This review serves as a valuable resource for researchers and practitioners and can help advance privacy-preserving in medical image analysis.
Entesar Hamed I. Eliwa, Amr Mohamed El Koshiry, Tarek Abd ElâHafeez, Ahmed Omar
Background: The global healthcare system faces challenges in diagnosing and managing lung and colon cancers, which are significant health burdens. Traditional diagnostic methods are inefficient and prone to errors, while data privacy and security concerns persist. Objective: This study aims to develop a secure and transparent framework for remote consultation and classification of lung and colon cancer, leveraging blockchain technology and Microsoft Azure cloud services. Dataset and Features: The framework utilizes the LC25000 dataset, containing 25,000 histopathological images, for training and evaluating advanced machine learning models. Key features include secure data upload, anonymization, encryption, and controlled access via blockchain and Azure services. Methods: The proposed framework integrates Microsoft Azureâs cloud services with a permissioned blockchain network. Patients upload CT scans through a mobile app, which are then preprocessed, anonymized, and stored securely in Azure Blob Storage. Blockchain smart contracts manage data access, ensuring only authorized specialists can retrieve and analyze the scans. Azure Machine Learning is used to train and deploy state-of-the-art machine learning models for cancer classification. Evaluation Metrics: The frameworkâs performance is evaluated using metrics such as accuracy, precision, recall, and F1-score, demonstrating the effectiveness of the integrated approach in enhancing diagnostic accuracy and data security. Results: The proposed framework achieves an impressive accuracy of 100% for lung and colon cancer classification using DenseNet, ResNet50, and MobileNet models with different split ratios (70â30, 80â20, 90â10). The F1-score and k-fold cross-validation accuracy (5-fold and 10-fold) also demonstrate exceptional performance, with values exceeding 99.9%. Real-time notifications and secure remote consultations enhance the efficiency and transparency of the diagnostic process, contributing to better patient outcomes and streamlined cancer care management.
Deep learning-based image generation has seen significant advancements with diffusion models, notably improving the quality of generated images. Despite these developments, generating images with unseen characteristics beneficial for downstream tasks has received limited attention. To bridge this gap, we propose Style-Extracting Diffusion Models, featuring two conditioning mechanisms. Specifically, we utilize 1) a style conditioning mechanism which allows to inject style information of previously unseen images during image generation and 2) a content conditioning which can be targeted to a downstream task, e.g., layout for segmentation. We introduce a trainable style encoder to extract style information from images, and an aggregation block that merges style information from multiple style inputs. This architecture enables the generation of images with unseen styles in a zero-shot manner, by leveraging styles from unseen images, resulting in more diverse generations. In this work, we use the image layout as target condition and first show the capability of our method on a natural image dataset as a proof-of-concept. We further demonstrate its versatility in histopathology, where we combine prior knowledge about tissue composition and unannotated data to create diverse synthetic images with known layouts. This allows us to generate additional synthetic data to train a segmentation network in a semi-supervised fashion. We verify the added value of the generated images by showing improved segmentation results and lower performance variability between patients when synthetic images are included during segmentation training. Our code will be made publicly available at [LINK].
Kar Balan, Andrew Gilbert, Alexander Black, Simon Jenni · 6 authors
We present DECORAIT; a decentralized registry through which content creators may assert their right to opt in or out of AI training as well as receive reward for their contributions. Generative AI (GenAI) enables images to be synthesized using AI models trained on vast amounts of data scraped from public sources. Model and content creators who may wish to share their work openly without sanctioning its use for training are thus presented with a data governance challenge. Further, establishing the provenance of GenAI training data is important to creatives to ensure fair recognition and reward for their such use. We report a prototype of DECORAIT, which explores hierarchical clustering and a combination of on/off-chain storage to create a scalable decentralized registry to trace the provenance of GenAI training data in order to determine training consent and reward creatives who contribute that data. DECORAIT combines distributed ledger technology (DLT) with visual fingerprinting, leveraging the emerging C2PA (Coalition for Content Provenance and Authenticity) standard to create a secure, open registry through which creatives may express consent and data ownership for GenAI.
Intelligent and sustainable healthcare systems can considerably benefit from applying Computational Intelligence (CI) and Artificial Intelligence (AI). These technological breakthroughs can reduce the ecological footprint and raise the bar for excellence. Yet, the broad adoption of such technologies for cutting-edge Internet of Things (IoT) applications generates enormous amounts of data, which can heavily strain the available computational resources. The major motivation behind this study is to provide evidence that Gated Recurrent Units (GRUs), a sophisticated subclass of Recurrent Neural Networks (RNNs), can outperform traditional RNNs. These technologies can be effective in identifying and treating breast cancer. This study collects data from tagged IoT devices and trains a GRU-RNN classifier. The Wisconsin Diagnostic Breast Cancer (WDBC) data tests the systemâs accuracy. The results show the proposed Internet of Medical Things (IoMT) is more effective than the current methods in recall, accuracy, and precision while preserving 95% of the original GRU-RNN.
K Rajeshkumar, Chidambaram Ananth, N. Mohananthini
Blockchain (BC) and Machine learning (ML) technologies have been investigated for potential applications in medicine with reasonable success to date. On the other hand, as accurate and early diagnosis of skin lesion classification is essential to gradually increase the survival rate of the patient, Deep-Learning (DL) and ML technologies were introduced for supporting dermatologists to overcome these challenges. This study designed a Blockchain Assisted Homomorphic Encryption Approach for Skin Lesion Diagnosis using an Optimal Deep Learning (BHESKD-ODL) model. The presented BHESKD-ODL model achieves security and proper classification of skin lesion images using BC to store the medical images of the patients to restrict access to third-party users or intruders. In addition, the BHESKD-ODL method secures the medical images using the mayfly optimization (MFO) algorithm with the Homomorphic Encryption (HE) technique. For skin lesion diagnosis, the proposed BHESKD-ODL method uses pre-processing and the Adam optimizer with a Fully Convolutional Network (FCN) based segmentation process. Furthermore, a radiomics feature extraction with a Bidirectional Recurrent Neural Network (BiRNN) model was employed for skin lesion classification. Finally, the Red Deer Optimization (RDO) algorithm was used for the optimal hyperparameter selection of the BiRNN approach. The experimental results of the BHESKD-ODL system on a benchmark skin dataset proved its promising performance in terms of different measures.
In recent years, COVID-19 has impacted millions of individuals worldwide, resulting in numerous fatalities across several countries. While RT-PCR technology remains the most reliable method for detecting COVID-19, it is both expensive and time-consuming. As a result, researchers have explored various machine learning and deep learning-based approaches to rapidly identify COVID-19 cases using X-ray images, with reduced costs and shorter processing times. However, preserving patient confidentiality poses challenges within third-party-controlled systems, potentially failing to safeguard patients from potential disgrace and discomfort. Nonetheless, blockchain technology offers the potential to securely store sensitive medical data anonymously, without requiring third-party intervention. Consequently, the combination of deep learning and blockchain could offer a viable solution to mitigate the spread of COVID-19 while ensuring patient privacy protection. In this paper, we propose a hybrid model of blockchain and deep learning model for automatically detecting COVID-19 using chest X-rays (CXR). The deep learning model includes a stacking ensemble of three modified pre-trained Deep Learning (DL) models: VGG16, Xception, and DenseNet169. The model obtained an accuracy of 99.10% and 98.60% for binary and multi-class respectively. Further, To ensure COVID-19 patientsâ privacy and security, the Ethereum blockchain has been adopted to store information related to COVID-19 cases. In addition, a smart contract on the blockchain has been designed for handling X-ray images in the Interplanetary File System (IPFS).
Open access
COVID-19 diagnosis using AI
Artificial Intelligence in Healthcare and Education
Sebastian Griewing, Michael Lingenfelder, Uwe Wagner, Niklas Gremke
This study aims at evaluating the use case potential of breast cancer care for artificial intelligence and blockchain technology application based on the patient data analysis at Marburg University Hospital and, thereupon, developing a digital workflow for breast cancer care. It is based on a retrospective descriptive data analysis of all in-patient breast and ovarian cancer patients admitted at the Department of Gynecology of Marburg University Hospital within the five-year observation period of 2017 to 2021. According to the German breast cancer guideline, the care workflow was visualized and, thereon, the digital concept was developed, premised on the literature foundation provided by a Boolean combination open search. Breast cancer cases display a lower average patient case complexity, fewer secondary diagnoses, and performed procedures than ovarian cancer. Moreover, 96% of all breast cancer patients originate from a city with direct geographical proximity. Estimated circumference and total catchment area of ovarian present 28.6% and 40% larger, respectively, than for breast cancer. The data support invasive breast cancer as a preferred use case for digitization. The digital workflow based on combined application of artificial intelligence as well as blockchain or distributed ledger technology demonstrates potential in tackling senological care pain points and leveraging patient data safety and sovereignty.
Open access
Artificial Intelligence in Healthcare and Education
The field of digital pathology produces a large number of images associated with patient metadata that are the raw material of computational pathology. The process of making images available with adequate privacy and data protection considerations takes a long time. Given that the Ethereum network associated with InterPlanetary File System (IPFS) promotes the exchange of information in a secure, private and decentralized manner, this association could be an important partner between digital and computational pathology. Therefore, here we propose and discuss a prototype with the aforementioned parts and the addition of neural compression, as an essential information preservation step. This prototype could constitute a link for the exchange of information in a secure way, providing transparency and reliability to the chain and empowering the field of manufacturing artificial vision solutions for the medical field
Muhammad Umar Nasir, Safiullah Khan, Shahid Mehmood, Muhammad Adnan Khan · 6 authors
Bone tumors, such as osteosarcomas, can occur anywhere in the bones, though they usually occur in the extremities of long bones near metaphyseal growth plates. Osteosarcoma is a malignant lesion caused by a malignant osteoid growing from primitive mesenchymal cells. In most cases, osteosarcoma develops as a solitary lesion within the most rapidly growing areas of the long bones in children. The distal femur, proximal tibia, and proximal humerus are the most frequently affected bones, but virtually any bone can be affected. Early detection can reduce mortality rates. Osteosarcoma's manual detection requires expertise, and it can be tedious. With the assistance of modern technology, medical images can now be analyzed and classified automatically, which enables faster and more efficient data processing. A deep learning-based automatic detection system based on whole slide images (WSIs) is presented in this paper to detect osteosarcoma automatically. Experiments conducted on a large dataset of WSIs yielded up to 99.3% accuracy. This model ensures the privacy and integrity of patient information with the implementation of blockchain technology. Utilizing edge computing and fog computing technologies, the model reduces the load on centralized servers and improves efficiency.
BACKGROUND: Recent advancements in digital pathology resulting from advances in imaging and digitization have increased the convenience and usability of pathology for disease diagnosis, especially in oncology, urology, and gastroenteric diagnosis. However, despite the possibilities to include low-cost diagnosis and viable telemedicine, digital pathology is not yet accessible owing to expensive storage, data security requirements, and network bandwidth limitations to transfer high-resolution images and associated data. The increase in storage, transmission, and security complexity concerning data collection and diagnosis makes it even more challenging to use artificial intelligence algorithms for machine-assisted disease diagnosis. We designed and prototyped a digital pathology system that uses blockchain-based smart contracts using the nonfungible token (NFT) standard and the Interplanetary File System for data storage. Our design remediates shortcomings in the existing digital pathology systems infrastructure, which is centralized. The proposed design is extendable to other fields of medicine that require high-fidelity image and data storage. Our solution is implemented in data systems that can improve access quality of care and reduce the cost of access to specialized pathological diagnosis, reducing cycle times for diagnosis. OBJECTIVE: The main objectives of this study are to highlight the issues in digital pathology and suggest that a software architecture-based blockchain and the Interplanetary File System create a low-cost data storage and transmission technology. METHODS: We used the design science research method consisting of 6 stages to inform our design overall. We innovated over existing public-private designs for blockchains but using a 2-layered approach that separates actual file storage from metadata and data persistence. RESULTS: Here, we identified key challenges to adopting digital pathology, including challenges concerning long-term storage and the transmission of information. Next, using accepted frameworks in NFT-based intelligent contracts and recent innovations in distributed secure storage, we proposed a decentralized, secure, and privacy-preserving digital pathology system. Our design and prototype implementation using Solidity, web3.js, Ethereum, and node.js helped us address several challenges facing digital pathology. We demonstrated how our solution, which combines NFT smart contract standard with persistent decentralized file storage, solves most of the challenges of digital pathology and sets the stage for reducing costs and improving patient care and speed of diagnosis. CONCLUSIONS: We identified technical limitations that increase costs and reduce the mass adoption of digital pathology. We presented several design innovations using NFT decentralized storage standards to prototype a system. We also presented the implementation details of a unique security architecture for a digital pathology system. We illustrated how this design can overcome privacy, security, network-based storage, and data transmission limitations. We illustrated how improving these factors sets the stage for improving data quality and standardized application of machine learning and artificial intelligence to such data.
BACKGROUND Recent advancements in digital pathology resulting from advances in imaging and digitization have increased the convenience and usability of pathology for disease diagnosis, especially in oncology, urology, and gastroenteric diagnosis. However, despite the possibilities to include low-cost diagnosis and viable telemedicine, digital pathology is not yet accessible owing to expensive storage, data security requirements, and network bandwidth limitations to transfer high-resolution images and associated data. The increase in storage, transmission, and security complexity concerning data collection and diagnosis makes it even more challenging to use artificial intelligence algorithms for machine-assisted disease diagnosis. We designed and prototyped a digital pathology system that uses blockchain-based smart contracts using the nonfungible token (NFT) standard and the Interplanetary File System for data storage. Our design remediates shortcomings in the existing digital pathology systems infrastructure, which is centralized. The proposed design is extendable to other fields of medicine that require high-fidelity image and data storage. Our solution is implemented in data systems that can improve access quality of care and reduce the cost of access to specialized pathological diagnosis, reducing cycle times for diagnosis. OBJECTIVE The main objectives of this study are to highlight the issues in digital pathology and suggest that a software architectureâbased blockchain and the Interplanetary File System create a low-cost data storage and transmission technology. METHODS We used the design science research method consisting of 6 stages to inform our design overall. We innovated over existing public-private designs for blockchains but using a 2-layered approach that separates actual file storage from metadata and data persistence. RESULTS Here, we identified key challenges to adopting digital pathology, including challenges concerning long-term storage and the transmission of information. Next, using accepted frameworks in NFT-based intelligent contracts and recent innovations in distributed secure storage, we proposed a decentralized, secure, and privacy-preserving digital pathology system. Our design and prototype implementation using Solidity, web3.js, Ethereum, and node.js helped us address several challenges facing digital pathology. We demonstrated how our solution, which combines NFT smart contract standard with persistent decentralized file storage, solves most of the challenges of digital pathology and sets the stage for reducing costs and improving patient care and speed of diagnosis. CONCLUSIONS We identified technical limitations that increase costs and reduce the mass adoption of digital pathology. We presented several design innovations using NFT decentralized storage standards to prototype a system. We also presented the implementation details of a unique security architecture for a digital pathology system. We illustrated how this design can overcome privacy, security, network-based storage, and data transmission limitations. We illustrated how improving these factors sets the stage for improving data quality and standardized application of machine learning and artificial intelligence to such data.
Rajesh Kumar, Wenyong Wang, Jay Kumar, Ting Yang · 7 authors
Deep learning, for image data processing, has been widely used to solve a variety of problems related to medical practices. However, researchers are constantly struggling to introduce ever efficient classification models. Recent studies show that deep learning can perform better and generalize well when trained using a large amount of data. Organizations such as hospitals, testing labs, research centers, etc. can share their data and collaboratively build a better learning model. Every organization wants to retain the privacy of their data, while on the other hand, these organizations want accurate and efficient learning models for various applications. The concern for privacy in medical data limits the sharing of data among multiple organizations due to some ethical and legal issues. To retain privacy and enable data sharing, we present a unique method that combines locally learned deep learning models over the blockchain to improve the prediction of lung cancer in health-care systems by filling the defined gap. There are several challenges involved in sharing that data while maintaining privacy. In this paper, we identify and address such challenges. The contribution of our work is four-fold: (i) We propose a method to secure medical data by only sharing the weights of the trained deep learning model via smart contract. (ii) To deal with different sized computed tomography (CT) images from various sources, we adopted the Bat algorithm and data augmentation to reduce the noise and overfitting for the global learning model. (iii) We distribute the local deep learning model wights to the blockchain decentralized network to train a global model. iv) We propose a recurrent convolutional neural network (RCNN) to estimate the region of interest (ROI) in theCT images. An extensive empirical study has been conducted to verify the significance of our proposed method for better prediction of cancer in the early stage. Experimental results of the proposed model can show that our proposed technique can detect the lung cancer nodules and also achieve better performance.
Dev P. Chakraborty, Robert M. Nishikawa, Colin G. Orton
One of the major roles of regulatory bodies is to enforce rules and thus maintain standards. They also often do research related to their missions, some of which might be used to establish the standards they are regulating and how they should be evaluated. This has led some to believe that, due to potential concerns of bias and conflicts of interest, regulatory bodies should not do evaluation methodology research related to their regulatory missions. This is the claim that is debated in this month's Point/Counterpoint. Arguing for the Proposition is Dev P. Chakraborty, Ph.D. Dr. Chakraborty earned his Ph.D. in solid state physics from the University of Rochester, New York in 1977 then, in 1979, began his career in medical physics working with Ivan Brezovich in the Department of Radiology, University of Alabama at Birmingham, AL, where he worked until 1988 before moving to the Department of Radiology, University of Pennsylvania, Philadelphia. He subsequently moved to the University of Pittsburgh, Pittsburgh, PA, in 1997, where he was Professor in the Department of Bioengineering before assuming his current position at ExpertCAD Analytics, LLC in 2016. He has published over 75 papers in peer-reviewed journals, many in the field of observer performance analysis. Arguing against the Proposition is Robert M. Nishikawa, Ph.D. Dr. Nishikawa received his B.Sc. in physics in 1981 and his M.Sc. and Ph.D. in Medical Biophysics in 1984 and 1990, respectively, all from the University of Toronto. While at the University of Chicago, he developed computer-aided diagnosis systems for classifying and detecting clustered calcifications in mammograms. He has seven patents on CAD-related technologies and has over 200 publications in breast imaging. He is currently a Professor and Director of the Clinical Translational Medical Physics Laboratory in the Department of Radiology at the University of Pittsburgh. He has won 24 awards including two for âbestâ paper, two innovation awards, and one teaching award. He is a fellow of the American Association of Physicists in Medicine, the Society of Breast Imaging, the College of American Institute for Medical and Biological Engineering, and a Distinguished Investigator, Academy of Radiology Research. His research interests are in computer-aided diagnosis, breast imaging, image quality assessment, and evaluation of medical technologies. The Food and Drug Administration (FDA) and the Center for Devices and Radiological Health (CDRH) both regulate imaging devices and claim leadership roles in how they are evaluated. To demonstrate that the CDRH leadership in imaging device evaluation research biases research in this area and results in suboptimal evaluation of new imaging devices, I will present a single extended example. CDRH scientists are leading proponents of FROC/ROC1, 2 methods for analyzing observer outcome studies. An alternative and often more efficacious approach is the JAFROC method3 pioneered in my laboratory. Does a computer-aided detection (CAD) manufacturer adopt evaluation methods developed by Chakraborty3 or does the manufacturer feel pressure to adopt the FDA's methods?1, 2 Chakraborty's methods/software (JAFROC) have been used in over 104 publications, but only 24 are from the US and none from the FDA. The chances that this low number is a fluke are astronomically small, especially given the much larger total numbers of published US studies relative to non-US studies. This is strong evidence the FDA has influenced US-researchers against using JAFROC. Most clinical trials, including the American College of Radiology Imaging Network (ACRIN) Digital Mammographic Imaging Screening Trial (DMIST),4 have used the lower power ROC paradigm for localization tasks, which is inappropriate and unethical:5 lower power means the study is either of dubious value or it is overly expensive. The location-specific method favored by the FDA1, 2 is based on the FROC curve: one can hardly do worse. FROC data consist of mark-rating pairs; marks are locations of suspicious regions and the rating is the associated confidence level. Based on a proximity criterion, a mark close to a lesion is scored as lesion localization (LL) and otherwise, it is non-lesion localization (NL). Lesion localization fraction (LLF) is defined as the number of LLs â„ threshold divided by the total number of lesions. The non-lesion localization fraction (NLF) is the number of NLs â„ threshold rating divided by the total number of images. The FROC curve (plot of LLF (ordinate) vs. NLF) rises with infinite slope from (0,0). The slope then decreases monotonically and the curve ends abruptly at an unpredictable point. The FROC is not contained within the unit square. This makes it impossible to define a meaningful area measure. The FROC is defined by marks: unmarked nondiseased cases, which represent perfect decisions, do not contribute to the area under the curve (AUC) under the FROC. In screening mammography, about 995 cases out of 1000 are nondiseased. The perfect radiologist, who marks all lesions and does not mark any nondiseased case, yields zero FROC AUC, receiving no credit for the 995 correct decisions. JAFROC is based on the AFROC (alternative-FROC) curve. The y-axis is similar to LLF, but the x-axis is the ROC false-positive fraction defined by the highest ratings on nondiseased cases, and the AFROC plot includes a connection from the uppermost operating point to (1,1). Unlike the FROC AUC, the AFROC AUC for the perfect observer is unity, not zero. JAFROC is ignored in FDA's Guidance Document,2 as are positive statements about JAFROC from the late Drs. Wagner and Metz,6 and there is not one reference to Chakraborty's work. The FDA's bias has doomed progress in breast cancer CAD (40,000 deaths/yr). Besides using incorrect FROC methodology, it has set a low (second reader) bar for CAD to be considered a âsuccessâ. The end result: massive clinical trials7 have shown that CAD is actually detrimental to the outcome and there has been a call to end CAD Medicare reimbursement.8 Regulation is necessary to balance the costs and benefits of implementing a product or activity. This raises two important issues. First, it is important to quantify costs and benefits accurately. Second, it is equally important for impartiality to acquire correct balances. The proposition directly addresses the second issue, but the first issue is necessary to discuss also. I will restrict my discussion to medical imaging devices for clarity. There are many well-established methods to determine the benefits of medical imaging devices.9 There are, however, situations where researchers need new evaluation methods, either for a new technology or to simplify tests for an existing type of technology. This requires research to develop and validate the new methodologies. The regulatory agencies need to understand the strengths and weaknesses of any tests presented to them as evidence for the effectiveness of a product. This would require regulatory agencies to either develop the expertise in-house or to rely on the scientific literature. That latter is insufficient for two reasons. First, regulatory science is not a well-funded branch of science. Therefore, unless the regulatory bodies perform the research, a disconnect may occur between developing the technologies and measuring their benefits and costs. This will either slow down approval of new technologies or lead to unbalanced regulations, or both. Second, reviewing the literature may be effective in understanding the basics of the evaluation methodology, but it is usually insufficient to understand the limitations of the method. Understanding the limitations is best done by applying the method, using simulations to a variety of situations, and evaluating the results. That is basically research and regulatory bodies benefit from conducting the studies themselves. While we can quantify benefits and costs, it is often difficult to decide on the proper balance of the two, particularly in an unbiased manner. Part of the difficulty arises from benefit and cost estimates not having the same units. A prime example of this, while not exactly in the regulatory domain, is the United States Preventative Services Task Force (USPSTF) recommendations on mammographic screening.10 We can evaluate the benefits of screening as lower mortality from breast cancer and costs as false-positive screens â recalling a woman for further imaging when, in fact, she does not have a breast cancer. It is not clear how to balance lives saved against more imaging and potentially an unnecessary biopsy. The USPSTF placed more weight on false-positive screens and chose not to recommend periodic screening for all women under the age of 50, compared to, for example, the American College of Radiology which supports annual screening of women 40 and older.11 Some proponents of screening argue that the USPSTF was biased in making their recommendations.12 There is no clear solution for this potential bias, but I do not believe that researching evaluation methodology is the right place to start. On the contrary, I believe there is less potential for bias when people are more knowledgeable â unless they are predisposed to a bias to begin with. Which is to say a bias can exist whether knowledge is obtained first hand or from reviewing the literature. I agree with my colleague that the FDA/CDRH needs to be current on the science. If regulatory science is not a well-funded branch of science, that makes it even more important to be current on the existing science, both from a revered in-house predecessor6 and from academia.3 I also agree that there is need for developing new evaluation methods, but then why is the new FDA/CRDH still wedded to the 1940s ROC paradigm; what is new about it? The âmechanisticâ approach13 that they are enamored with does not advance the state-of-the-art in general-paradigm multireader multicase (MRMC) analysis, rather it explains and generalizes the variance-component decomposition used in Dorfman/Berbaum/Metz analysis14 in a mathematically appealing way. But, and this is the serious limitation, it applies only to the Wilcoxon ROC statistic; it is not even applicable to fitted ROC curves, let alone FROC methodology. In my Opening Statement, I cited the âpowerâ imbalance when it comes to reviewing/vetting the work of the FDA/CDRH, and examples of questionable work. I could go on, especially how they validate methodologies. It is a brave and knowledgeable researcher who can properly review a paper15 listing as institution of origin: âNIBIB/CDRH Laboratory for the Assessment of Medical Imaging Systemsâ. Any applicant for an NIH grant in methodology development, and I see there is a recent funding opportunity announcement (PAR-17-125), would be well advised to cite this paper, never mind that it is about ROC analysis, while CAD provides FROC data, so at the very least the title of the paper is misleading. The cited work remains true to model observer philosophy, which assumes the lesion location is known, ignoring the fact that if location were known, there would be no need for a radiologist to find it. This entire debate would be of academic interest, but it was not for the implications for patient care: lives literally depend on the selection of proper imaging technology. Conducting ROC studies for search tasks is not only bad science but it is also unethical and a disservice to patients and taxpayers. My colleague Dev Chakraborty argues, I believe because it is not explicitly stated that the FDA, but principally the CDRH, is biased because it âforcesâ companies to use ROC analysis instead of JAFROC analysis, which Dev developed; and that this bias exists because members of the CDRH have done ROC research, but not FROC research. That is an interesting premise. Dev supports his assertion with statistics that are consistent with his view, but it does not constitute proof. Here is my prospective on Dev's claim of bias. First, I know many of the people at the CDRH. In my view, they are among the leaders in the field, both in terms of their scientific rigor and in their vision. The CDRH has a long history of significant and cutting edge research and establishing methodology for evaluating screen-film systems, digital systems, computer-aided diagnosis systems, ultrasound, and others. I have not seen signs of bias in my interactions with members of the CDRH. Certainly, the members have preferences, but they remain open-minded and fair. It is important to note that just as there are differences in approach between scientists in academia and industry, there are differences between scientists in the public service sector and academia (and industry). Scientists in the public are much more open to sharing data and ideas. Second, companies applying for FDA approval are, in my experience working with them, very conservative in their approach, and they basically follow any FDA precedent or previous approved applications. This is because the approval process can be time-consuming and expensive. Companies usually overpower their observer studies to include more readers and cases than what is required by an 80% power calculation. They do not want to risk having a null result because the observer study was underpowered. Furthermore, and more importantly, it is much easier and less risky just to copy a previously approved application. This will result in the same methods being perpetuated over time. So, when a company develops a new method, even if there are some benefits to it over existing techniques, they are less likely to use the new method in FDA submissions. This is the company's choice, not an FDA edict. So, while Dr. Chakraborty has presented evidence, it is all circumstantial and, until he produces a âsmoking gunâ, I believe that his assertion of bias at the CDRH is false. The authors have no relevant conflicts of interest to disclose.