| 1. |
- Tschandl, P., et al.
(författare)
-
Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study
- 2019
-
Ingår i: The Lancet Oncology. - 1470-2045 .- 1474-5488. ; 20:7, s. 938-947
-
Tidskriftsartikel (refereegranskat)abstract
- Background: Whether machine-learning algorithms can diagnose all pigmented skin lesions as accurately as human experts is unclear. The aim of this study was to compare the diagnostic accuracy of state-of-the-art machine-learning algorithms with human readers for all clinically relevant types of benign and malignant pigmented skin lesions. Methods: For this open, web-based, international, diagnostic study, human readers were asked to diagnose dermatoscopic images selected randomly in 30-image batches from a test set of 1511 images. The diagnoses from human readers were compared with those of 139 algorithms created by 77 machine-learning labs, who participated in the International Skin Imaging Collaboration 2018 challenge and received a training set of 10 015 images in advance. The ground truth of each lesion fell into one of seven predefined disease categories: intraepithelial carcinoma including actinic keratoses and Bowen's disease; basal cell carcinoma; benign keratinocytic lesions including solar lentigo, seborrheic keratosis and lichen planus-like keratosis; dermatofibroma; melanoma; melanocytic nevus; and vascular lesions. The two main outcomes were the differences in the number of correct specific diagnoses per batch between all human readers and the top three algorithms, and between human experts and the top three algorithms. Findings: Between Aug 4, 2018, and Sept 30, 2018, 511 human readers from 63 countries had at least one attempt in the reader study. 283 (55·4%) of 511 human readers were board-certified dermatologists, 118 (23·1%) were dermatology residents, and 83 (16·2%) were general practitioners. When comparing all human readers with all machine-learning algorithms, the algorithms achieved a mean of 2·01 (95% CI 1·97 to 2·04; p<0·0001) more correct diagnoses (17·91 [SD 3·42] vs 19·92 [4·27]). 27 human experts with more than 10 years of experience achieved a mean of 18·78 (SD 3·15) correct answers, compared with 25·43 (1·95) correct answers for the top three machine algorithms (mean difference 6·65, 95% CI 6·06–7·25; p<0·0001). The difference between human experts and the top three algorithms was significantly lower for images in the test set that were collected from sources not included in the training set (human underperformance of 11·4%, 95% CI 9·9–12·9 vs 3·6%, 0·8–6·3; p<0·0001). Interpretation: State-of-the-art machine-learning classifiers outperformed human experts in the diagnosis of pigmented skin lesions and should have a more important role in clinical practice. However, a possible limitation of these algorithms is their decreased performance for out-of-distribution images, which should be addressed in future research. Funding: None. © 2019 Elsevier Ltd
|
|
| 2. |
- Haenssle, H A, et al.
(författare)
-
Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists.
- 2018
-
Ingår i: Annals of Oncology. - : Elsevier BV. - 1569-8041 .- 0923-7534. ; 29:8, s. 1836-1842
-
Tidskriftsartikel (refereegranskat)abstract
- Deep learning convolutional neural networks (CNN) may facilitate melanoma detection, but data comparing a CNN's diagnostic performance to larger groups of dermatologists are lacking.Google's Inception v4 CNN architecture was trained and validated using dermoscopic images and corresponding diagnoses. In a comparative cross-sectional reader study a 100-image test-set was used (level-I: dermoscopy only; level-II: dermoscopy plus clinical information and images). Main outcome measures were sensitivity, specificity and area under the curve (AUC) of receiver operating characteristics (ROC) for diagnostic classification (dichotomous) of lesions by the CNN versus an international group of 58 dermatologists during level-I or -II of the reader study. Secondary end points included the dermatologists' diagnostic performance in their management decisions and differences in the diagnostic performance of dermatologists during level-I and -II of the reader study. Additionally, the CNN's performance was compared with the top-five algorithms of the 2016 International Symposium on Biomedical Imaging (ISBI) challenge.In level-I dermatologists achieved a mean (±standard deviation) sensitivity and specificity for lesion classification of 86.6% (±9.3%) and 71.3% (±11.2%), respectively. More clinical information (level-II) improved the sensitivity to 88.9% (±9.6%, P=0.19) and specificity to 75.7% (±11.7%, P<0.05). The CNN ROC curve revealed a higher specificity of 82.5% when compared with dermatologists in level-I (71.3%, P<0.01) and level-II (75.7%, P<0.01) at their sensitivities of 86.6% and 88.9%, respectively. The CNN ROC AUC was greater than the mean ROC area of dermatologists (0.86 versus 0.79, P<0.01). The CNN scored results close to the top three algorithms of the ISBI 2016 challenge.For the first time we compared a CNN's diagnostic performance with a large international group of 58 dermatologists, including 30 experts. Most dermatologists were outperformed by the CNN. Irrespective of any physicians' experience, they may benefit from assistance by a CNN's image classification.This study was registered at the German Clinical Trial Register (DRKS-Study-ID: DRKS00013570; https://www.drks.de/drks_web/).
|
|
| 3. |
- Sinz, Christoph, et al.
(författare)
-
Accuracy of dermatoscopy for the diagnosis of nonpigmented cancers of the skin.
- 2017
-
Ingår i: Journal of the American Academy of Dermatology. - : Elsevier BV. - 1097-6787 .- 0190-9622. ; 77:6, s. 1100-1109
-
Tidskriftsartikel (refereegranskat)abstract
- Nonpigmented skin cancer is common, and diagnosis with the unaided eye is error prone.To investigate whether dermatoscopy improves the diagnostic accuracy for nonpigmented (amelanotic) cutaneous neoplasms.We collected a sample of 2072 benign and malignant neoplastic lesions and inflammatory conditions and presented close-up images taken with and without dermatoscopy to 95 examiners with different levels of experience.The area under the curve was significantly higher with than without dermatoscopy (0.68 vs 0.64, P<.001). Among 51 possible diagnoses, the correct diagnosis was selected in 33.1% of cases with and 26.4% of cases without dermatoscopy (P<.001). For experts, the frequencies of correct specific diagnoses of a malignant lesion improved from 40.2% without to 51.3% with dermatoscopy. For all malignant neoplasms combined, the frequencies of appropriate management strategies increased from 78.1% without to 82.5% with dermatoscopy.The study deviated from a real-life clinical setting and was potentially affected by verification and selection bias.Dermatoscopy improves the diagnosis and management of nonpigmented skin cancer and should be used as an adjunct to examination with the unaided eye.
|
|
