Skin Cancer and Its Classification in the Aged: A Study of Melanoma and Squamous Cell Carcinoma Using OCT and AI

Frederick H. Silver ^1,2,*, Tanmay Deshmukh ²

1. Department of Pathology and Laboratory Medicine, Robert Wood Johnson Medical School, Rutgers, the State University of New Jersey, Piscataway, NJ 08854, USA

2. OptoVibronex, LLC., Bethlehem, PA 18104, USA

* Correspondence: Frederick H. Silver

Academic Editor: Taihao Quan

Received: February 21, 2026 | Accepted: May 27, 2026 | Published: June 07, 2026

OBM Genetics 2026, Volume 10, Issue 2, doi:10.21926/obm.genet.2602345

Recommended citation: Silver FH, Deshmukh T. Skin Cancer and Its Classification in the Aged: A Study of Melanoma and Squamous Cell Carcinoma Using OCT and AI. OBM Genetics 2026; 10(2): 345; doi:10.21926/obm.genet.2602345.

© 2026 by the authors. This is an open access article distributed under the conditions of the Creative Commons by Attribution License, which permits unrestricted use, distribution, and reproduction in any medium or format, provided the original work is correctly cited.

1. Introduction

There are 6.0 million new cases of skin cancer in the United States diagnosed every year with skin cancer the most prevalent type of cancer in the US in a study from 1990 to 2021 [1,2]. Despite the relative stability in deaths attributed to skin cancer, the global burden of skin cancer has increased continuously over the last 30 years [2]. One in five Americans will be diagnosed with a skin cancer by age 70 [1,2]. The incidence of skin cancers is reported to increase for all age groups up to and through 95 years of age. Since 2021, skin cancer is a substantial cancer burden in individuals 65 years or older and affects primarily male individuals [3]. Studies reveal that the incidence and mortality rates of various cancers in the elderly and extremely old individuals are on the rise worldwide, with most types peaking around the ages of 75 to 90, followed by a sharp decline [3].

Skin cancer is one of the costliest cancer treatments in the US, exerting substantial health and economic burdens [4]. Skin cancer types include basal cell carcinoma (BCC), squamous cell carcinoma (SCC), and melanoma [5,6,7,8]. Average annual total cost for skin cancer treatment increased by 126.2% during the 2007-2011 period, compared to a 25.1% increase for all other cancers [9]. The annual medical costs of treating skin cancer have reached $8.9 billion [1]. Early diagnosis, especially for melanoma, can save money and improve quality of life [1]. Early detection is critical; the estimated 5-year survival rate for melanoma drops from over 99% if detected in its earliest stages to about 14% if detected in its latest stages [2].

Population growth is indicated as the main driver for the increased rate of cancer development; however, increased exposure to UV radiation is also a cause [2]. Countries experiencing the greatest burden of skin cancer included Australia, New Zealand, and the US [5]. The predicted rates of incidence and prevalence attributable to BCC and SCC, may continue to rise through 2050, whereas the deaths because of SCC and the burden of melanoma may exhibit a decreasing trend [2]. The age-standardized prevalence rate of melanoma is expected to decrease by 45% by 2050. SCC exhibits the highest age-standardized rate of prevalence and deaths while BCC displayed the highest incidence rate with few deaths attributed to this skin cancer [2,6].

BCC represents the most predominant type of human malignancy globally, comprising roughly 70% to 80% of all non-melanoma skin malignancies. SCC ranks as the second most common keratinocyte carcinoma, accounting for roughly 20-25% of non-melanoma lesions, with a metastasis risk of 1-5% [2,6]. The U.S. Preventive Services Task Force USPSTF 2023, concluded that there is insufficient evidence to endorse or oppose routine screening for skin and related cancers [7]; however, at least one report suggests that cancer screening for older adults should be individualized [8]. Older patients with blue, green, or hazel eyes and blond hair can have multiple skin cancers by age 65 and therefore may warrant early screening as a precaution [8].

Although melanomas represent fewer than 5% of all skin cancers in the United States, they account for approximately 75% of all skin-cancer-related deaths and are responsible for over 10,000 deaths annually in the United States alone [2]. Computer models recently developed based on enlarged images of skin cancers allow medical practitioners the ability to detect skin cancers early in their growth. The rise in the number of skin cancer cases in the elderly underscores the need for additional skin screening of patients 65 and older [9].

Due to the rise in skin cancer cases in older individuals, new tests are needed that can be used to rapidly classify skin lesions. The purpose of this paper is to present a new rapid noninvasive method using optical coherence tomography and AI to classify skin cancers including melanoma, BCC, and SCC to handle the rising number skin cancers. This paper describes a new instrument (OptoScope) used in this study to classify skin cancers. OptoScope is used to noninvasively collect gray scale OCT images of normal skin in vivo and camera images of skin cancer biopsies in vitro. It consists of a modified Lumedica OQ 2.0 OCT (Lumedica Inc, Durham, NC) operating at a wavelength of 840 nm collecting 13,000 frames per second as described previously [10,11].

2. Methods

OCT gray-scale images were used to classify both normal skin and cancerous skin lesions. Lesions were studied immediately after a biopsy was taken and then processed and stained with H&E for standard histopathology and diagnosis by a board-certified dermatopathologist. All images were collected as part of IRB approved clinical studies on skin cancers at Summit Health (Berkeley Heights, NJ) and Rutgers Dermatology Center after obtaining informed consent from the patients. Clinical diagnoses were made by board-certified dermatopathologists after H&E staining and review of the tissue sections as part of routine clinical skin excisional protocols. Measurements were made on OCT images of control skin (N = 200), melanomas (N = 106), squamous cell (N = 73) and basal cell carcinomas (N = 80) on patients ranging in age from 60 to 95 years of age with Fitzpatrick skin types I and II. The OptoScope includes a handpiece that collects both white and infrared light reflected light from the sample and is connected to an OCT that operates using an I5 computer that is used to process gray-scale images.

All OCT images were created by scanning the sample cross sections using the volume scan app on the OptoScope. The samples are scanned vertically creating 128 gray-scale images at each vertical location within the sample camera image. The gray-scale-scans were color-coded using image J as reported previously [10,11]. The OCT gray-scale images were broken into green, blue, and red subchannel images based on the pixel density using a lookup table [10,11]. By breaking up the pixel intensity distribution at each point into low (green), medium (blue), and high (red) intensities, it is possible to examine differences in reflection of the different layers of skin and skin lesions. The green and blue images contain information about the stratum corneum, and intermediate filaments as described previously [10,11]. Loss of the green subchannel reflection is associated with formation of cellular and tissue aggregates found in skin cancers while increased blue subchannel surface reflections are associated with intermediate filament mutations and loss of hyporeflective regions in the sample image [10,11].

In this study, a transfer learning-based Convolutional Neural Network (CNN) model was used. The ResNet18 architecture using standard hyperparameters was implemented to map traditional Euclean classification of OCT images of normal skin versus melanoma, SCC, or BCC. The gray-scale images were converted into three-channel inputs to match the ResNet18 requirements, and data augmentation techniques such as random horizontal flipping, rotation, and resizing were applied to enhance generalization, while normalization followed ImageNet standards. The hyperbolic parameters were used for a Resnet18 CNN map using traditional Euclidean weights and activations into a non-Euclidean space. Using a 5-fold cross-validation strategy in which the data from the training model was broken into 5 parts and the training cycle was repeated 5 times, was used to ensure robust performance evaluation, the final classification layer of ResNet18 was modified to output a single node for binary classification. The network was trained using the Adam optimizer with Binary Cross-Entropy loss over multiple epochs, with the best-performing model saved based on validation accuracy. To provide interpretability, Gradient-weighted Class Activation Mapping (Grad-CAM) was employed to highlight the discriminative regions used by the model when distinguishing skin lesions from normal skin. Finally, model performance was assessed through test accuracy, confusion matrices, sensitivity, specificity, and ROC-AUOC analysis, offering both predictive power and clinical relevance. Each unknown lesion was compared to normal skin to calculate a probability that the sample was cancerous or like normal skin.

3. Results

The CNN models were able to differentiate BCC from, SCC, and from melanoma with specificities and sensitivities between 92% and 100% as reported previously [10,11]. OCT gray-scale images and AI data were collected on melanomas, SCCs, and BCCs to evaluate different cross-sections of skin cancers compared to normal skin. Figure 1 shows a color-coded OCT image of normal skin in vivo (A), and the green (B), blue (C) and red (D) subchannel images. The yellow surface color in A represents the stratum corneum, the pink and red illustrate the location of the granulating layers, and the blue is from the papillary dermis and basal cell layer [10,11]. These image details change in cancerous lesions and can be used along with AI to evaluate different locations within the sample.

Figure 1 Color-coded OCT image of normal skin in vivo (A), and the green (B), blue (C) and red (D) subchannel images. The yellow color represents the stratum corneum, the pink and red illustrate the location of the granulating layers, and the blue is from the papillary dermis and basal cell layer [10,11]. The green and blue subchannel images change in cancerous skin lesions while the red subchannel does not appear to provide any additional detailed information of changes in skin cancer.

As shown in Figure 2 color-coded OCT images of normal skin (A), basal cell carcinoma (B-BCC), squamous cell carcinoma (C-SCC), and melanoma (MEL-D) look different. These images were obtained by color-coding gray-scale images using image J. Note normal skin has a bright yellow stratum corneum, pink and yellow germinative layers, and blue basal and papillary layers. Note the lack of the stratum corneum in BCC (B), SCC (C), and melanoma (D).

Figure 2 Color-coded OCT images of normal skin (A), basal cell carcinoma (B-BCC), squamous cell carcinoma (C-SCC), and melanoma (MEL-D) obtained after color-coding gray-scale images. Note normal skin has a bright yellow stratum corneum (see arrow in A), pink and yellow germinative layers, and blue basal and papillary layers. Cancerous lesions show a reduced yellow stratum corneum (white arrow in B) and an altered basal cell/papillary dermis layer (red arrow in B). Note absence of yellow stratum corneum in SCC (C), and melanoma (D).

Changes in the green subchannel images of BCC, SCC, and melanoma associated with loss of the stratum corneum that occurs in skin cancer are shown in Figure 3. Notice how the green subchannel becomes faint and even almost disappears when a skin cancer is present (Figures 3B, 3C, and 3D).

Figure 3 The green OCT subchannel color-coded OCT pixel intensity images for normal skin (A), basal cell carcinoma (B), squamous cell carcinoma (C), and melanoma (D). Note green subchannel images for BCC (white arrow shows some stratum corneum while red arrow shows loss of the stratum corneum), SCC, and MEL. The cancerous lesions all have lower pixel intensities compared to normal skin (see arrow in A). Changes in the green subchannel image can be quantitatively assessed as using CNN modeling and AI analysis. The changes in skin cancer images occur because of deposition of cellular and tissue aggregates in cancerous tissues.

The blue OCT subchannel images also change in images of skin cancers. Figure 4 shows blue subchannel images of normal skin (A) and skin cancers (B-D) showing the disappearance of the hyporeflective (black) region that is associated with mutations in the intermediate filaments in the basal cell layer that is present in normal skin (see arrow in A). In the BCC lesion (B) the lesion is small (see red arrow) in contrast to areas of normal skin (see white arrow in A). These changes are more obvious in SCC (C) and melanoma (D) where very little of the hyporeflective regions are seen.

Figure 4 Blue OCT subchannel images for normal skin (A), basal cell carcinoma (B), squamous cell carcinoma (C), and melanoma (D). Note the blue subchannel image has a lower pixel intensity for BCC, SCC, and melanoma compared to normal skin (A). Note the hyporeflective blue subchannel image of normal skin is the black area (see arrow in A), while it is missing where the BCC lesion is observed (see red arrow in B), and is missing in SCC (C), and in melanoma (D). White arrow in B shows area of hyporeflective tissue that appears free of cancer. The differences in the blue subchannel images of normal skin and skin cancers can be quantitatively analyzed using AI in the CNN models.

Figure 5 shows a gross tissue camera image of a large skin excision from a patient with a melanoma diagnosed by a board-certified dermatopathologist. The diagnosis was based on histopathology conducted on tissue numbered sections shown in Figure 5. The excision consisted of most of the scalp of a male patient and the locations tested included 1 through 9 are listed on Figure 5. The locations 6, 7, 8 and 9 were free of cancer based on histopathology.

Figure 5 A gross tissue camera image of a large skin excision from a male patient with a suspected melanoma based on visual and dermoscopic visualization. Regions of melanoma were diagnosed based on histopathology of multiple tissue sections by a board-certified dermatopathologist. The edges of the lesion locations 6, 7, 8, and 9 were clear of the cancer based on histopathology as well as from OCT images and CNN model analysis. Points 1, 2, 3, 4, and 5 were analyzed in more detail using the CNN models and AI. Points 1, 2, 3, 4 and 5 were evaluated in detail in Figures 6-10 from 10× camera images, OCT images, and CNN models for melanoma.

Figure 6 Data from location 1 from lesion shown in Figure 5 includes the probability of the lesion being a melanoma (left table), color-coded OCT image (top middle left), OCT lesion camera image at 10× (top right), green OCT subchannel image (bottom center), and blue subchannel OCT image (bottom right) of cross-section #6. Based on the CNN and the camera image, the location of the melanoma is seen in vertical slices 6 and 7. Only OCT image of cross-section slice #6 is shown in Figure 6.

Figure 7 Data from location 2 shown in Figure 5 including probability of melanoma (left table), color-coded OCT image (top middle left), OCT camera image at 10× of lesion (top right), green OCT subchannel image (bottom center), and blue subchannel OCT image (bottom right). Based on the CNN model and the camera image the location of the melanoma is seen in vertical slice #2 (see vertical red line in camera image). Arrow in green channel image shows location of lesion based on loss of the green channel image while the arrow in the blue channel shows where the hyporeflective region is missing.

Figure 8 Data from location 3 from Figure 5 including probability of melanoma (left table), color-coded OCT image (top middle left), OCT camera image at 10× of lesion (top right), green OCT subchannel image (bottom center), and blue subchannel OCT image (bottom right). Based on CNN models and the camera image the melanoma is seen in vertical slices 6 and 7 (see vertical red lines in camera image). The OCT images shown are from slice #2.

Figure 9 Data from location 4 from Figure 5 including probability of melanoma (left table), color-coded OCT image (top middle left), OCT camera image at 10× of lesion (top right), green channel image (bottom center), and blue channel OCT image (bottom right). Based on CNN and camera image of the location contains no melanoma lesions. The OCT images shown are from slice #5.

Figure 10 Data from location 5 shown in Figure 5 including probability of melanoma (left table), color-coded OCT image (top middle left), OCT camera image at 10× of lesion (top right), green channel image (bottom center), and blue channel OCT image (bottom right). Based on CNN and the camera image of the location of the melanoma is seen in vertical slices 1, 2, and possibly 3 (see vertical red line in camera image). OCT images shown are from slice #2.

The scans shown are only a few of the 128 scans recorded by the OCT to simplify the analyses. The probabilities listed in the table were obtained from CNN models conducted on gray-scale images for each of the vertical scans. The presence of melanoma is based on a probability of at least 90% or greater. Note that several of the locations identified by the CNN model as melanoma showed dark pigmented spots in the OCT camera images. The slices defined as a high probability of being melanoma had irregular rough surfaces as seen in the OCT images.

Figures 6-10 show OCT images and data collected from locations 1 to 5 from Figure 5 are shown in Figures 6-10.

Figures 6-10 show camera images (10×) of excised scalp skin containing the melanoma shown in Figure 5, as well as color-coded OCT images, and green and blue subchannel images. The images were collected from locations 1, 2, 3, 4 and 5 of Figure 5, using vertical scans perpendicular to the lesion surface. CNN model predictions are shown in percent out of 100% of the probability that each vertical scan contained cancerous melanoma lesions. The red vertical line shows the location of cancerous lesions based on the CNN model in the camera image.

Data from a biopsy of a SCC is shown in Figure 11A. This lesion was diagnosed by a board-certified dermatopathologist to be a SCC. The SCC probability is shown (top left) as well as the color-coded OCT image with vertical lines where scans were done, and OCT green and blue subchannel images (bottom left and right) are shown. The probability that vertical slice #4 was a SCC was 97%. Note the green and blue subchannel images are different from that of normal skin. Figure 11B shows a BCC with camera image, OCT images of slice #1, and probability of it being a BCC based on the CNN model of BCC.

Figure 11 Data from a biopsy of a SCC (A) and a BCC (B). These lesions were diagnosed by a board-certified dermatopathologist to be SCC and BCC. The SCC probability is shown (top left) as well as the color-coded OCT image with vertical lines where scans were done, and green and blue OCT subchannel images (bottom left and right). The probability that vertical slice #4 was a SCC was 97%. Note the green and blue OCT subchannel images are different from that of normal skin. (B) shows a BCC with camera image, OCT images, and probability of it being a BCC based on the CNN model of BCC of all the slices was greater than 90%. The OCT image of slice #2 is shown in (B).

4. Discussion

The need to screen for skin cancer in subjects 65 years and older is critical for several reasons [12]. There are 56 million Americans 65 years and older in the US [13] and the median age for melanoma diagnosis is 66 years. 76.4% of cases are diagnosed in patients >55 years [14] with the highest incidence of melanoma is in subjects greater than 70 years of age [15]. The highest rate of death for melanoma is in the 65 to 74 age group, with the median age at death being 72.5 [16]. Clearly early melanoma detection is important for patients greater than 55 years of age. Beyond that rural versus urban living locations are important considerations in skin cancer diagnosis

Rural living locations are associated with lower rates of localized cancer at diagnosis and higher rates of distant disease [17]. The location of lesions on the skin is critical since those located in non-obvious places may be missed until patients are symptomatic [12]. Under these conditions the disease when diagnosed may be advanced or metastatic [12]. Elderly patients with aggressive skin cancer may not be able to receive treatment. Hispanic patients and Black patients are 45% and 58% less likely to visit an outpatient dermatologist for their known dermatologic condition [18]. Patients who live in rural areas are uniquely faced with the problem of long travel distances and travel times, which impacts their ability to seek timely diagnosis and receive optimal care once cancer is diagnosed [19].

Beyond considering the living location condition, the prevalence of multiple skin cancers in subjects over 65 years of age with blue, green and hazel eyes and/or blond or red hair is an important issue that has not been fully addressed. Not only is screening for skin cancer critical but the association between skin cancer and other non-primary malignancies such as lip, oropharyngeal, salivary gland, prostate, breast, and colorectal cancers have been reported [20]. Approximately 29% of patients with a first BCC will develop at least one or more BCCs during their lifetime [21]. There are few statistics published about the problem of repeated skin cancers but the need for rapid techniques to classify skin lesions is of critical importance since many senior dermatology patients complain of repeated visits to the dermatologist that are very time consuming. The screening of skin lesions can be facilitated by a rapid classification technique that can be done remotely.

We have developed a rapid noninvasive classification test for screening skin cancers based on gray-scale OCT images and CNN models of melanoma, SCC, and BCC. The difference between normal skin and skin cancers involves formation of cancer associated fibroblasts and fibrotic collagen as well as new thin blood vessels that change the texture, mechanical properties, and reflectance of the tissue [10,11]. These changes can be qualitatively assessed by visual review of the green and blue OCT subchannel images and quantitatively analyzed using CNN models of the gray-scale OCT images and AI. The results are both quantitative probabilities of the cancer classifications in addition to the ability to visually confirm the location of the lesions from the OCT images. This combination provides a visual mechanism to check AI model predictions with image observations. Beyond this the edges and depth of lesions can be determined by selecting different parts of the lesion and then running the CNN model to define the lesion classification.

The changes in cancerous tissues appear to reflect changes in the cellular organization observed from histopathology on cell-cell connections and cell-extracellular matrix interactions [22]. Cellular mutations in skin cancer cells precede stiffening of individual cells, changes in the intermediate filaments, and deposition of large amounts of fibrotic tissues [10,11]. These changes are likely caused by disruptions in connections between cells and the extracellular matrix that modify mechanotransduction pathways [22]. This leads to upregulation of TGF-beta synthesis of ECM resulting in increased local stiffness [22]. This also disrupts energy transmission throughout the skin which is required to maintain normal homeostasis [22].

The ability to noninvasively section the lesion and create 3D images of the tissue allows one to locate the cancerous tissue prior to surgery [23]. This technique provides a means to identify the clear edges and the centers of the lesions and allows for the entire cancer to be removed in one surgical procedure. It also provides a means for the pathology technician and pathologist to identify which parts of large friable biopsies to process and review first saving time and money.

Several study results are reported on the use of OCT to classify melanomas and SCCs [24,25,26,27]. However, none of these studies report green and blue subchannel images of the lesions that can be used to support AI results. This new method offers a rapid manner to classify lesions by certified medical technicians with the help of dermatologists. It takes only about 2 minutes to collect the OCT images and run the CNN models to classify if a lesion is cancerous or benign. OCT gray scale images can be collected by a trained technician and used as part of a telemedicine session, the results of which are reviewed and interpreted by a trained physician remotely [28]. Using this approach, large numbers of patients can be rapidly classified and evaluated, especially in remote areas using mobile healthcare units where dermatologists are in short supply. The digital image is then available to be viewed by experts who can then refer the patient for treatment.

The limitations of the methods described in this manuscript are as follows. The accuracy of the convolutional neural network models is improved by collecting data on additional cancerous as well as noncancerous skin lesions. Further modification of the models may prevent overfitting of the data and provide more accuracy. The technique is also limited by the lack of consensus among dermatopathologists on lesion diagnosis, especially of pigmented lesions including melanomas. Further studies with lesions with mixed components will be conducted to better define how the CNN models can be applied to complex skin cancers.

5. Conclusions

The need to screen for skin cancer in subjects 65 years and older is critical for several reasons. There are 56 million Americans 65 years and older in the US and the median age for melanoma diagnosis in the United States is 66 years. 76.4% of cases are diagnosed in patients >55 years with the highest incidence of melanoma in subjects greater than 70 years of age. The need to screen for skin cancer in subjects 65 years and older is critical. We report that OCT gray scale images and green and blue subchannel images it is possible to locate the cancerous portions of both of a melanoma, SCC and BCC with a lesion classification and location probability of between 90 and 99%. Using this approach, large numbers of patients can be rapidly classified and evaluated, especially in remote areas using mobile healthcare units where dermatologists are in short supply. The digital image is then available to be viewed by experts who can refer the patient for treatment if needed.

Acknowledgments

The authors would like to acknowledge the collaboration of Drs. Hari Nadaminti (Summit Health, Berkeley Heights, NJ) and Cindy Wassef (Rutgers Dermatology Center, Somerset NJ) for assistance with collecting OCT and camera images of BCC, SCCs and the melanoma.

Author Contributions

Conceptualization, F.H.S. and T.D.; methodology, F.H.S; formal analysis, F.H.S., TD.; investigation, F.H.S.; data curation, T.D.; writing—original draft preparation, F.H.S. and T.D.; writing—review and editing, F.H.S., T.D. Both authors have read and agreed to the published version of the manuscript.

Competing Interests

The authors have declared that no competing interests exist.

Data Availability Statement

Data availability at optovibronex.com.