Lithological mapping with pseudo-labelling: Promise or overestimation  in data-scarce settings?

Szilárd Szabó; Abdelmajeed A.  Elrasheed; Lilla Kovács; Imre J. Holb; Szilárd B. Likó; Dávid Abriha

doi:10.15201/hungeobull.74.4.1

Szilárd Szabó Department of Physical Geography and Geoinformatics, Faculty of Sciences and Technology, University of Debrecen. National Laboratory for Water Science and Water Safety, University of Debrecen, Debrecen, Hungary https://orcid.org/0000-0002-2670-7384
Abdelmajeed A. Elrasheed Department of Physical Geography and Geoinformatics, Faculty of Sciences and Technology, University of Debrecen, Debrecen, Hungary ; Department of Geology, University of Khartoum, Khartoum, Sudan https://orcid.org/0000-0001-6282-701X
Lilla Kovács Doctoral School of Geosciences, Faculty of Sciences and Technology, University of Debrecen, Debrecen, Hungary https://orcid.org/0009-0008-5007-5525
Imre J. Holb Institute of Horticulture, University of Debrecen, Debrecen, Hungary ; Plant Protection Institute, Centre for Agricultural Research, HUN-REN, Budapest, Hungary https://orcid.org/0000-0002-6368-4660
Szilárd Likó - https://orcid.org/0000-0001-7086-9957
Dávid Abriha Department of Physical Geography and Geoinformatics, Faculty of Sciences and Technology, University of Debrecen, Debrecen, Hungary https://orcid.org/0000-0003-1187-5892

DOI: https://doi.org/10.15201/hungeobull.74.4.1

Keywords: Multiple Adaptive Regression Splines (MARS), Random Forest (RF), self-training, probability, data augmentation

Abstract

Reference data are the most crucial points in model building. In geoscience, a scarcity of sufficient reference data is common. Pseudo-labelling (PL), i.e. incorporating high-probability data in the model-building process, offers a potential solution. We aimed to reveal the efficiency of PL in lithological mapping in a vegetation-free arid region of Sudan. Multiple Adaptive Regression Splines (MARS) and Random Forest (RF) were used to classify a Landsat 9 image. Reference data were collected during fieldwork and through visual interpretation. Image processing yielded classified maps with associated probability layers, from which 1000 additional traditional samples (PL data) were extracted at a 95 percent probability. A detailed accuracy assessment was conducted, and accuracy measures were evaluated using statistical analysis and visual inspection. MARS was found to be an ambiguous classifier because the probability was too optimistic related to the overall accuracy (OA) (81% of samples had above 99% probability, OA = 98.2%) compared to RF (21% above 99%, OA = 98.1%); that is, despite the high probability, the accuracy improvement was only 0.1 percent. At the class level, the correlation between probability and the F1-score was low (0.21%). The original and PL-based models resulted in different maps with improved accuracy, although the new model version showed lower probability values for both the classifiers. Visual inspection proved essential for better insights into the spatial patterns: expert knowledge is crucial for controlling the occurrence of rock types and identifying false classifications. The main finding is that probability should be handled carefully, as it does not guarantee high model performance in classification, although the PL approach can lead to more reliable maps.

References

ABDELKAREEM, M., HAMIMI, Z., EL-BIALY, M.Z., KHAMIS, H. and ABDEL WAHED, S.A. 2021. Integration of remote-sensing data for mapping lithological and structural features in the Esh El-Mallaha area, west Gulf of Suez, Egypt. Arabian Journal of Geosciences 14. (6): 497. https://doi.org/10.1007/s12517-021-06791-3

ABDELRAHMAN, S., IBRAHIM, M.A.E., LI, H., ABDEL RAHMAN, E.M. and FAISAL, M. 2024. Geochemical characteristics of Neoproterozoic meta-volcanic rocks of Ariab Auriferous Volcanogenic Massive Sulfide deposit, Red Sea hills, North-East Sudan. Journal of African Earth Sciences 216. 105305. https://doi.org/10.1016/j.jafrearsci.2024.105305

ABDELSALAM, M.G. and STERN, R.J. 1993. Tectonic evolution of the Nakasib suture, Red Sea Hills, Sudan: Evidence for a late Precambrian Wilson Cycle. Journal of the Geological Society 150. (2): 393–404. https://doi.org/10.1144/gsjgs.150.2.0393

ABDELSALAM, M.G., STERN, R.J. and BERHANE, W.G. 2000. Mapping gossans in arid regions with Landsat TM and SIR-C images: The Beddaho Alteration Zone in northern Eritrea. Journal of African Earth Sciences 30. (4): 903–916. https://doi.org/10.1016/S0899-5362(00)00059-2

ABRAMS, M. and YAMAGUCHI, Y. 2019. Twenty years of ASTER contributions to lithological mapping and mineral exploration. Remote Sensing 11. (11): 1394. https://doi.org/10.3390/rs11111394

AMUSUK, D.J., HASHIM, M., POUR, A.B. and MUSA, S.I. 2016. Utilization of Landsat-8 data for lithological mapping of basement rocks of plateau state North Central Nigeria. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-4-W1, 335–337. TC IV
International Conference on Geomatic and Geospatial Technology (GGT) 2016. Vol. XLII-4/W1. 3–5 October 2016, Kuala Lumpur, Malaysia. https://doi.org/10.5194/isprs-archives-XLII-4-W1-335-2016

ASGHAR, S., CHOI, J., YOON, D. and BYUN, J. 2020. Spatial pseudo-labelling for semi-supervised facies classification. Journal of Petroleum Science and Engineering 195. 107834. https://doi.org/10.1016/j.petrol.2020.107834

AYDAV, P.S.S. and MINZ, S. 2018. Classification of hyperspectral images using self-training and a pseudo validation set. Remote Sensing Letters 9. (11): 1109–1117. https://doi.org/10.1080/2150704X.2018.1511932

BACHRI, I., HAKDAOUI, M., RAJI, M., TEODORO, A.C. and BENBOUZIANE, A. 2019. Machine learning algorithms for automatic lithological mapping using remote sensing data: A case study from Souk Arbaa Sahel, Sidi Ifni Inlier, Western Anti-Atlas, Morocco. ISPRS International Journal of Geo-Information 8. (6): 1–20. https://doi.org/10.3390/ijgi8060248

BARSI, Á., KUGLER, ZS., LÁSZLÓ, I., SZABÓ, GY. and ABDULMUTALIB, H.M. 2018. Accuracy dimensions in remote sensing. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLII–3. 61–67. https://doi.org/10.5194/isprs-archives-XLII-3-61-2018

BELGIU, M. and DRĂGUŢ, L. 2016. Random forest in remote sensing: A review of applications and future directions. ISPRS Journal of Photogrammetry and Remote Sensing 114. 24–31. https://doi.org/10.1016/j.isprsjprs.2016.01.011

BOEHMKE, B. and GREENWELL, B.M. 2019. Hands-On Machine Learning with R. London, Chapman and Hall/CRC. https://doi.org/10.1201/9780367816377

BOEMHKE, B. and GREENWELL, B.M. 2020. Hands-On Machine Learning with R. London, Chapman and Hall/CRC https://bradleyboehmke.github.io/HOML/

BREIMAN, L. 2001. Random forests. Machine Learning 45. (1): 5–32. https://doi.org/10.1023/A:1010933404324

BUI, D.H. and MUCSI, L. 2022. Predicting the future land-use change and evaluating the change in landscape pattern in Binh Duong province, Vietnam. Hungarian Geographical Bulletin 71. (4): 349–364. https://doi.org/10.15201/hungeobull.71.4.3

CAO, C., CHICCO, D. and HOFFMAN, M.M. 2020. The MCC-F1 Curve: A Performance Evaluation Technique for Binary Classification. arXiv:2006.11278, arXiv. https://doi.org/10.48550/arXiv.2006.11278

CHEN, Y., SUI, Y. and SHAYILAN, A. 2023. Constructing a high-performance self-training model based on support vector classifiers to detect gold mineralization-related geochemical anomalies for gold exploration targeting. Ore Geology Reviews 153. 105265. https://doi.org/10.1016/j.oregeorev.2022.105265

CHICCO, D. and JURMAN, G. 2020. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics 21. (6): https://doi.org/10.1186/s12864-019-6413-7

COHEN, J. 2013. Statistical Power Analysis for the Behavioural Sciences. 2nd edition. London, Routledge. https://doi.org/10.4324/9780203771587

COLLINS, L., MCCARTHY, G., MELLOR, A., NEWELL, G. and SMITH, L. 2020. Training data requirements for fire severity mapping using Landsat imagery and random forest. Remote Sensing of Environment 245. 111839. https://doi.org/10.1016/j.rse.2020.111839

CONGALTON, R.G. 1991. A review of assessing the accuracy of classifications of remotely sensed data. Remote Sensing of Environment 37. (1): 35–46. https://doi.org/10.1016/0034-4257(91)90048-B

CRUZ, C., MCGUINNESS, K., PERRIN, P.M., O’CONNELL, J., MARTIN, J.R. and CONNOLLY, J. 2023. Improving the mapping of coastal invasive species using UAV imagery and deep learning. International Journal of Remote Sensing 44. (18): 5713–5735. https://doi.org/10.1080/01431161.2023.2251186

CURRAN, J. and HERSH, T. 2012. Hotelling: Hotelling’s T^2 Test and Variants. p. 1.0-8 [Dataset]. https://doi.org/10.32614/CRAN.package.Hotelling

DUNHAM, M.W., MALCOLM, A. and KIM WELFORD, J. 2020. Improved well-log classification using semi-supervised label propagation and self-training, with comparisons to popular supervised algorithms. Geophysics 85. (1): 1–15. https://doi.org/10.1190/geo2019-0238.1

EL-OMAIRI, M.A. and GAROUANI, A.E. 2023. A review on advancements in lithological mapping utilizing machine learning algorithms and remote sensing data. Heliyon 9. (9): https://doi.org/10.1016/j.heliyon.2023.e20168

EVANS, J.S., MURPHY, M.A. and RAM, K. 2023. spatialEco: Spatial Analysis and Modelling Utilities. Version 2.0-2. (Computer software). https://cran.r-project.org/web/packages/spatialEco/index.html

FOODY, G.M. 2009. Sample size determination for image classification accuracy assessment and comparison. International Journal of Remote Sensing 30. (20): 5273–5291. https://doi.org/10.1080/01431160903130937

FRIEDMAN, J.H. 1991. Multivariate adaptive regression splines. The Annals of Statistics 19. (1): 1–67. https://doi.org/10.1214/aos/1176347963

GAO, M., WANG, G., XU, Y., MOU, N., HUANG, L., ZUO, L. and WU, R. 2023. 3D mineral exploration Cu-Zn targeting with multi-source geoscience datasets in the Weilasituo-bairendaba district, Inner Mongolia, China. Frontiers in Earth Science 11. https://doi.org/10.3389/feart.2023.1102640

GRANDINI, M., BAGLI, E. and VISANI, G. 2020. Metrics for Multi-Class Classification: An Overview. arXiv:2008.05756, arXiv. https://doi.org/10.48550/arXiv.2008.05756

HAMIMI, Z., FOWLER, A.-R., COLLINS, A., LIÉGEOIS, J.-P., ABDELSALAM, M. and ABD EL-WAHED, M. 2021. The Geology of the Arabian-Nubian Shield. Regional Geology Reviews, Cham, Springer Nature. https://doi.org/10.1007/978-3-030-72995-0

HAN, W., ZHANG, X., WANG, Y., WANG, L., HUANG, X., LI, J., WANG, S., CHEN, W., LI, X., FENG, R., FAN, R., ZHANG, X. and WANG, Y. 2023. A survey of machine learning and deep learning in remote sensing of geological environment: Challenges, advances, and opportunities. ISPRS Journal of Photogrammetry and Remote Sensing 202. 87–113. https://doi.org/10.1016/j.isprsjprs.2023.05.032

HARVEY, A.S. and FOTOPOULOS, G. 2016. Geological mapping using machine learning algorithms. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLI-B8. 423–430. XXIII ISPRS Congress, Commission VIII, Volume XLI-B8. 19 July 2016, Prague, Czech Republic. https://doi.org/10.5194/isprsarchives-XLI-B8-423-2016

HESSELBARTH, M.H.K., SCIAINI, M., NOWOSAD, J. and HANS, S. 2025. Landscape Metrics for Categorical Map Patterns. p. 2.2.1 (Dataset). https://doi.org/10.32614/CRAN.package.landscapemetrics

JAMES, G., WITTEN, D., HASTIE, T. and TIBSHIRANI, R. 2013. An Introduction to Statistical Learning: With Applications in R. 1st edition. Corr. 7th printing 2017 edition. Cham, Springer.

KIM, E., MOON, J., SHIM, J. and HWANG, E. 2024. Predicting invasive species distributions using incremental ensemble-based pseudo-labelling. Ecological Informatics 79. 102407. https://doi.org/10.1016/j.ecoinf.2023.102407

KUHN, M. 2022. caret: Classification and Regression Training. https://CRAN.R-project.org/package=caret

LI, C., WANG, J., WANG, L., HU, L. and GONG, P. 2014. Comparison of Classification Algorithms and Training Sample Sizes in Urban Land Classification with Landsat Thematic Mapper Imagery. Remote Sensing, 6(2), Article 2. https://doi.org/10.3390/rs6020964

LI, J., SUN, B., LI, S. and KANG, X. 2022. Semi-supervised semantic segmentation of remote sensing images with consistency self-training. IEEE Transactions on Geoscience and Remote Sensing 60. 1–11. https://doi.org/10.1109/TGRS.2021.3134277

LIKÓ, S.B., HOLB, I.J., OLÁH, V., BURAI, P. and SZABÓ, S. 2024. Deep learning-based training data augmentation combined with post-classification improves the classification accuracy for dominant and scattered invasive forest tree species. Remote Sensing in Ecology and Conservation 10. (2): 203–219. https://doi.org/10.1002/rse2.365

LIU, H., WU, K., XU, H. and XU, Y. 2021. Lithology classification using TASI thermal infrared hyperspectral data with convolutional neural networks. Remote Sensing 13. (16): 3117. https://doi.org/10.3390/rs13163117

LUQUE, A., CARRASCO, A., MARTÍN, A. and DE LAS HERAS, A. 2019. The impact of class imbalance in classification performance metrics based on the binary confusion matrix. Pattern Recognition 91. 216–231. https://doi.org/10.1016/j.patcog.2019.02.023

MATCHARASHVILI, T., CZECHOWSKI, Z. and ZHUKOVA, N. 2019. Mahalanobis distance-based recognition of changes in the dynamics of a seismic process. Nonlinear Processes in Geophysics 26. (3): 291–305. https://doi.org/10.5194/npg-26-291-2019

MAXWELL, A.E., WARNER, T.A. and FANG, F. 2018. Implementation of machine-learning classification in remote sensing: An applied review. International Journal of Remote Sensing. 39. (9): 2784–2817. https://doi.org/10.1080/01431161.2018.1433343

MEAD, R.A., SHARIK, T.L., PRISLEY, S.P. and HEINEN, J.T. 1981. A computerized spatial analysis system for assessing wildlife habitat from vegetation maps. Canadian Journal of Remote Sensing 7. (1): 34–40. https://doi.org/10.1080/07038992.1981.10855007

MEYER, H. and PEBESMA, E. 2022. Machine learning-based global maps of ecological variables and the challenge of assessing them. Nature Communications 13. (1): 2208. https://doi.org/10.1038/s41467-022-29838-9

MILBORROW, S., HASTIE, T. and TIBSHIRANI, R. 2024. Package ‘earth’: Multivariate Adaptive Regression Splines. CRAN. https://doi.org/10.32614/CRAN.package.earth

NAGPAL, A. and SINGH, V. 2019. Coupling multivariate adaptive regression spline (MARS) and random forest (RF): A hybrid feature selection method in action. International Journal of Healthcare Information Systems and Informatics 14. (1): 1–18. https://doi.org/10.4018/IJHISI.2019010101

NAIR, P., SRIVASTAVA, D.K. and BHATNAGAR, R. 2023. Application of machine learning in mineral mapping using remote sensing. In IOT with Smart Systems. Eds.: CHOUDRIE, J., MAHALLE, P., PERUMAL, T. and JOSHI, A., Cham, Springer Nature, 27–35. https://doi.org/10.1007/978-981-19-3575-6_4

OTHMAN, A.A. and GLOAGUEN, R. 2014. Improving lithological mapping by SVM classification of spectral and morphological features: The discovery of a new chromite body in the Mawat Ophiolite Complex (Kurdistan, NE Iraq). Remote Sensing 6. (8): 30. https://doi.org/10.3390/rs6086867

QUIRÓS, E., FELICÍSIMO, Á.M. and CUARTERO, A. 2009. Testing multivariate adaptive regression splines (MARS) as a method of land cover classification of TERRA-ASTER satellite images. Sensors 9. (11): 9011–9028. https://doi.org/10.3390/s91109011

R Core Team, 2024. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. https://www.R-project.org/

ROTIGLIANO, E., MARTINELLO, C., AGNESI, V. and CONOSCENTI, C. 2018. Evaluation of debris flow susceptibility in El Salvador (CA): A comparison between multivariate adaptive regression splines (MARS) and binary logistic regression (BLR). Hungarian Geographical Bulletin 67. (4): 361–373. https://doi.org/10.15201/hungeobull.67.4.5

SHAHARE, Y.R., SINGH, M.P., SINGH, S.P., SINGH, P. and DIWAKAR, M. 2024. ASUR: Agriculture soil fertility assessment using random forest classifier and regressor. Procedia Computer Science 235. 1732–1741. https://doi.org/10.1016/j.procs.2024.04.164

SHAKER, A. 2023. Chapter 6 Effect sizes | STM1001 Topic 7: One-way ANOVA. https://bookdown.org/a_shaker/STM1001_Topic_7/6-effect-sizes.html

SHIM, J.W. 2024. Enhancing cross entropy with a linearly adaptive loss function for optimized classification performance. Scientific Reports 14. (1): 27405. https://doi.org/10.1038/s41598-024-78858-6

SHIRMARD, H., FARAHBAKHSH, E., HEIDARI, E., BEIRANVAND POUR, A., PRADHAN, B., MÜLLER, D. and CHANDRA, R. 2022. A comparative study of convolutional neural networks and conventional machine learning models for lithological mapping using remote sensing data. Remote Sensing 14. (4): 819. https://doi.org/10.3390/rs14040819

SIMARMATA, N., WIKANTIKA, K., TARIGAN, T.A., ALDYANSYAH, M., TOHIR, R.K., FAUZI, A.I. and FAUZIA, A.R. 2025. Comparison of random forest, gradient tree boosting, and classification and regression trees for mangrove cover change monitoring using Landsat imagery. The Egyptian Journal of Remote Sensing and Space Sciences 28. (1): 138–150. https://doi.org/10.1016/j.ejrs.2025.02.002

SZABÓ, SZ., HOLB, I.J., ABRIHA-MOLNÁR, V.É., SZATMÁRI, G., SINGH, S.K. and ABRIHA, D. 2024. Classification Assessment Tool: A program to measure the uncertainty of classification models in terms of class-level metrics. Applied Soft Computing 155. 111468. https://doi.org/10.1016/j.asoc.2024.111468

SZANIAWSKA, L. 2018. Lithological maps visualizing the achievements of geological sciences in the first half of the 19th century. Polish Cartographical Review 50. (2): 87–109. https://doi.org/10.2478/pcr-2018-0006

Willem, 2017 – https://stats.stackexchange.com/users/159052/willem, 2017, F1/Dice-Score vs IoU. https://stats.stackexchange.com/q/276144

ZHANG, L., YANG, L., MA, T., SHEN, F., CAI, Y. and ZHOU, C. 2021. A self-training semi-supervised machine learning method for predictive mapping of soil classes with limited sample data. Geoderma 384. 114809. https://doi.org/10.1016/j.geoderma.2020.114809

Lithological mapping with pseudo-labelling: Promise or overestimation in data-scarce settings?

Abstract

References

Clarivate - Web of Science

Scopus CiteScore

Scimago Journal & Country Rank

Abstracting & Indexing

Digital Archiving