Soil & Water Res., 2015, 10(4):218-227 | DOI: 10.17221/113/2015-SWR

Comparing different data preprocessing methods for monitoring soil heavy metals based on soil spectral featuresOriginal Paper

Asa Gholizadeh1, Luboš Borůvka1, Mohammad Mehdi Saberioon2, Josef Kozák1, Radim Vašát1, Karel Němeček1
1 Department of Soil Science and Soil Protection, Faculty of Agrobiology, Food and Natural Resources, Czech University of Life Sciences Prague, Prague, Czech Republic
2 Laboratory of Image and Signal Processing, Institute of Complex Systems, Faculty of Fisheries and Protection of Waters, University of South Bohemia in České Budějovice, Nové Hrady, Czech Republic

The lands near mining industries in the Czech Republic are subjected to soil pollution with heavy metals. Excessive heavy metal concentrations in soils not only dramatically impact the soil quality, but also due to their persistent nature and indefinite biological half-lives, potentially toxic metals can accumulate in the food chain and can eventually endanger human health. Monitoring and spatial information of these elements require a large number of samples and cumbersome and time-consuming laboratory measurements. A faster method has been developed based on a multivariate calibration procedure using support vector machine regression (SVMR) with cross-validation, to establish a relationship between reflectance spectra in the visible-near infrared (Vis-NIR) region and concentration of Mn, Cu, Cd, Zn, and Pb in soil. Spectral preprocessing methods, first and second derivatives (FD and SD), standard normal variate (SNV), multiplicative scatter correction (MSC), and continuum removal (CR) were employed after smoothing with Savitzky-Golay to improve the robustness and performance of the calibration models. According to the criteria of maximal coefficient of determination (R2cv) and minimal root mean square error of prediction in cross-validation (RMSEPcv), the SVMR algorithm with FD preprocessing was determined as the best method for predicting Cu, Mn, Pb, and Zn concentration, whereas the SVMR model with CR preprocessing was chosen as the final method for predicting Cd. Overall, this study indicated that the Vis-NIR reflectance spectroscopy technique combined with a continuously enriched soil spectral library as well as a suitable preprocessing method could be a nondestructive alternative for monitoring of the soil environment. The future possibilities of multivariate calibration and preprocessing with real-time remote sensing data have to be explored.

Keywords: heavy metals; preprocessing; support vector machine regression; visible-near infrared spectroscopy

Published: December 31, 2015  Show citation

ACS AIP APA ASA Harvard Chicago IEEE ISO690 MLA NLM Turabian Vancouver
Gholizadeh A, Borůvka L, Saberioon MM, Kozák J, Vašát R, Němeček K. Comparing different data preprocessing methods for monitoring soil heavy metals based on soil spectral features. Soil & Water Res.. 2015;10(4):218-227. doi: 10.17221/113/2015-SWR.
Share...
Download citation
  • BibTeX (.bib)
  • Bookends (.ris)
  • EasyBib (.ris)
  • EndNote (.enw)
  • EndNote 8 (.xml)
  • ISI WoS (.isi)
  • Medlars (.medlars)
  • Mendeley (.ris)
  • MODS (.xml)
  • Papers (.ris)
  • RefWorks (.txt)
  • RefManager (.ris)
  • RIS (.ris)
  • MS Word (.xml)
  • Zotero (.ris)
    • Open full article

    References

    1. Ben-Dor E., Banin A. (1990): Near-infrared reflectance analysis of carbonate concentration in soils. Applied Spectroscopy, 44: 1064-1069. Go to original source...
    2. Ben-Dor E., Banin A. (1995): Near infrared analysis (NIRA) as a rapid method to simultaneously evaluate several soil properties. Soil Science Society of American Journal, 59: 364-372. Go to original source...
    3. Ben-Dor E., Inbar Y., Chen Y. (1997): The reflectance spectra of organic matter in the visible near-infrared and short wave infrared region (400-2500 nm) during a controlled decomposition process. Remote Sensing of Environment, 61: 1-15. Go to original source...
    4. Borůvka L., Kozák J., Mühlhanselová M., Donátová H., Nikodem A., Němeček K., Drábek O. (2012): Effect of covering with natural topsoil as a reclamation measure on brown-coal mining dumpsites. Journal of Geochemical Exploration, 113: 118-123. Go to original source...
    5. Bradshaw A. (2000): The use of natural processes in reclamation - Advantages and difficulties. Landscape Urban Planning, 51: 89-100. Go to original source...
    6. Buurman P., Pape Th., Muggler C.C. (1997): Laser grainsize determination in soil genetic studies. Soil Science, 162: 211-218. Go to original source...
    7. Chen Q., Guo Z., Zhao J., Ouyang Q. (2012): Comparisons of different regressions tools in measurement of antioxidant activity in green tea using near infrared spectroscopy. Journal of Pharmaceutical and Biomedical Analysis, 60: 92-97. Go to original source... Go to PubMed...
    8. Chiang L.H., Pell R.J., Seasholtz M.B. (2003): Exploring process data with the use of robust outlier detection algorithms. Journal of Process Control, 13: 437-449. Go to original source...
    9. Chu X.L., Yuan H.F., Lu W.Z. (2004): Progress and application of spectral data pretreatment and wavelength selection methods in NIR analytical technique. Progress in Chemistry, 16: 528-542.
    10. Clark R.N., Roush T.L. (1984): Reflectance spectroscopy: quantitative analysis techniques for remote sensing application. Journal of Geophysical Research, 89: 6329-6340. Go to original source...
    11. Clark R.N., King T.V.V., Klejwa M., Swayze G.A., Vergo N. (1990): High spectral resolution reflectance spectroscopy of minerals. Journal of Geophysical Research, 95 (B8): 12653. Go to original source...
    12. Dalal R.C., Henry R.J. (1986): Simultaneous determination of moisture, organic carbon, and total nitrogen by near infrared reflectance spectrophotometry. Soil Science Society of American Journal, 50: 120-123. Go to original source...
    13. Duckworth J. (2004): Mathematical data preprocessing. In: Roberts C.A., Workman J.,Jr., Reeves III, J.B. (eds): Near-Infrared Spectroscopy in Agriculture. Madison, ASA-CSSA-SSSA: 115-132. Go to original source...
    14. Gholizadeh A., Borůvka L., Saberioon M.M., Vašát R. (2013): Visible, near-infrared, and mid-infrared spectroscopy applications for soil assessment with emphasis on soil organic matter content and quality: State-of-theart and key issues. Applied Spectroscopy, 67: 1349-1362. Go to original source... Go to PubMed...
    15. Gholizadeh A., Borůvka L., Vašát R., Saberioon M.M., Klement A., Kratina J., Tejnecký V, Drábek O. (2015): Estimation of potentially toxic elements contamination in anthropogenic soils on a brown coal mining dumpsite using reflectance spectroscopy: A case study. Plos One, 10: e0117457. Go to original source... Go to PubMed...
    16. Hidaka Y., Kurihara E., Hayashi K. (2011): Near infrared spectrometer for a head feeding combine for measuring rice protein content. Japan Agricultural Research Quarterly, 45: 63-68. Go to original source...
    17. Ji J.F., Balsam W., Chen J., Liu L.W. (2002): Rapid and quantitative measurement of hematite and goethite in the Chinese loess-paleosol sequence by diffuse reflectance spectroscopy. Clays and Clay Minerals, 50: 208-216. Go to original source...
    18. Ji W.J., Li X., Li C.X., Zhou Y., Shi Z. (2012): Using different data mining algorithms to predict soil organic matter based on visible near infrared spectroscopy. Spectroscopy and Spectral Analysis, 32: 2393-2398. Go to PubMed...
    19. Kemper T., Sommer S. (2002): Estimate of heavy metal contamination in soils after a mining accident using reflectance spectroscopy. Environmental Science and Technology, 36: 2742-2747. Go to original source... Go to PubMed...
    20. Kokaly R.F., Despain D.G., Clark R.N., Livo K.E. (2003): Mapping vegetation in Yellowstone National Park using spectral feature analysis of AVIRIS data. Remote Sensing of Environment, 84: 437-456. Go to original source...
    21. Kooistra L., Wehren R., Leuven R.S.E., Buydens L.M.C. (2001): Possibilities of visible-near-infrared spectroscopy for the assessment of soil contamination in river flood plains, Analytica Chimica Acta, 446: 97-105. Go to original source...
    22. Kooistra L., Wanders J., Epema G.F., Leuven R., Wehrens R., Buydens L.M.C. (2003): The potential of field spectroscopy for the assessment of sediment properties in river floodplains. Analytica Chimica Acta, 484: 189-200. Go to original source...
    23. Leone P.L., Sommer S. (2000): Multivariate analysis of laboratory spectra for the assessment of soil development and soil degradation in the Southern Apennines (Italy). Remote Sensing of Environment, 72: 346-359. Go to original source...
    24. Madejova J., Komadel P. (2001): Baseline studies of the clay minerals society source clays: infrared methods. Clays and Clay Minerals, 49: 410. Go to original source...
    25. McGrath S.P., Cunliffe C.H. (1985): A simplified method for the extraction of metals Fe, Zn, Cu, Ni, Cd, Pb, Cr, Co and Mn from soils and sewage sludges. Journal of the Science of Food and Agriculture, 36: 794-798. Go to original source...
    26. Meyer D., Dimitriadou E., Hornik K., Weingessel A., Leisch F. (2012): e1071: Misc Functions of the Department of Statistics (e1071), R Package Version 1.6-1. Wien, TU Wien.
    27. Moros J., de Vallejuelo S.F.O., Gredilla A., de Diego A., Madariaga J.M., Garrigues S., de la Guardia M. (2009): Use of reflectance infrared spectroscopy for monitoring the metal content of the estuarine sediments of the Nerbioi-Ibaizabal River (Metropolitan Bilbao, Bay of Biscay, Basque Country). Environmental Science and Technology 43: 9314-9320. Go to original source... Go to PubMed...
    28. Murray I. (1988): Aspects of interpretation of NIR spectra. In: Creaser C.S., Davies A.M.C. (eds): Analytical Application of Spectroscopy. London, Royal Society of Chemistry: 9-21.
    29. Nayak P., Singh B. (2007): Instrumental characterization of clay by XRF, XRD and FTIR. Bulletin of Materials Science, 30: 235-238. Go to original source...
    30. N'Guessan Y.M., Probst J.L., Bur T., Probst A. (2009): Trace elements in stream bed sediments from agricultural catchments (Gascogne region, S-W France): where do they come from? Science of the Total Environment, 407: 2939-2952. Go to original source... Go to PubMed...
    31. Nocita M., Stevens A., Noon C., Van Wesemael B. (2013): Prediction of soil organic carbon for different levels of soil moisture using Vis-NIR spectroscopy. Geoderma, 199: 37-42. Go to original source...
    32. Pearson R.K. (2002): Outliers in process modeling and identification. IEEE Transactions on Control Systems Technology, 10: 55-63. Go to original source...
    33. R Development Core Team. (2011): R: A language and environment for statistical computing. R foundation for Statistical Computing. Available at http://www.R-project.org
    34. Reeves J.B. III (2010): Near versus mid infrared diffuse reflectance spectroscopy for soil analysis emphasizing carbon and laboratory versus on-site analysis: Where are we and what needs to be done? Geoderma, 158: 3-14. Go to original source...
    35. Reeves III J.B., McCarty G.W., Mimmo T.V., Reeves V.B., Follet R.F., Kimble J.M., Galletti G.C. (2002): Spectroscopic calibrations for the determination of C in soils. Transactions of the 17th World Congress of Soil Science, 10: 1-9.
    36. Ren H.Y., Zhuang D.F., Singh A.N., Pan J.J., Qid D.S., Shi R.H. (2009): Estimation of As and Cu contamination in agricultural soils around a mining area by reflectance spectroscopy: A case study. Pedosphere, 19: 719-726. Go to original source...
    37. Rinnan A., van den Berg F., Engelsen S.B. (2009): Review of the most common pre-processing techniques for near-infrared spectra. Trends in Analytical Chemistry, 28: 1201-1222. Go to original source...
    38. Savitzky A., Golay M.J.E. (1964): Smoothing and differentiation of data by simplified least squares procedures. Analytical Chemistry, 36: 1627-1639. Go to original source...
    39. Song Y., Li F., Yang Z., Ayoko G.A., Frost R.L., Ji J. (2012): Diffuse reflectance spectroscopy for monitoring potentially toxic elements in the agricultural soils of Changjiang River Delta, China. Applied Clay Science, 64: 75-83. Go to original source...
    40. Stevens A., Udelhoven T., Denis A., Tychon B., Lioy R., Hoffmann L., Van Wesemael B. (2010): Measuring soil organic carbon in croplands at regional scale using airborne imaging spectroscopy. Geoderma, 158: 32-45. Go to original source...
    41. Vapnik V. (1995): The Nature of Statistical Learning Theory. New York, Springer-Verlag. Go to original source...
    42. Vasques G.M., Grunwald S., Sickman J.O. (2008): Comparison of multivariate methods for inferential modeling of soil carbon using visible near infrared spectra. Geoderma, 146: 14-25. Go to original source...
    43. Viscarra Rossel R.A., Behrens T. (2010): Using data mining to model and interpret soil diffuse reflectance spectra. Geoderma, 158: 46-54. Go to original source...
    44. White W. (1971): Infrared characterization of water and hydroxyl ion in the basic magnesium carbonate minerals. American Mineralogist, 56: 46-53.
    45. Williams P. (2003): Near-infrared Technology - Getting the Best out of Light. Nanaimo, PDK Projects.
    46. Wu Y., Chen J., Wu X., Tian Q., Ji J., Qin Z. (2005): Possibilities of reflectance spectroscopy for the assessment of contaminant elements in suburban soils. Applied Geochemistry, 20: 1051-1059. Go to original source...
    47. Xie X., Pan X.Z., Sun B. (2012): Visible and near-infrared diffuse reflectance spectroscopy for prediction of soil properties near a Copper smelter. Pedosphere, 22: 351-366. Go to original source...

    This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International (CC BY NC 4.0), which permits non-comercial use, distribution, and reproduction in any medium, provided the original publication is properly cited. No use, distribution or reproduction is permitted which does not comply with these terms.


    Return to the content