Final Report Summary - E-SOTER (Regional pilot platform as EU contribution to a global soil observing system)
Executive Summary:
Soil and land information is needed for policy-making, hands-on management of land resources and monitoring of the environmental impact of development. Lack of comprehensive information about land resources - globally, nationally or locally - means uninformed policies, continuing degradation of land and water resources, unnecessary carbon emissions to the atmosphere and no likelihood of achieving the Millennium Development Goals (MDGs). The viability and cost of vital infrastructure is affected just as much as food and water security and response to environmental change. In the case of the European soil thematic strategy, the operational measures laid down in the Framework Directive and impact assessment are hamstrung by lack of accessible, easy-to-use, consistent, harmonised and relevant soil data. The Group on Earth Observations (GEO) plans a Global Earth Observing System and, within this framework, the E-SOTER project addresses the felt need for a global soil and terrain database. As the European contribution to a Global Soil Observing System, it will deliver a web-based regional pilot platform with data, methodologies and applications, using remote sensing (RS) to validate, augment and extend existing data.
The collaborative project addressed four major barriers to a comprehensive soil observing system:
1. morphometric descriptions - enabling quantitative mapping of landforms as opposed to crude slope categories
2. soil parent material (PM) characterisation and pattern recognition by RS - enabling separation of soil processes within the landscape
3. soil pattern recognition by RS
4. standardisation of methods and measures of soil attributes to convert legacy data already held in the European Geographical Soil Database and various national databases to a common standard - so that they may be applied, e.g. in predictive and descriptive models of soil behaviour.
The project objectives that contribute to the lifting of the four major barriers described above are:
1. morphometric descriptions of the landforms both in an enhanced SOTER DEM methodology as well in newly developed DEM analysis using natural breaks. The existing DEM that will form the basis for the morphometric analysis will be filtered and enhanced to obtain an artefact-free product. The end products have been several landform layers in the window and pilot areas.
2. soil PM characterisation using RS (RS) and legacy data generated two PM classifications relevant for soil development and PM pattern within the window and pilot areas.
3. soil pattern recognition uses existing data and converted these into a standardised SOTER format.
Using RS generate additional predictors of soil properties. End products will be a soil layer in the window and pilot areas with standardised soil attributes for the 1:1 000 000 scale and the 1:250 000 scale.
Additional project objectives had been:
1. quality assessment of E-SOTER by performing a validation and uncertainty analysis
2. applications of the newly acquired E-SOTER in the field of major soil threats and comparisons with applications based on the European Soil Database.
3. dissemination of the results of the project through stake-holder conferences at the Food and Agriculture Organisation (FAO) and elsewhere and through web-based services (see http://www.esoter.net and http://portal.esoter.net).
The end product is a regional pilot platform with methodologies, concepts and applications that, together, facilitates:
1. an enhanced SOTER database methodology at scale 1:1 000 000 for Europe and the world;
2. ways of generating finer-scale maps of specific soil and terrain attributes and digital data, based on legacy soil survey data and RS;
3. a framework for new, cost-effective field survey and monitoring programmes.
Project Context and Objectives:
Soil is the basis of our daily life; we walk on it, growth our food on it and therefore depend on it for our life. Soil and terrain information is needed for many interpretations for example in the field of agriculture, environment, watershed management, infrastructure, etc. but available data are often inaccessible, incomplete, or out of date. The GEO plans a Global Earth Observing System and, within this framework, the E-SOTER project addresses the felt need for a global soil and terrain database. As the European contribution to a Global Soil Observing System, it delivered a web-based regional pilot platform with data, methodologies and applications, using RS to validate, augment and extend existing data. The project objectives were the following:
1. Morphometric descriptions of the landforms
2. Soil PM characterisation using RS and legacy data
3. Soil pattern recognition using existing data and RS
4. Quality assessment
6. Application and
6. Dissemination of the results.
In detail the final result of the E-SOTER project is a pilot platform and a portal that provides open access to:
1. a methodology to create 1:1 000 000 SOTER databases and an enhanced soil and terrain database at scale 1:1 000 000 for the four windows
2. an artefact-free 90m DEM
3. methodologies to create 1:250 000 scale enhanced SOTER databases and the databases themselves for four pilots
3. advanced RS techniques to obtain soil attribute data
4. validation and uncertainty propagation analysis
5. dedicated applications related to some major threats, like soil erosion by water and soil compaction, to soil quality and performance.
The use of the methodologies by soil and other natural resources institutes will have an impact on the soil and terrain information within Europe as well outside. Initially it was thought that the project will result in an European SOTER Database as an update to the existing 1:1 000 000 European Soil Database. The European Soil Bureau Network together with the Joint Research Centre of the European Commission said in the final stakeholder meeting in Rome that they want to deploy the developed methodology for the 1: 250 000scale in Europe to create an complete E-SOTER product for Europe. Currently the developed E-SOTER methodologies are further applied between ISRIC the Netherlands (NL), Cranfield United Kingdom (UK) and Scilands Germany, small and medium sised enterprise (SME) to apply and test it in Burkina Faso for a possible up scaling to all over west Africa if resources permit.
Towards the global contributions, E-SOTER contributed to one of the Global Data Tasks of the Group on Earth Observation's Work Plan by co-leading the task Global Soil Data (DA-09-03e - now IN02-C2) and will submit the portal and its results to Global Earth Observing Systems DataCore. The project has organised a group of potential contributors to this task (e.g. European Partners). Once operational this task will increase the exposure of the products compiled by the project and by others. The main PI applies the developed methodologies currently in west Africa. The E-SOTER project had technical contacts with the 'GlobalSoilMap.net' consortium for possible collaboration on technical issues.
Project Results:
WP1 developed a methodology for a global platform for terrain and soil parent data at a scale of 1:1 000 000 and conforming to SOTER criteria. Data preparation developed an artefact-free DEM based on shuttle radar topography mission (SRTM 0 thermal noise reduction) which was used for the morphometric characterisation of landforms, Next soil PM within these landform units, based on low-resolution satellite imagery, e.g. advanced very high resolution radiometer (AVHRR), moderate resolution imaging spectroradiometer (MODIS), SPOT Vegetation and DEM data, legacy soil was derived, lastly combining these into terrain units. Significant results were:
1. artefact-free SRTM DEM for the four window areas;
2. artefact-free SRTM DEM for the rest of Europe South of 60° almost completed;
3. revised PM classification focussed on soil forming processes;
4. procedure for PM properties characterisation and delineation for areas with limited or no data;
5. grid layers of the derived PM and physiographic parameters;
6. ArcInfo macro language (AML) script to run the physiographic delineation procedure and aggregation procedure;
7. Arc geographic information system (ArcGIS) tool to develop the SRTM base PM genetic layers;
8. 1:1 000 000 scal E-SOTER geometric databases of the terrain units (landform and PM) for the windows in Europe, Morocco and S-China;
9. report describing the whole procedure (Deliverable D3).
In detail the following results have been successfully obtained:
Task 1.1 developed by Partner SciLands delivered an artefact-free DEM for the four window areas. The algorithms for artefact removal were developed and applied for the four E-SOTER windows and the cleaned SRTM data of the windows were uploaded to the team site and were used as inputs for th E-SOTER unit development for the windows. A complete European coverage has not been created as the pattern of terrain and forest distribution outside the E-SOTER windows is sometimes totally different from the distribution that was found inside. Algorithms to reduce the surface of forest canopies to earth's surface where developed for the E-SOTER windows with satisfying results. But for some regions outside the E-SOTER windows (e.g. in Poland) the results are not satisfying. These new deviating situations needed new tools for handling: adaptation of the applied algorithm. The enhancements of the algorithms for handling forest canopies outside the window areas are still on-going research after the closure of the project.
Selected Result: Bock, M. and Köthe, R., 2009. SRTM DEM pre-processing. Paper presented at Geomorphometry 2009, 28 August to 2 September, Zürich, Switzerland.
Task 1.2 created by Partner University of Miskolc and INRA France delivered Landform classification according to th E-SOTER criteria and the development of terrain units based on the reference material of Dobos et al. (2005). There were two major modifications in the procedure:
1. modification/simplification of the potential drainage density (PDD) procedure
2. removal of PDD as differentiating criteria for areas having a relief intensity higher than 100 m
3. tile size of processing was decreased.
These changes have been applied and inserted into the Dobos et al (2005) procedure. The AML for running the procedure has been updated and applied at several locations throughout the world (e.g. E-SOTER project, Canada, Burkina Faso). For some specific areas, neither the terrain, nor the simplified PM classification resulted in an adequate delineation and huge polygons remained insufficiently disaggregated. In these cases, the soil regions delineated in WP2 were used to further divide the polygons. Further research is needed to address these issues.
Selected Result: AML with documentation available from the E-SOTER website
Task 1.3 performed research to improve delineation of soil PM units across the partners Misc, BGR and ISRIC. The sub-task had two major work lines. The first was to revise the existing lithological classification of SOTER (van Engelen and Wen, 1995) to allow a better correlation with the legacy data from geological maps and to make the classes easier to interpret for soil data users. The second aim was to develop a quantitative procedure for PM delineations for areas with no or limited PM data using RS data (MODIS) and SRTM modelling approaches. Third, the two different approaches were harmonised and integrated into a common data framework and structure.
Fulfilling the first aim of revising and simplifying the PM classification, a revised lithological classification was developed (ISRIC report 2013/6). It describes the unconsolidated materials as soil PMs, using the major PM properties that drive the soil formation processes. This new structure has independent properties to be mapped. All properties are combined in the final step. In this way, all potential input data can be imported into the delineation procedure regardless of the detail of the original PM data description.
The second part of the work was focusing on the quantitative PM data development for limited/no-data areas using MODIS data. A limited set of PM characteristics was selected and procedures were developed to create the appropriate layers. The quantitatively derived PM layers have been completed for the four windows and been passed onto the last step of the procedure development, namely the combination of the physiographic (derived in task 1.2.) and the PM units.
Selected Results:
1. Ulrich Schuler, Rainer Baritz, Jan Willer, J.A. Dijkshoorn, Harald G. Dill, 2013, ISRIC Report 2013/6 A revised approach to classify PM for soil mapping ISRIC Report 2013/6
2. ArcGIS based algorithm to derive the SRTM based PM genetic layers.
Task 1.4 finally combined landform and soil PM units into terrain units by Partners Unimis and INRA. The physiographic units and the PM units were combined to develop the final SOTER-unit polygon system. A thematic distance based rule system was developed to drive the aggregation procedure and derive the final SOTER polygons.
Selected Results:
1. Algorithm incorporated in Task 1.2 results
2. Dobos, E. 2010. A quantitative procedure for building physiographic units supporting a global soil database, Hungarian Geographical Bulletin, 59/2.
WP2 performed research to integrate legacy data with new data from RS. Legacy soil data from two windows in Europe, one in Morocco and one China have been used to compile soil mapping units within the WP1 terrain units, enhanced by MODIS imagery). Th E-SOTER soil component data structure has been revised to allow inclusion of new soil attributes with a complete coverage of th E-SOTER soil units (1:1 000 000). Significant results are:
1. Updated SOTER soil component data structure
2. three methods for harmonisation and correlation of national soil data, namely translation algorithms (prototype) for national soil data; correlation method of the national classification units with world reference base (WRB); decision-tree based method incorporating the centroid concept
3. assignment of representative profiles to the soil components of th E-SOTER units created by WP1
4. deliverable D2.1 (D5) a first version of the harmonised soil data base for the 1:1 000 000 scale windows (except China) was submitted in January 2011.
Task 2.1 by partners SIU, BGR, CULS, CU, ISSCAS and INRA Maroc generated spatial soil information for the 1:1 000 000 window. The major activities of the task were the collection of legacy data for the window areas, collection of RS data and development of methodology for the spatial definition of soil units.
A new E-SOTER approach for compiling spatial soil information has been developed. The approach attempts to estimate the spatial occurrence probability for the mapped area using RS, digital terrain data and pre-processed legacy data as training dataset.
The collection of legacy soil information for most of the research area enabled the development of Pedotransfer rules to harmonise data of various origins within the pilot windows and from national system into a generalised one for the central European (CE) window. French data are missing due to intellectual property (IP) restrictions. Data were derived from: national data - provided by the window partners, ISRIC World Inventory of Soil Emission Potentials (WISE) international soil profile dataset, SOTER dataset.
The collection of RS data was based on MODIS multi-temporal eight days composites over five dates covering the none-snow-covered period, evenly distributed over the vegetation period. Additionally, the WP1 DEM was used to derive terrain derivatives.
Lastly, a methodology for the spatial definition of soil units is based on the defined (from legacy data) master building units of the WRB classification system such as the diagnostic properties or horizons (DPDH). The approach attempts to estimate the spatial occurrence probability for the mapped area using RS, digital terrain data and pre-processed legacy data as training dataset. The success and detail of the approach depend largely on the quantity and quality of the input training set. In general, some of the related/similar classes have to be combined and new, more general classes are created to characterise the soil resources of the polygons. As the WRB includes numerous diagnostics, a limited set of significant units is defined by an expert group based on the existence and significance of horizons, properties and materials. Training datasets for this group of diagnostics are derived from legacy data, mainly using profile data or representative large scale database windows. Each training datasets consists of points or areas with known existence or absence of the property in question. Using these training datasets for classifying a complex MODIS/SRTM based image results in numerous continuous layers for each property having the probabilities of the existence of the diagnostic property. The major advantage of this approach is that it provides the needed thematic information on important or master soil properties, like texture, organic matter, salt content etc. At the end, a WRB-based simplified classification scheme is developed to identify the WRB soil types for each pixel. This raster database is used at the end to calculate the percentage of coverage of each soil type within each polygon, the so called definition of soil components. This approach is suggested to be used, when no contiguous coverage of legacy data is available for the area. The traditional approach should have a priority over the E-SOTER approach, except when the data quality of questionable or non-satisfactory.
Selected result:
Harmonised soil data base for the 1:1 000 000 scale windows at the portal website.
Task 2.2 performed research on the compilation of the soil data base for the 1:1 000 000 scale windows by partners SIU, BGR, Unimis, ISRIC, CU. The e-SOTER database is derived from and compatible with the Global Soil and Terrain (SOTER) database (van Engelen and Wen 1995). It is composed of a set of tables in a relational database management system (RDBMS) linked to the geometric database (GIS file). The database is composed of various tables; the terrain, terrain component and soil component tables that describe the attributes of the spatial parts of SOTER, while the profile, horizon and WRB diagnostics describe the attributes of the profile (point) data. The E-SOTER database has undergone a number of improvements in its structure, compared to the original SOTER database.
One of the major improvements has been the application of the updated standards for soil descriptions and soil classification. The master horizon designation, subordinate characteristics and site descriptions follow 2006 edition of the FAO Guidelines for soil description (FAO, 2006). The classification of soils and the related diagnostic horizons, properties and materials are described and coded according to the WRB for soil recourses (IUSS Working Group WRB, 2006, 2007). All profiles (representative and reference) are given both the WRB classification, to link to new developments and the classification according to the Revised Legend of the Soil Map of the World (FAO 1988) to link to the former SOTER databases. To facilitate using SOTER as a profile database also PM, land use and vegetation are given now in the profile table.
A new table, the WRB diagnostics, was included to describe properly the WRB diagnostic horizons, properties and materials as well as the depth of occurrence. All diagnostic horizon or property can apply for the same horizon and can be described. The qualifiers are listed together with the reference group.
Another improvement has been the inclusion of the small-scale map legend using the WRB for soil resources (addendum to WRB, 2010) in the soil component table. The WRB Legend and the Revised Legend (FAO 1988) are directly linked now to the geometric SOTER database and enable the user to derive various maps e.g. a dominant soil without too much database manipulation.
Translation and harmonisation of national soil data into the newly defined E-SOTER standards required a range of translation algorithms and correlation methods to be developed and to be applied to the existing data:
1. based on original field description and laboratory data,
2. based on taxonomic distance calculation of the national classification units with the WRB and
3. a decision-tree based method incorporating the centroid concept. The availability and the completeness of the received data from the window partners was variable and in several cases insufficient for proper correlation with the WRB. Sometimes, due to limited data availability the closest profile could occur in other country or continent. Further research is needed to increase the freely globally available soil data.
Selected Results:
1. Eberhardt, E. and Pietsch, D., 2009. Classifying soils according to WRB with national soil legacy data. Abstract of the papers of the Conference 'From the Dokuchaev School to numerical soil classifications', 18 September, 2009 Gödöllo, Hungary
2. Waltner, I. and Micheli, E., 2008. Computer aided classification of some selected soils of Hungary. Abstract of the papers of the International Soil Classification Conference, Santiago, Chile, 2008
3. Láng, V., Fuchs, M., Waltner, I., Michéli, E., 2010. Pedometrics application for correlation of Hungarian soil types with WRB, 19. IUSS World Congress, Brisbane, Australia
4. Láng, V., Fuchs, M., Waltner, I., Michéli, E., 2009. Correlation possibilities based on taxonomic distance measurement. 25th SSSEA Conference, Moshi, Tanzania
5. Láng, V., Fuchs, M., Waltner, I., Michéli, E., 2010. Taxonomic distance measurements applied for soil correlation, Agrokémia és Talajtan (in press)
6. Waltner, I., Láng, V., Fuchs, M. and Michéli, E., 2010. Application of a centroid-based concept for the correlation of national soil classification with the WRB fourth Global Workshop on Digital Soil Mapping, 24-26 May 2010 - Rome, Italy, abstract
7. Eberhardt, E. and Waltner, I., 2010. Finding a way through the maze - WRB classification with descriptive soil data 19th World Congress of Soil Science, 1-6 August 2010, Brisbane, Australia, abstract.
WP3 provided a high-resolution E-SOTER database for 1:250 000 scale pilots using state of the art DEM analysis (hierarchical classification, natural breaks). PM has been derived from RS using multispectral imagery and airborne gamma spectrometry. Europe-wide additional predictors of soil properties at various spatial, spectral and temporal scales have been integrated in the developed terrain units, their underlying PM and the respective semantic soil information. Some of the significant results include:
1. provision of draft methodology for alternative ways of derivation of physiographic units at 1: 250 000 scale and application of it
2. revision of the FAO PM classification (Schuler et al. 2013); includes the coordination with internationally agreed geological terminology and application thereby delivering a Proof of concept to utilise geological map data for E-SOTER
3. development of the soil component for E-SOTER using digital soil mapping and comparison with soil maps and representative soil profile data
4. a new task force on superficial deposits was founded under the umbrella of EuroGlobal Earth Observing Systemurveys. The idea is to develop a PM map for Europe as a result from the E-SOTER project results
5. use of RS for the further refinement of better defined lithological strata providing, if possibly, some information on key mineralogical features.
Several different approaches were investigated by the different partners to fulfil the objective of new classification algorithm of landform types (Task 3.1).
Cranfield University investigated how hill shed and slope break analysis for landform classification can be combined. Slope break (hill slope) analysis is an extension to hill shed analysis in the sense that it aims at classification of slopes within hill sheds. Classification is based on the most distinct breaks in slope and assigns three major classes within the length of slope. Hill slope analysis is particularly useful in correcting the 'bleeding effect' within peak sheds that were originally used in landform classification approaches. The 'bleeding effect' occurs due to the fact that peak sheds comprise areas between passes, or passes - bottoms of valleys resulting in incorporation of flatlands into peak sheds located at the border between hills and valleys.
Partner Scilands developed new algorithms for the advanced classification of continuous morphometric terrain parameters were developed. These algorithms allow the identification of 'natural breaks', using segmentation techniques as known from RS. It is now possible to identify homogeneous units (areas) of terrain parameters - as a base for terrain classification. In contrast to many segmentation techniques in RS the developed algorithms work without any 'seeds' what we regard as an advantage. 'Seeds' (seed grid cells) have to be given to common segmentation algorithms as input data. These seeds produce sometimes units which are unintended or some relevant units are missing (caused by missing seeds). The new algorithms in contrast allow the identification of 'natural breaks', using segmentation techniques as known from RS. The segmentation of the continuous terrain parameters results in a classified data set, which contains the 'right' delineations but also just as many those are not the intended. The challenge now is to identify homogeneous units (areas) and to combine the units in a plausible way.
The parameter Scale is an important property in any landform classification. Therefore a third attempt was dedicated especially to find scale-independent landform classification parameter datasets for all kinds of landscapes in Europe. The concept of terrain classification follows a pragmatic-deductive approach based on geomorphological knowledge (including knowledge of the genesis of landforms) by including - in first step simple - geological information. In a first stage the classification procedure distinguishes between two groups:
1. areas with consolidated rock as the PM; and
2. areas with unconsolidated rock as the PM.
Based on these initial delineations, methods of terrain classification follow a pragmatic approach and vary to calculate (delineate) the desired terrain units. The objective of all methods is to work with self-adjusting thresholds and natural breaks (both within ranges of values of the terrain parameters according to local terrain conditions, followed by an generalisation procedure with:
1. generalisation with regard to the content (types of the terrain units) and
2. generalisation regarding the size and shape of the individual terrain units. The system of terrain classification consisted of the following terrain units: 'Flats': several levels of 'Bottom Flats', several levels of 'Flats and Terraces' and 'Mountainous Summit Flats', 'Slopes': 'Divergent Slopes, Ridges and Bumps', 'Convergent Slopes and Small Valleys' and 'Upper Slope', 'Mid Slope' and 'Lower Slope, Gentle Slope'.
PM is one of the input drivers into the E-SOTER methodology. Unfortunately, it is not readily available from Geological Information. Task 3.2 investigated if RS analysis for soil PMs can be applied through a set of partners (BGR, JRC, CU, CULS). In central Europe, soil PM mapping based on satellite data is hindered by the vegetation cover. The only exceptions are bare arable land and rock outcrops. In addition, pronounced landscape formation features help to identify PMs at the landscape level. Such complex landforms can be identified from DEMs (e.g. karst towers, cruesta shaped landscapes). This E-SOTER task searched to optimise the use of different data sources to delineate types of PM in the context of soil mapping.
Research to improve the development of a SOTER PM layer pursues two main approaches, RS and the use of legacy geology data.
RS focussed on the use of hyper spectral, such as the advanced spaceborne thermal emission and reflection radiometer (ASTER) or Hyperion and gamma-ray sensing in different windows (e.g. Morocco, Germany). Gamma ray was clearly able to distinguish different geological settings and with that some PM.
During the process it became apparent that the existing (also in WP1) PM classification has some deficiencies. The existing FAO/SOTER hierarchy was adapted to the requirements of the E-SOTERWP1 definition, but also in relation to other existing soil PM classifications. In addition, the consistency with the OneGeology terminology was maintained, so that geology legends can be easily understood and re-interpreted by soil scientist. On that basis, geology map legends can be easily aggregated to the level required by E-SOTER and other applications in digital soil mapping, for example GlobalSoilMap. Besides parent rock, a second table on genetic aspects ('surface processes') has been added, so that the unconsolidated sediments can be further specified. On that basis, the vertical differences of unconsolidated, weathered or re-located layers can be described together with the properties of the underlying parent rock. Thus, the revised version is not only consistent with the geochemical quality of rocks and the geological terminology, but also suitable for field work (soil profile description). Geological maps can now easily be re-interpreted for SOTER mapping.
In the Research on RS for soil attribute data (Task 3.3) a wide range of activities have been involved (sampling, prediction, evaluation). During the collection of the field data using the constrained Latin hypercube sampling (LHS) aiming to acquire a sample representing the full range of covariate space (in this case three principal components and elevation) under budget and accessibility constraints it became clear that strong local variability is preferentially sampled which is preferred for Regression Tree/CART models, however might causes problems if the data are to be used for variogram estimation in a Global Earth Observing Systemtatistical analysis. Finally, the hyper spectral data proved efficient together with a developed a linear spectral trend model to predict the main composite mineralogy based on VNIR spectroscopy. For further details please refer to deliverable D7 and the papers by Mulder et al.
To integrate all the different products (geomorphic terrain units, PM and semantic soil information for each pilot area).
In the frame of WP3 an alternativ E-SOTER map was generated, merging the Geomorphographic map (1:1 000 000 scale) with the PM map and the map of the surface processes. The preservation of streamlines and major slope breaks gives this SOTER map a clearer structure of the landscape than th E-SOTER map of WP1. Additionally, 14000 soil augering observations were used to develop a methodology to derive a substrate unit map using classification tree (CART algorithm). The substrate map clearly shows a gradient from silty substrates in the lowlands to sandy and loamy substrates at elevated parts in the Ore Mountains. The analysis of different relief positions dependent of parent rock revealed a dominance of silty substrates for all relief positions, except for medium to steep slopes. This means that on steep slopes the loess cover is eroded and the soils are stronger influenced by the bedrock.
The E-SOTER map developed in the frame of WP3 represents both soil PM and relief. The high correspondence of the sand content distribution according to th E-SOTER map with both the geological map 1:50 000 and the radiometric maps show clearly that th E-SOTER map is suitable for mapping of soils and soil properties at small scale.
Selected results:
1. Mulder, V.L. de Bruin, S., Schaepman, M.E. and Mayr, T.R. 2011. The use of RS in soil and terrain mapping - A review. Geoderma, 162(1-2):1-19
2. Schuler, U., R. Baritz, J. Willer, J.A. Dijkshoorn, H. G. Dill (2013). A revised approach to classify PM for soil mapping. ISRIC Reports. ISRIC-Report 06-2013.
WP4 objective was the validation and uncertainty analysis with independent validation data for different scale windows. A comparison was made between the accuracy obtained with products of WP1 and WP2 with that of WP3. Sensitivity of SOTER methodologies have been identified using state of the art statistics (e.g. probability distribution functions, Monte Carlo simulation).
1. development of a sound methodology for statistical validation of digital soil maps and E-SOTER products including purity, user's accuracy and producer's accuracy of the E-SOTER soil maps as well as how aggregation and generalisation steps deteriorate the quality of E-SOTER landform maps
2. a Monte Carlo error propagation methodology was designed and implemented with which the propagation of DEM errors to the E-SOTER landform maps can be traced
3. correlation procedures between the national soil classification systems for England and Wales, Germany and the Czech Republic with that of the WRB were established and automated.
Research in WP4 investigated validation and uncertainty propagation analysis of E-SOTER products by comparing the E-SOTER soil and landform maps with independent validation data and by analysing how errors in a DEM propagate to the E-SOTER landform map.
Landform validation was done by comparing the true landform in the western European (WE) and CE windows with the landform as depicted on the E-SOTER maps. The various simplification and generalisation steps of the E-SOTER landform classification methodology cause discrepancies between the true and predicted landform. Validation showed that the accuracy of the E-SOTER landform maps is high for landform classes 'elevation', 'relief intensity' and 'flatness', with agreement between true and predicted class in 81 to 98% of cases. For landform class 'slope' the agreement is only around 50%, which means that in 1 out of 2 cases the E-SOTER map does not agree with reality. This is caused by the highly fragmented spatial pattern of the true slope map, a feature that cannot be reproduced in the E-SOTER map because it must generalise the map to the 1:1 000 000 scale. Comparison of landform validation results between the WE and CE windows did not show large or meaningful differences.
Soil validation was based on a comparison of the dominant soil classes on the E-SOTER maps with independent soil observations derived from existing legacy data. This first required a conversion of soil classes from the local classification system to the WRB. For this purpose correlation tables were prepared. The validation results show that the E-SOTER soil map of the UK pilot area reproduced the large scale patterns and had an agreement with the 'true' soil class of 51%. The E-SOTER map overrepresented Histosols and Podzols and lacked Leptosols as a dominant soil group. Validation results in the German / Czech pilot area revealed an overall agreement of only 32%. The rather low purity in this case can be assigned to high variability of the soil cover and often low dominancy of the dominant soil unit in th E-SOTER units. Also, in this case the more detailed validation data allow the use of more strict validation criteria than in the UK case. Overall, the fairly low agreements between E-SOTER soil maps and validation data in both pilot areas show that the soil maps have large uncertainties, even at the coarse soil group level that was considered here. Part of the disagreement may be caused by errors in the validation data, but this is unlikely to be the major cause of the discrepancy.
The uncertainty propagation analysis showed that DEM uncertainty mainly affects the E-SOTER slope class. The real elevation is noisier than the smoothed DEM and this causes the DEM-derived slope to be too small. Elevation was hardly affected while flatness is only sensitive to DEM errors in relatively flat areas and relief intensity in areas with a more intense relief (i.e. the CE pilot area). Overall DEM uncertainty does not seriously impair landform accuracy and is mainly restricted to zones along class boundaries. It is a smaller source of uncertainty than the simplification and generalisation steps that were analysed in the landform validation procedure.
Selected Results:
1. Brus, D.J. B. Kempen and G.B.M. Heuvelink (2011), 'Sampling for validation of digital soil maps', European Journal of Soil Science 62, 394 407.
2. Algorithms to reproduce validation and uncertainty research available at the algorithm database of the E-SOTER portal.
WP 5's objectives are applications of the E-SOTER database. It investigated whether use of the E-SOTER database has improved evaluation of threats to soil quality. Appropriate models that can measure the threats (EU Soil Thematic Strategy) at the relevant scales were selected and run with E-SOTER data and any additional data required. Identification of threats, data collection, a comparison of the threats based on E-SOTER and on pre-existing data sources considering both spatial patterns and at statistical trends and a comparison between existing data on threats and the results obtained from the 1:1 000 000 and 1:250 000 scale windows by running models for the most important threats determined was performed. Significant results are:
1. selection of methods to evaluate major soil threats and collection of data to run the models
2. development of evaluation method by elicitation
3. comparison of model outputs from legacy data and E-SOTER with the conclusion that the model outputs for the soil threats concerned based on the E-SOTER database are similar to once derived from legacy databases.
Task 5.1 with partners Alterra, JRC, BGR, CULS, INRA, INRA-Maroc WP5 members identified several threats as the most important ones in the different study areas.
In these lists, three threats are mentioned more than once, namely soil erosion (by water), loss of organic matter and compaction. Landslides are only important if the Moroccan pilot area is shifted northwards and desertification is currently not one of the soil threats listed by the EU Soil Thematic Strategy. Task 5.2 collected the data necessary to run the models. Thereby, e.g. a conversion strategy has been developed to reclassify different land use classifications. Unfortunately, in-depend validation data have not be available for the scale of the models. For these reasons, expert opinion was used in this study to evaluate which data base was performing better in the evaluation of soil erosion by water. A method for evaluating modelling results using expert opinion was devised.
Experts were asked in Task 5.3 to indicate the spatial distribution of the target variables 'sensitivity to soil erosion', 'potential soil loss' and 'inherent susceptibility of subsoil compaction' in a set of randomly selected nomenclature of territorial units three (NUTS3) units in the three windows. For each combination of window and soil threat, 3 specific experts in the field of soil erosion and/or compaction and with regional knowledge of the window, were invited to respond. A statistic referred to as D, that is defined as the maximum difference between the cumulative probabilities of the (discrete) spatial distributions of model outputs and/or expert assessments has been used. The larger D, the more distinct the spatial distributions. Statistic D varies from 0 (distributions are identical) to 100 (distributions are totally different).
Differences between expert judgements expressed as D vary between the windows and target variables. In general, larger values and larger variations of D were found for all target variables in the WEU window compared to the CEU and MOR windows except for soil compaction, for which a large variation in D was also observed in the other windows. This observation implies that experts agreed less on the target variables for the WEU window compared to the other two windows and might also indicate that soil compaction is hard to model and hard to assess by the experts.
In all three windows, larger values of D and a larger variation of D were observed for soil compaction compared to soil sensitivity to soil erosion and potential soil loss, with values of D mostly above 50%. This implies that experts agreed less on the susceptibility to soil compaction than on the target variables referring to soil erosion. The areas of the administrative units that were assessed did not seem to influence the differences between expert judgements very much, except for the potential soil loss in the CEU window, where the experts seemed to agree more for larger NUTS3-units.
Overall, the deviation of the model output for all target variables is large compared to the expert responses, with values of D up till 100% for erosion sensitivity and susceptibility to subsoil compaction in the CEU and WEU windows. Four reasons may explain this observation:
1. the experts were provided with less detailed spatial information on the land properties relevant to the soil threat, whereas the models were fed with this information at the level of 1 km2 pixels,
2. the experts estimated the coverage of the classes of the target variables visually ('eye-ball estimates'), whereas the coverage of classes was exactly determined in the analysis software
3. the experts used process knowledge, external information and field knowledge, that was not available to or incorporated in the model applications and
4. the experts were insufficiently informed on the areas selected for the questionnaire.
Differences between the value of D from model applications using the E-SOTER database (compared to expert judgment, DE-SOTER) versus the legacy databases (compared to expert judgment, Dlegacy) were calculated to assess if outputs based on the E-SOTER database are more in agreement with the expert assessments than model outputs based on the legacy databases. If these differences would be negative, this would imply that using the E-SOTER database would result in model outputs for the soil threats in better agreement with the expert judgments. The results show that both positive and negative differences between DE-SOTER and Dlegacy occur, indicating that model outputs based on the E-SOTER database are not always better according to the experts than those based on legacy databases. This observation applies to assessments from experts irrespective of their informedness, meaning that the results would not change if only experts who considered themselves to be well informed were considered. Model outputs for the susceptibility to soil compaction show the largest differences in the use of the E-SOTER or the legacy database when compared to the expert assessments. This may be due to the larger disagreement between the experts with regard to the susceptibility to soil compaction and that it is probably harder to model/estimate soil compaction than the other properties.
The results of this study showed that the model outputs for the soil threats concerned based on the E-SOTER database are not always better than those based on legacy databases. This may be due to the identified differences in the soil properties in both databases that are input to the models, which in some E-SOTER units result in model outputs more in agreement to the expert judgment using the E-SOTER database and in other E-SOTER units in model outputs more in agreement to the expert judgement if the legacy database is used. Furthermore, several other reasons may explain the result:
1. Contrary to the legacy databases, the E-SOTER database does not fully cover the administrative units in the windows. As a consequence, estimates by the experts often pertain to a larger (and therefore different) area than the model outputs based on the E-SOTER database.
2. The models only use a part of the soil data in the databases and therefore the comparison of the databases only refers to the input variables of the models that differed between the databases.
3. Model outputs are on ordinal scales (ordered classes). Differences between the databases providing the model inputs may therefore be tempered.
Selected result:
1. Hessel, R., J.Daroussin S.Verzandvoort D.Walvoort. Suitability of two different soil data bases to assess soil erosion using MESALES for three areas in Europe and Morocco (in prep.)
WP6s objective was the development of an E-SOTER dissemination platform on the Infrastructure for Spatial Information in the European Community (Inspire) principles. Task (T6.1) analysed the data specification and exchange rules (XML) and prepared the SoTerML. Next (T6.2) profile and analytical data management for existing legacy data and newly generated data was performed. Lastly, (T6.3-5) created a European E-SOTER Portal, including RDBMS structures, algorithm database and data dissemination services, cookbook, algorithm database, hosted now at ISRIC for long term management. Based on the results in D6.1 any other party/third party country (e.g. as part of Development Aid) might deploy a similar system. Significant results are:
1. SoTerML schema along with its documentation for soil data exchange available at schema.isric.org for long term maintenance
2. E-SOTER parser package to convert legacy SOTER data
3. Creation of a global/European E-SOTER Portal, including RDBMS structures, algorithm database and data dissemination services
4. Cookbook which allows recreation of the E-SOTER portal functionality (e.g. as part of development work in another country).
Research in Task 6.1 with partners Unott, JRC, BGR, CU and ISRIC performed an Analysis of data specification and exchange rules (XML) and the development of concept rules for SoTerML (OGC complained). The SoTerML serves as data exchange and standardisation format. The design of the XML format so called SoTerML was taking in account two competing aspects. The first one is the various soil attribute 'profiles' potentially used in SOTER databases such as the SOTER, WRB, the FAO classification schemas and on the other hand the interoperability developments led by OGC about geographical dataset. On one side its goal is to be able to facilitate understanding and transfer from existing format and model and one the other side is to provide exchange of dataset in a harmonised and standardised fashion also compliant with existing standards. Development of the platform was based/considered on agreed International Organisation for Standardisation (ISO) standards (ISO19136 (GML), ISO19110 (Feature Catalogues), Global Earth Observing SystemciML and ISO group /TC190/SC 1 N140 'Recording and Exchange of Soil-Related Data', the Global Earth Observing Systems subtask DA-09-03e on global soil data and on soil data model developments of the GlobalSoilMap.net project.
All existing legacy data were transferred to the new SOTER-ML language using a custom made 'E-SOTERParser' package, which converts the legacy SOTER database Access file and then the accompanying ESRI GIS Shape file (Task 6.2). This was a prerequisite for the successful execution of Development, setup and implementation of a global/ European E-SOTER portal by partners ISRIC, JRC and CU.
Development of the E-SOTER approach has culminated in the 'E-SOTER' project, which has sought to create harmonised SOTER database windows in western Europe between France and England; the Chemnitz region between the Czech and German borders; northern Morocco; and China (see http://portal.esoter.net/ and http://eusoils.jrc.ec.europa.eu/projects/E-SOTER/). As the European contribution to a Global Soil Observing System (part of GEO), a key element to these phases of activity has been the concurrent development of an E-SOTER web portal and related information spatial data infrastructure, designed to hold, manipulate and disseminat E-SOTER databases. The portal is running on a Virtual Machine based on a Linux Server Distribution and the content has been created using several open source tools which are freely available under General Public License (GPL) with great support from open source communities. The portal deploys Joomla as a Content Management System, PostgreSQL/PostGIS as its Database System, GeoNetwork for its MetaData Search and Portal Facility, Global Earth Observing Systemerver for WebMappingService (WMS) and WebFeatureService (WFS) and trac-project/mercurial for the algorithm database.
The technical solutions have been tested by JRC and ISRIC and data exchange connections have been tested against other services (at FAO, ISRIC, JRC). Generally, the portal products will be hosted at ISRIC World Soil Information. ISRIC as the world data centre for soil has a long-term mandates beyond the lifecycle of the E-SOTER project. One of the main problems in research today is to produce/maintain documentation of steps how a specific task has been performed (reproducible research). As the E-SOTER project developed a range of new methods, therefore a stringent documentation of these methods was desired. Due to the different software development environments, the solution could only be to store algorithm in a more descriptive way, while maintaining the source code in the specific software. Most of the algorithms developed in the project have been introduced into the algorithm database, while one specific one (Bulk Density estimation) has been tested for further implementation directly on the E-SOTER database.
To distribute the installation instructions of the E-SOTER portal an electronic cookbook has been set up which allows a much more dynamic and up-to-date information to be disseminated. This cookbook is distributed online via the E-SOTER portal - website. The steps in the cookbook describe the process of preparing a server to host the tools necessary to operate and run the E-SOTER web Portal and associated components.
1. Selected Results:
1. A. Pourabdollah, D. G. Leibovici, D, M. Simms, P. Tempel, S. H. Hallett, M. J. Jackson (2012) Towards a standard for soil and terrain data exchange: SoTerML, Computers & Global Earth Observing Systemciences, doi:10.1016/j.cageo.2011.11.026
2. E-SOTER portal cookbook http://portal.esoter.net/downloads/esoter_Portal_Cookbook_V5_6.pdf
WP7s objective was dissemination of results at different stakeholder conferences, presenting the results of the project. The projects midterm conference was held in Brussels as a special session of the annual meeting of the European Soil Bureau Network in Brussels on 22 September 2010. Symposium participants came from almost all national soil institutes of the EU member states as well as from the United States of America (USA) Natural Resources Conservation Service
The projects final conference was held in conjunction with a large FAO Global Soil Partnership workshop in Rome ('Towards Global Soil Information: activities within the GEO Task Global Soil Data'.). It enabled dialogue between project team members and user groups for around 80 people. The E-SOTER methodology was adopted by the European Soil Bureau Network as the recommended approach for updating soil information at the European level and as tools for European contributions to the Global Earth Observing Systems Global Soil Data task. Significant results are:
1. Mid-term stakeholder's conference with participants from European Soil Bureau Network in 2010.
2. E-SOTER methodology accepted by the Steering Committee of the European Soil Bureau Network in 2012 as the methodology to apply for the update of the 1:1 000 000 European Soil Database.
3. Presentations at several science meetings / June 2011 issue of International Innovation.
Potential Impact:
The final result of the E-SOTER project is a Pilot Platform and a portal that provides open access to:
1. a methodology to create 1:1 000 000 SOTER databases and an enhanced soil and terrain database at scale 1:1 000 000 for the four windows
2. an artefact-free 90m DEM for the windows
3. methodologies to create 1:250 000 scale enhanced SOTER databases and the databases themselves for four pilots
4. advanced RS techniques to obtain soil attribute data
5. validation and uncertainty propagation analysis
6. dedicated applications related to some major threats, like soil erosion by water and soil compaction, to soil quality and performance.
The use of the methodologies by soil and other natural resources institutes has an impact on the soil and terrain information within Europe as well outside. The E-SOTER methodology e.g. was accepted by the Steering Committee of the European Soil Bureau Network in 2012 as the methodology to apply for the update of the 1:1 000 000 European Soil Database.
E-SOTER contributed to one of the Global Data Tasks of the Group on Earth Observation's Work Plan by co-leading the task Global Soil Data (DA-09-03e). The project has organised a group of potential contributors to this task, drafted a task sheet and organised a first meeting for contributors. Once operational this task will increase the exposure of the products compiled by the project and by others.
E-SOTER is a new generation of soil and terrain mapping and database management technologies - with vastly improved applications to support environmental policies, impact assessment and management at regional and global levels.
E-SOTER responds directly to the Global Earth Observing Systems proposal for the Global Soil Data task - embracing global monitoring of environment and security and support for investments in urban and industrial infrastructure.
E-SOTER fully complies with OGC standards and INSPIRE rules and in following the interoperability standards by OGC, E-SOTER will allow data sharing, information management and optimisation of information between observation systems within Europe and worldwide.
List of Websites:
Project website: http://www.esoter.net http://portal.e-soter.net
Contacts: hannes.reuter@isric.org hannes.reuter@wur.nl
Soil and land information is needed for policy-making, hands-on management of land resources and monitoring of the environmental impact of development. Lack of comprehensive information about land resources - globally, nationally or locally - means uninformed policies, continuing degradation of land and water resources, unnecessary carbon emissions to the atmosphere and no likelihood of achieving the Millennium Development Goals (MDGs). The viability and cost of vital infrastructure is affected just as much as food and water security and response to environmental change. In the case of the European soil thematic strategy, the operational measures laid down in the Framework Directive and impact assessment are hamstrung by lack of accessible, easy-to-use, consistent, harmonised and relevant soil data. The Group on Earth Observations (GEO) plans a Global Earth Observing System and, within this framework, the E-SOTER project addresses the felt need for a global soil and terrain database. As the European contribution to a Global Soil Observing System, it will deliver a web-based regional pilot platform with data, methodologies and applications, using remote sensing (RS) to validate, augment and extend existing data.
The collaborative project addressed four major barriers to a comprehensive soil observing system:
1. morphometric descriptions - enabling quantitative mapping of landforms as opposed to crude slope categories
2. soil parent material (PM) characterisation and pattern recognition by RS - enabling separation of soil processes within the landscape
3. soil pattern recognition by RS
4. standardisation of methods and measures of soil attributes to convert legacy data already held in the European Geographical Soil Database and various national databases to a common standard - so that they may be applied, e.g. in predictive and descriptive models of soil behaviour.
The project objectives that contribute to the lifting of the four major barriers described above are:
1. morphometric descriptions of the landforms both in an enhanced SOTER DEM methodology as well in newly developed DEM analysis using natural breaks. The existing DEM that will form the basis for the morphometric analysis will be filtered and enhanced to obtain an artefact-free product. The end products have been several landform layers in the window and pilot areas.
2. soil PM characterisation using RS (RS) and legacy data generated two PM classifications relevant for soil development and PM pattern within the window and pilot areas.
3. soil pattern recognition uses existing data and converted these into a standardised SOTER format.
Using RS generate additional predictors of soil properties. End products will be a soil layer in the window and pilot areas with standardised soil attributes for the 1:1 000 000 scale and the 1:250 000 scale.
Additional project objectives had been:
1. quality assessment of E-SOTER by performing a validation and uncertainty analysis
2. applications of the newly acquired E-SOTER in the field of major soil threats and comparisons with applications based on the European Soil Database.
3. dissemination of the results of the project through stake-holder conferences at the Food and Agriculture Organisation (FAO) and elsewhere and through web-based services (see http://www.esoter.net and http://portal.esoter.net).
The end product is a regional pilot platform with methodologies, concepts and applications that, together, facilitates:
1. an enhanced SOTER database methodology at scale 1:1 000 000 for Europe and the world;
2. ways of generating finer-scale maps of specific soil and terrain attributes and digital data, based on legacy soil survey data and RS;
3. a framework for new, cost-effective field survey and monitoring programmes.
Project Context and Objectives:
Soil is the basis of our daily life; we walk on it, growth our food on it and therefore depend on it for our life. Soil and terrain information is needed for many interpretations for example in the field of agriculture, environment, watershed management, infrastructure, etc. but available data are often inaccessible, incomplete, or out of date. The GEO plans a Global Earth Observing System and, within this framework, the E-SOTER project addresses the felt need for a global soil and terrain database. As the European contribution to a Global Soil Observing System, it delivered a web-based regional pilot platform with data, methodologies and applications, using RS to validate, augment and extend existing data. The project objectives were the following:
1. Morphometric descriptions of the landforms
2. Soil PM characterisation using RS and legacy data
3. Soil pattern recognition using existing data and RS
4. Quality assessment
6. Application and
6. Dissemination of the results.
In detail the final result of the E-SOTER project is a pilot platform and a portal that provides open access to:
1. a methodology to create 1:1 000 000 SOTER databases and an enhanced soil and terrain database at scale 1:1 000 000 for the four windows
2. an artefact-free 90m DEM
3. methodologies to create 1:250 000 scale enhanced SOTER databases and the databases themselves for four pilots
3. advanced RS techniques to obtain soil attribute data
4. validation and uncertainty propagation analysis
5. dedicated applications related to some major threats, like soil erosion by water and soil compaction, to soil quality and performance.
The use of the methodologies by soil and other natural resources institutes will have an impact on the soil and terrain information within Europe as well outside. Initially it was thought that the project will result in an European SOTER Database as an update to the existing 1:1 000 000 European Soil Database. The European Soil Bureau Network together with the Joint Research Centre of the European Commission said in the final stakeholder meeting in Rome that they want to deploy the developed methodology for the 1: 250 000scale in Europe to create an complete E-SOTER product for Europe. Currently the developed E-SOTER methodologies are further applied between ISRIC the Netherlands (NL), Cranfield United Kingdom (UK) and Scilands Germany, small and medium sised enterprise (SME) to apply and test it in Burkina Faso for a possible up scaling to all over west Africa if resources permit.
Towards the global contributions, E-SOTER contributed to one of the Global Data Tasks of the Group on Earth Observation's Work Plan by co-leading the task Global Soil Data (DA-09-03e - now IN02-C2) and will submit the portal and its results to Global Earth Observing Systems DataCore. The project has organised a group of potential contributors to this task (e.g. European Partners). Once operational this task will increase the exposure of the products compiled by the project and by others. The main PI applies the developed methodologies currently in west Africa. The E-SOTER project had technical contacts with the 'GlobalSoilMap.net' consortium for possible collaboration on technical issues.
Project Results:
WP1 developed a methodology for a global platform for terrain and soil parent data at a scale of 1:1 000 000 and conforming to SOTER criteria. Data preparation developed an artefact-free DEM based on shuttle radar topography mission (SRTM 0 thermal noise reduction) which was used for the morphometric characterisation of landforms, Next soil PM within these landform units, based on low-resolution satellite imagery, e.g. advanced very high resolution radiometer (AVHRR), moderate resolution imaging spectroradiometer (MODIS), SPOT Vegetation and DEM data, legacy soil was derived, lastly combining these into terrain units. Significant results were:
1. artefact-free SRTM DEM for the four window areas;
2. artefact-free SRTM DEM for the rest of Europe South of 60° almost completed;
3. revised PM classification focussed on soil forming processes;
4. procedure for PM properties characterisation and delineation for areas with limited or no data;
5. grid layers of the derived PM and physiographic parameters;
6. ArcInfo macro language (AML) script to run the physiographic delineation procedure and aggregation procedure;
7. Arc geographic information system (ArcGIS) tool to develop the SRTM base PM genetic layers;
8. 1:1 000 000 scal E-SOTER geometric databases of the terrain units (landform and PM) for the windows in Europe, Morocco and S-China;
9. report describing the whole procedure (Deliverable D3).
In detail the following results have been successfully obtained:
Task 1.1 developed by Partner SciLands delivered an artefact-free DEM for the four window areas. The algorithms for artefact removal were developed and applied for the four E-SOTER windows and the cleaned SRTM data of the windows were uploaded to the team site and were used as inputs for th E-SOTER unit development for the windows. A complete European coverage has not been created as the pattern of terrain and forest distribution outside the E-SOTER windows is sometimes totally different from the distribution that was found inside. Algorithms to reduce the surface of forest canopies to earth's surface where developed for the E-SOTER windows with satisfying results. But for some regions outside the E-SOTER windows (e.g. in Poland) the results are not satisfying. These new deviating situations needed new tools for handling: adaptation of the applied algorithm. The enhancements of the algorithms for handling forest canopies outside the window areas are still on-going research after the closure of the project.
Selected Result: Bock, M. and Köthe, R., 2009. SRTM DEM pre-processing. Paper presented at Geomorphometry 2009, 28 August to 2 September, Zürich, Switzerland.
Task 1.2 created by Partner University of Miskolc and INRA France delivered Landform classification according to th E-SOTER criteria and the development of terrain units based on the reference material of Dobos et al. (2005). There were two major modifications in the procedure:
1. modification/simplification of the potential drainage density (PDD) procedure
2. removal of PDD as differentiating criteria for areas having a relief intensity higher than 100 m
3. tile size of processing was decreased.
These changes have been applied and inserted into the Dobos et al (2005) procedure. The AML for running the procedure has been updated and applied at several locations throughout the world (e.g. E-SOTER project, Canada, Burkina Faso). For some specific areas, neither the terrain, nor the simplified PM classification resulted in an adequate delineation and huge polygons remained insufficiently disaggregated. In these cases, the soil regions delineated in WP2 were used to further divide the polygons. Further research is needed to address these issues.
Selected Result: AML with documentation available from the E-SOTER website
Task 1.3 performed research to improve delineation of soil PM units across the partners Misc, BGR and ISRIC. The sub-task had two major work lines. The first was to revise the existing lithological classification of SOTER (van Engelen and Wen, 1995) to allow a better correlation with the legacy data from geological maps and to make the classes easier to interpret for soil data users. The second aim was to develop a quantitative procedure for PM delineations for areas with no or limited PM data using RS data (MODIS) and SRTM modelling approaches. Third, the two different approaches were harmonised and integrated into a common data framework and structure.
Fulfilling the first aim of revising and simplifying the PM classification, a revised lithological classification was developed (ISRIC report 2013/6). It describes the unconsolidated materials as soil PMs, using the major PM properties that drive the soil formation processes. This new structure has independent properties to be mapped. All properties are combined in the final step. In this way, all potential input data can be imported into the delineation procedure regardless of the detail of the original PM data description.
The second part of the work was focusing on the quantitative PM data development for limited/no-data areas using MODIS data. A limited set of PM characteristics was selected and procedures were developed to create the appropriate layers. The quantitatively derived PM layers have been completed for the four windows and been passed onto the last step of the procedure development, namely the combination of the physiographic (derived in task 1.2.) and the PM units.
Selected Results:
1. Ulrich Schuler, Rainer Baritz, Jan Willer, J.A. Dijkshoorn, Harald G. Dill, 2013, ISRIC Report 2013/6 A revised approach to classify PM for soil mapping ISRIC Report 2013/6
2. ArcGIS based algorithm to derive the SRTM based PM genetic layers.
Task 1.4 finally combined landform and soil PM units into terrain units by Partners Unimis and INRA. The physiographic units and the PM units were combined to develop the final SOTER-unit polygon system. A thematic distance based rule system was developed to drive the aggregation procedure and derive the final SOTER polygons.
Selected Results:
1. Algorithm incorporated in Task 1.2 results
2. Dobos, E. 2010. A quantitative procedure for building physiographic units supporting a global soil database, Hungarian Geographical Bulletin, 59/2.
WP2 performed research to integrate legacy data with new data from RS. Legacy soil data from two windows in Europe, one in Morocco and one China have been used to compile soil mapping units within the WP1 terrain units, enhanced by MODIS imagery). Th E-SOTER soil component data structure has been revised to allow inclusion of new soil attributes with a complete coverage of th E-SOTER soil units (1:1 000 000). Significant results are:
1. Updated SOTER soil component data structure
2. three methods for harmonisation and correlation of national soil data, namely translation algorithms (prototype) for national soil data; correlation method of the national classification units with world reference base (WRB); decision-tree based method incorporating the centroid concept
3. assignment of representative profiles to the soil components of th E-SOTER units created by WP1
4. deliverable D2.1 (D5) a first version of the harmonised soil data base for the 1:1 000 000 scale windows (except China) was submitted in January 2011.
Task 2.1 by partners SIU, BGR, CULS, CU, ISSCAS and INRA Maroc generated spatial soil information for the 1:1 000 000 window. The major activities of the task were the collection of legacy data for the window areas, collection of RS data and development of methodology for the spatial definition of soil units.
A new E-SOTER approach for compiling spatial soil information has been developed. The approach attempts to estimate the spatial occurrence probability for the mapped area using RS, digital terrain data and pre-processed legacy data as training dataset.
The collection of legacy soil information for most of the research area enabled the development of Pedotransfer rules to harmonise data of various origins within the pilot windows and from national system into a generalised one for the central European (CE) window. French data are missing due to intellectual property (IP) restrictions. Data were derived from: national data - provided by the window partners, ISRIC World Inventory of Soil Emission Potentials (WISE) international soil profile dataset, SOTER dataset.
The collection of RS data was based on MODIS multi-temporal eight days composites over five dates covering the none-snow-covered period, evenly distributed over the vegetation period. Additionally, the WP1 DEM was used to derive terrain derivatives.
Lastly, a methodology for the spatial definition of soil units is based on the defined (from legacy data) master building units of the WRB classification system such as the diagnostic properties or horizons (DPDH). The approach attempts to estimate the spatial occurrence probability for the mapped area using RS, digital terrain data and pre-processed legacy data as training dataset. The success and detail of the approach depend largely on the quantity and quality of the input training set. In general, some of the related/similar classes have to be combined and new, more general classes are created to characterise the soil resources of the polygons. As the WRB includes numerous diagnostics, a limited set of significant units is defined by an expert group based on the existence and significance of horizons, properties and materials. Training datasets for this group of diagnostics are derived from legacy data, mainly using profile data or representative large scale database windows. Each training datasets consists of points or areas with known existence or absence of the property in question. Using these training datasets for classifying a complex MODIS/SRTM based image results in numerous continuous layers for each property having the probabilities of the existence of the diagnostic property. The major advantage of this approach is that it provides the needed thematic information on important or master soil properties, like texture, organic matter, salt content etc. At the end, a WRB-based simplified classification scheme is developed to identify the WRB soil types for each pixel. This raster database is used at the end to calculate the percentage of coverage of each soil type within each polygon, the so called definition of soil components. This approach is suggested to be used, when no contiguous coverage of legacy data is available for the area. The traditional approach should have a priority over the E-SOTER approach, except when the data quality of questionable or non-satisfactory.
Selected result:
Harmonised soil data base for the 1:1 000 000 scale windows at the portal website.
Task 2.2 performed research on the compilation of the soil data base for the 1:1 000 000 scale windows by partners SIU, BGR, Unimis, ISRIC, CU. The e-SOTER database is derived from and compatible with the Global Soil and Terrain (SOTER) database (van Engelen and Wen 1995). It is composed of a set of tables in a relational database management system (RDBMS) linked to the geometric database (GIS file). The database is composed of various tables; the terrain, terrain component and soil component tables that describe the attributes of the spatial parts of SOTER, while the profile, horizon and WRB diagnostics describe the attributes of the profile (point) data. The E-SOTER database has undergone a number of improvements in its structure, compared to the original SOTER database.
One of the major improvements has been the application of the updated standards for soil descriptions and soil classification. The master horizon designation, subordinate characteristics and site descriptions follow 2006 edition of the FAO Guidelines for soil description (FAO, 2006). The classification of soils and the related diagnostic horizons, properties and materials are described and coded according to the WRB for soil recourses (IUSS Working Group WRB, 2006, 2007). All profiles (representative and reference) are given both the WRB classification, to link to new developments and the classification according to the Revised Legend of the Soil Map of the World (FAO 1988) to link to the former SOTER databases. To facilitate using SOTER as a profile database also PM, land use and vegetation are given now in the profile table.
A new table, the WRB diagnostics, was included to describe properly the WRB diagnostic horizons, properties and materials as well as the depth of occurrence. All diagnostic horizon or property can apply for the same horizon and can be described. The qualifiers are listed together with the reference group.
Another improvement has been the inclusion of the small-scale map legend using the WRB for soil resources (addendum to WRB, 2010) in the soil component table. The WRB Legend and the Revised Legend (FAO 1988) are directly linked now to the geometric SOTER database and enable the user to derive various maps e.g. a dominant soil without too much database manipulation.
Translation and harmonisation of national soil data into the newly defined E-SOTER standards required a range of translation algorithms and correlation methods to be developed and to be applied to the existing data:
1. based on original field description and laboratory data,
2. based on taxonomic distance calculation of the national classification units with the WRB and
3. a decision-tree based method incorporating the centroid concept. The availability and the completeness of the received data from the window partners was variable and in several cases insufficient for proper correlation with the WRB. Sometimes, due to limited data availability the closest profile could occur in other country or continent. Further research is needed to increase the freely globally available soil data.
Selected Results:
1. Eberhardt, E. and Pietsch, D., 2009. Classifying soils according to WRB with national soil legacy data. Abstract of the papers of the Conference 'From the Dokuchaev School to numerical soil classifications', 18 September, 2009 Gödöllo, Hungary
2. Waltner, I. and Micheli, E., 2008. Computer aided classification of some selected soils of Hungary. Abstract of the papers of the International Soil Classification Conference, Santiago, Chile, 2008
3. Láng, V., Fuchs, M., Waltner, I., Michéli, E., 2010. Pedometrics application for correlation of Hungarian soil types with WRB, 19. IUSS World Congress, Brisbane, Australia
4. Láng, V., Fuchs, M., Waltner, I., Michéli, E., 2009. Correlation possibilities based on taxonomic distance measurement. 25th SSSEA Conference, Moshi, Tanzania
5. Láng, V., Fuchs, M., Waltner, I., Michéli, E., 2010. Taxonomic distance measurements applied for soil correlation, Agrokémia és Talajtan (in press)
6. Waltner, I., Láng, V., Fuchs, M. and Michéli, E., 2010. Application of a centroid-based concept for the correlation of national soil classification with the WRB fourth Global Workshop on Digital Soil Mapping, 24-26 May 2010 - Rome, Italy, abstract
7. Eberhardt, E. and Waltner, I., 2010. Finding a way through the maze - WRB classification with descriptive soil data 19th World Congress of Soil Science, 1-6 August 2010, Brisbane, Australia, abstract.
WP3 provided a high-resolution E-SOTER database for 1:250 000 scale pilots using state of the art DEM analysis (hierarchical classification, natural breaks). PM has been derived from RS using multispectral imagery and airborne gamma spectrometry. Europe-wide additional predictors of soil properties at various spatial, spectral and temporal scales have been integrated in the developed terrain units, their underlying PM and the respective semantic soil information. Some of the significant results include:
1. provision of draft methodology for alternative ways of derivation of physiographic units at 1: 250 000 scale and application of it
2. revision of the FAO PM classification (Schuler et al. 2013); includes the coordination with internationally agreed geological terminology and application thereby delivering a Proof of concept to utilise geological map data for E-SOTER
3. development of the soil component for E-SOTER using digital soil mapping and comparison with soil maps and representative soil profile data
4. a new task force on superficial deposits was founded under the umbrella of EuroGlobal Earth Observing Systemurveys. The idea is to develop a PM map for Europe as a result from the E-SOTER project results
5. use of RS for the further refinement of better defined lithological strata providing, if possibly, some information on key mineralogical features.
Several different approaches were investigated by the different partners to fulfil the objective of new classification algorithm of landform types (Task 3.1).
Cranfield University investigated how hill shed and slope break analysis for landform classification can be combined. Slope break (hill slope) analysis is an extension to hill shed analysis in the sense that it aims at classification of slopes within hill sheds. Classification is based on the most distinct breaks in slope and assigns three major classes within the length of slope. Hill slope analysis is particularly useful in correcting the 'bleeding effect' within peak sheds that were originally used in landform classification approaches. The 'bleeding effect' occurs due to the fact that peak sheds comprise areas between passes, or passes - bottoms of valleys resulting in incorporation of flatlands into peak sheds located at the border between hills and valleys.
Partner Scilands developed new algorithms for the advanced classification of continuous morphometric terrain parameters were developed. These algorithms allow the identification of 'natural breaks', using segmentation techniques as known from RS. It is now possible to identify homogeneous units (areas) of terrain parameters - as a base for terrain classification. In contrast to many segmentation techniques in RS the developed algorithms work without any 'seeds' what we regard as an advantage. 'Seeds' (seed grid cells) have to be given to common segmentation algorithms as input data. These seeds produce sometimes units which are unintended or some relevant units are missing (caused by missing seeds). The new algorithms in contrast allow the identification of 'natural breaks', using segmentation techniques as known from RS. The segmentation of the continuous terrain parameters results in a classified data set, which contains the 'right' delineations but also just as many those are not the intended. The challenge now is to identify homogeneous units (areas) and to combine the units in a plausible way.
The parameter Scale is an important property in any landform classification. Therefore a third attempt was dedicated especially to find scale-independent landform classification parameter datasets for all kinds of landscapes in Europe. The concept of terrain classification follows a pragmatic-deductive approach based on geomorphological knowledge (including knowledge of the genesis of landforms) by including - in first step simple - geological information. In a first stage the classification procedure distinguishes between two groups:
1. areas with consolidated rock as the PM; and
2. areas with unconsolidated rock as the PM.
Based on these initial delineations, methods of terrain classification follow a pragmatic approach and vary to calculate (delineate) the desired terrain units. The objective of all methods is to work with self-adjusting thresholds and natural breaks (both within ranges of values of the terrain parameters according to local terrain conditions, followed by an generalisation procedure with:
1. generalisation with regard to the content (types of the terrain units) and
2. generalisation regarding the size and shape of the individual terrain units. The system of terrain classification consisted of the following terrain units: 'Flats': several levels of 'Bottom Flats', several levels of 'Flats and Terraces' and 'Mountainous Summit Flats', 'Slopes': 'Divergent Slopes, Ridges and Bumps', 'Convergent Slopes and Small Valleys' and 'Upper Slope', 'Mid Slope' and 'Lower Slope, Gentle Slope'.
PM is one of the input drivers into the E-SOTER methodology. Unfortunately, it is not readily available from Geological Information. Task 3.2 investigated if RS analysis for soil PMs can be applied through a set of partners (BGR, JRC, CU, CULS). In central Europe, soil PM mapping based on satellite data is hindered by the vegetation cover. The only exceptions are bare arable land and rock outcrops. In addition, pronounced landscape formation features help to identify PMs at the landscape level. Such complex landforms can be identified from DEMs (e.g. karst towers, cruesta shaped landscapes). This E-SOTER task searched to optimise the use of different data sources to delineate types of PM in the context of soil mapping.
Research to improve the development of a SOTER PM layer pursues two main approaches, RS and the use of legacy geology data.
RS focussed on the use of hyper spectral, such as the advanced spaceborne thermal emission and reflection radiometer (ASTER) or Hyperion and gamma-ray sensing in different windows (e.g. Morocco, Germany). Gamma ray was clearly able to distinguish different geological settings and with that some PM.
During the process it became apparent that the existing (also in WP1) PM classification has some deficiencies. The existing FAO/SOTER hierarchy was adapted to the requirements of the E-SOTERWP1 definition, but also in relation to other existing soil PM classifications. In addition, the consistency with the OneGeology terminology was maintained, so that geology legends can be easily understood and re-interpreted by soil scientist. On that basis, geology map legends can be easily aggregated to the level required by E-SOTER and other applications in digital soil mapping, for example GlobalSoilMap. Besides parent rock, a second table on genetic aspects ('surface processes') has been added, so that the unconsolidated sediments can be further specified. On that basis, the vertical differences of unconsolidated, weathered or re-located layers can be described together with the properties of the underlying parent rock. Thus, the revised version is not only consistent with the geochemical quality of rocks and the geological terminology, but also suitable for field work (soil profile description). Geological maps can now easily be re-interpreted for SOTER mapping.
In the Research on RS for soil attribute data (Task 3.3) a wide range of activities have been involved (sampling, prediction, evaluation). During the collection of the field data using the constrained Latin hypercube sampling (LHS) aiming to acquire a sample representing the full range of covariate space (in this case three principal components and elevation) under budget and accessibility constraints it became clear that strong local variability is preferentially sampled which is preferred for Regression Tree/CART models, however might causes problems if the data are to be used for variogram estimation in a Global Earth Observing Systemtatistical analysis. Finally, the hyper spectral data proved efficient together with a developed a linear spectral trend model to predict the main composite mineralogy based on VNIR spectroscopy. For further details please refer to deliverable D7 and the papers by Mulder et al.
To integrate all the different products (geomorphic terrain units, PM and semantic soil information for each pilot area).
In the frame of WP3 an alternativ E-SOTER map was generated, merging the Geomorphographic map (1:1 000 000 scale) with the PM map and the map of the surface processes. The preservation of streamlines and major slope breaks gives this SOTER map a clearer structure of the landscape than th E-SOTER map of WP1. Additionally, 14000 soil augering observations were used to develop a methodology to derive a substrate unit map using classification tree (CART algorithm). The substrate map clearly shows a gradient from silty substrates in the lowlands to sandy and loamy substrates at elevated parts in the Ore Mountains. The analysis of different relief positions dependent of parent rock revealed a dominance of silty substrates for all relief positions, except for medium to steep slopes. This means that on steep slopes the loess cover is eroded and the soils are stronger influenced by the bedrock.
The E-SOTER map developed in the frame of WP3 represents both soil PM and relief. The high correspondence of the sand content distribution according to th E-SOTER map with both the geological map 1:50 000 and the radiometric maps show clearly that th E-SOTER map is suitable for mapping of soils and soil properties at small scale.
Selected results:
1. Mulder, V.L. de Bruin, S., Schaepman, M.E. and Mayr, T.R. 2011. The use of RS in soil and terrain mapping - A review. Geoderma, 162(1-2):1-19
2. Schuler, U., R. Baritz, J. Willer, J.A. Dijkshoorn, H. G. Dill (2013). A revised approach to classify PM for soil mapping. ISRIC Reports. ISRIC-Report 06-2013.
WP4 objective was the validation and uncertainty analysis with independent validation data for different scale windows. A comparison was made between the accuracy obtained with products of WP1 and WP2 with that of WP3. Sensitivity of SOTER methodologies have been identified using state of the art statistics (e.g. probability distribution functions, Monte Carlo simulation).
1. development of a sound methodology for statistical validation of digital soil maps and E-SOTER products including purity, user's accuracy and producer's accuracy of the E-SOTER soil maps as well as how aggregation and generalisation steps deteriorate the quality of E-SOTER landform maps
2. a Monte Carlo error propagation methodology was designed and implemented with which the propagation of DEM errors to the E-SOTER landform maps can be traced
3. correlation procedures between the national soil classification systems for England and Wales, Germany and the Czech Republic with that of the WRB were established and automated.
Research in WP4 investigated validation and uncertainty propagation analysis of E-SOTER products by comparing the E-SOTER soil and landform maps with independent validation data and by analysing how errors in a DEM propagate to the E-SOTER landform map.
Landform validation was done by comparing the true landform in the western European (WE) and CE windows with the landform as depicted on the E-SOTER maps. The various simplification and generalisation steps of the E-SOTER landform classification methodology cause discrepancies between the true and predicted landform. Validation showed that the accuracy of the E-SOTER landform maps is high for landform classes 'elevation', 'relief intensity' and 'flatness', with agreement between true and predicted class in 81 to 98% of cases. For landform class 'slope' the agreement is only around 50%, which means that in 1 out of 2 cases the E-SOTER map does not agree with reality. This is caused by the highly fragmented spatial pattern of the true slope map, a feature that cannot be reproduced in the E-SOTER map because it must generalise the map to the 1:1 000 000 scale. Comparison of landform validation results between the WE and CE windows did not show large or meaningful differences.
Soil validation was based on a comparison of the dominant soil classes on the E-SOTER maps with independent soil observations derived from existing legacy data. This first required a conversion of soil classes from the local classification system to the WRB. For this purpose correlation tables were prepared. The validation results show that the E-SOTER soil map of the UK pilot area reproduced the large scale patterns and had an agreement with the 'true' soil class of 51%. The E-SOTER map overrepresented Histosols and Podzols and lacked Leptosols as a dominant soil group. Validation results in the German / Czech pilot area revealed an overall agreement of only 32%. The rather low purity in this case can be assigned to high variability of the soil cover and often low dominancy of the dominant soil unit in th E-SOTER units. Also, in this case the more detailed validation data allow the use of more strict validation criteria than in the UK case. Overall, the fairly low agreements between E-SOTER soil maps and validation data in both pilot areas show that the soil maps have large uncertainties, even at the coarse soil group level that was considered here. Part of the disagreement may be caused by errors in the validation data, but this is unlikely to be the major cause of the discrepancy.
The uncertainty propagation analysis showed that DEM uncertainty mainly affects the E-SOTER slope class. The real elevation is noisier than the smoothed DEM and this causes the DEM-derived slope to be too small. Elevation was hardly affected while flatness is only sensitive to DEM errors in relatively flat areas and relief intensity in areas with a more intense relief (i.e. the CE pilot area). Overall DEM uncertainty does not seriously impair landform accuracy and is mainly restricted to zones along class boundaries. It is a smaller source of uncertainty than the simplification and generalisation steps that were analysed in the landform validation procedure.
Selected Results:
1. Brus, D.J. B. Kempen and G.B.M. Heuvelink (2011), 'Sampling for validation of digital soil maps', European Journal of Soil Science 62, 394 407.
2. Algorithms to reproduce validation and uncertainty research available at the algorithm database of the E-SOTER portal.
WP 5's objectives are applications of the E-SOTER database. It investigated whether use of the E-SOTER database has improved evaluation of threats to soil quality. Appropriate models that can measure the threats (EU Soil Thematic Strategy) at the relevant scales were selected and run with E-SOTER data and any additional data required. Identification of threats, data collection, a comparison of the threats based on E-SOTER and on pre-existing data sources considering both spatial patterns and at statistical trends and a comparison between existing data on threats and the results obtained from the 1:1 000 000 and 1:250 000 scale windows by running models for the most important threats determined was performed. Significant results are:
1. selection of methods to evaluate major soil threats and collection of data to run the models
2. development of evaluation method by elicitation
3. comparison of model outputs from legacy data and E-SOTER with the conclusion that the model outputs for the soil threats concerned based on the E-SOTER database are similar to once derived from legacy databases.
Task 5.1 with partners Alterra, JRC, BGR, CULS, INRA, INRA-Maroc WP5 members identified several threats as the most important ones in the different study areas.
In these lists, three threats are mentioned more than once, namely soil erosion (by water), loss of organic matter and compaction. Landslides are only important if the Moroccan pilot area is shifted northwards and desertification is currently not one of the soil threats listed by the EU Soil Thematic Strategy. Task 5.2 collected the data necessary to run the models. Thereby, e.g. a conversion strategy has been developed to reclassify different land use classifications. Unfortunately, in-depend validation data have not be available for the scale of the models. For these reasons, expert opinion was used in this study to evaluate which data base was performing better in the evaluation of soil erosion by water. A method for evaluating modelling results using expert opinion was devised.
Experts were asked in Task 5.3 to indicate the spatial distribution of the target variables 'sensitivity to soil erosion', 'potential soil loss' and 'inherent susceptibility of subsoil compaction' in a set of randomly selected nomenclature of territorial units three (NUTS3) units in the three windows. For each combination of window and soil threat, 3 specific experts in the field of soil erosion and/or compaction and with regional knowledge of the window, were invited to respond. A statistic referred to as D, that is defined as the maximum difference between the cumulative probabilities of the (discrete) spatial distributions of model outputs and/or expert assessments has been used. The larger D, the more distinct the spatial distributions. Statistic D varies from 0 (distributions are identical) to 100 (distributions are totally different).
Differences between expert judgements expressed as D vary between the windows and target variables. In general, larger values and larger variations of D were found for all target variables in the WEU window compared to the CEU and MOR windows except for soil compaction, for which a large variation in D was also observed in the other windows. This observation implies that experts agreed less on the target variables for the WEU window compared to the other two windows and might also indicate that soil compaction is hard to model and hard to assess by the experts.
In all three windows, larger values of D and a larger variation of D were observed for soil compaction compared to soil sensitivity to soil erosion and potential soil loss, with values of D mostly above 50%. This implies that experts agreed less on the susceptibility to soil compaction than on the target variables referring to soil erosion. The areas of the administrative units that were assessed did not seem to influence the differences between expert judgements very much, except for the potential soil loss in the CEU window, where the experts seemed to agree more for larger NUTS3-units.
Overall, the deviation of the model output for all target variables is large compared to the expert responses, with values of D up till 100% for erosion sensitivity and susceptibility to subsoil compaction in the CEU and WEU windows. Four reasons may explain this observation:
1. the experts were provided with less detailed spatial information on the land properties relevant to the soil threat, whereas the models were fed with this information at the level of 1 km2 pixels,
2. the experts estimated the coverage of the classes of the target variables visually ('eye-ball estimates'), whereas the coverage of classes was exactly determined in the analysis software
3. the experts used process knowledge, external information and field knowledge, that was not available to or incorporated in the model applications and
4. the experts were insufficiently informed on the areas selected for the questionnaire.
Differences between the value of D from model applications using the E-SOTER database (compared to expert judgment, DE-SOTER) versus the legacy databases (compared to expert judgment, Dlegacy) were calculated to assess if outputs based on the E-SOTER database are more in agreement with the expert assessments than model outputs based on the legacy databases. If these differences would be negative, this would imply that using the E-SOTER database would result in model outputs for the soil threats in better agreement with the expert judgments. The results show that both positive and negative differences between DE-SOTER and Dlegacy occur, indicating that model outputs based on the E-SOTER database are not always better according to the experts than those based on legacy databases. This observation applies to assessments from experts irrespective of their informedness, meaning that the results would not change if only experts who considered themselves to be well informed were considered. Model outputs for the susceptibility to soil compaction show the largest differences in the use of the E-SOTER or the legacy database when compared to the expert assessments. This may be due to the larger disagreement between the experts with regard to the susceptibility to soil compaction and that it is probably harder to model/estimate soil compaction than the other properties.
The results of this study showed that the model outputs for the soil threats concerned based on the E-SOTER database are not always better than those based on legacy databases. This may be due to the identified differences in the soil properties in both databases that are input to the models, which in some E-SOTER units result in model outputs more in agreement to the expert judgment using the E-SOTER database and in other E-SOTER units in model outputs more in agreement to the expert judgement if the legacy database is used. Furthermore, several other reasons may explain the result:
1. Contrary to the legacy databases, the E-SOTER database does not fully cover the administrative units in the windows. As a consequence, estimates by the experts often pertain to a larger (and therefore different) area than the model outputs based on the E-SOTER database.
2. The models only use a part of the soil data in the databases and therefore the comparison of the databases only refers to the input variables of the models that differed between the databases.
3. Model outputs are on ordinal scales (ordered classes). Differences between the databases providing the model inputs may therefore be tempered.
Selected result:
1. Hessel, R., J.Daroussin S.Verzandvoort D.Walvoort. Suitability of two different soil data bases to assess soil erosion using MESALES for three areas in Europe and Morocco (in prep.)
WP6s objective was the development of an E-SOTER dissemination platform on the Infrastructure for Spatial Information in the European Community (Inspire) principles. Task (T6.1) analysed the data specification and exchange rules (XML) and prepared the SoTerML. Next (T6.2) profile and analytical data management for existing legacy data and newly generated data was performed. Lastly, (T6.3-5) created a European E-SOTER Portal, including RDBMS structures, algorithm database and data dissemination services, cookbook, algorithm database, hosted now at ISRIC for long term management. Based on the results in D6.1 any other party/third party country (e.g. as part of Development Aid) might deploy a similar system. Significant results are:
1. SoTerML schema along with its documentation for soil data exchange available at schema.isric.org for long term maintenance
2. E-SOTER parser package to convert legacy SOTER data
3. Creation of a global/European E-SOTER Portal, including RDBMS structures, algorithm database and data dissemination services
4. Cookbook which allows recreation of the E-SOTER portal functionality (e.g. as part of development work in another country).
Research in Task 6.1 with partners Unott, JRC, BGR, CU and ISRIC performed an Analysis of data specification and exchange rules (XML) and the development of concept rules for SoTerML (OGC complained). The SoTerML serves as data exchange and standardisation format. The design of the XML format so called SoTerML was taking in account two competing aspects. The first one is the various soil attribute 'profiles' potentially used in SOTER databases such as the SOTER, WRB, the FAO classification schemas and on the other hand the interoperability developments led by OGC about geographical dataset. On one side its goal is to be able to facilitate understanding and transfer from existing format and model and one the other side is to provide exchange of dataset in a harmonised and standardised fashion also compliant with existing standards. Development of the platform was based/considered on agreed International Organisation for Standardisation (ISO) standards (ISO19136 (GML), ISO19110 (Feature Catalogues), Global Earth Observing SystemciML and ISO group /TC190/SC 1 N140 'Recording and Exchange of Soil-Related Data', the Global Earth Observing Systems subtask DA-09-03e on global soil data and on soil data model developments of the GlobalSoilMap.net project.
All existing legacy data were transferred to the new SOTER-ML language using a custom made 'E-SOTERParser' package, which converts the legacy SOTER database Access file and then the accompanying ESRI GIS Shape file (Task 6.2). This was a prerequisite for the successful execution of Development, setup and implementation of a global/ European E-SOTER portal by partners ISRIC, JRC and CU.
Development of the E-SOTER approach has culminated in the 'E-SOTER' project, which has sought to create harmonised SOTER database windows in western Europe between France and England; the Chemnitz region between the Czech and German borders; northern Morocco; and China (see http://portal.esoter.net/ and http://eusoils.jrc.ec.europa.eu/projects/E-SOTER/). As the European contribution to a Global Soil Observing System (part of GEO), a key element to these phases of activity has been the concurrent development of an E-SOTER web portal and related information spatial data infrastructure, designed to hold, manipulate and disseminat E-SOTER databases. The portal is running on a Virtual Machine based on a Linux Server Distribution and the content has been created using several open source tools which are freely available under General Public License (GPL) with great support from open source communities. The portal deploys Joomla as a Content Management System, PostgreSQL/PostGIS as its Database System, GeoNetwork for its MetaData Search and Portal Facility, Global Earth Observing Systemerver for WebMappingService (WMS) and WebFeatureService (WFS) and trac-project/mercurial for the algorithm database.
The technical solutions have been tested by JRC and ISRIC and data exchange connections have been tested against other services (at FAO, ISRIC, JRC). Generally, the portal products will be hosted at ISRIC World Soil Information. ISRIC as the world data centre for soil has a long-term mandates beyond the lifecycle of the E-SOTER project. One of the main problems in research today is to produce/maintain documentation of steps how a specific task has been performed (reproducible research). As the E-SOTER project developed a range of new methods, therefore a stringent documentation of these methods was desired. Due to the different software development environments, the solution could only be to store algorithm in a more descriptive way, while maintaining the source code in the specific software. Most of the algorithms developed in the project have been introduced into the algorithm database, while one specific one (Bulk Density estimation) has been tested for further implementation directly on the E-SOTER database.
To distribute the installation instructions of the E-SOTER portal an electronic cookbook has been set up which allows a much more dynamic and up-to-date information to be disseminated. This cookbook is distributed online via the E-SOTER portal - website. The steps in the cookbook describe the process of preparing a server to host the tools necessary to operate and run the E-SOTER web Portal and associated components.
1. Selected Results:
1. A. Pourabdollah, D. G. Leibovici, D, M. Simms, P. Tempel, S. H. Hallett, M. J. Jackson (2012) Towards a standard for soil and terrain data exchange: SoTerML, Computers & Global Earth Observing Systemciences, doi:10.1016/j.cageo.2011.11.026
2. E-SOTER portal cookbook http://portal.esoter.net/downloads/esoter_Portal_Cookbook_V5_6.pdf
WP7s objective was dissemination of results at different stakeholder conferences, presenting the results of the project. The projects midterm conference was held in Brussels as a special session of the annual meeting of the European Soil Bureau Network in Brussels on 22 September 2010. Symposium participants came from almost all national soil institutes of the EU member states as well as from the United States of America (USA) Natural Resources Conservation Service
The projects final conference was held in conjunction with a large FAO Global Soil Partnership workshop in Rome ('Towards Global Soil Information: activities within the GEO Task Global Soil Data'.). It enabled dialogue between project team members and user groups for around 80 people. The E-SOTER methodology was adopted by the European Soil Bureau Network as the recommended approach for updating soil information at the European level and as tools for European contributions to the Global Earth Observing Systems Global Soil Data task. Significant results are:
1. Mid-term stakeholder's conference with participants from European Soil Bureau Network in 2010.
2. E-SOTER methodology accepted by the Steering Committee of the European Soil Bureau Network in 2012 as the methodology to apply for the update of the 1:1 000 000 European Soil Database.
3. Presentations at several science meetings / June 2011 issue of International Innovation.
Potential Impact:
The final result of the E-SOTER project is a Pilot Platform and a portal that provides open access to:
1. a methodology to create 1:1 000 000 SOTER databases and an enhanced soil and terrain database at scale 1:1 000 000 for the four windows
2. an artefact-free 90m DEM for the windows
3. methodologies to create 1:250 000 scale enhanced SOTER databases and the databases themselves for four pilots
4. advanced RS techniques to obtain soil attribute data
5. validation and uncertainty propagation analysis
6. dedicated applications related to some major threats, like soil erosion by water and soil compaction, to soil quality and performance.
The use of the methodologies by soil and other natural resources institutes has an impact on the soil and terrain information within Europe as well outside. The E-SOTER methodology e.g. was accepted by the Steering Committee of the European Soil Bureau Network in 2012 as the methodology to apply for the update of the 1:1 000 000 European Soil Database.
E-SOTER contributed to one of the Global Data Tasks of the Group on Earth Observation's Work Plan by co-leading the task Global Soil Data (DA-09-03e). The project has organised a group of potential contributors to this task, drafted a task sheet and organised a first meeting for contributors. Once operational this task will increase the exposure of the products compiled by the project and by others.
E-SOTER is a new generation of soil and terrain mapping and database management technologies - with vastly improved applications to support environmental policies, impact assessment and management at regional and global levels.
E-SOTER responds directly to the Global Earth Observing Systems proposal for the Global Soil Data task - embracing global monitoring of environment and security and support for investments in urban and industrial infrastructure.
E-SOTER fully complies with OGC standards and INSPIRE rules and in following the interoperability standards by OGC, E-SOTER will allow data sharing, information management and optimisation of information between observation systems within Europe and worldwide.
List of Websites:
Project website: http://www.esoter.net http://portal.e-soter.net
Contacts: hannes.reuter@isric.org hannes.reuter@wur.nl