Spatineo Quality Awards for Iberian SDIs
As a result of the INSPIRE Directive implementation efforts made by these 2 member states in the Iberian peninsula, Spain and Portugal, and the general need from the society for services containing spatial data, more than six thousand spatial web services are currently harvested by the Spatineo monitoring agent in the Iberian peninsula, of which approximately 30% are INSPIRE services1, varying from Viewable and Downloadable service types, providing data sets for several spatial data themes2. The amount of spatial services seems to be at a good level, but how good is the overall quality of the services provided?
In order to investigate the Quality of Service for Iberian spatial services and also give credit for those organisations going the extra mile for providing high-quality and reliable usage experience for their users, we used the extensive service availability records we have harvested and kept over the last 8 years while operating Spatineo Monitor, to award the best available Iberian SDIs in general, and added service description and validation information to rank and award the best providers of INSPIRE services in the region.
Based on a consistent and transparent quality-of-service evaluation method, we revealed the winners of the 1st Edition of Spatineo Quality Awards for Iberian SDIs and Iberian INSPIRE Service providers. Similar work has been presented in 2017 and 2018 INSPIRE Conference editions.
Here we describe the evaluation methodology used for INSPIRE and non-INSPIRE services’ quality rankings, the challenges and solutions in running this ranking calculations, some statistics about the quality of services according to our database, and a detailed breakdown on the different Quality of Service indicators of the top ranking organisations in the analysed period (October 2019-September 2020).if you are not familiar with the monitoring system developed by Spatineo, before you go further into this article, I recommend a reading as for a basic reference about how our harvest and monitoring system works, in our previous blog post.
Read More: Why Spatineo is monitoring Open Spatial Web Services?
Directory of Spatial Web Services as backbone of Quality Awards
From the public page https://directory.spatineo.com/, it is possible to see a ranking of individual services based on their availability level per country. With that info on hand, we listed the organisations having the best average of availability among all their services listed (and measurable) during the analysed period. The scoring system used to rank was in the range 0-1.0, 1.0 representing 100% availability. Meaning that an organisation with 2 services listed, having 1 service with say, 98.5% and the other with 100% availability levels, would get the score of 0.9925. We also used a criteria that overall per organisation there should be a minimum of 10 datasets among all the services included in the calculation.
For the INSPIRE services, we also used the availability levels to base the first step of the calculations, but we added a metadata validation step according to the INSPIRE Reference Validator3. The results of that validation were then compiled in the form percentages of successful tests, and applied to a function to normalize the results in the same range of 0-1.0 described above. The assumption used was that the 5% top services with less errors would imply 1% less in the score system for validation, meaning achieving 0.99, e.g.
The final score for INSPIRE service providers was calculated multiplying the scores obtained in availability (0-1.0) and validation (0-1.0). Then, the final ranking was an average of the sum of these individual final scores per service of the organization.
Curious Facts about spatial web services
When running these calculations, we actually face some curious facts along the way that we would like to share. Here they go: There were 4380 individual spatial web services listed in total for the calculations, 1850 of them were INSPIRE services. 3906 total and 883 INSPIRE services qualified for the period. There 74 different organisations listed, 30 of them with INSPIRE services following the criteria. The minimal cut of 10 datasets, made these numbers shrink to 69 and 22, respectively. One particular stat noticed was that 29 (out of 74) organizations had achieved an average of 99% or above in availability. This, combined with the fact that some interruptions were scheduled and not unexpected events, make this number even more meaningful.
Now, with no further ado, we are happy to share the list of the awarded organisations, the first 3 ones in each list, along with the list of the top 10 for both INSPIRE services providers and the total number of services in the Iberian península:
INSPIRE Services:
- Junta de Castilla y León
- MAPAMA
- Ayuntamiento de Barcelona
- Junta de Andalucía
- Consell Insular de Menorca
- IDE de Andalucía
- Diputación Foral de Gipuzkoa
- Região Autónoma da Madeira
- Direção-Geral do Território
- Gobierno Vasco
Services – Total:
- Ajuntament de Vilanova
- Universidad de Alicante
- Diputación de Badajoz
- Universidade de Vigo
- IDEGranCanaria
- GRAFCAN
- EPRINSA
- IDEC – Catalunya
- Diputación de Jaén
- Confederación Hidrográfica del Segura
We welcome all the organisations to congratulate those listed here and if your organisation is part of the Iberia peninsula, but it’s not listed here, and it wants to participate in next year’s ranking or simply curious about how the spatial services are performing in your SDI, here it is what we recommend you to do. First things first, check the Spatineo Directory page and see if the services are already identified by our monitoring agent. If they are not, most likely the agent will find the URL associated with the service or its Capabilities Document.
Once identified and monitored, it’s time to look at the availability, reliability and performance of the services, which ultimately translate into quality, frequently represented by QoS. Hence, to improve QoS we can list a summary of recommendations from our technical team. Obviously, this list does not intend to cover all the possibilities, but provide a fairly good head start. Here it goes:
- Describe the Service: Tell what data the service provides, and for which use. Describe the whole service and datasets. Make sure the service has a title, an abstract and keywords.
- Organize the datasets: Bear in mind that a high number of datasets in a service may affect performance. Rule of thumb, more than 500 datasets in the same service should raise a red flag. But on the other hand it is also not reasonable to have 1 single data offer for each service, especially if there are hundreds of them.
- Assure functionality: This is where most organisations that face issues in their infrastructures usually lack of diligence. Tests should be made with several client applications before services are published. A reliable capacity definition test is key to a successful operation of the service in the real usage world. Make sure the service and data is compliant with industry standards (OGC, INSPIRE), both from the service configuration and data validation point of view. And last but not least, ensure that each service works in a fast and reliable manner, so users get a smooth experience when accessing the information.
We would also like to open the discussion about the best methods and ideas for improving the quality of Spatial Web Services of the SDIs in the region. We invite the GIS community to share ideas and comments to improve the ways we can currently (or in future) better assess the quality and reliability of spatial web services of the Iberian SDIs.
