Validation and accuracy

Bias and error validation of Solcast data against surface measurements

During early and mid-2019, extensive analysis was performed comparing Solcast data with publicly available surface measurements from around the world, across all climate types except polar climates.

The internal Solcast analysis presented here can be referenced against a growing list of validation papers listed on the publications page. If you would like a copy of the validation timeseries data, a sample for one site is provided here on our website, and you are welcome to make a request for us to send you the data for all sites (be sure to tell us about your intended use case). This validation and accuracy information is correct to the best of our knowledge, but should not be interpreted as any form of guarantee or warranty. If you’d like help with further analysis of uncertainty at your specific site(s), please reach out to us to speak about a consulting study.

Site Selection

For validation of satellite-derived irradiance estimates, granular time series data from uniform, high quality, well maintained, long-period surface measurement sites are required. The Baseline Surface Radiation Network (BSRN) was selected to ensure the consistency of sensor specifications and the measurement calibration and quality control approach. A number of BSRN stations are equipped to measure solar and atmospheric radiation, using sensors of the highest available accuracy and a high sample rate. The BSRN network was set up specifically to provide high-quality observational data to the scientific community, making it ideal for validating satellite-based estimates of solar radiation. The use of a single, research-grade dataset for validation ensures the below statistics are of the highest possible quality and an accurate representation of the Solcast database.

BSRN_stations_used.PNG BSRN sites used in this validation, by climate type. Two mountain-top sites (Izana and Sonnblick) have been excluded from this study, since Solcast data is not designed for steep mountain areas.

Within the total set of BSRN stations, sites outside Solcast data coverage areas were excluded (i.e. removing oceanic and polar sites), and comparison was limited to sites with a sufficient record in time during the Solcast data availability period of approximately 12 years from 2007 (Solcast finds that prior to the mid-2000’s, satellite data was of substantially lower quality in general to warrant inclusion, even with quality control and other steps applied). Two mountain-top sites were excluded, Sonnblick, Austria (SON), and Izaña, Spain (IZA), which have extreme local topography that is not representative of solar development sites. Solcast data is not recommended for use in steep mountain terrain with local relief of more than 1000m.

Indicators

The most common statistical indicators in the solar industry used to evaluate solar radiation models fall into two categories; bias measures and error measures.

Bias measures

Bias measures are important for long term energy yield calculations, providing a base to estimate the accuracy of energy simulations by understanding a possible error in the long-term estimate. Mean bias error (MBE) provides information about the aggregate difference between the Solcast dataset compared against ground observations. It is a measure of the dataset’s systematic tendency to over- or underestimate the solar resource. Deviation of bias error indicates the spread of this error. It indicates the likelihood of the error at a specific site occurring within a confidence interval.

Error measures

Error measures are important for short term variability work. They help to develop an understanding of the accuracy of operational calculations based on energy simulations, covering recent performance and evaluation, equipment calibration, plant monitoring and forecasting. Mean absolute error (MAE) measures the average magnitude of the difference between the ground station and the models. Root mean squared error (RMSE) is another measure of these differences, but uses quadratic weighting. This means larger errors have a disproportionately larger effect on RMSE, and consequently this measure is very sensitive to large errors.

Results Overview

The following tables present the global total summary statistics of the Solcast dataset compared to the surface measurements, at the hourly average time granularity.

Global bias statistics - Hourly GHI  
Mean bias error (MBE) -0.02 W/m2
Relative mean bias error (rMBE) 0.00%
Standard deviation of bias error ±2.02%
80% of sites have bias smaller than ±1.93%
90% of sites have bias smaller than ±2.50%
Global mean error statistics - Hourly GHI  
Mean absolute error (MAE) 47.44 W/m2
Relative mean absolute error (rMAE) 10.89%
Root mean square error (RMSE) 72.38W/m2
Relative root mean square error (rRMSE) 16.5%
Correlation coefficient 0.91

Solcast Validation Tool

The world is a big place, but the BSRN measurement sites are limited. To estimate the likely error characteristics of Solcast data for your particular site, use our validation tool below.

The two inputs you’ll need are:

  • Your climate type. The tool uses your selected climate type (equatorial, arid, temperate, cold) to only include sites with the same climate type.
  • Your continent/region. The tool uses your selected continent/region to only use sites from the same types of satellites that go through the same type of processing.

Select climate type and/or continent:

Validation summary of your selected sites

sites match your selection criteria
Mean Minimum Maximum
Bias
RMSE

North & South America

Station Bias RMSE

Europe, Africa & West Asia

Station Bias RMSE

East Asia & Oceania

Station Bias RMSE

Test now, free

Get free historical data credits when you sign up, so you can try our data for yourself. You can download data automatically within around 1 to 9 minutes of submitting your request.