Evaluating accuracy I: Satellite-derived irradiance data

Step 1: Review validation materials

When selecting a data provider for satellite-derived irradiance information, it's essential to review their validation materials. Any reputable provider should make their validation data and methodology publicly accessible. Look for standard error metrics such as nMAE, MAPE, and nMBE that are easily comparable between vendors. It's also important that the validation materials clearly define any sites that have been excluded from the validation process and provide the exclusion criteria.

Some providers offer the opportunity to review data for specific validation sites upon request. They may also work with customers to carry out independent validations using third-party data sources, like SURFRAD sites, or even against the customer's own measurement data.

Solcast has published our validation statistics here, to allow anyone to understand how we operate our validation process and review and analyse the data themselves. If you would like to review the data for any of these validation sites, get in touch with our team and we’ll share that data for you to assess yourself. Our commercial team will provide you with as much data as you need to complete your assessment. If you would like to discuss an assessment, request to speak with our commercial team here.

Step 2: Read third party peer-reviewed publications or reports

Satellite derived irradiance data is available to academics and researchers, and as a result can be reviewed through the applications used in peer-reviewed publications and third party reports. These resources should be independent, and may either directly or indirectly validate the quality of the data source.

Solcast is proud to support researchers and academics conducting science in the field, and we provide access free of charge and no-strings attached for research purposes. Solcast provides links to some of the many articles and reports where our data has been used. You can read them here.

Step 3: Review existing commercial applications

Satellite-derived irradiance data has a wide range of applications across various use-cases, including some beyond solar energy. To further validate the data, consider how it's being used commercially. Depending on how you intend to use the data, there may be other operators in your industry that have used the data for similar purposes and applications.

Solcast can provide you with a comprehensive list of customer references, including case studies, from organisations that have conducted in-depth validation assessments themselves.

After completing the above reviews, if you want to validate the data yourself, our modelling team have outlined their process for a detailed validation study of satellite derived irradiance data.

Running your own Validation

Timeseries Alignment

The first step in comparing Solcast and measurement irradiance is to align the timeseries. See our example notebook 3.1 Comparing to Measurements: Timezone and Time Period alignment for a demonstration using real-world data.

1. Resample your measurement timeseries.

Solcast timeseries generally comes as period-average labelled with the end of the period (hence why period_end is the timestamp label in our API responses.). You will generally have to resample your measurement timeseries to match the period of your Solcast timeseries. Resampling timeseries will cause the timeseries to appear shifted, and longer period averages will “smooth out” the variability.

2. Relabel period_start/period_end

Once your measurements have been resampled to the correct period average, then you need to ensure your timestamps between your Solcast and Measurement timeseries have the same labelling convention. Here is an example for how to convert to the end of the period:

period_end = period_start + period
period_end = period_mid + period/2

‍‍3. Time-zone shift Solcast timeseries

Solcast timeseries are generally provided with UTC timestamps in ISO8601 format. If you want to work in the local timezone of your measurements (which can make some tasks, such as taking daily totals, more intuitive), then you will need to time shift the Solcast timeseries. This is an example of what the end result looks like, so that the midday peak and nighttime periods alight between the Solcast and measurement data.

Quality Control (QC) Measurement Data

Irradiance measurement data often has corrupt periods that need to be removed, due to for example:

• Dust soiling on sensors.
• “Stuck” measurements due sensor malfunction.
• Sensor drop-outs in multi-sensor setup (especially if there is a bug in the algorithm that combines multiple sensors into one measurements timeseries)
• Shading (either consistent from e.g. nearby buildings, or temporary from site works).
• Spiders, birds, other unintended interference.

See our notebook at 3.2 Comparing to Measurements: Quality Controlling and Gap Filling Measurements with Solcast Actuals for an example on how to quality control measurements.

Evaluating Quality

After aligning and quality-controlling your measurements, there are two main approaches which you should employ when evaluating the quality of Solcast timeseries compared to your measurement irradiance timeseries:

• Qualitative Assessment: Manually inspecting timeseries to observe correlation and peaks of the timeseries
• Quantatitve Assessment: Here you calculate metrics such as normalised bias (nMBE), mean absolute error (nMAE, MAPE), correlation or others. See our notebook at 3.3 Comparing to Measurements: Accuracy Metrics for an example of how to conduct this assessment.

‍