Inputs and algorithms

How we estimate and use satellite, aerosol and weather sources to make our data

Solcast has spent four years developing a system to produce data with very low bias and error, providing you with the lowest uncertainty, and the most “bankable” data possible. Our database lowers risk for developers, operators, and investors, ensuring decisions on the development or operation of high-value solar assets are made confidently.

Solcast’s global solar database is produced using high-resolution (1-2km) imagery from a range of geostationary meteorological satellites, from which we diagnose our own clouds using an innovative 3D approach. The data is created using a combination of peer-reviewed, industry-standard models, and algorithms developed in-house. Read more about how we make our data here, and test for yourself against your own ground measurements using the free data credits you receive when you register.

Solar Irradiance Modeling

The Solcast method for estimating solar irradiance from geostationary weather satellites consists of three major steps. First is the detection of cloud cover and the characterisation of that cloud cover in terms of its impact on solar radiance. Second is modeling the available solar radiance under clear skies, including treatment of aerosols (dust, salt, etc.) and water content. And third is the final, combined estimate of the amount of solar irradiance reaching the Earth’s surface after it passes through the clouds (if any are present).

Methodology.PNG

DETECTING CLOUD COVER AND THICKNESS

Solcast runs its own proprietary and globally consistent cloud detection system, using the highest spatial resolution satellite data available. The core input (see tables below) we use for detecting clouds from space is the imagery produced by geostationary meteorological satellites. The latest generation of satellites from NOAA, EUMETSAT and JMA (satellites in 5 different primary positions) produce scans of the Earth every 5 to 15 minutes at resolutions as fine as 500m. Theses satellites provide coverage anywhere on the globe, except Antarctica.

coverage.PNG

The raw imagery is first standardised on a satellite-by satellite basis so that the rest of the processing can be agnostic to the quirks of individual satellites. The imagery is georeferenced and projected onto a regular grid. The channels are converted to regular units. Visible channel data is converted from observed radiance to (bi-directional) reflectance, the fraction of reflected incoming radiation, which is independent of solar zenith during daytime periods. Infrared channel data is converted to brightness temperature, which is closely correlated to cloud-top temperature where there are clouds, and surface skin temperature elsewhere. Automated quality control is applied to catch imagery artefacts, such as swathes of empty data and striping, before they can corrupt downstream processing.

The standardised imagery is then converted into an uncalibrated initial estimate of cloud opacity. The visible channel reflectance is differenced with a pre-computed estimate of the clear-sky surface reflectance (based on a synthesis of recent past satellite imagery and other surface analyses including for snow presence and depth). This estimate is then further processed using information from a combination of IR imagery and NWP forecast data to account for confounding phenomena such as sun glint over tropical oceans, snow, salt pans, and other artefacts. Data are gap-filled across periods of satellite outages or bad satellite scans, using a combination of surrounding good satellite data and also weather reanalyses, in a combination that depends on the duration of the gap.

The cloud opacity estimate is then calibrated according to long-term historical comparisons to surface pyranometer data. As well as the initial uncalibrated estimate, the calibration is a function of solar position and satellite to account for small residual differences when standardizing the raw satellite imagery. This serves to minimize bias to the already low variance of the analysis cloud opacity.

Dataset Provider Satellite Fleet Period Coverage Resolution of data used
NOAA GOES Entire Solcast Record, 2007 to date North and South America, Hawaii 1-2 km
EUMETSAT Meteosat Entire Solcast Record, 2007 to date Europe, Africa, Middle East and South Asia 1-2 km
JMA MTSAT, Himawari Entire Solcast Record, 2007 to date South East Asia, China, Japan, Oceana 1-2 km

AVOIDING OLDER, LESS RELIABLE SATELLITES

Because the quality of satellites and sensors was significantly lower prior to the mid 2000’s, we avoid using or providing data for dates prior to 2007. This provides a 12 year (and growing) record, which provides a long period at a higher level of certainty.

Prior to the mid 2000’s, even with the best quality control and algorithms, data is generally not sufficient for what we judge to be an acceptable level of quality in satellite derived irradiance estimates. The issues include: long data gaps (which some data suppliers fill using weather model data that is much lower precision and can cause bias), less frequent imagery resulting in undersampling, geolocation issues resulting in spatial errors, and a myriad of bad data artefacts that can silently affect accuracy of the resulting irradiance estimates.

CLEAR-SKY MODEL

Solcast’s clear sky module utilises the REST2v5 model (Gueymard 2008) with proprietary improvements. This model relies on the input of the following variables to be computed;

Data Type Dataset Provider Period Coverage Resolution
Aerosols MERRA-2 Reanalysis NASA Entire Solcast Record Global 50km
Water Vapour ERA-Interim Reanalysis ECMWF Entire Solcast Record Global 30km

SATELLITE DERIVED SOLAR IRRADIANCE

Solar radiation components are then computed, utilising the cloud and clear sky inputs. The aerosol and moisture varying clear sky radiation from the modified REST2 clear sky model is first combined with the computed and calibrated cloud opacity to produce global horizontal irradiance. The Engerer2 (Engerer 2015, Bright & Engerer 2019) model is then used to separate this into direct and diffuse components.

Model Type Solcast Algorithm Description Citation
Clear sky irradiance REST2v5 (with proprietary modifications) Models the irradiance with no clouds, as a function of aerosols, time and location Gueymard 2008
Cloud detection Solcast proprietary method Estimates the presence and opacity of clouds, using multispectral satellite data Not Published
Radiation separation model Engerer2 (with proprietary modifications) Separates the total irradiance into its direct and diffuse components Engerer 2015, Bright & Engerer 2019

Other Weather Parameters

Data Type Dataset Provider Period Coverage Resolution
Temperature (TEMP) ERA-Interim Reanalysis ECMWF Entire Solcast Record Global 30km
Wind Speed (WS) ERA-Interim Reanalysis ECMWF Entire Solcast Record Global 30km
Wind Direction (WD) ERA-Interim Reanalysis ECMWF Entire Solcast Record Global 30km
Atmospheric Pressure (AP) ERA-Interim Reanalysis ECMWF Entire Solcast Record Global 30km
Precipitable Water (PWAT) ERA-Interim Reanalysis ECMWF Entire Solcast Record Global 30km
Relative Humidity (RH) ERA-Interim Reanalysis ECMWF Entire Solcast Record Global 30km
Dew Point (DWPT) ERA-Interim Reanalysis ECMWF Entire Solcast Record Global 30km
Snow Depth (SWDE) ERA-Interim Reanalysis ECMWF Entire Solcast Record Global 30km

ELEVATION CORRECTIONS

Because Solcast is designed for the Solar Energy community, and solar development sites are generally in relatively homogeneous and non-mountainous terrain, we do not spend effort applying elevation corrections to our irradiance data. The elevation data we use is an average at a 5km resolution, designed to be representative of the surrounds of the site. If your site is not on a mountain-top, and not in very rugged terrain with local relief of more than 1,000 metres - then our data will work very well for you. For example, if you’re in a large high-altitude valley, our data will capture the elevation effect whereby your site will receive higher irradiance due to the lower amount of atmosphere above the site.

We also do not apply any terrain shading corrections to our data, because the true horizons at (and within/across) your particular site can vary greatly based on local factors that are not captured even in 10 metre topographic data sets like trees, small mounds, buildings and other structures. Good PV modelling software like PV Syst allows you to edit your horizons precisely, and take control of shading for yourself.

TMY Generation

The Typical Meteorological Year (TMY) is a collation of multiparameter weather data that provides a representation of the typical conditions at a specific location. This one-year period of data is selected from a historical dataset so that it presents the range of weather phenomena, while still giving annual averages that are consistent with the long-term averages. The use of long term 10+ year data is necessary to reduce uncertainty from interannual weather variability.

TMY data is generated from Solcast’s global solar database of satellite-derived and modeled meteorological data. For each of the twelve months of the year, data is selected from the Solcast historical record that is closest to that month’s long term average. These representative months correspond to the P50, or the average year. Solcast’s standard choice of representative months is based on GHI averages, reflecting the focus on modeling and forecasting of PV systems. Other selection and weighting criteria (e.g. DNI preferred for CSP modeling) can also be customized.

References

Bright, J. & Engerer, N. 2019, ‘Engerer2: Global re-parameterisation, update, and validation of an irradiance separation model at different temporal resolutions’, Journal of Renewable and Sustainable Energy, vol. 11, no. 3, 10.1063/1.5097014.

Engerer, N. 2015, ‘Minute resolution estimates of the diffuse fraction of global irradiance for southeastern Australia’, Solar Energy, vol. 116, pp. 215-237.

Gueymard, C. 2008, ‘REST2: High-performance solar radiation model for cloudless-sky irradiance, illuminance, and photosynthetically active radiation – Validation with a benchmark dataset’, Solar Energy, vol. 82, no. 3, pp. 272-285.

Test now, free

Get free historical data credits when you sign up, so you can try our data for yourself. You can download data automatically within around 1 to 9 minutes of submitting your request.