The limitations of numerical weather models for solar irradiance forecasting

The next time you're speaking with your forecasting vendor, ask them this little question: "to what extent (and how) do you rely on Numerical Weather Prediction models for solar forecasting"?

Why is this an important question? Well that gets directly to the point of this article, which is an outline of the Solcast answer to this question, which I hope you will find useful for screening forecast vendors or building your own models.

The standard answer from most forecasting vendors will usually be something like "We use a range of models and work out the best forecast", or "We use XYZ model, it's the best"; possibly followed by "we put the models into a neural network, and [more unhelpful jargon]". This may be true, and could be interesting, but is rarely helpful for producing a high quality solar irradiance or power forecast.

What is a weather model?

Firstly, what is a weather model? A Numerical Weather Prediction (NWP) model is a computer simulation of the weather. You can think of NWP as being like a video game, a simplified but impressive computer representation of reality. NWP models consume measurements of weather that are around 1 to 12 hours old, blending these with a previous forecast to estimate what was happening about 1 to 3 hours ago (called an analysis). Then, the future is forecasted by starting at the analysis and running the simulation. These simulations then use simplified physics and race against time to generate the forecast before the actual weather happens! NWP is amazing technology, and can be very useful when used well. Overall forecast accuracy of general weather patterns from NWP models has been improving steadily since the 1970s.

Solar irradiance forecasting - The limitations of weather models

These NWP models are fairly good (and improving), so let's just take the forecast made by the model and use it - right?

Wrong, especially for solar irradiance forecasting - a mistake made by many. There's a fundamental property of weather models, one which is often forgotten or glossed over. The map is not the territory.

What does this mean? Allow me to explain further. First, all NWP models have to project reality into their own state space. This means that a group of hills near your site may not be in the weather model at all, or a mountain range nearby may be differently shaped to simplify the forecasting process.

A great example is morning fog, which often impacts solar farm site in low lying or coastal areas. Even if the fog is observed by a weather station nearby, there may be a totally different representation of the local conditions inside the model. What this means is that even if the model produces its own version of this fog layer tomorrow morning, without substantial interpretation (human or machine learned) it won't be correctly forecast. Weather model improvement efforts track general weather pattern accuracy, but these projection issues can be very localised (such as for our fog illustration). This is just one example of how a model may contain information, but the signal needs to be projected back from the model space into the real world. This is not an easy problem to deal with when using NWP to make solar forecasts, especially for clouds. The traditional weather parameters like temperature and wind exist inside the NWP models and are relatively easy to "correct" from the models to reality in a linear fashion, but clouds are highly localised and many clouds aren't able to be properly represented by the models.

Below: An example of low lying cloud and fog over Northern California

Solar irradiance forecasting is cloud cover driven

When it comes to solar irradiance forecasting, clouds are usually the most important driver, but clouds don't really exist in weather models, at least not fully. Cloud processes operate on tiny scales, because they consist of tiny droplets and ice crystals. Weather models generally run at between 1km and 20km resolution, which is millions of times bigger than many cloud processes. These processes are "parameterised" in the model, meaning they are simplified in order to represent the average behaviour of the clouds at the model's grid scale. Given this simplification, it is crucial to not take the model forecasting data too literally. Even as model resolution improves, cloud forecasts can actually get worse for a given point, since the model starts to resolve individual small clouds - making overly confident forecasts about the exact timing and location of future cloud events.

Can numerical weather model forecast data be interpreted by machine learning?

Many folks will work to try to find a way around these problems, by using a machine learning or regression method that takes uses NWP forecast data along with solar plant measurements. This can be useful, and may improve results overall, but still has at least one major pitfall.

Almost invariably, and regardless of method, the result will be a forecast that is improved over raw NWP outputs for the first few hours, but often remains a worse prediction than simply looking out the window! This is because it these methods are not at all based on the cloud cover conditions that actually exist at a given time, or those that are about to form over next few tens to hundreds of minutes. Such a (fine tuned) NWP-reliant forecast may also contain serious blind spots for the rest of today and tomorrow. Simply put, if you want to generate a good solar irradiance forecast for the next few hours: It's all about tracking the actual clouds!

Rapid update satellite forecasts: critical for good solar irradiance forecasts

This need to track what the cloud cover is actually doing at any given time, is why Solcast runs its own Rapid-Update forecasting service, re-computing our forecast models every 10 or 15 minutes based on the real clouds. We detect these from raw satellite data using our own algorithms, and then focus on the details of the cloud situation. By using the latest imagery, we avoid as many big assumptions as we can, learning from the wealth of high resolution satellite data and solar irradiance ground measurements.

Below: Rapid-update cloud forecasting over the Southeast Asia region

Day-ahead solar irradiance forecast improvements from satellite data

We also use our detected cloud data to avoid problems and assumptions in the day-ahead forecasting for your solar facility. By tracking cloud cover conditions, we can more easily deal with bad or polluted plant measurement data. This allow Solcast to be more explicit about what is due to the model and what is due to the real characteristics of your plant, and we can make more accurate forecasts for the many PV plants where measurements cannot be readily obtained or shared.

Enough from me. Back to making a better forecast, and making it easier for you to test and validate for your site.

Access cloud cover driven solar irradiance forecasts in just minutes!

If you're after a solar irradiance forecasting service that combines the best of what satellite imagery and NWP models can offer, the Solcast API service has you covered! You can register for our API service for free, and be testing our Rapid Update solar irradiance forecasting data within minutes. We even offer FREE data access for researchers, and FREE solar forecasting for your home Rooftop Solar PV system.