Big accuracy gains for Europe

Solcast forecasts are now more accurate over Europe! We have achieved a relative ~10% error improvement over our already market-leading forecasts thanks to the addition of a high-resolution, rapid-update weather model.

The challenge: Searching for the next big gain

At Solcast, accuracy never sleeps. Sometimes we call our Data Science (or Modelling) team our “Accuracy team”. We constantly search for more high-quality surface measurements, more competitive trials, anything that enables us to measure-up and learn. We love product features, we love API capabilities, but we love nothing more than accuracy.

From winning many competitive forecast trials in Europe, we knew that we were already market-leading on accuracy in general, however we could see that occasionally we were not always the best, especially in winter-time.

We turned to our strengths. Firstly, we knew that our solar-specific capabilities like real aerosol inputs and state-of-the-art irradiance models were very effective in minimising errors in irradiance and PV power forecasting. And secondly, we knew that our bankable cloud-tracking “actuals” were a proprietary advantage for our machine learning, teaching our ML to reduce the errors of the four top global weather models we use as inputs. During 2023, some projects aimed at improving these strengths only showed small improvements, suggesting to us that the bigger improvements lay elsewhere. But where?

Remembering the lake

I remembered the many hours last June sitting by the Steinhuder Meer, a lake in Saxony, waiting for the wind to come. It was a windsurfing event, and without enough wind the racing would be postponed, leading to a lot of sitting around waiting. As a former meteorologist, others were asking me - what will the wind do next, James?

Proving it

We knew there were cases where our forecasts could be beaten, we knew this model was used by some players already, and I had my lived experience by the lake. Our head of modeling Dr Hugh Catcher agreed - it was enough evidence to warrant turning our team to this task.

We looked at the ICON-EU and the other available models. Clearly this model was faster to update than the top global models we use, with 3-hourly updates compared to 6- or 12-hourly. It also had higher resolution, at 6.5km horizontal resolution compared to 9-20 km. There are higher resolution regional weather models out there with finer resolution, but they tend to suffer from a conundrum of resolving only half the convective scales, leading to noise and error, plus they cover only smaller areas.

ICON-EU model coverage. All Solcast forecasts over this wide region are benefiting from this improvement.

Upon inspection, we could see that ICON-EU was also doing a better job of using the available measurements to better reflect reality. We saw a number of cloud events it captured, events that the other models missed.

So our team trained our algorithms to use this model. They developed algorithms against our own satellite-derived cloud actuals to find what seemed the best implementation. Then they looked at how best to blend this new model with our existing satellite cloud-tracking “nowcasts” (which are used in the first four hours of our forecasts), and with our existing longer-horizon forecasts driven by the slower-updating top global weather models.

With a good approach tested, it was time to verify. We used data from some surface irradiance measurement networks across Europe, data that we hadn’t trained our model on.

The results were good! We could see a large GHI forecast error reduction, about 10% relative and 3% absolute. At the top end of solar forecasting, that’s a big improvement!

Improvements in +24 forecasts across the Netherlands and Germany

Live in operations!

This forecast improvement went live in operations on 7th May 2024. All Solcast forecast data over this wide region of Europe are benefiting from this improvement, and also some surrounding parts of Africa and Asia. As with all our accuracy improvements, users don’t need to upgrade, and don’t need to change their API requests.

Solcast’s multi-year historical forecast archive, which is commonly used to evaluate and to train models, is now being updated using past ICON-EU forecasts. This will allow our users to evaluate and train on the same forecast formulation that we are using in operations. The ETA for this historic forecast upgrade is early June 2024 - contact us to register for this if you want to try it out!

What’s next?

Similar rapid-update weather models are run over many countries, allowing us the opportunity to scale this improvement globally. We are already working on a similar forecast improvement project for North America. Stay tuned for updates!

Want to see how the updated forecast models perform for your assets? Start your evaluation today, using the Solcast toolkit or speak to our team to get some more comprehensive data.