Three Time-saving Tips for Solar Forecast Validation

Solar forecasting validation doesn't have to be hard

In case you don't already know me - I'm Nick, the Chief Technology Officer at Solcast. I've been working on solar radiation modelling and forecasting for the past 10 years.

Today, I want to leverage that experience to share three tips that will save you time, energy and sanity when completing your next forecasting validation exercise.

Tip #1: Don't (mis)place all your effort into finding the absolute best forecast

Why on earth would I say that? My reasoning is based on experience, and watching one too many energy analysts spend months of their time comparing several forecasting options.

The point here is to focus on the outcome you want. You need to answer the question - is the forecast from the provider fit for purpose?

Start with the accuracy requirements and forecast horizon you need to achieve your goal (the purpose). If your grid operator requires <15% error at 1-hour ahead (our nRMSE at 1-hour ahead is usually <5%), then that's the outcome you want. Then ask: can the forecast provider I am evaluating meet this requirement? ​ ​If the answer is yes, great then sign-up for a trial and give them a try. Job done. You just saved yourself months of headache writing scripts, managing time-stamps, wrangling a database, dealing with several varying FTP/API sources and then writing all sorts of equations to assess them.

Bottom line - don't be a hero. Just get the job done.

Tip #2: If getting access to a vendor's forecasting data is difficult, choose a different vendor

You've done your research, made a list of all of the forecasting tools available in your region, and you're ready to get to testing (maybe you already forgot tip #1 and are getting ready to conduct an expensive validation...). You fill out some forms, send some emails and... the waiting begins. A few days later (or a week or more), you're exchanging emails with someone in a different timezone, being asked to complete some paperwork (possibly even pay a fee or make a commitment), and perhaps be (somewhere) on your roundabout way to accessing their SFTP server.

If your early engagements with a forecasting vendor don't match your experience signing up for an online software service (free trial, free data to try without asking a human), then what should you expect moving forward? Old brittle systems, slow response times, difficult engagements with far too many hands-on interventions.

Bottom line - It's 2020 people! Make your forecasting vendors act like it!

Tip #3: You don't need to collect every forecast to complete a validation - just the right ones

​ Forecasting data should change often so that it captures the latest weather conditions (Solcast regenerates our forecasts every 5-minutes). But that doesn't mean that your validation process has to grab new forecasts every five minutes.

Start-off with tip #1, and know the outcome you need to achieve. If you have a day-ahead unit commitment that needs to be submitted by 11AM the day-prior, set yourself a calendar reminder and download the forecasting data at that time each day for a period of 1-2 weeks. Need to make a precision 5-minute ahead forecast for your energy market dispatch target? Pick a day where there is some cloud activity over your site and grab new forecasts every 5 minutes over the course of 30-minutes for a few different days.

The bottom line - 1) don't put off completing a forecast validation by making it too hard AND 2) don't drown yourself in data analysis; forecast methodologies are either fundamentally solid or they aren't (i.e. if you can observe them working well during your "snapshot", that is likely to continue!).

That's it from me, I hope these 3x tips were useful!

PS - a bonus tip! If your forecast validation outcomes look really poor - share the analysis with your vendor. You're likely to find a mistake somewhere, and both parties will benefit.

Go build that solar-powered future!