Historical and TMY data in the API Toolkit: the next step in the Solcast mission

27 September 2019

For the past three years, our team at Solcast has been united under a singular, transformative purpose - deploying data and tools for the solar powered future. It’s what gets us up for work each day, motivates our team’s culture & interactions, and most importantly, what drives our creative thinking for ‘what’s next?’.

Back in the early days of our company journey, James (our CEO) and I settled on this mission and it deeply informed the scale of our thinking. As we built up our team of software developers, meteorologists and modellers - it was this mission that set the scope for the deployment of our solar forecasting technology. It was our ambition to deliver a global solar forecasting system, one that would track and predict cloud cover conditions all around the world.

So our team set upon this challenge, and quickly scaled a cloud-based solution that now generates over 600 million forecasts every hour, rapid-updating every 5 to 15 minutes with new imagery from five weather satellites and integrated with output from numerical weather models. Our philosophy of enabling others with this data mean that it also needed a top notch API framework to make that data accessible to (now more than three-thousand!) users all around the globe.

As that global forecasting system’s user base grew quickly, and the system architecture matured, our team once again looked to that mission to answer that inevitable human question - ‘what’s next?’. What could we build that would further our ability to enable the hard-working folks of the solar energy industry? Where would our data and tools for the solar future naturally fit next?

The answer came from our customers and fit perfectly with the mission

‘Building the solar-powered future’ - its quite literally what the solar energy industry does. Solar farm developers scout for new sites, design and engineer them, before securing finance through rigorous due diligence processes. At several of these steps, there is a need for solar data services. Naturally, as our customer base grew, a common theme of question emerged - “Do you have historical data?” or “Can you create TMY datasets?” or “Are monthly averages available?”.

Our mission, half complete?

We quickly realised that we’d only completed roughly half of the ‘data and tools’ our mission required. As these requests came in with consistency, Solcast took a close look at the way solar farm financiers, developers and due diligence firms were accessing the historical and climatological solar data services they needed. Our conclusion? The whole process was overdue for overhaul. So we rolled up those metaphorical sleeves (we don’t really get our hands dirty writing code 😂…) and started working hard to design and deliver another suite of products that were global in scale and meaningful in enabling others to get the job done.

The Solcast API: A natural fit for historical and climatological solar data

Practically, Solcast is building new ‘historical’ data each day. With every satellite scan our database of ‘estimated actuals’ grows as we model solar radiation components with the latest information on aerosols and our unique ability to model clouds in three-dimensions. However, delivering on a meaningful amount of historical solar radiation data meant digging through archived satellite and weather model outputs from up to 15 years in the past. This was no small task, as it involved processing many tens of terabytes of data through our cloud detection and modelling algorithms, while storing the very large volume of outputs into a database that would remain responsive and quick to service customer requests. While it took more than a year of dedicated work - **the outcome was built to match the expectations of our brand: easy, fast and of the highest quality we could manage. **

The Solcast API Toolkit: A new way to transact on historical and TMY data services

historical and tmy solar data.png

In the end, we built an entirely new way to test, validate, purchase and download the TMY (typical meteorological year) and historical Time Series data the solar farm developer, investor and due diligence community needs to get the job done. This new experience for transacting on historical and climatological data is now available as a feature of the Solcast API Toolkit, with data formats that are ready to integrate directly with PVsyst, SAM and take only minutes to download.

How to learn more about the historical and TMY data offering

But as with any new product launch, our team at Solcast still has plenty of work to do on informing our audience of these changes, and instilling the confidence those customers will need to incorporate these data into their investment and modelling decisions. So, as I draw this latest chapter in the Solcast story to a close, let us assure you - it is our intention to provide a steady stream of supporting information to you via multiple channels over the coming months.

Over the next few months, Solcast will be regularly sharing information on our new offering by:

Please ‘like’, ‘subscribe’ or ‘follow’ Solcast on the above links to keep informed on the latest!

In closing, I will leave you with a list of helpful content we’ve published to our website. We look forward to seeing you test out the data in the API Toolkit.

Dr. Nick Engerer

Dr. Nick Engerer

Solcast Co-founder • Author

Nick is an expert in the field of solar radiation and distributed solar PV modelling, and has co-founded Solcast out of a sincere desire to enable others to build the solar-powered future.

Read more about Dr. Engerer's scientific papers

Solar Power Forecasting