What Sandia’s 2025 validation study demonstrates about terrain inputs and modelling discipline

In late 2025, Sandia National Laboratories published the results of a blind performance assessment comparing seven commercially used PV modelling tools - including SolarFarmer. The study evaluated POA irradiance accuracy and full-year energy yield predictions across a lab-scale system in Albuquerque and a utility-scale system in Germany, using sub-hourly measured data.

It is a well-run first study: onymous, blind, and featuring modelled power submitted directly by PV modelling software vendors. Independent validation like this is exactly what the industry needs. It holds all of us to account - and when we examined SolarFarmer's results, we were surprised by the results, but have found something worth discussing. 

What we identified was a modelling error - a mismatch between the terrain input and the physical site, not a model error. Our analysis shows that SolarFarmer's transposition and shading algorithms performed correctly for the surface they were given. The distinction matters, and it carries a broader lesson for anyone running yield simulations. 

The observation 

Sandia's POA comparison revealed that SolarFarmer's predictions showed a slight temporal offset relative to other tools. On a clear-sky day in the available data (25 October, image below), the SolarFarmer trace appeared to lag by approximately six minutes, most visibly around peak irradiance. SolarFarmer also recorded the highest POA NRMSE among participants for the Albuquerque site. 

 

 

For a reader, and for our team reading the paper, this raises the question, reasonably, about whether SolarFarmer's transposition calculations contained a fault. 

The root cause: a modelling error, not a model error 

The answer was straightforward, once we looked in the right place. 

The Albuquerque validation site is a small research installation on essentially flat ground. No topography information was provided to participants - the paper notes that this led each provider to make their own assumptions about the terrain. Six vendors assumed a flat, horizontal surface. SolarFarmer requires the user to define a terrain surface, either through a simple plane assumption, imported survey or LiDAR data, or an SRTM download. In this case, the team used SRTM, a globally available dataset at approximately 30-metre resolution, which introduced a gentle northwest-to-southeast slope across the site. 

 

 

By design, SolarFarmer places racks on the terrain surface as defined by the elevation data available in the simulation. When that surface has even a slight slope, the rack normal vectors shift - in this case, picking up a small westerly component instead of pointing due south. That changes the solar incidence angle through the day, delaying the time of minimum incidence (and therefore maximum POA) by roughly six minutes under clear-sky conditions. 

This was a modelling error: in the absence of site-specific terrain data, the team used SRTM, and that data didn’t represent the physical site. It was not a model error: SolarFarmer calculated POA irradiance correctly for the surface geometry it was given. The tool did exactly what it was designed to do, it simply wasn’t given inputs that matched the reality. 

A note on what this means for the expected results 

Whilst our teams are confident that this has materially impacted the performance, without access to all the data used in the paper, we can’t reproduce comparison results. The value of a blind validation study lies in the fact that it tests real workflows - including the setup decisions, assumptions, and input choices that practitioners make before they hit "run." Retrofitting inputs after seeing the published results would undermine that. 

We did re-run the Albuquerque case on flat terrain for our own diagnostic purposes, and the POA offset disappeared - confirming that terrain representation was the sole cause. However we don't think it would be appropriate to present that as a replacement result. The study captured a genuine modelling error, and there is more to learn from being transparent about that than from quietly correcting it.

What to check when a model produces an unanticipated result

When a model produces an unexpected result, the instinct is to ask "what's wrong with the tool?" Experienced practitioners know that's often the wrong starting point. 

The more productive question - and the one this episode reinforces is: do my inputs accurately represent what exists in the real world? 

Outputs are only as reliable as the inputs that drive them. SolarFarmer's terrain-aware design means it faithfully represents whatever elevation data you provide. That's a strength when the data is accurate. It becomes a source of discrepancy when it isn't —  as in this case.  

This principle doesn't just apply to terrain. It extends to meteorological datasets, module specifications, soiling assumptions, and every other parameter that shapes a yield estimate. When results look unexpected, validate your inputs and their relationship to reality first.  As model physics improve, inputs become the primary source of remaining uncertainty - which means the modeller's responsibility for input quality only increases.  

Why terrain-aware modelling matters - and what we recommend 

The fact that this discrepancy was caused by terrain modelling doesn't diminish the importance of explicitly modelling terrain — it reinforces why it exists and why it needs to be used deliberately. 

Real PV sites are rarely flat. For every site developers face decisions about whether to grade a site or build on existing topography, and that decision directly affects energy yield, shading interactions, and LCOE. A tool that assumes flat terrain by default removes that analysis from the design process entirely. 

SolarFarmer's approach - importing terrain from sources such as SRTM, LiDAR, or ground surveys and automatically incorporating it into rack placement, shading, and incidence angle calculations, gives developers the ability to quantify those trade-offs. This is particularly important for SolarFarmer's explicit row-to-row shading calculations, where terrain slope directly influences when and how shadows fall across adjacent rows. Being able to model these scenarios inside SolarFarmer allows pre-construction design decisions to be based on fully physical yield models of the scenarios being considered.  

The Sandia study inadvertently demonstrated just how sensitive those calculations are: a barely perceptible slope on a small research site produced a measurable shift in POA timing. On a site with meaningful topographic variation, the effect on yield and design decisions would be significantly larger.

Our recommendation is clear: before modelling with SolarFarmer, confirm the local terrain conditions and whether the site is to be graded. If earthworks are planned, model the post-grading surface. If the site will be built on existing terrain, use the highest-resolution elevation data available for your project stage - SRTM for early screening, survey-grade data for pre-construction estimates. SolarFarmer does not generate terrain data; it uses what you supply. The quality of the output is directly linked to the quality of that input. 

Energy yield: what the broader results show 

Sandia's analysis extended beyond POA to compare full-year energy predictions across both sites. 

At the lab-scale site in Albuquerque, SolarFarmer aligned closely with the industry consensus, with both annual energy error and NRMSE centred near zero. Under controlled conditions and with well-matched inputs, the core modelling approach performs in line with established expectations. 

At the utility-scale site in Germany, SolarFarmer appeared as a conservative outlier, predicting approximately 3.3% less energy than the mean of all tools. SolarFarmer was the only tool that used uneven terrain for this site, and it explicitly models loss mechanisms - including complex shading interactions and sub-module mismatch — that not all tools represent with the same granularity. More conservative results from a tool with broader loss coverage is a predictable outcome, not necessarily an anomaly. 

It's also important to note that the study measures deviation from the mean of all participating tools, not from measured production data. Being an outlier relative to a peer average is not, by itself, evidence of a quality problem — particularly when the inputs, derate decisions and modelling scope differ between participants.

Closing 

Sandia's study is a valuable contribution to the industry, and its central finding reinforces what experienced modellers already know: as system size and complexity increase, modelling assumptions and user-defined inputs become the dominant source of divergence between tools. The differences aren't primarily about which engine is better, they're about what each practitioner chose to feed it. 

For our part, the study taught us something concrete about our own process: when terrain data is available as a default option, it's easy to apply it without verifying whether it matches the site. That's a workflow discipline issue and we've taken it on board. 

SolarFarmer's terrain-aware modelling remains a capability we stand behind. But capability without input discipline is just complexity. The tools give you the fidelity. Your job, and ours, is to make sure the inputs deserve it.