Winds of change in weather forecasting

In a world beset by ‘extreme’ weather, meteorological foresight has gained greater value: globalised, digitally interdependent economies constantly seek new ways to manage risk and avoid adverse conditions.

While the smartphone turns every handset owner into a citizen meteorologist who both consumes and supplies weather updates, the imperatives driving technological innovation in weather data gathering come from business.

All kinds of organisations buy tailored weather forecasts that inform their strategic plans and day-to-day decision-making. Their information requirement is not just to know about the weather, but about the impact of it on specific aspects of their business. They want to find out how their operations will be affected by meteorological outcomes and how they can counteract likely damages. This is called ‘weather intelligence’.

This booming market for weather intelligence is reflected in the growing number of independent weather information bureaux: AccuWeather, Katestone, MetraWeather, Planalytics, Panasonic Weather Solutions and The Weather Company, to name a few. Another, MeteoGroup, gained media headlines when it ousted the UK’s Met Office as supplier of weather services for the BBC.

Such challengers have entered a space previously dominated by governmental agencies, helped by  technologies previously affordable only for state-sponsored meteorological bureaus.

Cloud computing, for instance, provides the high-performance computing power required to reduce processing times for weather data to be used in meaningful predictive reports. Developments in satellite technology still provide the orbital views of transcontinental meteorological patterns as they unfold and, with new low-Earth orbit (LEO) satellite constellations, sensors will be close enough to the Kármán line (the altitude of about 100m above sea level at which outer space ‘officially’ begins) to measure meteorological elements such as pressure, humidity, mass, radiation, temperature and airflows.

These will be positioned to network the volumes of data being sent from meteorological and other sensors (that form part of the Internet of Things) attached to ocean-going vessels, in-flight aircraft, smart buildings, etc, in ways that have been neither practicable nor affordable before.

“There’s been a tremendous shift in weather information,” says Christine Killip, MD at meteorology services company Katestone Environmental Consultants. “Customers demand more meaningful weather information that is relevant for their business or industry. And as climates become more unsettled and ‘extreme’, the reliance on accurate and relevant climate data to manage business risks becomes an important tool. It’s recognised that adverse climatic conditions and the associated risks such as heat stress, wind, dust, odour and lightning have massive impacts on businesses and economies.”

This heightened attention to weather intelligence’s capacity to help manage financial impacts places extra pressure on providers. “There are always challenges of improving the underlying physics, ingesting a diversity of data and being able to produce the information faster and more reliably,” says Lloyd Treinish, chief scientist of environmental modelling, climate & weather at IBM Research. “A new extension to this is the ability to predict the impacts of weather. For example, predicting when and where the power network will go down due to storms, and what resources are required to restore power.”

‘There’s been a tremendous shift in weather information. Customers demand more meaningful weather information that is relevant for their specific business or industry’

For decades, high-end computing has driven weather forecasting. Meteorological agencies have been some of supercomputer vendors’ best customers. The UK Met Office’s XC40 Cray Supercomputers, for example, can perform in excess of 23,000 trillion calculations-per-second, and take in hundreds of thousands of weather observations from which it runs atmospheric models that contain over a million lines of code. Yet it’s the kind of power that will become available to anyone who can afford high-performance computing services delivered via cloud models.

“Cloud computing is the driving factor in changing the way we do things in meteorology, both in terms of throughput and data ingestion,” explains Dennis Schulze, chief meteorology officer at MeteoGroup. “National meteorological agencies are still able to make more use of innovative observation techniques. This data is then ingested into the models which form the basis of our own proprietary forecasting techniques.”

The sources of data that Schulze alludes to seem destined to expand almost infinitely, as sensors are attached to both purpose-designed data gatherers and other static and moving objects well-positioned to capture and transmit climate information.

Panasonic Weather Solutions’ (PWS’s) weather-forecasting capabilities have been based on a combination of proprietary atmospheric data collection systems and a weather-predictive modelling platform that’s run on a supercomputer called Sora. The company has worked with software firm Safety Line to develop efficient airline and airport operations, focused on fuel and CO2 emissions reduction.

This solution is based on weather forecasts from real-time weather data collected by PWS’s TAMDAR (tropospheric airborne meteorological data reporting) sensors fitted to the fuselages of scheduled aircraft and linked to satellite communications company Iridium’s FlightLink solution.

TAMDAR data provides observations of wind, temperature, pressure, icing and moisture every five seconds as 250 TAMDAR-equipped aircraft descend and ascend at 300 airports across North America every day. It also collects 3,500 profiles from several hundred additional airports globally.

PWS meteorologists use these atmospheric datasets, allied to its proprietary forecasting models, to provide Safety Line with forecasts. A system called OptiClimb from Safety Line then uses the real-time-refreshed PWS weather forecasts to offer optimised flight climb profiles for each aircraft’s ascent.

Airlines can cut fuel consumption by up to 10 per cent during ascent with OptiClimb’s guidance, it’s claimed. The system uses a combination of machine learning performance models for each flightpath and the computing of optimised climb profiles issued ahead of each journey.

Getting airborne sensors as close as possible to atmospheric conditions and the weather that develops from them has long proved a challenge. Balloons remain a staple transport, but are largely uncontrollable and prone to bursting at high altitudes.

Crewed aircraft, such as Nasa’s ‘Flying Laboratories’ – specially kitted-out DC-8s airliners used in the agency’s studies of emissions and atmospheric composition programme, which aims to determine how pollution, storms and climate interact – mustn’t get too close to the eyes of any storms they follow, for obvious reasons.

Unmanned aerial vehicles (UAVs) are an alternative that meteorologists expect to improve close-up weather reporting. In the US, Nasa launches UAVs that monitor weather events such as hurricanes. In October 2016, for instance, Nasa’s Global Hawk UAV tracked Hurricane Matthew off the coast of Florida, dropping sensors straight into the category 5 storm.

Much of a tornado’s deadly force comes from the unpredictability of its cause, the speed at which it forms, and its direction. Some destruction could be reduced if people located on the periphery of a twister’s path have sufficient warning to move cars, caravans and boats before it hits.

Typically, outlying weather monitors and observers give around 15-20 minutes warning. CLOUD-MAP is a project that unites four US universities to develop applications in meteorology and atmospheric physics that might expand that margin.

Oklahoma State University (OSU) is researching ways in which remotely piloted UAVs could analyse tornadoes by in-flight data-gathering of their characteristics. The research team is also developing UAVs that could fly into the core of a tornado’s vortex to capture data that could be used to work out how future tornadoes will likely manifest, and derive ways to provide extended warning periods. Other OSU UAVs fly in formation around a tornado to collect multiple data points simultaneously.

At present it’s unfeasible for a UAV flown into a tornado funnel to send out data in real-time: captured data must be retrieved afterwards. This means the aircraft must be tough enough to withstand the pummelling of wind, rain, hail and debris, and – as it’s unlikely to manoeuvre out – the impact of being flung to the ground.

“UAVs need [modification to make them] more robust and [able] to fly at higher altitudes,” says Dr Jamey Jacob, director of Unmanned Systems Research Institute at the OSU’s School of Mechanical and Aerospace Engineering. “However, we have already shown that these systems can make a tremendous impact on observations and resulting forecasting.”

Tornado-proof UAVs cannot scramble at short notice. They must be prepared more than an hour before take-off. Other work at OSU is developing a tornado early-warning system based on low-frequency sound detection. Before a tornado fully forms, it emits a low-frequency sound. Directional microphones on OSU’s campus pick-up that vibration and forward the signal for analysis to tell the UAV ground crew where to pilot their craft to gain best measurements.

Vaporous clouds are both a symptom and a cause of weather patterns, and better dimensional data showing what causes them to form and transform would result in more insightful understanding of their role in shaping cloudy weather patterns.

Key to this is a better understanding of cloud structure. Meteorologists have long been left high and dry when it comes to the processes that occur as clouds move and change shape, what causes their fluctuations in moisture density and temperature.

Researchers at Stony Brook University School of Marine and Atmospheric Sciences (SoMAS) and Brookhaven National Laboratory have made use of radar and meteorological tech to take an ‘MRI’ scan of clouds. In the same way magnetic resonance imaging (MRI) uses magnetic fields and radio waves to produce detailed images of inside the human body, the project aims to helps scientists gain visualisation of what happens within clouds.

The goal is to use data from scans of clouds to improve the ability of weather modelling to forecast the type of precipitation and amount on the ground with levels of accuracy and precision not achievable by previous meteorological methods.

“Using these technologies, we can pinpoint and highlight different components of the interior of clouds,” says Professor Pavlos Kollias of Applied Radar Science Group at SoMAS. “We can now ‘see’ how precipitation forms and grows in clouds, and better predict if they mean rain or snow, and also how much precipitation may accumulate. The radar we use operates at mm-wavelengths, as compared to weather radars that operate at cm-wavelengths. The difference makes these systems more sensitive to small particles and offers superior temporal and spatial resolution – the high-definition aspect of the cloud MRI.”

SoMAS sensors do not cover large horizontal distances, but offer superior resolution in the vertical dimension. Kollias adds: “The vertical dimension is where particles fall and grow under the action of complex microphysical processes. We use several different radars that look at the same part of the sky and allow us to do enhanced imaging of certain hydrometeor properties the same way MRI uses different frequencies to achieve resolution and improved contrast.”

The better we know the state of the atmosphere, the better the forecast will be, says Katestone’s Killip: “Emergent satellite technology and the extensive weather station network allow us to estimate what is going on. Improvements in high-performance computing power make it more possible to simulate the atmosphere, water and land more reliably and accurately. Considerable effort has been made to improve data assimilation and model the dynamics of the atmosphere and the physical processes that occur.”

Of the technology providers, IBM arguably has established the furthest reach into the field of weather intelligence. In 2015 IBM bought weather forecasting IT specialist The Weather Company, and subsequently has consolidated its own proprietary weather intelligence platform Deep Thunder into its acquisition’s operations.

Work in this discipline is underpinned by IBM’s supercomputing experience, with its Watson HPC resource available for weather modelling, weather analytics and investigations into how artificial intelligence (AI) and machine learning can be adapted to do the grunt work of weather forecasting.

“Deep Thunder’s projects are based upon the notion of advancing the science, working with better measurements, and using more computing power,” says IBM Research’s Treinish. “We have a version of Deep Thunder that produces global forecasts. Another example is Deep Thunder driving multiple predictive models to assess changes in water quality. We have Deep Thunder operating in a lake watershed coupled to both a predictive model of runoff over the land, and another of the three-dimensional circulation of the lake. With the latter operating at a resolution of tens of metres, we can look in detail at the transport of pollutants in the lake.”

“We are never going to have a perfect forecast because some processes which occur at very small scales, such as cloud formation, need to be simplified,” Killip adds, “and we have to consider the atmosphere as a chaotic system.”

Attempts at global weather models, where this knock-on effect could be scoped, call for computing resources that are challenging even for the most powerful agencies