The Square Kilometre Array radio telescope is expected to generate twice as much data each day as the world's current daily Internet traffic (credit IBM RESEARCH)

SKA project drives first 'extreme Big Data' computing systems

ASTRON and IBM have embarked on a €32.9M project to develop the computer systems needed to support the SKA radio telescope.

Called DOME, the five-year project ASTRON/IBM initiative will be tailored from the outset to meet SKA’s requirement for massively big data management and storage, and will begin by mapping the full range of computing technologies that will be needed by the time it is fully operational in 2024.

SKA will be the world’s largest radio telescope, with a collection area equivalent to one square kilometre, made up of individual receptors in an array extending up to 3000km from the centre.

It will combine signals received from thousands of antennas to simulate one radio telescope capable of very high sensitivity and angular resolution.

The SKA Organisation - a company owned by the nations backing the project - has been assessing two possible locations for the array: South Africa with neighbouring states and Australia together with New Zealand.

Members were expected to make a decision on 3 April, but in a surprise move they issued a statement in favour of an inclusive approach that would “maximise the value from the investments made by both candidate host regions”.

A scientific working group will now report on compromise options in mid-May.

SKA is expected to generate 10EB (exabytes) of data each day, of which at least 1PB (petabyte) will need to be stored permanently.

Chipsets needed to process data access and SKA applications will need to be capable of 20-25 exaflops of processing power, according to IBM Research’s Ton Engbersen, DOME scientist and project leader.

“[DOME will take] Big Data analytics to the extreme,” Engbersen added.

“Take the current global daily Internet traffic, double it, and you are in the range of the data set that the SKA will collect each day.”

This would equate to around 40,000Pb every 24 hours.

Engbersen believes that SKA represents a new generation of ‘large scientific instruments’ (LSIs) that will drive innovation in computer technology across a range of disciplines, from advanced accelerators and 3D stacked chips, through tape storage to optical interfaces and energy-efficient computing.

“With radio astronomy the two key issues are I/O – getting data into and out of the processors at the speed required – and the compute performance of the processors themselves, added Engbersen.

“Balancing the ratio between these is crucial, because if they are not balanced optimally you introduce the same inefficiencies inherent to most traditional computing architectures.”

DOME will also be developing intelligent data storage algorithms that will eventually enable the anticipation of data request patterns shown by SKA personnel.

“DOME will ‘learn’ how people use the SKA data, and decide how to store it accordingly,” Engbersen explained.

LSIs are not the only developments that will be generating extremely big data over the next decade, Engbersen pointed out.

“Increasingly, any application that uses sensors to accumulate massive data sets will benefit from some of the innovations that DOME will drive,” he said.

“Natural water management [here in the Netherlands] is a good example of this, where sensors are being fitted to the country’s system of dykes and canals to monitor volume levels and flows.”

DOME will be based at the newly-established ASTRON & IBM Centre for Exascale Technology in Drenthe in the north-east of Holland.

The multidisciplinary team will probably comprise 15 IBM Research and 10 ASTON team-members.

Further information:

High-end routers rise to meet Exabyte challenge

Data management - will we ever press 'delete'?

Life begins at 40GBit/s

Sign up to the E&T News e-mail to get great stories like this delivered to your inbox every day.

Recent articles