Universities renew big data agreement for 5 years

The agreement forming the Inter-University Institute for Data Intensive Astronomy has been renewed by the universities of Cape Town, the Western Cape and Pretoria for a period of five years. The institute is the main place for South African university researchers to process the very large data sets coming from the South African Radio Astronomy Observatory’s MeerKAT telescope.

The Inter-University Institute for Data Intensive Astronomy (IDIA) was set up in 2015 to establish a leading centre of research and innovation in South Africa to address the scientific and technical challenges of big data in astronomy, and to ensure that both the infrastructure and the skills pipeline for data-intensive research are developed at South African universities as the country prepares to host the Square Kilometre Array (SKA).

The vision was, and still is, to enable South African researchers to become global leaders on MeerKAT large-survey science projects and large projects on other SKA pathfinder telescopes.

“The Square Kilometre Array, a global mega-science project that is coming to South Africa within the next few years, is the biggest scientific data challenge of this decade,” says Professor Russ Taylor, director of IDIA and the SKA Research Chair in Radio Astronomy at the University of Cape Town (UCT) and the University of the Western Cape (UWC). “With renewal of the IDIA agreement among the partner universities, we can build upon our first five years to ensure that researchers and students at South African universities have the capabilities and data science skills to be leaders in this new era of big data astronomy.”

Professor Sue Harrison, UCT’s deputy vice-chancellor for research and internationalisation, confirms: “We are very pleased that the next phase of the IDIA agreement is in place. It is taking us forward to the next exciting leg of the big data platform, which will change the way we do science and open the door to real-time global collaboration.”

Milestones

In its first few years of operation, the institute reached several major milestones, including setting up a first-in-Africa cloud computing facility for data intensive research, building a big data processing pipeline for MeerKAT observations, and facilitating the first science results from the telescope, which was inaugurated in 2018.

“The Square Kilometre Array, a global mega-science project that is coming to South Africa within the next few years, is the biggest scientific data challenge of this decade.”

From discovering that black holes are spinning in unison to processing data for MeerKAT’s launch image of the centre of the galaxy, the achievements of IDIA researchers are as numerous as they are impressive. And they have been made possible by some equally impressive technology developments, such as the development of scientific software optimised for running on cloud computing infrastructure, and innovations in data visualisation using immersive technologies. These, and other key successes, are documented in the first IDIA activity report published in 2019.

IDIA is also home to a big data visualisation laboratory where immersive scientific data analysis is made possible, using technologies like virtual reality (VR) and the Iziko digital planetarium. IDIA has an active development and outreach department, whose purpose is to engage the public and to ensure maximum socio-economic benefits from IDIA programmes.

ilifu

A major spin-off of IDIA is the establishment of ilifu, a scientific cloud computing facility serving both astronomy and bioinformatics. This new partnership includes Sol Plaatje University, the Stellenbosch University, the Cape Peninsula University of Technology and the South African Radio Astronomy Observatory as well as UWC and UCT through its computational biology department.

“Cloud technologies are key to data-intensive research in many fields, including astronomy and bioinformatics. The containerisation of entire analysis pipelines and the increasingly easy interfaces with scientific software, through for example Jupyter Notebooks, make cloud computing an essential tool for scientific research today and in the future,” says Professor Rob Simmonds, associate director in new technologies at IDIA and professor in computer science at UCT.

Platform of the future

“To remain competitive in today’s world of data intensive research, we need new ways of doing science and cloud computing is just that,” says Professor Carolina Ödman, associate director for development and outreach at IDIA. “Not just because of the large amounts of data, but also because it helps the integrity of the scientific process to be able to reproduce scientific results that ultimately inform policy and how we design our future.

“By training a generation of scientists on the platform of the future, we develop skills that are increasingly in demand in all science fields and open the door to careers in the knowledge economy.”

Hand-in-hand with other projects, such as the MeerKAT telescope and the H3Abionet initiative, the renewal of the IDIA agreement is an important milestone in South Africa’s recent scientific history – one that will yield yet unforeseen results and develop the country’s growing talent in science and technology.