Finding big data solutions through teaching and learning

As a partner in the African Research Cloud (ARC) and the Inter-University Institute for Data-Intensive Astronomy (IDIA), UCT is home to some powerful computing facilities. These facilities, which allow researchers to work with enormous sets of data, are helping us to prepare for the era of big-data science.

But when these facilities first arrived, someone needed to forge the way and figure out how best to harness them, share access to them and demonstrate their usefulness. Dr Bradley Frank, a senior researcher at IDIA, was among the team who led this charge. As project scientist for ARC’s astronomy demonstrator project, and a member of the IDIA team working to develop and deploy tools to analyse astronomical data generated by the Square Kilometre Array (SKA) and its precursor, MeerKAT, Frank faced a range of challenges. One particular challenge centred on ensuring comprehensive access for astronomers to the telescopes’ data and to the tools to process and analyse it.

As he is also the SKA lecturer at UCT, Frank realised that he could try out any solution on a group of benign test subjects: his second-year students.

Overlapping challenges and solutions

Frank spotted an opportunity to use the ARC and IDIA facilities to enrich his teaching. He set out to find a way to provide access to the software tools and data required to study astronomical data for the 45 students in his astronomy techniques class.

Although there is a variety of computer facilities available at UCT for student training, none of them provided the ideal hardware and software capabilities for in-depth computing for a large number of participants. Frank needed another solution.

In one fell swoop, Frank planned to use the data-intensive computing facilities of the ARC and IDIA to both teach his students and demonstrate teaching and learning as a use-case for the facilities, as well as interrogating the system to see where processes might break or fall short.

Teaching and learning on the cloud

The solution was facilitated by eResearch’s technical specialist, Timothy Carr. Carr was able to set up – and subsequently manage – a powerful, cloud-based virtual machine, or hub, on the ARC and IDIA servers. Frank’s 45 students could then access the data, software, instructions and computing power they needed to complete their analyses via a web-based interface and on any device with a web browser.

It worked so well that Frank used the same system with his students the following year. He even worked with Carr to solve problems on the fly; for example, when the students ran out of RAM on the virtual machine Carr had set up.

This sort of teaching and learning intervention is one of the first examples of its kind in the country. Such interventions are transferrable, Frank believes: “They could be used by researchers in other fields at UCT, but also across South Africa, to provide access to skills development in maths, physics, statistics and computing for a huge audience: anyone with a mobile device.

“This is an enormously powerful tool, with the potential to develop skills in science, engineering and technology in South Africa.”

For more stories like this, take a look at our 2017-2017 UCT eResearch annual report.