Deep within the structure of the Mathematics and Computer building (MC) at the University of Waterloo lies one of the most sophisticated and important digital research computing systems in Canada. The newly refreshed system, named Nibi, recently replaced the Graham supercomputer.
The powerful system has more than 700 nodes and 140,000 CPU cores allowing Nibi to undertake countless simultaneous computational jobs. It ranks in the top 500 research computing systems in the world.
"Nibi's launch represents a defining moment for Canada's digital research infrastructure," said Mélanie Joly, Minister of Industry and Minister responsible for Canada Economic Development for Quebec Regions. "It equips our researchers with the advanced computing power they need to push the boundaries of discovery, strengthen our economy, and lead on the global stage. With its cutting-edge, energy-efficient design, Nibi is not only supporting today's transformative projects, it is laying the foundation for the next generation of Canadian innovation from coast to coast to coast."
"Nibi is the equivalent of 35,000 desktop computers working in harmony," according to John Morton (BMath '94), who is the director of Technology for SHARCNET and the technical lead for the Graham cluster. "Harnessing that power allows for ground-breaking research through batch computing, storage, visualization, GPU acceleration, cloud VMs, DBaaS and more being available to support Canadian research."
While data centres are known to be huge users of power, the designers of Nibi have sought to use leading-edge technology to reduce cooling costs. The system uses water rather than air to be more efficient and sustainable. Instead of paying to get rid of heat generated by high-performance computing, the heat is captured and repurposed to heat the Mike and Ophelia Lazaridis Quantum-Nano Centre.
To recognize this sustainable approach, designers consulted with local Indigenous communities and decided to name the system Nibi. The name means water in Anishinaabemowin (Ojibwe).
Nibi provides GPU-equipped nodes each with eight H100 Nvidia GPUs to support large AI models. With more than 25 Petabytes of purely flash-based storage space, Nibi provides increased performance and reliability from previous storage solutions.
"Ontario is home to some of the brightest researchers in the world who are driving Ontario's key industries forward," said Nolan Quinn, Minister of Colleges, Universities, Research Excellence and Security. "Through Compute Ontario, our government is proud to support the University of Waterloo's innovative supercomputer, ensuring that researchers have the tools they need to make critical advancements in science, medicine, and technology, saving and improving lives across the province."
Since 2017, Waterloo's supercomputer has supported thousands of researchers from multiple universities to advance innovation across science, medicine and technology. As a member of SHARCNET, a multi-university consortium in Ontario, Nibi is projected to support more than 4,000 researchers each year.
"At Waterloo, we're proud to collaborate with our funding and research partners to advance the next wave of Canadian innovation," says Charmaine Dean, vice-president, Research & International at Waterloo. "Nibi allows researchers across Canada the opportunity to explore and discover what's possible, and we are pleased to continue Waterloo's legacy as a computation leader."
Waterloo has a rich history of computing dating back to its early years. Thanks in large part to pioneering researchers such as Don Cowan and Wes Graham, the institution was the first in Canada to house the most powerful computer of its time. The IBM 360/75 was the same model that NASA used to send astronauts to the Moon.
Since then, Waterloo has continued to be a leader in computing. To ensure that it stays on the cutting-edge, Waterloo researchers work closely with a variety of partners. Nibi is funded by Innovation, Science and Economic Development through the Digital Research Alliance of Canada, and the Ministry of Colleges, Universities, Research Excellence and Security through Compute Ontario.
