It's not easy for a robot to get around in a greenhouse.
According to Shengrong Bu, Associate Professor of Engineering at Brock University, fixtures, elevated planters and narrow, winding corridors can stop machines in their tracks.
"Self-driving machines operating in greenhouses have different sensors collecting different information," says Bu. "The information gathered needs to be fused together so that the machines can better understand their surrounding environment to avoid collisions."
Bu, Professor of Engineering Shahryar Rahnamayan and their team have developed an obstacle detection system enabling robots to successfully navigate complex greenhouse layouts.
Facilitating the project was the Greenhouse Technology Network, leveraging support from the Government of Canada through the Federal Economic Development Agency for Southern Ontario (FedDev Ontario).
The researchers partnered with Kevares Autonomous Services, an Oshawa-based company developing modular autonomous robots for tasks such as cargo hauling, lawn mowing, floor scrubbing, snow removal and more in parking lots, on sidewalks and in parks and green fields.
Robots in greenhouses support a wide range of tasks, from transporting supplies and produce including reverse logistics to planting, watering, seeding, harvesting and monitoring plant health, growth and disease detection.
While Kevares had already developed robots capable of navigating large and open environments, the project offered an opportunity to expand the company's ability to work in the distinct environment of a greenhouse.
"Our system was not yet tasked with near-field obstacle perception and precise navigation for cramped spaces such as a greenhouse and that's exactly what we advanced with professors Rahnamayan and Bu and their teams," says Kevares CEO, Joel Nascimento.
Kevares' mobile robots are equipped with multiple sensors that generate a detailed profile of the robot's surrounding environment.
The team, which included robotics research assistant Nili Rahmani and then-postdoctoral researcher Junfei Li, began by evaluating the performance of Kevares' existing sensors.
These sensors include:
- Depth cameras, which measure the distance from the robot to nearby objects and between objects.
- RGB (red, green, blue) cameras, which capture visible-light imagery.
- Ultrasound sensors, which calculate distance by emitting ultrasonic sound waves.
- LiDAR (Light Detection and Ranging), which uses laser beams to measure distances and detect movement in real time.
The team then developed a sensor fusion algorithm, created a software package that integrates data from ultrasound, RGB imagery, depth sensing and other streams, and trained and maintained artificial intelligence models.
"Using our system, Kevares' robots can detect obstacles that pose a collision risk," says Bu. "The robot may halt in front of, or avoid, objects laying on the floor or upright, enabling the machine to make informed navigation decisions based on its immediate environment in real time."
She says the system can be adapted to various greenhouse types, "ultimately boosting productivity and improving crop quality."
Nascimento says the system developed by the Brock team has opened up new markets for his company.
"The ability to sense and discern obstacles from as close as 10 centimetres away, instead of requiring one or two metres, is a foundational safety feature for the greenhouse industry a capability we've now added to our product, for both the farming and other industries, too," he says.
"For us, it was a special experience working with the Yousef Haj-Ahmad Department of Engineering at Brock, which has an extremely professional, collaborative team," adds Nascimento, noting that his company has since hired Rahmani to continue advancing Kevares' work.
The team's project, "AI-Powered Smart Vision: Enhancing Greenhouse Robotics with Deep Learning and Computer Vision," is particularly relevant for the region, says Chris Bittle, Member of Parliament for St. Catharines.
"Agriculture is the mainstay of Niagara's economy, with the greenhouse industry being a major driver of the region's growth in agri-business," says Bittle. "This research partnership between Brock and Kevares will lead to greater efficiency and productivity in the region's greenhouse operations."
According to Niagara Economic Development, the region had more than 21 million square feet of greenhouse space in 163 greenhouses as of 2021.
