Visual Odometry

My main goal is to use visual odometry to localize the robot on an internal virtual map. OpenCV has algorithms that can be used for this and I will augment it with extra sensors to improve accurracy. The idea is that we can compare two visual frames (images) with an overlap to determine where we are and how far we we have moved.

Key areas that need addressing:

• Consolidating the virtual map with the real world map (what the camera sees)
• Create a coverage path that will determine where we want to go
• Corrections that need applying as we work out where we actually are
• A compass is available for PID feedback. Maybe it will help using it in the path planning stage?
• Account for drift.

Monocular Visual Odometry

This means we have a single camera. It can be running video or take successive photos. The overlap between two adjacent images can be used for localization to determine position and pose.

The key data that can be retrieved from the Monocular Visual Odometry is the robot's pose (T matrix).This can be used to determine:

• The direction the robot is facing - the Rotation R Matix. (in 3D space, we only need two)
• The translation vector (t) (or how far the robot has moved)

The T matrix contains both the R and t data.

Limitations with Monocular Visual Odometry

It is not possible to detrmine real scale with regard to the translation vector. We can only say "we have moved 5 units", not "we have moved 4 metres". Translation is relative to the grid being used so it can be inaccurate and drift over time. The drift in translation also means there will be a drift in the rotational vector as we might not be where we think we are. If the images used are sparse or featureless (a huge field), it can be difficult to localize. In order to improve these shortfalls, we need extra sensors to augment the odometry.

To correct for translation, the proposed solution is to use periodic April Tag recognition. This will give us absolute location in the real world which we can feed back to correct our path planning. To correct for rotational drift, we can use a digital compass and integrate it into the planning algorithm (sensor merging via a filter). When we have a planned path, we can send instructions to move at a particular speed and particular angle to move the robot.

Proposed solution for kupe with Monocular VO:

• Single camera attached to Raspberry Pi
• April Tag recognition, processed on the Pi
• Compass used to augment planning phase. The compass is attached to the Pi
• The arduino is reduced to following instructions issued from the Pi

Stereo Visual Odometry

Uses two cameras with a known distance between them (baseline) to capture synchronized images. This stereoscopic vision alows for depth perception. This gives us these advantages:

• Depth Perception: Directly computes depth by triangulating corresponding points in the left and right images
• Scale-Aware: Provides metric scale estimation without additional sensors, as depth is derived from the stereo baseline and disparity. We can find real distance traveled in metres rather than grid-units
• Feature Matching: Requires matching features between the left and right images - requires careful camera configuration an placement
• Scale Drift: Less of a problem than Monocular VO as we are measuring real distance
• Rotational inaccurracy is less of a problem as we are measuring real distances
• Better in feature-sparse environments as we are comparing two synchronized images

Since Stereo VO gives real-world depth perception, the rotational drift issue is also addressed. This means we do not need the compass as part of Coverage Path Planning. I still think it prudent to incorperate April Tags as that give a hard-resync if something goes very wrong (robot gets turned dramatically).

Proposed solution for kupe with Stereo VO:

• Two cameras mounted a fixed distance apart attached to the Raspberry Pi
• April Tag correction, processed by the Pi
• Compass can be relegated to the Arduino to improve straight-line traction (as part of the PID).

April 2026