RGB tells you how much red, green, blue is in a pixel — but not what colour it "feels" like. HSV and LAB reshape the same pixel into hue, saturation, brightness — the axes humans actually use to describe colour. This is the foundation of colour-based tracking.
Three grayscale maps: H (which colour), S (how vivid), V (how bright). Because H is a wheel, its map is coloured — same hue at 30° and 390° means the same red.
Hue is an angle (0-360°) around the colour wheel. Saturation is how pure vs. washed-out the colour is. Value is brightness. Split brightness from colour identity, and "find all yellow pixels" becomes one simple check on H — instead of a three-way RGB comparison that breaks the moment the lighting shifts.
LAB was designed so that equal numerical distance = equal perceived difference. L is pure lightness. a runs green-to-red, b runs blue-to-yellow. There is no way to make a red look green in LAB without going through neutral first — which makes it excellent for measuring colour difference the way humans see it.
Colour trackers (a ball on a stick, a highlighted marker, a green screen) almost always convert to HSV first. Skin-tone detectors and film-look colour grading almost always convert to LAB. RGB is how images are stored; HSV and LAB are how algorithms reason about them.
Your lighting keeps changing — sun goes behind a cloud, then out again — and you're trying to track a red toy across the frame. Which space handles the shifting brightness most gracefully?