Foundations 01Technical ArticleGaitAI Research8 min read
From Walking Video to Movement Intelligence
What actually happens inside GaitAI
Capture, pose estimation, gait features, sensor fusion, quality control and output — the sequence of transformations that turns an ordinary walking sequence into something a clinician, researcher or operator can act on.
Before you scroll: this essay follows one walking sequence from pixels to something a clinician or operator can act on.
Drag the track, tap a stage, or use ← →
Text description
An eight-stage progression, one walking figure throughout. Raw video: recorded frames from the same recorded walking sequence. Person: a detection box around the walker, tracked across frames. Pose: a small set of landmarks — head, shoulders, hips, knees, ankles, wrists — replaces appearance. Skeleton: the landmarks joined into bones; geometry only. Temporal trajectory: earlier poses ghosted behind the walker and the paths the ankle and wrist trace through time. Movement features: normalized image-space ankle, wrist and hip positions from the same frames. No cadence or symmetry is measured. Context: the same signal set beside an illustrative personal baseline and history, with a capture-quality note. Decision support: a report for review, marked as decision support and not a diagnosis.
Short on time? Read the 2-minute version
- Capture is a signal, not an answer: video, CCTV and wearable streams all start as raw movement that has to be constructed into something measurable.
- Pose estimation gives geometry — joints in space. A single frame shows a posture; it does not show a gait.
- Gait features live in time: cadence, stride rhythm, symmetry and variability only exist across a sequence.
- Fusion adds resilience where one modality is weak. It does not automatically add accuracy.
- Signal-quality control decides whether a number means anything, which is why captures are gated rather than scored regardless.
- A metric becomes intelligence only against a baseline and a history — and it is offered as decision support, never as a diagnosis.
The full article follows, with the diagrams and the caveats.
A walking video looks simple.
A person crosses a room. A camera records the movement. To the human eye it is immediately recognizable as walking — and immediately interpretable. We read effort, hesitation and confidence from a gait without consciously measuring anything.
For an AI system, useful movement intelligence does not exist in the pixels automatically. It has to be constructed.
The journey from an ordinary walking sequence to an interpretable movement signal runs through a series of transformations: capture, pose estimation, temporal analysis, feature extraction, multimodal reasoning, quality control, and finally a form of output that a clinician, researcher or operator can actually use.
This is the pipeline behind GaitAI.
Movement begins as a signal
Before anything can be measured, movement has to be observed. The observation itself is a design decision — what is recorded, from where, and for how long shapes everything downstream.
Depending on the deployment, the input to a movement-intelligence system may be any of the following:
- A short walking video — a few seconds of natural walking, captured on request
- A smartphone camera — the most widely available capture device there is
- A fixed camera or CCTV stream — where the setting and its governance make that appropriate
- A smartwatch or wrist-worn wearable — continuous, low-friction, already on the person
- IMU and other compatible sensor streams — accelerometer, gyroscope and related channels
From pixels to pose
Raw RGB frames are not a usable representation of movement. They carry an enormous amount of information that has nothing to do with how a person walks: clothing, lighting, background, camera noise. The first substantive transformation strips that away.
reduces each frame to a small set of anatomical landmarks — points whose positions can be compared across frames and across people:
- Shoulders
- Hips
- Knees
- Ankles
- Elbows and wrists
- Head and neck reference points
Why the temporal sequence matters
A single frame gives a posture. Gait is not a posture — it is a rhythm. Step timing, the alternation between left and right, the fraction of the cycle spent with both feet grounded, the smoothness of a swing: none of these exist in one frame. They exist only in the relationship between frames.
A single frame shows a posture. Only a sequence shows a gait.
This is also why tracking matters as much as detection. Landmarks are only comparable over time if the system knows they belong to the same person throughout the sequence.
Text description
A single skeleton at mid-stance stands alone: one frame, which shows posture. While the "Hold to reveal gait" control is held, four more skeletons appear on either side in stride order — heel strike, loading, mid-stance, toe-off, swing — and the ankle's path is drawn through all five. Released, they fade and one frame remains. The point: cadence, symmetry and variability exist only across this sequence, never in one frame.
A skeleton is still not gait intelligence
A temporal pose sequence is geometry, not insight. The next transformation converts that geometry into gait-related descriptors — quantities describing how the walking is organized rather than where the joints happened to be.
- — steps per unit time
- Step timing — the duration and consistency of individual steps
- Stride rhythm — the regularity of the full gait cycle
- — how closely the two sides mirror each other
- — how much the pattern fluctuates from cycle to cycle
- Posture — trunk orientation and its stability through the cycle
- — the paths described by individual joints
- Walking speed, where the capture geometry makes it recoverable
- Balance-related proxies — derived indicators associated with stability
Why fusion matters
Different sensors see different aspects of the same movement, and none of them sees all of it.
Vision
Spatial structure
- Body geometry and joint relationships
- Step length and spatial symmetry
- Posture through the gait cycle
- Degrades under occlusion and poor framing
Wearable / IMU
Inertial dynamics
- Acceleration and rotation at the sensor site
- Fine-grained step timing
- Continuous, viewpoint-independent capture
- Limited view of whole-body geometry
is the mechanism for combining those views. It is also the part of a multimodal system easiest to get wrong, because adding a modality always adds parameters, and adding parameters usually moves a benchmark number.
Fusion should not mean “more data is automatically better.” It should mean different evidence is combined when it improves the task.
That distinction is worth testing rather than assuming. We return to it in When Fusion Looks Better Than It Is, which sets out the questions a multimodal result should be able to survive.
Quality before intelligence
Real captures are not clean. A movement-intelligence system that assumes otherwise will produce its most confident outputs on exactly the observations that deserve the least confidence.
The failure modes are mundane and frequent:
- Occlusion — furniture, other people, or the person's own body hiding key landmarks
- Incomplete body visibility — feet or head outside the frame for part of the walk
- Poor camera placement — an angle that collapses the movement being measured
- Low-quality signal — motion blur, low light, heavy compression, dropped frames
- A missing modality — one expected stream simply not present
- A — data that arrives, but damaged
- Very short recordings — too few gait cycles to establish a rhythm at all
A responsible system therefore needs a second capability alongside measurement: the ability to recognize when the evidence is not good enough to measure from. That means scoring explicitly, flagging degraded captures, attenuating confidence, and declining to emit a result where the input does not support one.
A clean, well-framed capture: every signal can be computed. Switch a problem on to see what a responsible pipeline should withhold.
Text description
Five capture problems can be switched on: occlude legs, crop feet, low light, motion blur, poor camera angle. For seven movement signals — cadence, stance/swing timing, left/right symmetry, ankle trajectory, stride length, posture, walking speed — a qualitative verdict is shown: available, degraded, unreliable or unavailable. With a clean capture every signal is available. Hiding the legs removes timing, symmetry, trajectory and stride length while cadence and posture survive. Cropping the feet removes the ankle trajectory and makes stride length unreliable. Low light degrades everything a little. Motion blur makes timing and trajectory unreliable. A poor angle removes stride length and makes symmetry and speed unreliable. This is an illustrative teaching table, not an analysis of any capture.
From metric to decision support
Even a clean, well-measured gait feature is not yet intelligence. A number on its own has no reference point.
Measurement is not the same thing as intelligence.
What makes a measurement interpretable is context — a baseline to compare it against, a history to place it in, and a presentation that reaches the right person in a usable form.
The value sits in the middle of that chain, not at either end. Why the longitudinal step matters so much is the subject of A Fall-Risk Score Is Not Enough.
One engine, different outcomes
Capture, pose, gait features, fusion and quality control are shared. What changes between applications is the interpretation layer that sits on top of them.
In a clinical or care setting, a change in stride variability is read against a person's own movement history and surfaced for review — the framing behind MobilityCare. In a safety or spatial-analytics setting, the same underlying movement processing supports a different question about events and flow in a monitored space — the framing behind SecureVision.
The measurements can be identical. The interpretation, the governance and the output are not, and treating them as interchangeable is how movement systems drift outside their intended purpose. The product family and the use cases set out where each interpretation applies.
Movement is the input.
The real intelligence lies in what happens between observation and action.
Written by GaitAI Research. GaitAI produces movement measurements and decision-support outputs for research, clinical and operational teams — not diagnoses. Published research is listed in the publications library.
GaitAI Foundations · 1 of 5
You now know how a walking video becomes movement intelligence.
Next
What else can one walk reveal?
Foundations 02Your Walk Is More Than a BiometricWhy gait is more than identityContinue