Measuring angles in perspective

The last experimental version of Kinovea introduced the ability to use the perpective grid as a coordinate system, which allows all kind of measurements on the grid. Here I’ll describe on what we have done with it in the context of podiatrics.

The experiment

For an experiment in podiatrics, we needed to measure how much people rotate when asked to perform 50 steps in the same spot, blindfolded. This is known as the Fukuda stepping test, it is generally used to detect neuropathologies but it has the interesting feature of being reproducible. I’ll not go into much details over the experiment itself or the rest of the setup as it is still unpublished and not my work. I’ll focus on the use of the perspective grid.

For this experiment, we developped a custom tool tailored for the Fukuda stepping test. The tool itself might be dissected in a future post, but the important thing here is that it is capable of measuring the angle of rotation of the person relatively to his initial direction (Σ on the tool).

 

Using the perspective grid’s ability to act as a coordinate system was very interesting here. Not only it lifted the need to mount the camera to the ceiling to get the view from above, but it actually allowed us to take series of images weeks apart without fearing measurement errors due to small changes in how the camera was fixed or how the experiment was set up. We could have taken the plastic track and set it up anywhere, even outdoors, and still have reproducible measures.

Once a person had performed the test (about 1 minute), we marked his feet position and took a picture. The picture was later processed in Kinovea to find the rotation angle.

Calibration

To be able to make measurements on the perspective grid, we first needed to calibrate it. To do this we measured the physical size of four segments on the plastic track that would be guaranteed to always be visible on any image we would take.
After laying down the grid and aligning its corners on these four points, we open the calibration dialog and enter the corresponding measures.

Note that the quadrilateral in the calibration dialog will have the same shape as the real one. This can be handy to give the segments the correct measure, especially when we take a picture from a completely different angle, rotate the camera, etc.

After the calibration is done, we can lay down the custom tool and make our measurement. (We generally do this on a different key image and disable the persistance for the grid so the grid doesn’t get in the way of the tool positionning).

 

Accuracy

During the course of the experiment we tested for the accuracy of the system.
To get the best results, we need to have the most precise calibration possible. It’s one of these cases where the number of megapixels does matter, and we used a still images camera rather than a camcorder to collect the measurement pictures.

To adjust the corners of the grid perfectly, we zoom to the max and adjust at pixel level.

To test the reproductibility, we started with the result of  a single person and took several pictures from various angles around the room. We then processed the images independently from one another and compared the resultts. We also measured the angle manually, to act as the “ground truth”.

We got approximately ±1° margin of error, which was perfectly acceptable with regards to the experiment.

 

How the perspective grid works

3D geometry tells us that any quadrilateral on a plane can be seen as the projection of a rectangle from another plane. In other words, a rectangle can be mapped to an aribitrary quadrilateral and back, with a transform matrix. In Kinovea, the pixel coordinates of the grid on the screen represent the quadrilateral, while the physical position of the corners on the ground represent the rectangle.

This means that after the calibration is done, any coordinate on the screen plane can be converted to its coordinate in the ground plane. It’s as if Kinovea could see the ground from above.

To measure angles in perspective, we first convert the screen coordinates of the three points to physical world coordinates, and we make the measurement from there. After the conversion step all the measurements use regular 2D geometry. A similar approach is used to measure distances, speed, etc.

Future

Many things can be done when we can measure snakesthings on a plane. Statistical analysis of ball impacts in table tennis? Distribution of scoring angles in football? Tell us what you did with the perspective grid! And report what parts of the tool can be improved to further streamline the process.

Notes

 

 

 


Creating a custom tool

(Note: this was originally posted in november 2012, I’m reposting it here to give it more visibility and to revive the blog).

This tutorial describes the process of designing and creating a custom tool for Kinovea (version 0.8.18 or later needed). It follows the actual creation of the “angle-to-vertical” tool shipped in 0.8.19. The process is currently manual and involves editing XML files.

Draw it on paper first

The first thing to do is to draw a draft of the tool on a piece of paper. Take notes of what type of interactions you want, angles to display, etc. Associate numbers to every point (movable by the user or not), starting from 0.

Here is my draft:

This tool is to check the angle of a segment relatively to the vertical axis.

Point 1 is the driving point and I want to measure the angle of segment [1,3] relatively to the vertical axis represented by [0,2].
Here is the final tool in a bike fit setup:

(Image courtesy of Synergy-ProTraining.de)

The spec of the tool is that when the knee point is moved, it drags the vertical line with it. This way the angle is always measured relatively to the vertical axis. (The numbers are for illustration, they are not part of the final tool).

Start with an existing file

Chances are that an existing tool resembles what you want to do, or has the right number of points. It’s always easier to start with an existing shell and fill in the blanks. The existing tools are found in the program directory, for example:
“C:\Program Files\Kinovea\DrawingTools”

Copy one of the existing XML file to a more convenient place and rename it. The name of the file is only relevant to the order of appearance in the menu, but it’s good practice to give it the same name as the tool name. To test the tool, save it back in Program files (you’ll be asked for admin rights for the copy) and restart Kinovea.

Open the file in Notepad or your favorite text editor.

1. Change the name. This is what will be displayed in the menu.
2. Create a unique GUID for the tool. Go to http://createguid.com/ or http://www.guidgenerator.com and copy the result in the ID attribute.
3. Fill in the total number of points.

<Name>Angle to vertical</Name>
<Id>B0D88F59-FA46-4B8A-A2B3-F28A66FE2163</Id>
<PointCount>4</PointCount>

Segments

Segments connects points together. When referencing points, use the numbers on your draft. Remember that the first point is “0″ not “1″.

Here we only have 2 segments, one going from “0″ to “2″, and one going from “1″ to “3″. (we will describe the relation between point “1″ and the [0,2] segment later).

The line style can be dashed or solid. By convention virtual segments like verticals, symmetry lines, etc. are dashed, and real segments are solid.

<Segments>
<Segment point1="0" point2="2" style="Dash" width="2"/>
<Segment point1="1" point2="3" style="Solid" width="2"/>
</Segments>

Angles

The Angles section defines between which segments we want to display angles.
origin, leg1 and leg2 attributes refers to Point indices (starting at 0 as always).

Angles with the relative attribute are always less than 180° and switch sides when crossing the reference leg (segment from origin to leg1).

For non relative angles, the order of the “legs” are important since they indicate the angle orientation. An angle is defined as the angle going from leg1 to leg2 clockwise. If you have a 90° angle and you mess up the orientation, it will show as 270°. Just invert the legs in the XML.

Angles with the tenth attribute to true will display tenth of degrees instead of integer values.

<Angles>
<Angle origin="1" leg1="2" leg2="3" relative="true" tenth="true"/>
</Angles>

Handles

At the simplest, a handle just references a point that the user can move around.
(There are also segment handles and ellispes handle, not discussed here).

Handles can be extended with constraints and impacts. “Constraints” express in which ways the point itself can be moved around. “Impacts” define what happen to other points when we move this handle.

Here, we want the handle “0″ and “2″ to be constrained on the vertical axis, this is done with VerticalSlide.

<Handle type="Point" reference="0">
<Constraint type="VerticalSlide"/>
</Handle>

<Handle type="Point" reference="2">
<Constraint type="VerticalSlide"/>
</Handle>

Point “1″ can go anywhere, but when it moves, the vertical segment [0,2] must moves with it, and stay vertical. Basically, we want point “0″ and point “2″ to always have the same “x” coordinate than point “1″. This is done with the HorizontalAlign impact.

<Handle type="Point" reference="1" trackable="true">
<Impact type="HorizontalAlign">
<HorizontalAlign point="0"/>
</Impact>
<Impact type="HorizontalAlign">
<HorizontalAlign point="2"/>
</Impact>
</Handle>

This is equivalent of saying: “Each time point 1 moves, align point 0 and point 2 with it, on the horizontal coordinate”.

The handle on Point “3″ is completely free from any constraint and has no impact on other points.

<Handle type="Point" reference="3" trackable="true"/>

That’s almost it. The impact and constraint engine is the most advanced part but after you’ve done a few tools you will see the patterns.

Icon

The Icon section can be omitted, it will use the default icon:
To have a custom icon you must use a 16×16 image and convert it to Base64 encoding.

Go to this Base64 online encoder, input your 16×16 icon from your hard drive, click Encode, and paste the output to the XML file as is.

Initial configuration

The initial configuration is the shape of the drawing when first added to the video.
The numbers are coordinates in pixels on a virtual plane of 800×600. They will be adapted to the size of the video.

<InitialConfiguration>
<Point>100;100</Point>
<Point>100;150</Point>
<Point>100;200</Point>
<Point>150;200</Point>
</InitialConfiguration>

Tracking

You may have noticed that I added the attribute trackable=”true” to the handle for point “1″ and point “3″ :)

Trackability is new in version 0.8.19. The goal is that everything becomes trackable, including the angle tool, line tool, magnifier, custom tools, ect. I hope the combined power of custom tools and trackability will enable great things.

Be careful with which points you allow tracking on though. Generally a point with a constraint probably shouldn’t be trackable. It’s the tracking of other points that will move it, respecting the constraint along the way. A trackable but constrained point could cause problems when the tracking go outside the constraint. Experiment!

A copy of the final file is also online here.

Next

As a custom tool designer, be sure to report your experiments and special needs! The custom tool framework is still new and will be further enriched as special needs arise. Length display, arrows, rectangles, new constraints or impacts, each time something is added to support a particular tool, the toolbox gets richer and opens even more possibilities for all future tools to come.



Kinovea 0.8.15

We are excited to release Kinovea 0.8.15, the new version of your open source video analysis toolkit !

Now in 16 languages, with several new features, usability and functional improvements, as well as bug fixes.


Download Kinovea 0.8.15



New features include:
The Capture screen : directly stream live action from your camcorder into Kinovea; record images and videos; or delay the live stream for self coaching.
Observational references : overlay complex drawings or images as motion guides on top of the video.
Dual Export: save composite output from your comparison analysis.



See also:
- A more complete list of changes (What’s new in Kinovea 0.8.15)
- The Online Manual (Also in Italian, French).
- The Feature list (PDF) (Also in Italian, French).



With the introduction of live capture this version is an important milestone for the project, and another leap forward.
As always, early adopters feedback has been crucial to drive the developement: Thanks !






Feature freeze / Translations

babel tower

OK, the new features and improvements over 0.8.7 are now pretty stable and mature enough for prime time. The current cycle will be mostly focused on updating translations, user manual and bug fixing.

Translating Kinovea is as simple as downloading a spreadsheet and filling in the gaps. (learn more on this page).
If you start a new language please post in this thread indicating which language you are starting to work on.

idea – A Polish speaking contributor is needed to update the Polish locale (currently 52% complete).

contributorThe most wanted list :
(Languages not already in Kinovea sorted by number of visitors of on the website (based on country of origin))
1. Japanese
2. Russian
3. Swedish
4. Arabic
5. Korean

Thanks for your help !