Topic: Kinovea 0.9.1 (beta)

This is Kinovea 0.9.1.
This version introduces capture-and-replay automation, improves capture performances, especially for delayed video, and adds many other improvements. This version requires .NET 4.8.

 

Many thanks to everybody that helped with testing this version. Special thanks to rkantos, Faultyclubs, and Reiner Hente, for their patience in spite of my continuous requests for testing buggy builds tongue

Plenty of changes aren't described here, please consult the full changelog for details.

 
1. Capture automation

We can now trigger recording based on microphone volume, and stop recording after a specific duration. This enables a hands-free, continuous recording workflow.

http://www.kinovea.org/screencaps/0.9.1/091-capture-automation2.png

Recording footage around an event of interest is natively supported with the existing delay feature. For example, say we are filming a golf swing and want to capture from 3 seconds before impact to 2 seconds after it. We set the video delay to 3 seconds in the capture screen, and set "stop recording by duration" to 5 seconds. When the club impact triggers the recording, it will start saving the video stream from 3 seconds ago.

When using multiple instances of Kinovea, each instance now has a deterministic name. By default it will be a number in sequence but you can also use the `-name` argument on the command line for full control. Each instance can use its own preferences file. This is useful to create advanced setups, for example having an instance dedicated to capture and another to replay, or for instrumenting Kinovea from other programs. Multiple Kinovea instances can listen to the same microphone for synchronized recording by audio trigger.

> kinovea.exe -name replay

http://www.kinovea.org/screencaps/0.9.1/091-naming.png

We can also run a script on the resulting file after the capture is complete. This can be useful to copy the file somewhere else or process it further.

 
2. Capture performances

A lot of care went into the performance of delayed capture, and it should now be almost on par with real-time capture. You still need to toggle the option under Preferences > Capture > Recording.

The act of compressing the images for storage is usually the main bottleneck when recording with the typical cameras used in Kinovea (high-end webcams and machine vision cameras). We can now bypass this compression step entirely and record uncompressed videos. Be mindful that uncompressed videos take a lot of storage space. This option is under Preferences > Capture > General.

Modern storage options like SSD, NVMe or RAMDisks all have higher bandwidth than the USB link of the camera on the other side, so hopefully whatever the camera can send to the PC can be recorded without drops. The simulator camera and the infobar above the capture area can be used to diagnose issues.

http://www.kinovea.org/screencaps/0.9.1/091-captureinfobar.png

On top of recording uncompressed videos, we can now record "raw" video streams if the camera supports it. This records the raw sensor images, grayscale with color implicitly encoded in a Bayer grid pattern. The player has a new option to rebuild color images from raw files under menu Image > Demosaicing. The advantage of doing this is the storage bandwidth is only that of a grayscale video, so it cuts requirements by a factor of 3.

http://www.kinovea.org/screencaps/0.9.1/091-debayering3.png

 
3. Replay folders

This is a new concept, completing support for a fully hands-free capture-and-replay workflow. In this mode a playback screen is associated with an entire folder and any new video file created in this folder, usually by the capture module, will be instantly loaded and start playing.

http://www.kinovea.org/screencaps/0.9.1/091-openreplayobs.png

Typically we would use this within a single instance of Kinovea, but as it is based on the file system, we can also have a separate instance of Kinovea dedicated to replay. It should even be possible to put the replay instance on a different machine on the network, copying over the captured files using a post-recording command.

http://www.kinovea.org/screencaps/0.9.1/091-replayobserver.gif

In the above screencast, the left screen is a camera filming the stopwatch. The right screen is open using a replay folder observer on the folder where the captured videos are saved. In this case the capture was configured to stop by itself after 2 seconds. As soon as the capture is completed, the playback automatically starts in the other screen.

 
4. Time origin and relative clock

Many analysis scenarios involve a specific moment within the video that everything else is related to. A golfer's club-ball impact, a baseball pitcher's release point, a long jumper's take-off, the start of a race, etc. We can now navigate to this precise moment and mark it as the zero point, the origin of all times for the clip. Every other moment will now be relative to this origin, using negative time before the event and positive time after it.

http://www.kinovea.org/screencaps/0.9.1/091-timeorigin.png

A new simple clock tool lets you see relative time directly on the image.

http://www.kinovea.org/screencaps/0.9.1/091-relativeclock.gif

 
5. Annotation importers

We can now import .srt subtitles and OpenPose keypoint files.

OpenPose is a deep learning software stack for human posture recognition. The result of OpenPose 25-point body model is automatically imported into a dedicated custom tool in Kinovea. At this point this is not meant to be used for measurements but more for general posture assessment.

http://www.kinovea.org/screencaps/0.9.1/091-openpose.gif

 

Thanks!
Don't hesitate to post feedback, questions, feature requests, bug reports, either in this thread or in dedicated threads.

Re: Kinovea 0.9.1 (beta)

please add audio playback, then I and a whole community of social science researchers would be able to use it!

Re: Kinovea 0.9.1 (beta)

hi joan, great stuff on the automation. it works flawlessly.

i'm not sure if this is a bug or not but i wanted to bring it to your attention just in case. when i switch from "camera: records real time frames" to "Delayed: records delayed frames" the cpu usage goes from 4-5% up to 34-35% when using two usb cameras running at 640x480 @100fps. in kinovea version 0.8.15 using the same setup, i see around 17% cpu usage.

thank you for all of your hard work.

Re: Kinovea 0.9.1 (beta)

Hi Inorkuo,
I already did a lot of performance analysis of the new version. Depending on the CPU-type the load is different. More Cores and threads decrease the loading a lot. I just work on an load/Bandwidth overview of different processors/camera settings and will publish it the next time. In short CPU/Load (delayed mode) for 1 Camera/1Kinovea:
Pentium Core2: 45%, i5-9600k: 25-35%, i9-9900: 12%

If Kinovea is in delayed mode (for us golfers a must have), it is quasi under full load. If record is pressed, the load will NOT increase any more, it stays at the same level. If in camera mode, the load goes down to below 5%, but if record is pressed, it goes up to the same level as in the delayed mode. Therefore, it doesn't matter if delayed mode is used anyway.

Re: Kinovea 0.9.1 (beta)

thanks reiner,

i was hoping that bc kinovea 0.8.15 does not use as much cpu in delayed mode, there would be a way to decrease the load in 0.0.1. as it is, i can run everything i need on my golf sim computer at around 95% cpu occasionally bumping up to 100% which is ok.

one other problem i had with 0.9.1 is when i save a composite video, it does not use the speed setting like it did in 0.8.15.

Re: Kinovea 0.9.1 (beta)

Hi Inorkuo,
to reduce load you could try several things.
First, reduce the display fps in the settings down to i.e. 1fps. That will give you some percent.

Try to use 2 (TWO) Kinovea instances, each using 1 camera for video acquisition. Under some circumstances I found that using two times 1 camera exclusively to 1 Kinovea has a lower CPU-load than 2  cameras shared to 1 Kinovea.

Regarding the speed it maybe a misbehavior of 2 cameras in 1 Kinovea. When using 1 camera/1kinovea the videos are nicely synchronized.
Using 2 Kinovea for acquisition, also if different cameras are used the new settings of Threshold/Replace can be "misused" to correct individual behavior of each camera. I.e. if acquisition speed is 100fps, Threshold can be set to 80 and replacement to 30. Then, the header of the resulting video contains the value of 30fps replay speed and replays in slow motion (30/100).

Re: Kinovea 0.9.1 (beta)

reducing the display fps down to 1 helps quite a bit with cpu usage.

I am using two kayeton global shutter cameras at 640x480 120fps. in the stable version of kinovea, when you save a single video, kinovea will ask if you want to take slow motion into account. when you save two videos as a composite video, it does not ask, but automatically applies the slow motion setting to the video. hopefully the beta kinovea can be made to work this way.

i'm still not sure what the threshold and replacement settings do but I will play around with it.

there are a few more issues I've run across. in dual playback mode, the "synchronize videos on the current frames" does not work.

the last issue is with automatic playback with two playback screens, when new videos are created, often only one will play and I have to manually press play to get both to play together. it seems that there is a correlation to the difference in the length of the two videos. I have the "stop recording by duration" set to 3 seconds but sometimes, one video will be 3.1s and the other will be 2.5s. it seems that when the difference is small, both videos will play. when the difference is larger, only one will play.

Re: Kinovea 0.9.1 (beta)

inorkuo wrote:

i'm still not sure what the threshold and replacement settings do but I will play around with it.

Typically when capturing a high speed camera, the camera is sending let's say 300 fps, in the final file we don't put 300 fps but something more reasonable like 30 fps so the player can read it without trying to decode at 300 fps which would be too intensive. Physical high speed cameras or phones do the same when saving.

The first line, "Framerate replacement threshold" is the value above which this behavior is active. By default it's 150 fps. Anything under this will keep the value in the final file. Anything above this value will be replaced by something else. I had a hard time with the wording, it's hard to convey the meaning succinctly.

The second line, "Replacement framerate" is the actual framerate that will be written in the video metadata.

The old behavior is that anything above 100 fps was silently converted to 30fps.

inorkuo wrote:

there are a few more issues I've run across. in dual playback mode, the "synchronize videos on the current frames" does not work.

Can you describe how the problem manifests itsedf? What this is supposed to do is create a "time origin" point in each video at the respective current location. Then synchronization works based on these time origin.

- When you hit the button, does it change the timing in individual videos (they should now have negative times before the sync point)?
- Does it work if you move videos to the desired point and set up each individual videos instead, by using "Mark current time as time origin", (button or right click in each video)?

inorkuo wrote:

the last issue is with automatic playback with two playback screens, when new videos are created, often only one will play and I have to manually press play to get both to play together. it seems that there is a correlation to the difference in the length of the two videos. I have the "stop recording by duration" set to 3 seconds but sometimes, one video will be 3.1s and the other will be 2.5s. it seems that when the difference is small, both videos will play. when the difference is larger, only one will play.

Are the camera captured to the same folder? This is not supported in replay at the momentq as it just looks for the most recent one, they should be saved to different folders and each replay screen should be pointed to the respective folder. But even then both screens should load the same video and start playback, not sure what's going on.

One video being 2.5 indicates another issue. Either there are frame drops or the camera isn't sending a true 120 fps.

Re: Kinovea 0.9.1 (beta)

synchronization works if I mark the time origin for each video by right clicking. when I use the synchronize button in the dual player control bar it does the time origin for the left video correctly but the right video time shifts and then the time origin is set.

I have the cameras capturing to different folders. most of the time auto play back works. would the timing of the creation of the videos affect playback?

Re: Kinovea 0.9.1 (beta)

Thanks, I think I can reproduce both issues. The Syncronize button is definitely broken for the right video, this looks systematic, collateral damage of recent changes in this area, will fix asap.

I can also reproduce the auto playback issue by manually recording each camera to force a gap in file creation time. The dual playback needs to be robustified. In theory when the late video starts it should force the other one to move back and they should continue in sync. This seems to work when done manually but not when the videos are started automatically.

There is also a third issue you might encounter from times to times, it may happen that both videos start replaying, but not properly locked and one of them start to drift away after a few iterations.

Re: Kinovea 0.9.1 (beta)

i tried setting the replacement rate to 24 with the threshold set to 30 and it doesn't seem to do what you explained. when i save a composite video the file shows 100fps. when i save each video separately without taking slow motion into account, the files show 120fps. when i save the files separately, taking slow motion into account, the files show 24fps, which makes sense bc 24/120 = .20.

Re: Kinovea 0.9.1 (beta)

The threshold should floor at 60 fps, can you really set it to 30?
To be clear this mechanism is only used by the capture screen. When the video opens in playback, what does the header above the screen say? It should say 24 fps. If both files are showing 24 fps in the header the composite video should also be 24 fps. I'll double check the behavior when the files don't have the same framerate, it's possible there is some trickery that makes it fallback to 100 fps for some reason.

Re: Kinovea 0.9.1 (beta)

Sorry it works for the recording. I was trying to save a composite video from the dual playback screen. I should have known since the setting is under the recording tab.