811 Reply by joan

(9 replies, posted in Bug reports)

Ooh, very interesting :-)

I am deep into the capture code these days so this case is of utmost interest.

Basically it should work like this :
1. The camera driver exposes a number of formats that it supports, usually what the camera can source.
2. The capture code creates a "Sample Grabber" specifying format RGB24.
3. The capture code calls DirectShow to connect the camera driver to the sample grabber, and DirectShow adds the necessary converters inbetween (the path to go from source to destination might have several steps).
4. During capture, samples are grabbed in RGB24, a Bitmap object is created with the sample inside, and the bitmap is sent downstream to the rest of the code.

In 1, the source format from the driver can be RGB24 already, but usually it's in some form of YUV format like YUY2, I420 (a list here). It may also be in a compressed format if the camera has on-board compression (MJPEG or H.264).

What might happen is that DirectShow cannot find the proper color space converter to go from the YUV format to RGB24, but normally if there is no match the connection fails entirely.
Another possibility is that it can find the proper color conversion chain but somewhere something is interpreting the source wrongly.

I think I still have a working PS3Eye, I'll also see if I can find something.
I don't know if the CodeLabs driver is based on the Microsoft UVC driver like for most cameras, I'll check that later.

If you want to start to investigate, you can download GraphStudioNext. Add your camera and look at the properties of the output pin.

Go to forum: Bug reports

812 Reply by joan

(12 replies, posted in Cameras and hardware)

I also need to investigate these capture cards more. It seems the gaming market is creating a new demand for them which could drive the prices down.

If the use-case is display only, like delayed playback for example, it might be interesting.

If you want to record to disk it's more difficult, I'm afraid at the moment Kinovea isn't quite capable of recording full HD to disk in real time without dropping frames. Using the on-camera storage is a more appropriate option for "record" scenario.

Go to forum: Cameras and hardware

813 Reply by joan

(27 replies, posted in Cameras and hardware)

Some more experimentations on my side…

Drivers
On Windows 7 there are two capable drivers giving slightly different options.
The Microsoft driver that installs itself by default, and the Logitech driver that must be installed manually.

The MS driver provides the stream in YUY2 (an uncompressed format), MJPG and H264.
The Logitech driver provides RGB24, I420 (two uncompressed formats), MJPG and H264. (These options will be available for selection in the next version). I haven't yet found what the actual native uncompressed format of the camera is.

However there is a catch, the official Logitech driver provides the H.264 stream in an unusual way (on a secondary pin of the DirectShow filter), which makes it invisible to Kinovea and to the majority of DirectShow based capture applications.

On the upside, when using the Logitech driver, it is possible to get and set the exposure duration with 100µs granularity, with a lowest value at 300µs.

Both drivers lie to some degree with the list of framerates they support. For example when using MJPG output, the drivers report that they support 7 framerates, from 5 to 30 fps, but whatever we choose, it will always use 30 fps.

DirectShow internals
When we connect to the camera, there is some black magic done by DirectShow to find a suitable decoder for the camera output down to the application. DirectShow will search the installed codecs on the computer and look for the best ones to complete the connection.

This means that when the camera is outputting H264 for example, there is a specific codec that is selected to provide the decoding. It may be a third party codec installed as part of another application or one provided with Windows. The performance of this codec has a direct impact on performance and image quality.

The problem is that codecs register themselves with a "merit" value that changes their likelihood of being selected.
More tests are needed to see if it would be practical to try to bypass this system completely or find a set of known-to-work-well codecs for various camera output formats, or at least detect which codec ends up being used.

Go to forum: Cameras and hardware

814 Reply by joan

(27 replies, posted in Cameras and hardware)

So I got my C920 earlier this week and I've started experimenting with it. Let's make this the official Logitech C920 thread, and everyone should post their findings or issues with this camera. I'll keep in mind the holy grail of the thread, being able to record and preview two C920 at their full 1920×1080 @ 30 fps, simultaneously.

1. Exposure time.
The camera has a setting for exposure duration which is great. Extremely important for sport video as we already discussed in the forum. Combined with the HD frame size and the 30 fps it makes it a very interesting tool indeed.

To change the exposure time you have to go into the device property pages and to Camera control. I think the Logitech software also has a page where you can change it.

The mapping between the settings and actual durations is not published by Logitech though, and there are conflicting informations between the DirectShow spec and Logitech. They do have a lower level API where it's possible to set the duration by increments of 100µs, so I'll try to use that at some point in the future.

2. Framerate auto adjustment.
If you leave the exposure control on automatic, the exposure duration will adjust to the light conditions. As the exposure and framerate are partly interdependent, it may also alter the framerate. You don't want this.

Especially if you are indoors, the exposure will set itself to longer values, automatically decreasing the framerate.
To get the full 30 fps you need to set exposure on manual (and maybe get powerful artificial lights).

There seems to be a bug in the camera driver as it forgets the settings when you restart the streaming though.

3. Support in Kinovea
The list of available image sizes / framerates is somewhat limited in the current versions of Kinovea. I have started to change the code to get the full range of capabilities.

The camera has on-board compression and can stream to H.264 and MJPEG in addition to uncompressed.
At the moment Kinovea only exposes the configurations options corresponding to the H.264 stream due to some underlying limitation that I'm working on lifting.

USB 2.0 doesn't have the bandwidth for uncompressed full HD @ 30 fps, so getting the compressed stream is the only way to get the top score. However it means that the images have to be uncompressed on the computer to be displayed and then recompressed to save to disk. I'll later see if it's possible to directly save the MJPEG stream to disk.

Go to forum: Cameras and hardware

815 Reply by joan

(13 replies, posted in Cameras and hardware)

Follow-up:
After some testing and off-forum communication, the issue should be resolved in the next version of Kinovea.

Go to forum: Cameras and hardware

816 Reply by joan

(13 replies, posted in Cameras and hardware)

Hum, Point Grey have just changed their web site so most Google links are dead. It has become harder to find relevant documentation.

Anyway, in order to understand the problem better I would be interested in the following:
- Do you get the possibility to configure the stream size/framerate in V1 ?
- Could you download Graph Studio Next. It is an Open Source DirectShow utility working at a lower level.

1. Go to Graph > Insert Video Source > FlyCapture2 Camera.
It should add a box for the camera on the main panel. (At that point it crashes for me. The DirectShow filter apparently expect that a camera is actually attached).
2. How many capture pins are there on the right side of the box (little squares protruding) ?
3. If you right click the little capture pin and choose Properties, do you get the stream configuration tab (where you can change video format, image size and framerate) in the resulting dialog ?
4. And if you go to the Interfaces tab in this dialog, do you see the IAMStreamConfig entry ?

Thanks

Go to forum: Cameras and hardware

817 Reply by joan

(13 replies, posted in Cameras and hardware)

Yes I will make some experiments and get back to you for testing.

Go to forum: Cameras and hardware

818 Reply by joan

(27 replies, posted in Cameras and hardware)

Thanks for the details.
In the lab I was on they used USB-to-Ethernet extension cables so most of the wiring was inside the walls. It's another alternative for indoor settings.

Regarding multi-core processors it's definitely a game-changer. There are at least 3 intensive threads running in parallel: the image production in the camera driver/grabber, the display of images in the UI thread, and the compression/storage to disk in another background thread. If the CPU has to block any of these to run the others, performances will suffer.

I am making progress on a new architecture that will allow me to better control how everything works together at a low level and making sure the stream of images flows down to the disk with maximal fluidity.

Go to forum: Cameras and hardware

819 Reply by joan

(27 replies, posted in Cameras and hardware)

I spent a few hours on a computer with two Logitech C920 today.

I'm actually surprised you got this far :-)
On the system I tested the highest resolution available was 640x480 and the two cameras couldn't be active simultaneously, only one or the other.

This camera seems to be the current best-seller so I might buy one myself to make more tests and see if its support in Kinovea can be improved.

From what I can gather the higher resolutions are coming down a different channel. There is on-camera compression to H.264 (USB 2.0 bandwidth is about 35 MB/s which is not enough to transmit raw HD frames at 30 fps). Some softwares like Skype must be able to directly consume this H.264 stream. Whereas other softwares handle frames that have been uncompressed by some other underlying component.

Go to forum: Cameras and hardware

820 Reply by joan

(2 replies, posted in Ideas and feature requests)

I'll probably show my utter incompetence in Apple devices, but can you plug it in USB storage mode? When I use this option on an Android device I can browse the content from Kinovea file explorer.
Do you use a special software to copy the files or just Windows Explorer ?

Go to forum: Ideas and feature requests

821 Reply by joan

(2 replies, posted in Ideas and feature requests)

Hi,
The F5 shortcut key is bound to this functionality. As in Windows Explorer or Web browsers.

Go to forum: Ideas and feature requests

822 Topic by joan

(1 replies, posted in Français)

Version expérimentale, merci de remonter toutes les regressions éventuelles !

Installeur : Kinovea.Setup.0.8.23.exe
Version portable : Kinovea.Portable.0.8.23.exe

Le topic annonce sur le forum anglais avec tous les détails!

Go to forum: Français

823 Topic by joan

(2 replies, posted in General)

Experimental version. As always feedback is very appreciated! wink
Beware of regressions and report anything suspicious. Do not assume the issue is known.

Installer: Kinovea.Setup.0.8.23.exe
Portable version: Kinovea.Portable.0.8.23.exe

There are four important aspects to 0.8.23:

  1. A new "portable" version.

  2. Lens distortion correction.

  3. Kinematics plot for trajectories and scatter plot for markers.

  4. Polyline, arrows, and styling improvements.


1. Portable version
The portable version is an autonomous Kinovea that doesn't need to be installed in "Program Files".

When you double click the downloaded executable it will extract itself in a sub-directory of the current directory. You can then copy that new directory wherever you want, on a USB stick for example. To run, locate and start "Kinovea.exe" inside this directory.

This is nice for testing new releases or to carry Kinovea in your pocket.
It uses its own local preferences so you can run it in parallel to the installed ones without interferences.

2. Lens distortion correction
This new feature lets you calibrate your camera's lens distortion. Kinovea will then take that distortion into account for all coordinates and measurements.
This allows the processing of videos filmed with GoPro cameras for example.

There are more information on this feature in this topic, and I'll post a tutorial on how to perform the calibration on the blog in a few days.
http://www.kinovea.org/screencaps/0.8.x/lensdistortion/distortedsquash.jpg


3. Data analysis
A new "Data analysis" menu is available from the trajectory and the point/marker drawing context menus.
This opens a dialog window with plots and data export utilities.

For the Trajectory tool, it creates a time-series plot of a kinematic quantity (position, total distance, velocities, accelerations, etc.) You can change the plot from the "Data source" combo box in the upper right corner. The about tab explains the data filtering algorithms used.
http://www.kinovea.org/screencaps/0.8.x/dataanalysis/dataanalysis-traj.jpg

For the marker tool, it creates a scatter plot of all markers added in the video.
Combined with plane calibration, this can be used to map ball impacts, player locations, etc.
Here is a short tennis point with mapped ball impacts.
http://www.kinovea.org/screencaps/0.8.x/dataanalysis/dataanalysis-points.jpg

Both these windows have export options on the right. The save to file option will save a .csv file that you can open back in a spreadsheet application.

You will notice that the values exported do not exactly match the ones exported by the "File > Export to spreadsheet" menu. The latter do not use any filtering and are subject to noise. If you do not intend to perform your own filtering, you should always use the export function from within the data analysis windows.

4. Polyline, arrows and color profiles

Several changes related to drawing tools have also been introduced:

  • Some of the tools that were under the "custom tools" button have been relocated to more appropriate places.

  • The custom tools can now be individually styled from the color profile dialog.

  • The new polyline tool lets you draw a series of connected segments.

  • Several variations of arrow tools are now available under the arrow button.

http://www.kinovea.org/screencaps/0.8.x/arrows.png

The raw changelog is here.

Enjoy!

Go to forum: General

824 Reply by joan

(6 replies, posted in Cameras and hardware)

Yes I'll definitely come back to you for testing help, thanks.

Go to forum: Cameras and hardware

825 Reply by joan

(6 replies, posted in Cameras and hardware)

The issue with the capture side of things is that there is many different hardware and I can't test most of them.

I am starting to redesign the capture pipeline to integrate benchmarking capabilities and find where the bottlenecks are. At the moment I am just at the "measurability" stage. Trying to take every detail in account.

I want to answer things like:
- What is the the actual grabbing framerate ?
- Is it stable or does it varies widely ? What is its standard deviation ? Median ? 95th percentile ?
- What is the sustainable recording framerate ?
- What is the sustainable display framerate ?
- Does it matter if delayed-display is active ? How large should the buffers be ?
- Should we degrade the display framerate while recording, by how much ?
- What is the best tradeoff between compressing the images (takes CPU time) and not compressing (takes I/O bandwidth) for the current capture configuration ?
- What is the fastest method to write sequential data to the disk ?
- Are the bottlenecks the same between an HDD and an SSD ?
- etc.

I am considering the option of storing captured images to a dedicated format, optimized for fast writing, rather than going through a classic video format as is done currently.
I have a monochrome Basler camera capable of 2048x1084 @ 50fps. Uncompressed, this is about 106 MB/s. The goal is to fully support that kind of bandwidth, and then up from there.

Go to forum: Cameras and hardware