1 Topic by fajitas

  • fajitas
  • Member
  • Offline

Hi,

I'm trying to programmatically cut videos before using them in Kinovea. That is, I have a Excel/csv file with video filenames and start/stop frames manually established through Kinovea. I want to cut to EXACTLY these frames.

So far ffmpeg (ffmpeg.org) seems to be my best shot. Seems like Kinovea is using it as a building block, too.

My issue is:

1) If I happen to cut to a position which is not a keyframe (aka i-frame), Kinovea can not load the video anymore (screen appears black, yet e.g. VLC can still play it).

2) If I convert the video in ffmpeg to turn all frames into keyframes, the quality drops (in/output is MP4 and I'm not recording). I'm not sure why this is, but I'm new to video editing/encoding. I would have thought, that only file size and playback performance would worsen. I understand, that this is not an ffmpeg forum, but since Kinovea uses it internally, maybe the developers have some insight here?

3) Perhaps I could resolve the quality issue by exporting in ffmpeg to a lossless format. But I have no idea which can be read by Kinovea. I don't think any of those listed in the FAQ (kinovea.org/help/en/002.html) are lossless, correct?

--------
commands used in ffmpeg

1) Cut video unCut.MP4 from 18.44s to 20.44s (video is 100 Hz and times where manually picked in Kinovea (toe off in my case)
> ffmpeg -i unCut.MP4 -ss 00:00:18.440 -to 00:00:20.440 cut.MP4

2) Cut video while making every frame a keyframe (seems to reduce quality)
> ffmpeg -i unCut.MP4 -ss 00:00:18.440 -to 00:00:20.440 -g 1 cut.MP4

2 Reply by joan

  • joan
  • joan
  • Admin
  • Offline

Hi,
I'm no expert in ffmpeg command line. The issue I believe here is that the timestamp of the first frame in the output file is non-zero, possibly negative. Kinovea definitely doesn't like that. There is an attempt at handling these in the next version but I tried your command and I still have the same issue as you.

Maybe there is a way to change the PTS of the output frames. Otherwise I think you'll need to transcode in order to turn that first frame into a keyframe. I'm not sure you can cut at that exact frame without transcoding since the decoder will always miss some data for it and would likely have to start at the next keyframe.

Not sure why the quality would degrade with -g, maybe it's using the same bitrate as in the original but now all frames are fully encoded so it degrades them to keep the bitrate promise.

3 Reply by rkantos (edited by rkantos 2019-10-22 08:28:42)

  • rkantos
  • Member
  • Offline

I don't know how FFMPEG handles time to frame conversions when doing cuts like you described. Based on experience I have with FFMPEG I would recommend using the exact frames. Using codecs like MJPEG should make it possible to cut videos  from any frame to any other frame.

AFAIK as Kinovea uses FFMPEG, it should work with as many formats for playback as FFMPEG does. Of course players like VLC (which is also FFMPEG based) can have some other features that can allow playing files which don't have the correct header / metadata.

4 Reply by joan

  • joan
  • joan
  • Admin
  • Offline

My guess is @fajitas source video isn't MJPEG but H.264 so when asking for a specific time it lands on a non-keyframe, and the output is created with the first frame having missing reconstruction information and possibly a weird decoding timestamp if the file has B-frames (bi-directional prediction, info to rebuild the frame is stored in both adjacent keyframes). This trips Kinovea up. In any case I don't see how to turn this random frame into a full keyframe in the output without transcoding somehow...

In the next version there is a way to import a sequence of images as if it was a video, if they are named correctly like image001.png, image002.png, image003.png, etc. FFMpeg should be able to export that. Maybe an avenue to explore.

5 Reply by tosserberry

  • tosserberry
  • Spambot
  • Offline
rkantos wrote:

I don't know how FFMPEG handles time to frame conversions when doing cuts like you described. Based on experience I have with FFMPEG I would recommend using the exact frames. Using codecs like MJPEG should make it possible to cut videos  from any frame to any other frame.
funny shooter 2
AFAIK as Kinovea uses FFMPEG, it should work with as many formats for playback as FFMPEG does. Of course players like VLC (which is also FFMPEG based) can have some other features that can allow playing files which don't have the correct header / metadata.

If the input file contains B-frames, which are a type of bi-directional prediction in which the information needed to rebuild a frame is contained in both adjacent keyframes, the output will have a first frame that is corrupted with missing reconstruction information and perhaps an odd decoding timestamp.

6 Reply by Reiner

  • Reiner
  • Member
  • Offline

The native video format Kinovea uses is a format (codec) that contains a full image for every frame. This can be mjpeg or rawvideo, whereby all compressed formats are using the mjpeg codec.
The compressed format can be stored as mp4, mkv or avi container.
The uncompressed format can be stored as mkv or avi container (the mp4 container always uses compressed images)
All these formats have a full image stored in each single video frame that is perfect in doing video observation and editing frame by frame. Cutting the video is simple due to the existence of a full images in every frames.

However, some video programs (like windows media player) cannot replay these formats (codecs), if not an appropriate codec is installed separately.

In your case, the video format (or better named codec) is H.264 in a mp4 container. This common codec can be replayed by most of all video players. However, it contains K-frames, having a full image and B-frames in between, not containing a full image.

Therefore, a video with a H.264 codec cannot be cut at any position without having black frames. It has to be re-encoded to get a clean output video.

The quality of the output video depends on the parameters you are using during re-encoding. If they are not specified, ffmpeg uses default values (about medium quality).

Every frame in a video (full frame like mjpeg, K-frame or B-frame) contains a PTS (presentation time stamp) that determines the position of this specific frame in the video line. The first frame contains the value 0 (under standard conditions).
If you cut or trim a video starting after the first frame, the first frame in the output video contains a value of non 0. Therefore, you have to correct the PTS during the trimming procedure. If you omit this step, the video program often does not know when to start the video. Kinovea expects the first frame to have a PTS as 0. VLC may recognize the non 0 value and corrects it.

To control the quality of the output, parameters may be used depending on the type of codec.
If the target should be H.264 codec (mp4 container), the -crf (value) and -preset (value) parameters are appropriate
-crf 0-51: 0 is the best, 51 is the lowest quality
-preset (value) has a literal parameter like fast, slow,…
a list can be found at https://trac.ffmpeg.org/wiki/Encode/H.264

I’ve never used the -g (value) parameters. You might use -g 0 to prevent any B-frames between K-frames. But I think it is more consistent to either use the H.264 or the mjpeg codec.


In the following examples, a segment of a video is cut from position 0.5 to 0.8 second

Example of creating a "good“ quality output video with the H.264 codec might be:
FFmpeg -i Infile.mp4 -ss 0.5 -to 0.8 -c:v libx264 -crf 5 -preset slow outfile.mp4

If the target should be a mjpeg codec (mp4 container), the -crf and -preset may not be used. Instead use
-q:v (value), value ranging from 2-32, 2: highest quality, 32: lowest quality
Example of creating a "good“ quality output video with the mjpeg codec might be:
FFmpeg -i infile.mp4 -ss 0.5 -to 0.8 -c:v mjpeg -q:v 5 outfile.mp4

Cutting a video at an exact position:
If the source video contains the mjpeg codec, the start/end/duration time position can be used without re-encoding:
FFmpeg -i Infile.mp4 -ss 0.5 -to 0.8 outfile.mp4
or better
FFmpeg -i infile.mp4 -ss 0.5 -to 0.8 -c:v copy outfile.mp4

(the "copy" parameters prevents re-encoding)

Influence of time position on the exactness:
If the start position is used before the infile, ffmpeg uses an approximation to find the start position, but it is faster:
FFmpeg -ss 0.5 -i infile.mp4 -to 0.8 …
If the start position is used after the infile, ffmpeg uses a more exact frame position to find the start position, but is somehow slower:
FFmpeg -i infile.mp4 -ss 0.5 -to 0.8 …


To create a video having a correct PTS-value of 0 for the first frame, a video-filter has to be inserted (re-encoding is mandatory)
FFmpeg -infile.mp4 -ss 0.5 -to 0.8 -vf "setpts=PTS-STARTPTS“ -c:v mjpeg -q:v 5 outfile.mp4

To tell ffmpeg the frame rate of the invideo, -r (frame rate) may be used before the infile, specifying the frame rate of the infile
FFmpeg -r 100 -i infile.mp4 …

However, you should be sure that this value is correct.
You might determine it using the ffprobe- program that comes with ffmpeg, creating a batch-file (i.e. ffprobe2.bat)

@echo off
<path to ffprobe>\ffprobe.exe -v error -show_format -show_streams %1
pause

Then call it at the command line:
ffprobe2 infile.mp4
and it displays the metadata of the infile.mp4



My recommendation in your cases could be the following:

high quality (not highest) H.264 target video:
ffmpeg -i unCut.MP4 -ss 18.440 -to 20.440 -vf "setpts=PTS-STARTPTS"  -c:v libx264 -crf 3 -preset slow cut.MP4

high quality mjpeg target video:
ffmpeg -i unCut.MP4 -ss 18.440 -to 20.440 -vf "setpts=PTS-STARTPTS"  -c:v mjpeg -q:v 5  cut.MP4


REMARK:
Only use Quote ABOVE for the Videofilter.

Hope that helps