Ok, rendering on an external form seems good, with smooth playback with HD videos which before were issues. However it does jump now and then (it's not clean the code as it was a hack around).
It sounds super promising! ![]()
It would be nice to have some instrumentation on this, monitoring how many ms are spent on the rendering when using DX and when using GDI+.
One thing which I think must be changed is the use of the invoking the paint method on the picture box. This is not a fast method. It would be much better to call this function directly (and maybe from an external loop). Going through the operating system/control seems like a possible performance issue.
Yes, you're probably right. I don't quite remember the rationale for always going through the system's paint event. It's possible that it was simply the most convenient way to get a reference on the Graphic object we are painting on.
The timer itself runs in another thread, so asking for the next frame is done asynchronously and I don't think this can change. However, when we have the frame in question, rendering it on the screen can (should) be triggerred right away. As you point, no need to go through the Windows message queue, that can only slow things down.
I have a feeling the only way it will really work well (like high performance is rewriting this control). That way frames are extracted straight into the working canvas memory, and drawn from there. Other drawings on top are added to a list, which adds them to the rendering line to place on top of the image as it is renders. Maybe when you have more than one video, they should be rendered on the same surface (even if they have separate controls.
Does this sound right?
I'm not sure I followed everything, if I understood correctly you suggest extracting the frames from FFMPeg directly onto the canvas / graphic card memory, instead of using an intermediate Bitmap object. (?).
I don't know how this would work with the working zone concept. Currently the user may apply some image filters like "Auto Contrast" or "Sharpen" for example. These filters are applied on pre-extracted copies of the frames, to speed things up and add flexibility (combining filters, etc.).
I'm not sure we would be able to make the whole thing fast enough to allow for these filters to be applied in real time and still play smoothly…
The working zone and its pre-extraction of frames helps in many other regards too. For example, you can jump straight to any frame without having to decode all the ones in between (this wouldn't be quite possible without pre-extraction, we would have to seek to the closest keyframe (in the video coding sense) and then start decoding from there).
Also, some effects will always need to have a list of frames in memory anyway, e.g. the "overview" and "reverse" functions, and in the future the more complex special effects like chronophotography, etc. So far this model has proven very convenient.

