Yes the frame shift most likely depends on the format. Or even the encoder in the camera. If you want you can send me a file with the problem so that I can see if it's a bug that could be fixed somehow. Less than 5MB send to joan at kinovea dot org, if more than that host it somewhere else and send me the link.
Regarding filtering, if it's possible, I would still suggest to test a digitization of a file for which you have ground truth available if possible. Ideally coming from a physical measurement system, not from another optical based system. The filtering helps smoothing the minuscule noise introduced by the manual or automated tracking process, where even subpixel placement at 600% zoom might not be enough to get the correct coordinates. I would assume this to be universally beneficial for precision/repeatability.
Note that the radial distortion calibration will also not be perfect, and usually less accurate at the periphery. The tracking works only in 2D so deviation from the the plane of motion is also going to add errors. If you are computing derivatives the noise is going to increase the error. If you compute or save acceleration data for example, I would definitely try to evaluate the accuracy first, to know where you are standing.