Dropped Frames in DV

[jeff b]

To those of us who are accustomed to what happens when frames drop during capture with analog video, switching over to DV has an insidious little surprise for you: there's never a dropped frame during capture.

'Wow,' I said to myself...

Later on, at the final stage of a recent project, when I sent my work back out the IEEE-1394 port to the camcorder, I found little annoying 'snags' that were hard to figure out. On the one hand, it seemed like a frame might be missing. But on the other hand, there were no dropped frames during capture, and the 'analyze video' in MSP showed no missing frames on either the source clips or on the finished clip.

Curiouser and curiouser...

At this point, I began a three day system tweaking safari. I did anything and everything I ever did to make it smooth with my Rainbow Runner. Damn if my finished project didn't still have those little annoying 'snags' in a couple of spots.

Upon really REALLY close examination, looking at each individual frame in MSP on the timeline, I finally discovered what happens: instead of a frame missing or coming out with half the interlaced field missing ('evil frame'), I had a couple of spots where a frame WAS missing, but was replaced by the frame before it!

As it turned out, this happened during capture with the drive running with write-behind cache. This was a new WD 30.7gig drive, too!

So, I just thought I'd share that little tidbit for any of you who might be going the same route... the same system activities that used to bite into captures on analog systems can easily apply to DV, too.

[Pertti]

Jeff,

What drivers were you using, and did you use the correct field order (Field order B), through out the prosess ?

[jeff b]

The drivers are the latest update from the MSP6 install disk. With DV capture, there's no option that changes the captured DV file from its native format. Any of the options for frame size, frame rate, etc., are all greyed out when the capture plug-in is active for DV.

This frame skip/previous frame dup phenomenon was definitely an IEEE-1394 to hard disk throughput interruption problem, as the same clip off the camcorder captures without a hitch, now. I thought that I might have had a soft spot on the tape, actually, and recaptured suspect areas of the origninal clip four times, just to see if it would happen again, but it never did after I dropped the write behind caching on my system.

[Michel Carleer]

It is well known that for video capture, write behind caching should be disabled. With write behind on, when the cache is full it is suddenly flushed to disk in one step. If the cache is big, writing to disk will take too much time and dropped frames will happen.
This does not mean that write behind caching is less effective for normal burst work, it simply means it does not work correctly with a constant data stream where the PC has to constantly be ready to receive data. Which is not the case during cache flushing.
By the way, there are two different types of dropped frames: if the capture driver does not have enough time to handle (convert, compress, ...) a frame, but sufficient time to copy the previous frame, then you still have the correct number of frames in your movie and the capture software does not tell you about it. If on the other hand the program does not have time to even copy the previous frame, then frames will be missing and are be reported as dropped frames. At least that's how I understand things from my own experience and a bit of reading in the Video for Windows and Multimedia developpers documents.
Michka

[jeff b]

Well, now, I thought I had this one solved, but the subtleties to spotting seemingly random glitches or frame drops is never evident until the whole project goes back out the 1394 port to the camcorder.

It turns out that I can enable write behind cache, and read ahead optimization to my heart's content. The real problem appears to be in the MSP6 DV rendering engine. Every 1000 frames, the difference between 29.97fps and 30fps will be one frame. Bear this in mind as I continue...

Captured clips from VideoStudio4, MSP6, and DVIO can all be exported right back out to the camcorder via 1394 using either of the three pgms mentioned. Absolutely no problems. Absolutely beautiful, high quality video and audio, faithful to the last frame.

MSP6 created files for DV, however, will have frame 998 duplicated at frame 999, and this phenomenon recurs every 1000 frames.

My view is that if my Sony TRV-900 was exporting 30fps over the 1394 port, and MSP6 was rendering 29.97, then I'd simply have a missing frame every 33-1/3 seconds. Conversely, the effect of taking a 29.97fps file and rendering it out for 30fps, however, would have the effect of needing one additional frame every 33-1/3 seconds, which is exactly what I'm getting here.

Mediaplayer tends to smooth this over, but there is still an audio glitch. Going out over the 1394 port back to the camcorder, however, this is very noticable whenever it happens with enough motion at that spot. The audio glitch is also quite noticable as long as there's a steady audio at that point.

I've been able to faithfully duplicate this over and over with all the possible combinations of properties available to render out a DV file with MSP6, and have come to the conclusion that this is a bug in the rendering engine, or else it's a matter of Microsoft throwing a curve somewhere.

I do notice that DV files are not yielding any frame rate or audio rate info from within Windows Explorer. Mediaplayer 6.4, however, reports 'statistics' for all my DV files as having a frame rate of 30, whether captured or rendered, and then an 'actual achieved' (or something like that) of some crapped out calculation that ends up being 29.58 or 29.99, etc., depending on what file I look at.

If anyone's interested in pursuing some experiments with their own 1394 and MSP6 setups, it might yield different results that I would be interested in hearing about here in this thread. Meanwhile, I guess I'll just have to break down and do a clean install of my system before I attempt to get to the bottom of this.

The experiment goes thus: capture a couple of minutes of moderate to high action mini-DV via 1394. Play the raw clip back out to make sure that there are no skips or problems. Open MSP6, import the clip, and render out a new file with 'create'. Export the new file and watch for a glitch every 33-1/3 seconds (1000 frames). If you have to, edit the clip so that you know exactly where that first 33-1/3 second point is going to show up, and make sure that there's enough motion and audio to notice the phenomenon.

I haven't been able to see anyone posting on this problem here or anywhere else, so it may just be a system oddity on this end, thus the re-install. The projects I've done so far have been different from the one that really made this problem evident. I have a bunch of clips of a river that I shot last week, constantly panning with the roaring river noise as a constant on the audio track. Where lack of noise or lack of motion probably hid this phenomenon in previous projects, this one really showed it quite clearly.

[dgcom]

Hi!

As I correctly recall, the 'audio glitch every ~30 sec" problem was solved by MS about 3 month ago... Just check the version of qdv.dll you have.

It should be 6.1.5.321

If not, request fix Q243174 from MS...
It is also on MSPro 6 CD-ROM in "DV Patch" folder. File name is 243174UP.EXE

Try it... Of cause, it may be different issue...

DGCom

[Michel Carleer]

Yes Jeff b, that's a subtle one. As I am in Europe, I don't know a lot about this 29.97/30 fps problem: when should you use 29.97, when 30? Do TV broadcasters send at 29.97 or 30?
Anyway, I still believe write-behind caching should better be disabled on your A/V disk. Read-ahead has no reverse effect. Well, it probably does not really matter with a fast PC, but for the ones at the limit... And that's only valid for real time capture of course, not for editing.
Michka

[jeff b]

Dgcom: the dv patch from the MSP6 CD is installed on my system. The problem isn't just an audio glitch, it's a frame being duplicated every 1000 frames, as described above.

[dgcom]

How did you find that frame is duplicated?

So, you sure you have that version of qdv.dll?

DGCom

[jeff b]

Dgcom: The qdv.dll I have is in the windows/system subdirectory, 210kb dated 7/15/99. I'll have to see if I can obtain a more recent copy of that file. Apparently, the update on the MSP6 won't put anything more recent there, as I've run that update again and it didn't change. If the file is supposed to only be about 3 months old, obviously I don't have the latest copy.

Finding the duplicated frame is done by putting the clip on the MSP6 VEditor timeline with the view set to show all frames, and set to show every frame at a large enough size to inspect. When you know that the duplicated frame will occur every 1000 frames, of course, it's very easy to find.

[jeff b]

Dgcom: The Q243174 document describes a different problem dealing with the way audio is handled on a small number of applications, particularly with DVD. I don't have that problem.

[dgcom]

Hi!

That's exactly the problem you have.

First, you have to make sure the files you have match those listed in Q243174 KB article.

It may happen, that 243174UP.EXE won't update them. You need to unpack it (use "243174UP.EXE /c" from the command line) and replace files manually.

Try it. The MS DV codec in this update is sure the last edition (size 214,288, date 10/1/99). It is not exactly 3 month, but close to the date they released patch...

If you can't get it, I'll email files to you.

Let me know.

DGCom

P.S. MS was always hard in describing their own bugs... There is nothing to do with DVD in updated DV codec!
(And to check file version - right-click on it in explorer, select properties and go to Version tab)

[jeff b]

Thank you VERY much, dgcom!

Apparently, the update won't overwrite qdv.dll or qcap.dll if they exist. They have to be deleted first, then run the update. Also, just copying them manually won't update the registry.

[Keith Wiebe]

I don't have the dreaded 33.3 sec. hickup with my Raptor! I heard that this was a problem with ADS Pyro in the early stages. Get the Raptor, most of your problems will go away! The cheap-end firewire cards just aren't there yet! They probably will be eventually.

[dgcom]

jeff b: You are always welcome...


Keith: I'd better agree with Jeff (see his other post).

The problem here is an absence of good info/support. Each company wants to get it's own piece of "blanket" - consumer DV market.
Before introduction of OHCI-compatable boards and Microsoft Firewire stack, each company was making huge profit by selling relatively low-cost board and high-priced software bandle turned to work with it. Now, imagine, everybody is FREE to choose hardware and software. And if hardware is the same and priced about $70, then the main war is for software. And if particular company can't manage to write good, reliable soft - who want to buy it? That's right. Nobody.
Result: All low-cost consumer DV solutions are trying to keep their bandles - StudioDV works with their "own" card only (they are checking for paticular PCI vendor ID, {not true anymore, starting with 1.03}), ADS checks for presence of "their own" card during updates installation...
Who should provide software and updates? Microsoft? They are too huge to manage all of it... Thank you Ulead! the only company by now (shame on Adobe) to support this ocean of OHCI-compatable boards in PRO video-editing tool. And support for Windows 2000 - that's the second "thank you".

I'm sure, at the end of this year, all this war would be remembered by early OHCI/Pyro/Ulead adopters...

Will see.

DGCom

P.S. In no way I wanted to make Canopus look bad here - even they managed to make OHCI-compatable starter's solution. That's right - starter's! You want to become pro - learn something and go get MS Pro 6...
By that time, software DV codec will be the same speed as hardware. And, tell you what - software solution is ALWAYS more flexible than hardware...



[This message has been edited by dgcom (edited 14 March 2000).]