why AVI files with arbitrary headers ?!?

[ionutz]

I have aquired some clips from the VCR with the Marvel G400. Now I want to put them toghether. However, the AVI files are having different AVIFILEINFO informations. Some files have a dwScale of 333670, others of
333672, 333673 or 333667. They are all close to the NTSC 29.97 fps, but they are not identical.

I have 8 files and I want to get only one big AVI with different encoding for audio and video.

I have started making my own program for this (using VfW). The program can play in the present these files and I can easily select the frames I want to keep from each individual file.

The problem is that the files have good audio/video sycronisation and I am afraid I will loose this syncronisation if I simply copy audio and video segments but force them to have a single (mean) dwScale. How can I
glue toghether these files without loosing audio/video syncronisation ?

And why the Marvel saves files with different settings when I requested vidtools to use the same size/fps/compression ?

Am I forgetting something ?

TIA,

Ionutz

PS: VirtualDub also freaks out when I try to append these files. I guess this happenes for the same reason :-(

[Walrus]

ionutz:
I was having a conversation with Avery Lee about this very issue. The problem is that PCVCR drops frames, and it's these frame drops that are varying the frame rates. Try to capture using virtualdub to get around these issues. As for how to fix the files... In theory if you could add extra duplicate frames into the streams to make them exactly 29.970 that would do the trick, though I don't know if it's worth the effort. Since the difference between your files and being 29.970 is so small, you may just want to manually change the frame rates to 29.970 and then stretch the wave file using a cool edit or something to match the length of the file while staying at 44100. This may cause A/V synch to vary by a frame or so from time to time in the middle of the file, but it should keep the overall synch very close to perfect. (being off by a frame is usually hard to see since we're dealing with being off by very small fractions of a second)

[ionutz]

Hi,

Here are some better info about the problematic avis:

<pre>
FILE PR.00.AVI PR.00_01.AVI PR.01.AVI
---------------------------------------------------------------------
AVIFILEINFO

dwFlags 0 0 0
dwCaps 3 3 3
dwStreams 2 2 2
dwSuggestedBufferSize 0 0 0
dwWidth 352 352 352
dwHeight 240 240 240
dwScale 33373 33370 33372
dwRate 1000000 1000000 1000000
dwLength 51005 3166 52037
dwEditCount 0 0 0
szFileType "AVI Default File Handler"

---------------------------------------------------------------------
AVISTREAMINFO (audio)

fccType 1935963489 1935963489 1935963489
fccHandler 0 0 0
dwFlags 0 0 0
dwCaps 0 0 0
wPriority 0 0 0
wLanguage 0 0 0
dwScale 4 4 4
dwRate 88200 88200 88200
dwStart 0 0 0
dwLength 37533440 232960 3829480
dwInitialFrame 0 0 0
dwSuggestedBufferSize 44536 44536 44536
dwQuality 4294967295 4294967295 4294967295
dwSampleSize 4 4 4
rcFrame 006CF710 006CF710 006CF710
...

---------------------------------------------------------------------
AVISTREAMINFO (video)

fccType 1935960438 1935960438 1935960438
fccHandler 828534116 828534116 828534116
dwFlags 0 0 0
dwCaps 0 0 0
wPriority 0 0 0
wLanguage 0 0 0
dwScale 33373 33370 33372
dwRate 1000000 1000000 1000000
dwStart 0 0 0
dwLength 51005 3166 52037
dwInitialFrame 0 0 0
dwSuggestedBufferSize 49812 47856 48348
dwQuality 4294967295 4294967295 4294967295
dwSampleSize 0 0 0
...

---------------------------------------------------------------------
</pre>

Now the question is: is VirtualDub going to behave better than the Matrox software ? Is it going to aquire without dropping frames ?

From what I've seen, VirtualDub was not very stable with my Marvel when acquiring...

Using Cooledit is not a good solution: what I want is to get everything in one program.

This is way I've started doing my own. Now I can display the MJPEGs pretty nice (I think nicer than VirtualDub ). The only problem is that I still get dropped frames (on play) if the size of the AVI is too big. For 352x240, I get all frames, but for 704x480 I get only 1/2 of the frames. I think this is a problem with the size as this doesn't change with the compression level from the AVI. One of my TODO points is using DrawDib functions instead of normal DIB functions.

The compression is not done yet, because of the AVI header problem. Most probably, I'll start by copying the compression routines from VirtualDub. My program is going to be also GPL. If you want to see it, mail me, but in the present it is just a big mess . I use BCB3 to develop it.

Ionutz

[This message has been edited by ionutz (edited 19 April 2000).]

[Walrus]

ionutz: I should be more careful in the future to read your original post more carefully. VirtualDub (especially the latest versions) is actually quite good at capturing with VERY few lost frames. I also have a Marvel (G200) and I lose frames at a rate less than 1 per 1000 (@ 29.970 fps) which I consider more than adequate. A/V synch stays perfect even for very long captures.
The cool thing is it's pretty good for handling the lost frames and not losing synch because of it. You can even capture to multiple AVIs perfectly well to get beyond the 2gb limit. As for lost frames issues.. I know AVI_IO and VirtualDub disable a lot of Windows built-in caching and do the IO on a relativley low level. Anyways, VirtualDub is GNU so you're free to analyze any of its code. That said, Avery Lee is a very nice guy, and its always nice to not have to reinvent the wheel. You may want to get in contact with him about trying to modify VirtualDub's display and capture functionality to better handle your situation. I *THINK* that the problems with the source distribution of VirtualDub have been fixed so you no longer need MASM 6.14 to compile the code. However, you do need Visual C++ 6.0 (with SP3) no matter what to compile VirtualDub. (I will give you credit for using a better compiler
Still, I would be glad to offer you help in your project as well. I think I have a copy of BC++ Builder laying around (I purchased an educational license last year, and never ended up writing much with it since)
Anyways, I wish you the best of luck no matter what path you decide to follow.

[Walrus]

One more thing:
My personal solution to this problem is to frame server my processed data from VirtualDub to bbMPEG from my multiple files. This solution works perfectly well for me, but is far from ideal. (I think I have 7 frame servers working right now on this computer

[ionutz]

Hi,

Thanks for the additional infos.

I'll try to make a zip with my app one of these days and post it on the net.

The app is some kind of a bastard of "Peck's Power AVI joiner" (or what the name was) and VirtualDub.

The main interface will be similar with that of Peck's, letting you select the AVI files to join. It'll look nicer (at least this is what I think B) ). You'll have similar capabilities to set the mark in & out (when viewing individual files) and to view the whole joined stuff.

One thing already implemented is that opening/previewing files works faster with my program: I think it is related on when the compressor manager is called (and I call it on early times and release it very late).

From the VirtualDub I want to take the transcoding stuff. This is not present in Peck's program. Neither in my program for the moment.

One problem with Avery Lee is that I've sent him an e-mail about my problem with these AVIs and never got an answer. The other is that he uses VC++ .

OK. I'll see what I can do, including using VirtualDub for acquring. However, I was not very satisfied on how VirtualDub handles joined files.

Ionutz

[ionutz]

What is this frame server and what is its usage ? I've seen it mentioned around, but never found a definition...

[Walrus]

ionutz: One other thing to keep in mind about Avery Lee is he's VERY busy. You may want to politely re-email him if he never answered you. However, I have a feeling he will tell you what I am as well. The code that checks whether or not to allow joining of the files simply checks the dimensions, codecs used, audio rates and makes sure the frame rates match. It's the last one that's killing you probably. That said, I have some files captured with AVI_IO all of which report to be EXACTLY 29.970 that won't join either. I'm trying to work with him to figure out the problem with those files in particular. (its possible there is an additional bug in his joining code)
However, you are correct in that Virtualdub wasn't really designed to handle joined files (especially files that weren't designed to be together), and you're correct in that he uses VC++ (I do have to admit, I prefer Borland compilers)
As for Frame Server, until recently it has been one of the most obscure features of Virtualdub, but now that Avery fixed the code to work on NT/2K.. I'm VERY impressed. Basically, it allows you to pipe the processed output of Virtualdub directly to another program: for instance an MPEG encoder, without having to recompress the file into an intermediate file. It's an analog to piping the output from one program to another.
It's usage is a bit tricky, and I can't improve on the instructions Avery includes in the help system. But what I've been able to do is to set up multiple copies of virtualdub to act as frame servers for many of of the file segments I have and to queue up the place-holder files that point to the frame server's output in bbMPEG allowing me to generate a single MPEG file out of the processed output of several avis without having to do any additional compression. (the results are TERRIFIC)
It's still in the early stages, but what I've gotten out of it thus far make me VERY impressed.

[dgcom]

Hi!

Why don't you try to hexedit dwScale to make it the same among files? Try 33372. I did some calculations - for the small file increase will be only 0.006332 sec, which is less them one frame (0.0333667). For the file with dwScale=33373, decrease in length will be -0.051005 sec, which is about 1.5 frame. They will probably stay in sync.

DGCom

[ionutz]

Hi,

So you're suggesting simply ignore the dwScale variation and go with a mean value. I'll try this, as it is the single solution I have for now.

But I will probably use the 33370. The examples are from a part of my files. I have digitized a whole movie and, as I couldn't encode the movie from the start to the end, I've ended having 4 big files and 3 smaller ones wich overlaps over the transitions between the big files.

Something like:

<pre>
BBBBB BBBBBB BBBBB BBBB
sss sss sss

B:= big file
s:= small file
</pre>

Now I want to select the frames from all these 7 files and get a final MPEG-4 + MP3 avi. Eventually also split the final into 2 files to be able to put the files on CDs.

The dwScale varies between 33367 and 33373, so probably 33370 is the best thing. I'll try implement an automatic selection of dwScale that minimize the frame desyncronisation.

[dgcom]

Yeah, try this...

But get averege dwScale from your big files then. Small files have much less chances to get out of sync.

And I'm curious, what results would be... (in terms of audio sync)

DGCom