Frame based / Field order / Firewire capture ?

[SEN]

Hi, I have tried searching this forum .. but I didn't find the answer.

When creating video from MSP6, at some time I have to set frame-based or Field Order A or B.... but I have NO idea if my capture (DV via firewire) is the one or the other?

Is it correct that I should always set Field order B for playback to a TV nomatter if the final media is a DV tape or a MPEG2 file?

SEN

[wkulecz]

I don't know what is "correct" but I did a firewire capture and output back to the camcorder using my Pyro & MSPro6 and recorded the composite analog on VHS. The result was as good as VHS ever is, so I think the default is fine, which if I remember correctly, the DV 32KHz audio template default was "frame based".

It would be nice to know if this is built into the DV standard or if we can expect problems playing DV from one brand of camcorder back thru firewire onto another because of this potential issue.

For MPEG, I can't advise as I've so far not been satisified with anything I've encoded from either MJPEG or DV sources.

If you were transcoding from DV to MJPEG, I assume you'd want field order B for the Marvel so the MJPEG played back thru the hardware codec match the way it would have been captured. I did a transcode from DV to MJPEG using the picvideo codec this way, but when played back on my G200 Marvel it used the software codec and thus didn't play thru the composite out on the BOB. It looked fine on the computer display however.

I assume I have to use UWE's utility to change the fourcc code to dmb1 and match the data rate to something the Marvel would produce. This might be difficult and I've not played with it since. My next test will be to do the transcode on the system with the Marvel and see if it produces the proper dmb1 codec output.

--wally.

[SEN]

Hi wally

Maybe your answer created more questions than it solved ... well

I don't use a Marvel anymore, so the only options I have for making analog output is from my camcorder or from my Cinemaster hw MPEG2 decoder. After reading the guide2MPEG on the DTV newspage I have my doubts about the later.
The 'player' that comes with the Cinemater 3 does not allow you to play files, only DVD's, so what I need is a program that can convert my mpeg2 files into a DVD image (for writing on a CD-R). I have seen some NLE software stateing they were able to create DVD-ready output .... anyone tried?

[Grigory]

You should use correct field order if you want to get good rendered transitions.
What field order is correct for your DV card - depends on the card itself. You need to read the manual.
For my DV Raptor card, the "lower field first" order is correct.

You can get satisfactory results with frame based rendering, however, the quality of transitions will be lower (25/30 frames instead of 50/60 fields per second)

Grigory

[SEN]

Hi Grigory

The is (at least) one thing I don't understand. I got no dedicated DV card, I got a no-name fire-wire card, using eigther MS or TI driver.
I would expect the fire-wire card to know less than nothing about DV, it's simply a matter of transporting data. Therefor I would expect the source (camera), the driver (MS or TI) or the capture program (MSP6 og VS4) being responsible for framebased/field order capturing. What I can't get into my head is why isn't there a "standard" field order? I do under stand the diff. between frame based and field order depending on the output device, but what reason could there be for NOT capturing in the field order as the TV would expect?

Does your 'lower field first' correspond to 'field order B'?

Well I guess I have to make some experiments using transistions.

Thanks to all ..
SEN

[Grigory]

Hi Sen,

Field order is how the data is written in avi file. I believe (but not sure) that the DV data on tape is written as field pictures, each of 288(240) lines in height. One field contains odd lines, two its neighbors contain even lines. There is only a sequence of fields on tape track.
When the card get this data and makes avi, it MUST combine two fields in one frame. This is because avi format can store only frames. Depending on what pair of fields in a sequence are chozen to make one frame you get either odd-even or even-odd combinations. Both are valid, since both contain two adjacent fields of video.
The difference is that for odd-even frame the field with odd lines contains a picture that was shot <u>earlier</u> than the picture with the even lines. For even-odd pair the sequence is reversed.
The card driver knows the field order and can make backward transformation while writing DV avi to tape.

To get correct transition rendering, the editing application must know correct field order in avi file. For Canopus and DVsoft codecs this field order is lower first or B, depending on what words are in use in specific editor.
I suppose that another DV codecs also have this field order.
If you apply say cross disslove transition, the transparency of each clip should change in time from o to 1 and from 1 to 0.
If you use field rendering the time value used in calculations is the field number, counting from the beginning of transition. So, the editor must take a field image, apply transition with correct time value and then go to another field.
For this process it is important to know which field in a frame is first in time. For example, for a transition of 30 frames, you have 60 fields, with numbers 1,2,3...60. If you set correct field order, the transition is applied correctly: the field that comes to screen as N-th picture has N-th time parameters of transition: 1,2,3,4...59,60.
If you reverse field order in editior, you get for transition the following time values 2-1-4-3...60,59. So, the image begins flickering.
On unchanged fragments the image data is copied directly to output avi, so the field order remains correct.
If you set frame rendering, only frame numbers are used as time values. So, instead of 60 phases of transition you will make only 30 in the example above. Sometimes this produces not perfect smoothness of motion, especially on short transitions. It is noticeable with moving titles and similar elements.

Grigory

[SEN]

Thanks Grigory

I think I got 'the picture'. It's all down to the fact that AVI can't hold fields .. therefore .. whoever was at work the day they wrote the driver decides :P which field goes first into the frame ... well I can live with that, as you wrote: probable most codecs uses the same order anyway.
Now least I now know what to look for ... any why!

--SEN

I love this forum ... allways a lot of info within a few days.

[dgcom]

Grigory: Most of Firewire cards no nothing about data they transfer... So, they do not combine/split fields into frame. DV strean itself is self-contained video format - motly like mpeg. Most drivers just take audio from there and duplicate it in AVI 'auds' stream. DV stream IS interlaced and field based and, I think, "lower first" order is somewhat standard among camcorders. It does not really matter if fields are encoded as separate or as frame in DV stream - it's up to codec to interpret it correctly (as long as you are stay with DV).
Most problems happens when you, lets say, want to convert DV to Matrox MJPEG and output it to TV... Or, compress to MPEG-2 for DVD playback.

And most of Video editing software recommends using frame-based setting to edit DV.

DGCom

[Grigory]

dgcom,

Ok, the hardware itself know nothing about the data.
But... to make correct avi, the DRIVER for conversion of DV stream to avi or back MUST combine pairs of fields on frames, and write avi chunks of data. Otherwise this avi file will not play in Video for Windows software or its successors like activemovie, directmedia, what else MS will say for the same avi playback functionality.
So, the driver converts DV stream into avi file and in reverse.
The conversion rules may be different, but, because all such drivers originate from some driver that was the first, the field order in avi file is hopefully always the same. You still have to specify correct field order for any editing operation.

As for the recomendation, please tell me, how is it possible that most software editors recommend to edit with fields the interlaced video files, but, according to your words, make specific exclusion to interlaced DV avi files?. This sounds contradictory and is not correct. Canopus DV Raptor presets for ALL editors are for lower field first type of editing.
For some reason, Adaptec DVSoft presets are frame based, which, of course, is incorrect. I had to change them manually, according to Adobe recommendations.
Actually, anyinterlaced video MUST be edited with correct field order. DV is not an exclusion.

DV is modified MJPEG by means of DCT compression technique, nothing very special. Just one possible codec of many existing codecs that are used in avi files.

Grigory

[jeff b]

One of the curious experiences that users of generic OHCI compliant IEEE-1394 boards and MSP6 are having is that when you go to render a new Type 1 DV file, and you set for 'frame based' or 'field order A' or 'field order B', it doesn't seem to make any difference at all. I have no idea why this is, but that's what I've experienced, too. MSP6 defaults to 'frame based', however, when you go to the options for creating a Type 1 DV file.

One other curious thing that I noticed using Type 1 DV with a generic 1394 and MSP6 is when I start setting cues. The frame showing in the window will look a bit dull, but when I hit F5 to set a cue on it, and then set it in, the frame suddenly becomes very clear. It seems as if I'm suddenly viewing a full dual field frame, where before I was only viewing a smudged single field frame.

It would appear that if one uses Type 1 DV from start to finish of a project in MSP6, the field order selections are superfluous. However, if one were to take MJPEG source and save the project in Type 1 DV, then the field order selection might then be critical.

[dgcom]

Grigory: If you take out all chunks from DV AVI type 1 file and write them together, you'll get exactly the stream which flows over Firewire. Again, it does not matter what the file format is if you use the same codec to compress/decompress (and edit) video. Actually, most recommendations are for type 1 files... For type 2 (Canopus, DVSoft) it may vary. Why it is specific for DV files? Just because the codec you are using (MS DV) will handle fields in the frame properly when recompression occur. This may not be truth for Canopus...
By the way,Canopus codec theoretically, can invert fields order when decompressing, but this setting in registry does not work.

DGCom

[Grigory]

Just because the codec you are using (MS DV) will handle fields in the frame properly when recompression occur

I would like to make things clear.

I am not familiar with this type1 avi format, however the recompression and rendering are different things. If you render transition, the editor always:
1 takes compressed picture
2 converts it to RGB (unfortunately) format, using the codec;
3 applies effect and then
4 recompresses the picture using the codec.

For type2 avi, the picture term above is frame, and contains two fields. The editor has no idea which field comes first, so we have to set correct field order. This is not related to CODEC directly, but happens because the codec has no way to tell the editor which field order (if any) it uses. When you specify the field order, the editor extracts both fields as separate images and applies effect on them according to their order. Then it combines fields back in single frame and sends this picture to the codec.

Note, when you specify field rendering for a frame based movie, the editor will also extract odd and even lines and use them as separate images in rendering. This may result in creation of interlacing artefacts on pure PC video with no fields at all. To check this, apply siple moving text effect and render SIF size movie with fields. You will see interlacing lines on characters.

If the type 1 avi contains fields as separate pictures, then the movie inside this avi is simply 50/60 fps frame based 288/240 lines video. The codec can, however, reconstruct this video on PC screen and show it as full height interlaced frames, or it actually shows only one field from two, resized to full height. Note this if in the beginning. I do not know how the video data is stored and displayed on PC screen. However, following this assumption the correct field order MUST be set to no fields, because there is no need to the editor to extract fields from frames.

If DV avi type 1 contains frames, then the field order must be set correctly, and you have to specify it.

I do not know what assumption is correct for type1 DV.

recompression to mjpeg or mpeg. Here the most important is the output codec field order.
The editor decompresses frame to bitmap and then compresses it AS IS into output codec format. If the playback codec has built-in field order that does not match the input codec order, you get a problem. This happens with Canopus DV to LSX mpeg2 encoding. To solve it, it is enough to shift the frame up by one pixel. This reverses the field order.

You can do this for mjpeg too.

Concerning the field order switching for Canopus, I only can imagine that the codec can try to do the same one pixel shift of the output frame. There is no another way to change the field order when the video is stored per frames in avi file. I also have no idea how this can help in editing. Maybe this is the reason why Canopus did not made this feature working.

Regards,

Grigory

[michigan___bv]

Finally, an explanation of the difference between Field order A and B. I've been looking for it for years now with no success until today!! Hopefully someone can advise me how to proceed. I'm capturing video with the rainbow runner g-series at the highest quality MJPEG settings for SVCD. When I encode to MPEG-2 using Premiere5 or MSP6, should a still choose field order b(lower field first), or frame based. Also, should I use the MSP6 or Premiere5 deinterlacing option, or does it conflict with or negate the field order settings? I love this forum! Lots of good tips and hints from the veterans, thanks!

[Grigory]

michigan___bv,

If you only encode your video, there is no difference with field order settings. Premiere decodes frame from MJPEG movie, and sends it to mpeg encoder. If you don't apply transitions, you cannot see the difference.

Field-based video playback on PC screen is always frame-based actually. The player displays entire frame at a time, and might attempt to do deinterlacing. Anyway, it shows FRAMES and you cannot detect incorrect field order.

TV output devices, such as hardware DVD/mpeg decoders, or DH G400 will show field based movie with fields on TV screen. Here you can get troubles and have to set correct field order for encoding software.

Encoding plugin settings may have independent from Premiere field order settings. Bbmpeg gives you a choice to use frame (progressive) encoding, of field encoding. For frame encoding there is no choice of field order, or is it not working. For field encoding you must choose the field order that matches your clip's field order - lower first. Otherwise you get reversed field order on playback.
LSX or Panasonic encoders are compressing FRAMES. You cannot change the field order. The decoder, such as Hollywood+ uses its default field order when it plays mpeg files that are frame encoded.
For unknown reason LSX and Panasonic mpeg1 files have correct field order, but mpeg2 video from LSX has reversed field order. Since mpeg1 and 2 encoding in LSX use the same compression technique, there is no difference between LSX mpeg1 and 2 in quality, if you use the same encoding settings. So, mpeg2 is only useful if you really need to make DVD/SVCD disk for standalone player. In all other cases mpeg1 files from Panasonic or LSX are better, not only because of good quality, but also because of correct field order and ability to play them on any PC.

If you use different encoder(s), you might need to find correct field order settings, if any.

Grigory

[dgcom]

Hi!

Thanks, Grigory! You made me do some checking and not to relay on what others are saying (even in documentation).

First, I have to point out, that it is definitely does not matter, how video is stored in file. Decoder will extract full picture from it (regardless if this was field-based or frame-based video).

You are absolutely correct, that most video editors will process fields in video separately and apply changes to two consecutive fields as if they where fps*2 frames… (Simplified)

Also, I confirmed, that OHCI-compatible cards (actually, camcorders, connected to it) produce field-based video with Frame Order A, which is “bottom (even) field first”.
It is relatively easy to check by yourself. Just capture a pure frame from your footage, which contain fast motion of contrast vertical object. Open it in any graphic editor (i.e. MS Paint), Zoom In (about 400%). Move your cursor to the part of this vertical object showed first. Since most editors shows lines starting 0, if you see vertical cursor position as an odd number, this would mean it is using field order A, bottom field first. If cursor position shows even number, your video is in field order B. All this is true if decoder is not swapping fields for some reason.

But when you start editing video, how to interpret source depends on what would be a destination of it.
Let’s say, we are using Ulead MSPro 6. If target is recording back to DV tape, you are better off with marking all source DV clips on timeline as frame order A and use the same field option when creating video output. If target is playback on computer screen, use field order A on timeline as well, but check “deinterleace” check box and use “frame-based” when creating output. If target is a full D1 (720x480/576) MPEG-2 for playback via hardware decoder, and you are using Ligos encoder, you’ll need to swap fields due to the bug in Ligos MPEG-2 compressor – the flag to play top field first is always set. You can swap fields by selecting field order B (if source clips are set to field order A) when creating output video.

That’s in short, my findings on field order…

DGCom