[Linphone-developers] Re: [Ffmpeg-devel] Problems ffmpeg==>Openh323

[Dan Dennedy]

Earlier, Derek wrote:

>  I want to bring you guys up with the status of using your code in the
> OpenH323 project (www.sf.net/projects/openh323)
> 
> It has been a bit of a battle, but we finally have ffmpeg working
> reliably in a windows and linux environment.

Hi, I have been working on video support for Linphone (SIP) using
ffmpeg.  This is my first time implementing something like this, so I am
facing perhaps some similar issues as you.

On Thu, 2003-07-31 at 19:16, Derek Smithies wrote:

> 2)Initially, we tried to get ffmpeg to work in rtp_mode.
>   This generated nicely sized packets, which were then sent to the 
>   network. However, ffmpeg did not always respect the rtp_payload_size
>   variable, and some fatally large packets were generated.

I noticed this too, but I am not totally aware of the impact of packets
larger than MTU - RTP header size. I just figured they would be split at
the link layer based upon MTU. For now, it seems to work ok as long as
there is not too much UDP packet lossage. 

To use the rtp callback, I added a generic data pointer arg to the
callback parameter list. I make it point to a structure I need to use
within my callback to enqueue the packets for RTP transmission. A patch
was submitted here but not yet committed.

>   So, we worked hard and split those fatally large packets.

Based purely on size or are any other factors involved?

>   Encoding worked fine - all the packets were being transmitted Ok.
>   At the decoder, all the received packets were concatenated into one big 
>   block. This block would be identical to the block generated by:
>     avcodec_encode_video

I use the same approach. I use the GOB number to keep track of start of
a new frame.

>   But, there was a problem resulting from transmission on the network.
>    You see, the decoder would receive X of N packets in a frame.
>   Typically, X is equal to N.
>   However, sometimes X is less than N. In this situation, we just 
>   sent the smaller block to avcodec_decode_video and hoped.

Yes, due to UDP packet loss.

>   Experiments showed that this was unwise. I found experimentally that 
>   windows was less stable than linux (when receiving smaller blocks)
>   This maybe due to the windows machine being slower, and more likely to 
>   drop blocks, but....
> 
>   Consequently, I took a simple approach. After encoding the video, I 
>   manually split it into N packets, where packet size is < 1400 bytes.
>   At the decoder, the packets are reassembled into a block.
>   If one packet is missing, the block is discarded, and not decoded.

Hmm... interesting. I could do this easily by checking continuity of GOB
after reading out of the jitter buffer; however, it would fail to detect
missing GOBs at the end of the frame. I have not looked too deeply into
the decoder, but I assume it can parse each concatenated GOB and attempt
to make the best out of missing GOBs. However, perhaps the results are
not best in the end and sacrificing frame rate is more acceptable.