[Linphone-developers] H263 RTP packet assembling

[Uwe Zipf]

Hello,

I tried to use video with something other (Windows Messenger 5.1
for now) than Linphone (1.1.0) as the remote party. Video works from
Messenger to Linphone but the Messenger only shows an empty black
image instead of the image send by Linphone. Also on the Linphone
side the video decoder constantly complains about
"Bad picture start code" or "bad marker" "header damaged".
Anyone else tried something similar?

I have found some things Linphone does not do as the RFCs say while
searching for the problem.
Unfortunately changing them has not solved it yet...


* Marker Bit in the RTP header has to be set if end of frame in packet

  RFC2190:
  "Marker bit (M bit): The Marker bit of the RTP fixed header is set to
   1 when the current packet carries the end of current frame; set to 0
   otherwise."

  If the rtp_payload_size of the encoder is set to cover the maximum size
  that occurs this is always true because one packet contains one complete
  frame.
  If I interpret the RFC right (a packet with a complete frame carries the
  end of the "current" frame) this means we can say:
  "if the payload is H.263 I always set the bit".

  Therefor I inserted the line

  if(rtp->paytype == 34) rtp->markbit = 1;

  into rtp_session_sendm_with_ts() in rtpsession.c just before the
  call of ortp_rtp_send().


* Two Problems in msavencoder.c ms_AVencoder_rtp_callback()

  1 The comment says, that there is set a payload header according to
    RFC 2429. This is imho the header format for H.263+. But the payload
    type given in the RTP header is that of H.263 (34) according RFC3551.
    The encoder is also set to H.263, not to H.263+.
    H.263+ does not have fixed payload type number. Instead ist is defined
    as "dynamic" wich means, that the applications have to choose one
    or get one from somewhere. So an application receiving this data must
    think this is H.263, interpret the header as H.263 and get invalid
    values.

  2 The payload header ist written directly into the data provided by the
    encoder and overwrites the H.263 frameheader.
    As far as I have learned, the data from the encoder begins with the
    H.263 frameheader (if it is the begin of a frame) starting with a
    start code containing the bit value 00000000 00000000 100000
    (frame start code).

    RFC2190:
    "4. Usage of RTP
       When transmitting H.263 video streams over the Internet, the output
       of the encoder can be packetized directly. For every video frame, the
       H.263 bitstream itself is carried in the RTP payload without
       alteration, including the picture start code, the entire picture
       header, in addition to any fixed length codes and variable length
       codes."

    I modified the function to append the payload header in front of the
    frameheader as follows:

    static void ms_AVencoder_rtp_callback (AVCodecContext *ctx, void *data, int 
size, int packet_number)
    {
      MSAVEncoder *r = MS_AVENCODER(ctx->opaque);
      MSQueue *outq = r->q_outputs[0];
      MSMessage *outm;
      guint32 *p = (guint32 *) data;
      gint gob_num = (ntohl(*p) >> 10) & 0x1f;
      gchar *dat = (gchar *)data;
      guint32 tr;

      ms_trace("ms_AVencoder_rtp_callback: received %08x %08x", ntohl(p[0]), 
ntohl(p[1]));
      /* Set the H.263 Payload Header (RFC 2190)*/
      /* Get 4 more byte for it */
      outm = ms_message_new(size+4);
      /* Get the "temporal reference" from the H.263 frameheader.
         Use bitshifting if you don't like "* 64" and "/ 4" */
      tr = ((dat[2] & 0x03) * 64) + ((dat[3] & 0xfc) / 4);
      /* Construct payload header.
         Set videosize to QCIF = 0x00400000 (Windows Messenger uses this size)
         and set the temporal reference to that of the frame */
      ((guint32 *)outm->data)[0] = ntohl(0x00400000 | (tr & 0x000000ff));
      /* Append the framedata after the payload header */
      memcpy(&(outm->data[4]),data,size);
      /* Set p to the new data for the ms_trace (I don't use this, I
         look at the packets on the network instead) */
      *p = (guint32 *)outm->data;
      ms_trace("ms_AVencoder_rtp_callback: sending %08x %08x", ntohl(p[0]), 
ntohl(p[1]));
      ms_queue_put(outq, outm);
    }

    Of course the extra byte must be handled in the receiving function.
    This should solve the "Bad picture start code" messages mentioned
    above.
    As I tried to get Windows Messenger to work as receiver I have not
    tested that yet.


Uwe