[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Linphone-developers] Do you have the plan to integrate the VP9 supp

From: jim
Subject: Re: [Linphone-developers] Do you have the plan to integrate the VP9 support in the near future?
Date: Thu, 2 Jul 2015 17:54:29 +0800

Ghislain, thanks for your kindly help!

One more question: 

- Need I add something in the makefile if I have added the new vp9 filter? 

And can you help to talk more about “the main new features of the VP9 codec are not really useful for real-time communications 

- What are the main new features you means about VP9? Do you means the temporal scalability/spatial scalability?

- Not useful for the real-time communication? Do you means that it is just suitable for the video on demand service or others but video call?



 Original Message 
Sender: Ghislain MARY<address@hidden>
Recipient: jim<address@hidden>
Date: Wednesday, Jun 17, 2015 16:54
Subject: Re: [Linphone-developers] Do you have the plan to integrate the VP9support in the near future?

Hi Jim,

We have no current plan to support VP9 in the near future.
We have studied the question and concluded that it's not worth the quite big amount of work. The reasons for this are that:
 - the VP9 codec does not seem to be quite stable yet (and there is no communication from WebM around it since the end of 2013)
 - it requires a lot more processing power than VP8 and a lot of devices already have some difficulties to do the processing for VP8
 - the work done so far on the codec has not targeted the real-time encoding but rather the video encoding with several passes
 - the RTP format to transport the VP9 bitstream is not the same as the one for VP8 and is still a draft and may change in the future
 - the main new features of the VP9 codec are not really useful for real-time communications
 - it appears VP9 is not adopted by a lot of people yet, except for WebRTC that also supports VP8 (and in fact the support for VP9 in WebRTC is really basic and does not use at all the new features, providing a user experience much lower than VP8)

However if you want to work on supporting this codec in linphone the steps you have listed are a good starting point. What you missed is that it will be needed to write a vp9rtpfmt to handle the RTP format to transport the VP9 bitstream as it has been done in vp8rtpfmt for the VP8 codec. This RTP format for VP9 is described in the RFC.


Le 17/06/2015 09:04, jim a écrit :

Hi Ghislain,

1) Can you help to answer if linphone team have the plan to implement the VP9 support in the near future?

2) If we want to integrate the VP9 support, can you help to comment on the steps of my understanding?
    step 1 - add support in ORTP module:
    a) Add declaration of payload type for vp9, e.g. payload_type_vp9 in avprofile.c 
    b) Add  VAR_DECLSPEC PayloadType payload_type_vp9 in payloadtype.h
    c) Add rtp_profile_set_payload() for payload_type_vp9 in av_profile_init() in avprofile.c
    step 2 - add support in mediastream2 module:
    a) Add MSFilterDesc for the encoder, e.g. ms_vp9_enc_desc, and implement the members for this structure, and finally export this structure.
    b) Add MSFilterDesc for the decoder, e.g. ms_vp9_dec_desc, and implement the members for this structure, and finally export this structure
    c) Add id for the encoder and decoder in allfilter.h
    d) Add the ms_vp9_enc_desc and ms_vp9_dec_desc in ms_filter_descs[]

    step3 - create instance for encoder and decoder by ms_filter_create_encoder() and ms_filter_create_decoder() respectively.

Your help will be appreciated!

 Original Message 
Sender: Ghislain MARY<address@hidden>
Recipient: jim<address@hidden>
Cc: linphone-developers<address@hidden>
Date: Wednesday, May 27, 2015 15:40
Subject: Re: [Linphone-developers] Please help on why there no any RPSI whenavpfis enabled.

Hi Jim,

Here is an explanation:
 - The inviting party proposes the use of PLI, SLI, RPSI and FIR.
 - The answering party only accepts the use of FIR.

FIR is a feedback message to force the other party to generate a keyframe.
There can be a lot of reasons why the other party does not accepts the other feedback message types: it does not support them, it is not configured to use them...


Le 27/05/2015 04:56, jim a écrit :

Thanks Ghislain for your reply!


I have some questions about the media session negotiation, here are the sdp detail:


m=audio 7078 RTP/AVPF 96 101

a=rtpmap:96 SILK/16000

a=rtpmap:101 telephone-event/16000

a=rtcp-fb:* trr-int 5000

m=video 9078 RTP/AVPF 96 97

a=rtpmap:96 VP8/90000

a=rtpmap:97 H264/90000

a=fmtp:97 profile-level-id=42801F

a=rtcp-fb:* trr-int 5000

a=rtcp-fb:96 nack pli

a=rtcp-fb:96 nack sli

a=rtcp-fb:96 nack rpsi

a=rtcp-fb:96 ccm fir


m=audio 16836 RTP/AVPF 96

a=rtpmap:96 SILK/16000

a=fmtp:96 maxaveragebitrate=20000

a=fmtp:96 usedtx=0

a=fmtp:96 useinbandfec=1

a=silenceSupp:off - - - -




m=video 18598 RTP/AVPF 96

a=rtpmap:96 VP8/90000

a=rtcp-fb:* ccm fir


Can you help to explain:

1) Does it means that the other party do not confirm with the type of RPSI/PLI/SLI ? 

2) What is the possible reason for the other party do not support such kinds of Feed Back types? 

3) And What’s the line a=rtcp-fb:* ccm fir in  200OK means for?

Your help will be appreciated!

 Original Message 
Sender: Ghislain MARY<address@hidden>
Recipient: linphone-developers<address@hidden>
Date: Tuesday, May 26, 2015 15:50
Subject: Re: [Linphone-developers] Please help on why there no any RPSI whenavpf is enabled.


Enabling AVPF only means that you propose it when calling. But there is a negociation with the other party and therefore it may not be enabled. There is even a negocation for each AVPF packet types (RPSI, PLI, SLI) so RPSI may be disabled by the other party.
But if the negociation is successful, Linphone sends a RPSI for every golden and/or altref frame being received.


Le 26/05/2015 05:59, jim a écrit :

Hi Jehan,

I set the avpf_enabled to be TRUE by calling linphone_proxy_config_enable_avpf(), but I do not find the decoder sending any Reference Picture Selection Indication. Is that anything wrong with it? Must the decoder respond to the encoder a RPSI when it 

recieve a golden frame or a alfref frame?



 Original Message 
Sender: jehan monnier<address@hidden>
Recipient: linphone-developers<address@hidden>
Date: Friday, May 15, 2015 17:50
Subject: Re: [Linphone-developers] Please help on how to get lastsuccessfully decoded frame for application to display whilethe the current frame is corrupt when decoding in dec_process

Hi Jim,

With Linphone, you can achieve same result with linphonerc parameter:


Best regards

Le 15 mai 2015 à 03:35, jim <address@hidden> a écrit :

Hi experts,

This is Jim and I am using linphone for a voip project.  To improve the user experience for video , I want to make some change in the mediastream2 layer: if we find the current frame is corrupt, just use the last successfully decoded frame to display.

I try this way for vp8 in vp8.c, the line in blue is what I add :
static mblk_t *dec_last_successfully_decoded_frame = NULL;

static void dec_process(MSFilter *f) {
mblk_t *im;
DecState *s=(DecState*)f->data;

while( (im=ms_queue_get(f->inputs[0]))!=0) {
mblk_t *m;

dec_unpacketize(f, s, im, &s->q);

vpx_codec_err_t err;
vpx_codec_iter_t  iter = NULL;
vpx_image_t *img;

err = vpx_codec_decode(&s->codec, m->b_rptr, m->b_wptr - m->b_rptr, NULL0);
if (err) {
ms_warning("vpx_codec_decode failed : %d %s (%s)\n", err, vpx_codec_err_to_string(err), vpx_codec_error_detail(&s->codec));

if ((f->ticker->time - s->last_error_reported_time)>5000 || s->last_error_reported_time==0) {
if (s->first_image_decoded == FALSE) {
/* if no frames have been decoded yet, do not try to browse decoded frames */


                 if (VPX_CODEC_CORRUPT_FRAME == err||VPX_CODEC_UNSUP_BITSTREAM == err) {


                     /* do not try to browse decoded frames, use last successfully decoded frame instead */
                     if (dec_last_successfully_decoded_frame) {
                         ms_queue_put(f->outputs[0], dec_last_successfully_decoded_frame);

/* browse decoded frames */
while((img = vpx_codec_get_frame(&s->codec, &iter))) {
int i,j;

if (s->yuv_width != img->d_w || s->yuv_height != img->d_h) {
if (s->yuv_msg)
s->yuv_msg = ms_yuv_buf_alloc(&s->outbuf, img->d_w, img->d_h);
s->yuv_width = img->d_w;
s->yuv_height = img->d_h;

/* scale/copy frame to destination mblk_t */
for(i=0; i<3; i++) {
uint8_t* dest = s->outbuf.planes[i];
uint8_t* src = "">
int h = img->d_h >> ((i>0)?1:0);

for(j=0; j<h; j++) {
memcpy(dest, src, s->outbuf.strides[i]);

dest += s->outbuf.strides[i];
src += img->stride[i];


//ms_queue_put(f->outputs[0], dupmsg(s->yuv_msg));


                 if (dec_last_successfully_decoded_frame) {
                     dec_last_successfully_decoded_frame = NULL;



                ms_queue_put(f->outputs[0], dupmsg(s->yuv_msg));
                 dec_last_successfully_decoded_frame = dupmsg(s->yuv_msg);


if (ms_video_update_average_fps(&s->fps, f->ticker->time)) {
ms_message("VP8 decoder: Frame size: %dx%d", s->yuv_width, s->yuv_height);
if (!s->first_image_decoded) {
s->first_image_decoded = TRUE;

But it does not work for me, Can anyone help with me or give some advice on it.
Looking forwards to your answer, your kind help will be appreciated!


Linphone-developers mailing list

Linphone-developers mailing list

reply via email to

[Prev in Thread] Current Thread [Next in Thread]