[Gnash-dev] GSoC - Optimizing the IO Pipeline for AV Data Update

[Jonas 'Sortie' Termansen]

Hi.

I'm Jonas 'Sortie' Termansen, and I want to work on Gnash for Google's
Summer of Code 2011. I've written a proposal for my project, which strk
suggested I'd post here. If anyone is interested, I have included my
proposal below. I have based it on GNU's guideline template for GSoC 
applications. Comments would be welcome. :)


Optimizing the IO Pipeline for AV Data
----
Summary:
This project is about optimizing the playback of audio and video (AV) in
Gnash. In particular, I will change AudioDecoder to a more efficient
push/pop/pull model like already implemented in VideoDecoder. I will
also track down unneeded copy operations and memory allocations in the
IO pipeline to further reduce CPU usage, which can be quite high.

Benefits:
In the recent years, it has become popular to stream videos online
through sites such as Youtube. Unfortunately, in my experience, Gnash
can have a high CPU usage which leads to a lagging experience. Improving
the performance of the audio and video data decoding pipeline would be a
key step towards smooth playback. Some users may decide against using
Gnash if it doesn't run smoothly, and may even decide to use non-free
software instead.

Deliverables:
I will mainly be modifying sources in the libmedia/ directory. To port
AudioDecoder to a push/pop/pull model, I will need to redefine the
AudioDecoder interface and modify all implementing subclasses to the new
schematics. Gnash has several decoding backends (FFmpeg, gstreamer), and
I will need to port each of them to the new system. To remove needless
copying of decoded data, I will need to follow the data flow for audio
and video frames and modify it as necessary. I will rewrite
documentation for changes that modify the internal API, which will be
easy as we generate API docs from source code.

Plan:
The first week will be spent familiarizing myself with the code base and
figuring the exact details on how to implement the improvements. In
particular, I will analyze the existing implementation to identify
bottlenecks and weak spots that will need to be fixed. I will discuss
implementation ideas with other project developers to get feedback.

I will attempt to make smaller, incremental changes that can be pushed
upstream so I can get used to the code base, before I implement larger
API-breaking changes to the interfaces. This will also allow my mentor
to keep track of my progress.

Benchmarks will be required to evaluate my changes against the existing
implementation. Depending on the debugging tools already available, I
may need to code these myself.

When the mid-term evaluation is reached, I will have a sample
implementation ready for testing. The sample implementation should show
a performance improvement. Any remaining problems should be known and
fixable in the remaining time span.

The remaining time should be used for fixing bugs in the contribution.
Since we are dealing with advanced parsers for a variety of backends,
testing the code properly for a number of configurations might prove
time consuming, and possibly difficult. If the code base is ready early,
I can use the time working on more experimental designs or other related
tasks.

Once the project is finished it should display a clear performance
improvement, as well as being stable enough for usage, and stable enough
to be merged into the main repositories.

If I am unable to complete the project, the partial contributions will
still be useful since I am dealing with optmizing an existing system:
anyoptimization is still an improvement.

I don't have any vacation plans this summer, andI w ill be able to work
on the project most of the time. I might be away a week at one point,
but I won't leave the country, so I might still be able to work on the
project.

Communication:
I will use the gnash-dev mailing list to discuss my project, as well as
using IRC to contact my mentor and other developers. (Emailing my mentor
is naturally also possible). I work well independently, but naturally I
will provide status updates when code is finished, or ask when unsure
how a feature works. We use Git for version control and I will provide
incremental patches regularly. This enables peer review, as well as
keeping other developers up to date with my work.

Qualification:
Flash has become widespread proprietary software used on the Internet.
Sadly, many common websites require it to function. I like the idea of
developing a free software replacement for Flash, so that websites such
as Youtube can be used on fully free systems.
I have developed my own flexible IO streaming frameworks for my own
home-brewn free software projects. One of my experimental projects was
building a media player and rendering it on 3D surfaces inside a game
engine. This involved using libavcodec (FFmpeg) to decode audio and
video and using OpenAL-Soft as a sound system. Because of these
qualifications, I found the task of hacking on the Gnash AV pipeline
intriguing. I have not worked with some of the base libraries used in
Gnash, such as Boost. I will need to learn these APIs, but that
shouldn't be a problem. I may continue to contribute to Gnash once the
project has been completed, depending on how much time I need for the CS
studies after the summer break.