Software codec optimization

Proposal and action points as registered in Multimedia mini summit (June 2011) - http://pad.ubuntu.com/RGCdnXTSNB

= Optimization Forum Proposal =
* Wikipage to hold existing optimized software implementations (codecs, encryption, checksumming)
   * Also capture requests for software-optimizing
   * How to message it? NEON-optimizing usefully constrains optimization type; maybe ARM optimizing; take into account that OpenCL is another optimization avenue
   * Aggregate per domain? For instance, Codecs/formats available, and then noting what is optimized
      * Potentially grade optimization level of codecs
* Mailing list to hold conversation around site-specific optimization
* Potentially hold a day or set of sessions in Cambridge August sprint
* Look at libjpeg-turbo and the optimizations that can be done there
  * Mans, Tom, Mandeep to work with Darrell on -turbo

== ACTIONS ==
* ACTION: Study how Android does hardware capability selection to dynamically load NEON vs non-NEON components?
* ACTION: Look at OpenSSL on ARM; evaluate NEON optimization potential
* ACTION: Look at libjpeg Android loading: how is support for hardware codecs handled?
* ACTION: Investigate how libjpeg-turbo could cater for handling hardware decoding
   * Switching would need to be dynamic because for smaller images the latency overhead could negate the benefit
   * How would the hardware codec be exposed to libjpeg-turbo
   * What about decoding hundreds of images in a gallery
* ACTION: Investigate OpenCV implementation -- how much of it is vectorizable; what has been SSE'd; how hard will it be to add NEON routines (Mandeep: some Freescale work done)
* ACTION: Do a broad sweep of Ubuntu packages that are using SSE and missing NEON for candidates for work
   * pixman, skia probably already have NEON code
   * transcoding for MMS might be another use case
* ACTION: Look at use of composition/blending and color conversion in SurfaceFlinger; potential for NEON work or handled by the GPU already?
* ACTION: Investigate which upstream projects could accept NEON optimizations
* ACTION: NEON: study if we need to look at anything beyond what we have checked so far (AAC, VP8, JPEG)?
  * Skype SILK - is there a NEON version for that? (No, interesting to optimized by neon, low priority)
  * Low latency voice codecs (which ones are interesting to look at)? (Speex neon optimization, low priority)
  * MP3 - do we want to do the effort to optimize it (not low hanging fruit, and not big improvement expected).