Improving Mozilla translations imports and exports in Rosetta

Facts:
 * Upstream uses mercurial VCS to store source code (http://hg.mozilla.org/releases/) and translations in different branches.
 * Upstream translation teams have individual branches for each language (http://hg.mozilla.org/l10n-central/)
 * Upstream translation branches are a 'temporary storage' and cannot be trusted for doing the upstream translation imports. Instead the published upstream translation XPIs need to be used.
 * The published XPIs have a different directory layout in the embedded .jar archive, than the upstream mercurial branches use. When feeding translation improvements back to upstream, they need to be in the branch directory layout in order to be applied by upstream translators. If patches to the XPI get submitted to upstream, they tend to get ignored, since the translators are not able to map the file locations back to their branches.
 * When building Firefox and Thunderbird packages, the en-US XPI files get built and fed into Rosetta to become the templates for translations. However, we don't have the mapping of the file locations between the location in the XPI and the location in the upstream branch.
 * When exporting Firefox translations from Rosetta, we export them into a XPIPO format, where the file and string locations like they would be found in the XPI are stored in the comments. The current po2xpi tool in langpack-o-matic uses this information to reconstruct XPI files out of the XPIPO files.
 * Due to a bug in Rosetta, users cannot upload any XPIPO files or XPI files into Rosetta.
 * XPIPO format is useful for translators if they want to use their offline tools (e.g. poedit) to edit .po files and then upload the result back into Rosetta.

Ideas how to improve this:
 * When generating the Firefox and Thunderbird packages, we compute the file location like they would be in the upstream translation branches and store that information in the files itself as a comment before we generate the en-US.xpi
 * Rosetta reads those comments and stores each file as a separate template with the file paths takes from those comments. This enables us to export the files in the upstream branch format.
 * Instead of having one single template for Firefox/Thunderbird, we have multiple templates, one for each file in the XPI.
 * Meta data from the install.rdf is stored in a specific metadata template. When exporting that template into upstream branch format, the data gets exported into the required toolkit/defines.inc and browser/defines.inc files, which are needed to build the XPIs out of the branch format.
 * The templates should be exportable in two formats: XPIPO and upstream branch format.
 * User imports (Rosetta) should accept, both formats, XPIPO and native .dtd a d .properties files.
 * Exports in upstream branch format can be auto-exported into a bzr branch, so we can diff the changes to the upstream branch and submit the changes back to upstream.
 * When building XPIs for the Ubuntu language-packs, we should use the upstream branch format and use the Firefox/Thunderbird source code to build the XPIs, just like upstream does. Advantage: when upstream changes the code to generate XPIs, we are in sync and don't have to reengineer the changes into our langpack-o-matic tool.

How to compute the location of the files in the upstream branch:
1. in the $BUILDDIR: find ./ -name en-US
-> This will outout a list like this:
mozilla-1.9.2/security/manager/locales/en-US
mozilla-1.9.2/other-licenses/branding/firefox/locales/en-US
mozilla-1.9.2/netwerk/locales/en-US
mozilla-1.9.2/extensions/spellcheck/locales/en-US
mozilla-1.9.2/extensions/irc/locales/en-US
mozilla-1.9.2/extensions/venkman/locales/en-US
mozilla-1.9.2/extensions/reporter/locales/en-US
mozilla-1.9.2/toolkit/locales/en-US
mozilla-1.9.2/dom/locales/en-US
mozilla-1.9.2/browser/locales/en-US
mozilla-1.9.2/browser/branding/nightly/locales/en-US
mozilla-1.9.2/browser/branding/unofficial/locales/en-US

For each path, $DESTDIR = strip mozilla-1.9.2/ and /locales/en-US (e.g. security/manager), then append the rest of the path of each file.
E.g. "mozilla-1.9.2/security/manager/locales/en-US/chrome/pipnss/pipnss.properties" will become "security/manager/chrome/pipnss/pipnss.properties"

Thunderbird will have something like this:

comm-1.9.2/other-licenses/branding/thunderbird/locales/en-US
comm-1.9.2/other-licenses/branding/sunbird/locales/en-US
comm-1.9.2/mail/locales/en-US
comm-1.9.2/mail/branding/nightly/locales/en-US
comm-1.9.2/mail/branding/unofficial/locales/en-US
comm-1.9.2/suite/locales/en-US
comm-1.9.2/suite/debugQA/locales/en-US
comm-1.9.2/editor/ui/locales/en-US
comm-1.9.2/calendar/locales/en-US
comm-1.9.2/calendar/sunbird/branding/nightly/locales/en-US

(Strip comm-1.9.2/ and /locales/en-US)

and

./comm-1.9.2/mozilla/security/manager/locales/en-US
./comm-1.9.2/mozilla/other-licenses/branding/firefox/locales/en-US
./comm-1.9.2/mozilla/netwerk/locales/en-US
./comm-1.9.2/mozilla/layout/forms/resources/locale/en-US
./comm-1.9.2/mozilla/extensions/spellcheck/locales/en-US
./comm-1.9.2/mozilla/extensions/irc/locales/en-US
./comm-1.9.2/mozilla/extensions/venkman/locales/en-US
./comm-1.9.2/mozilla/extensions/reporter/locales/en-US
./comm-1.9.2/mozilla/extensions/inspector/resources/locale/en-US
./comm-1.9.2/mozilla/extensions/layout-debug/ui/locale/en-US
./comm-1.9.2/mozilla/extensions/metrics/locale/en-US
./comm-1.9.2/mozilla/toolkit/locales/en-US
./comm-1.9.2/mozilla/dom/locales/en-US
./comm-1.9.2/mozilla/testing/extensions/community/chrome/locale/en-US
./comm-1.9.2/mozilla/browser/locales/en-US
./comm-1.9.2/mozilla/browser/branding/nightly/locales/en-US
./comm-1.9.2/mozilla/browser/branding/unofficial/locales/en-US

Here the ./comm-1.9.2/mozilla/ part needs to be stripped in addition to /locales/en-US.

2. This should be stored as the first line of the .dtd and .propertis files, like this:
 a) .dtd files:
<!-- [launchpad-branch-path:$PATH] -->
 b) .properties:
#### [launchpad-branch-path:$PATH]
3. build the en-US.xpi like usual

Meta data template:
 * from the upstream translation XPI install.rdf file: store language name and contributers as two msgids.
 * on export into branch format, the language name needs to go into toolkit/defines.inc and the contributers list needs to go into browser/defines.inc
 * store the mapping between .jar file path and target file path in the header. Then when importing upstream translation XPIs, read this mapping and import the files into the correct templates.

Work items:
[mozillateam] Add code to the build rules of Firefox and Thunderbird packages, which computes the location of the files and puts them as a special comment like defined above into the files itself, before building the en-US.xpi
[rosetta] enable .dtd and .properties file exports
[rosetta] break down XPI file into .dtd and .properties files during import and extract the target filepath out of the special comments.
[rosetta] store the mapping between XPI location of each .dtd and .properties file and the target filepath in the meta data template in order to be able to correctly import upstream translation XPIs.
[rosetta] add a switch to export into XPIPO or branch format tarball. Individual template exports should be in XPIPO format or native .dtd/.properties formats.
[rosetta] enable native .dtd and .properties file imports for users
[rosetta] export branch format into a bzr branch
[rosetta] handle include statements in .dtd files (remember their locations)