Integrate Autotesting with LibreOffice Upstream QA
LibreOffice has an extensive testing infra. To make our testing valuable for them we need to integrate with existing solutions:
http://
http://
http://
http://
This session is to evaluate:
- what we can do
- what upstream plans to do
- how this all is scheduled
- what can be achieved for this cycle
- what we can do in the long term
In the discussion we should involve from upstream: David Ostrovsky and Norbert Thiebaud early on. They did a lot already to push this forward upstream. I will hope they might join via IRC.
Blueprint information
- Status:
- Not started
- Approver:
- Jason Warner
- Priority:
- High
- Drafter:
- Björn Michaelsen
- Direction:
- Approved
- Assignee:
- Björn Michaelsen
- Definition:
- Approved
- Series goal:
- Accepted for raring
- Implementation:
-
Unknown
- Milestone target:
-
ubuntu-13.04-month-6
- Started by
- Completed by
Whiteboard
Kinds of tests
Three kinds of tests in LibreOffice ("LO")
Normal tests: quick tests that every developer should tolerate and give attention.
Slow tests: important tests that are too slow for most people to care to give attention to.
"Subsequent": tests that require full installation, and can't be run in source tree.
ubuild and jenkins runs all three kinds.
bjoern will make package that gets tests to run against installed LO. Including "junit" tests
Only Java tests can be run against installed version. As tests have problems, people are migrating Java versions of tests to C++ and removing the Java tests. bjoern will try to find a way to get C++ tests to run against installed version.
How can we give tests to upstream?
We have Tinderbox tests running continuously.
"Raleigh" is interesting to upstream.
One problem is the interleaving of LO releases with Ubuntu releases. LO releases about mid-U cycle. It's hard to keep the testing straight. We want to continue to test the active development ("master", as LO calls it), where dependencies change often, but we want to use stable code in Ubuntu.
Since debian source "orig" tarballs are precious and immutable, we keep internal libraries in the source tar even if we do not use it, because we might discover that the preferred library, in Ubuntu system, is buggy and should be abandoned and be replaced with the internal library in debian packaging script.
Can we build on ARM? It takes 20+ hours to build on native hardware. QEMU -j10 takes less time. We would like to.
Are there UI tests? Not yet. The "subsequent" tests can be run in head-ed mode (it isn't stable right now), but that can work. It would be a lot of work. Can we test Unity integration functionality only? Maybe. Instead of relying on tests to find right timing, can we record X events during execution and then inspect the events to see if what we expected happened? Not very useful because it won't catch some bugs.
There's a code-review system upstream, and we're using for manual reviews, but not automated. By end of 2012, we hope to use automated also to inform developers of success or failure of tests after application of change to tree.
Work Items
Work items:
[bjoern-michaelsen] setup build on master branch using libreoffice internal libraries: POSTPONED
[bjoern-michaelsen] setup build on release branch using system libs whereever possible (as does the package in the end): POSTPONED
[jibel] investigate QEMU/ARM build/test (LP: #1103405): BLOCKED
[bjoern-michaelsen] integrate master builder with tinbuild2 and http://
[bjoern-michaelsen] watch out for upstream gerrit developments: DONE
discuss upstream VM setup for manual bibisect testing with shm_get and jibel: DONE
[jibel] Implement and setup bibisect testing in the QA Lab : DONE
[jibel] Publish bibisect repository automatically on people.c.c (http://
[jibel] Move the builder to the DMZ and stop the git/http rewriting dance: DONE
[jibel] Allow copy from DMZ to people.c.c: DONE
