Gap Coverage

= Platform-api =
qt<something>-opensource-src do not run unit tests during package build (verified with qtbase-opensource-src, qtsensors-opensource-src, qtmultimedia-opensource-src-touch). It is possible that it is an issue with the build system as running 'make check' should run the tests.

List of upstream qt packages used by Ubuntu for phone:
qt3d-opensource-src
qtbase-opensource-src
qtdeclarative-opensource-src
qtfeedback-opensource-src
qtgraphicaleffects-opensource-src
qtjsbackend-opensource-src
qtlocation-opensource-src
qtmultimedia-opensource-src
qtmultimedia-opensource-src-touch
qtpim-opensource-src
qtscript-opensource-src
qtsensors-opensource-src
qtsvg-opensource-src
qtsystems-opensource-src
qtwebkit-opensource-src
qtxmlpatterns-opensource-src

(
one-liner to run on Trusty to get the list of source packages used on the phone
-8<--------8<--------8<--------8<--------8<--------8<--
manifest=http://cdimage.ubuntu.com/ubuntu-touch/daily-preinstalled/pending/trusty-preinstalled-touch-armhf.manifest; \
apt-cache showsrc $(lynx -dump $manifest|awk '$1 !~ /click:/ {print $1}') \
| awk '/^Package:/ {print $2}'| sort -u
->8-------->8-------->8-------->8-------->8-------->8--
)

Force each component of the platform API write at least 3 system tests that cover the area of the component.

Proposition of patch to enable make check during the build:
======
--- qtxmlpatterns-opensource-src-5.0.2/debian/rules 2013-04-20 19:27:10.000000000 +0200
+++ qtxmlpatterns-opensource-src-5.0.2.tests/debian/rules 2013-11-29 16:14:37.551517163 +0100
@@ -29,3 +29,6 @@

 override_dh_builddeb:
  dh_builddeb -- -Zxz
+
+override_dh_auto_test:
+ cd tests/auto/ && ( test -e Makefile || qmake auto.pro -o Makefile ) && make -f Makefile check
======

This must be applied to debian/rules of all packages called qt<someinthg>...-src . I tried on qtxmlpatterns and qtsystems but test suites failed because they require a specific setup (a modificatoin of /etc/hosts in one case and a database in the other case)

Work items for ubuntu-13.11:
[jibel] Estimate the effort it would take to enable those existing tests into CI (without fixing the failing tests, this include the Qt Wrapper and backend): DONE
[jibel] Estimate the effort for providing the automatic coverage metrics in CI: DONE

Work items for ubuntu-13.12:
[allanlesage] Enable and run unit test for qt3d-opensource-src: DONE
[chris.gagnon] Enable and run unit test for qtbase-opensource-src: DONE
[om26er] Enable and run unit test for qtdeclarative-opensource-src: INPROGRESS
[om26er] Enable and run unit test for qtfeedback-opensource-src: DONE
[chris.gagnon] Enable and run unit test for qtgraphicaleffects-opensource-src: DONE
[allanlesage] Enable and run unit test for qtjsbackend-opensource-src: DONE
[allanlesage] Enable and run unit test for qtlocation-opensource-src: INPROGRESS
[om26er] Enable and run unit test for qtmultimedia-opensource-src: DONE
[chris.gagnon] Enable and run unit test for qtpim-opensource-src: DONE
[iahmad] Enable and run unit test for qtscript-opensource-src: DONE
[iahmad] Enable and run unit test for qtsensors-opensource-src: DONE
[chris.gagnon] Enable and run unit test for qtsvg-opensource-src: DONE
[om26er] Enable and run unit test for qtsystems-opensource-src: DONE
[allanlesage] Enable and run unit test for qtwebkit-opensource-src: DONE
[iahmad] Enable and run unit test for qtxmlpatterns-opensource-src: DONE
[om26er] Analyze how to enable coverage: DONE
[allanlesage] Publish coverage metrics from jenkins to dedicated site: INPROGRESS
[allanlesage] Verify if binaries with/wiouth coverage enabled are identical: DONE
[chris.gagnon] Investigate how to extract coverage data from the build logs: DONE
Seed initial integration test for packages of platform-api not already covered by a test and publish results to CI dashboard: TODO
[chris.gagnon] Create a PPA to build packages with coverage enabled and talk to webops to enable build on non-virtualized ARM: POSTPONED
[chris.gagnon] Create generic jenkins test runner that adds coverage bits to packages then compiles package to get results: DONE

Work items for ubuntu-14.03:
[jibel] Enable autopkgtest for firefox-next and firefox-security PPA on stable releases: DONE