Core Apps Test Review

    Let's review each of the core apps and there current status in regards to tests.
Are more tests needed?
Do we have 100% feature coverage?

    how do we determine coverage?

    At leats 1 test per user story

    How do we define a user story?

    something that gives value to the user.

    A user story can be split in more than one test.

    if no design document, then ?

    4 or 5 core apps do have some design document

    Some features don't give direct value to the user, like changing a config setting.

    We can test them with QtTest or autopilot, depending on how hard to write and maintain the code would be.

- needs-autopilot-test bugs on LP: https://bugs.launchpad.net/ubuntu-phone-coreapps/+bugs?field.tag=needs-autopilot-test
Are new tests being added as features are added?
 - Test addition/modification should be required in an MP before it is approved
Do the tests run and pass reliably?

    we don't know why tings fail when they fail; we don't keep track

    we often pursue quick fixes for failures

    tests should be written with failures in mind so failures are more obvious

    can the CI team provide data or be masters of the data?

Are the test suites compatible with both python 2 *and* python 3?
   If not, how can we make that happen in a timely manner?
Need to access only local sources for data!
ubuntu-weather-app-autopilot 2013/11/20 15:47 17 14 3 0 0 82.4%

    Leo wants to refactor

    remember to mockup for network issues

    2 tests need to be re-enabled and tested (swipe to delete emulator)

    test_locationmanager.TestLocationManagerWithLocation.test_cancel_remove_location

    test_locationmanager.TestLocationManagerWithLocation.test_remove_location

    missing tests

    test to swipe for different hours in the day

ubuntu-terminal-app-autopilot 2013/11/20 15:39 9 9 0 0 0 100%

    all good :-)

ubuntu-rssreader-app-autopilot 2013/11/20 15:32 5 2 3 0 0 40%

    forces us to use internet resources which can cause failures

    https://bugs.launchpad.net/ubuntu-rssreader-app/+bug/1248737

    needs smaller tests, a bit of refactoring

ubuntu-filemanager-app-autopilot 2013/11/20 15:17 25 8 17 0 0 32%

    remove patching home

    refactor to use new folder

ubuntu-clock-app-autopilot 2013/11/20 15:10 11 11 0 0 0 100%

    refactor to only use local sources

    land mp's

    https://code.launchpad.net/~nik90/ubuntu-clock-app/use-default-swipe-delete/+merge/191905

    https://code.launchpad.net/~nik90/ubuntu-clock-app/more-stopwatch-tests/+merge/193922

    alarm tests

    UI alarm test in autopilot

    lower level tests for push notifications

ubuntu-calculator-app-autopilot 2013/11/20 14:59 30 30 0 0 0 100%

    some tests better as unit tests, but they all work and features covered

dropping-letters-app-autopilot 2013/11/20 14:55 5 5 0 0 0 100%

    Good, no code changes, tests rock solid

music-app-autopilot 2013/11/20 14:48 8 7 1 0 0 87.5%

    shuffle implementation and tests have been an issue

    Needs tests for interfacing with Aritsts, Albums, Songs tabs

    Needs tests for each tab to add an item to a playlist or the queue

    Needs tests to create and modify playlists

    Needs tests for swipe to delete of playlist items and queue items

calendar-app-autopilot 2013/11/20 14:41 7 2 5 0 0 28.6%

    3 mp's need to land / merge.

    once landed all features should have tests

    https://code.launchpad.net/~acerisara/ubuntu-calendar-app/failing-tests/+merge/193706

    https://code.launchpad.net/~vthompson/ubuntu-calendar-app/merge-ap-fix-branches has the AP 1.4 fixes as well as a few others. It is green as far as Jenkins can see. I think balloons has a merge that will fix a test that's failing on the device.

ubuntu-docviewer-app
sudokutouchgame

    not in public jenkins view

    can we add?

stock-ticker-mobile-app

    not in public jenkins view

    can we add?

    https://code.launchpad.net/~veebers/stock-ticker-mobile-app/porting-autopilot-1.4/+merge/193718
    https://code.launchpad.net/~veebers/stock-ticker-mobile-app/mock-server-for-testing/+merge/194971

Work items:
[nskaggs] add work items for above failures ^^!: DONE
[elopio] get the weather app to 100% green.: DONE
[popey] Add table to wiki page for <expand for all coreapp> listing features, implementation status and test coverage: TODO
[nskaggs] Fill in feature tables with current test coverage: TODO
Run test suite for <expand for all coreapp> using python 3, file bugs for any errors that are different from python 2: DONE
[elopio] make sure the SDK emulators work in python 2 and python 3.: TODO
[nskaggs] Make sure all core apps have MP test passing enforced before landing in trunk: DONE
[thomir-deactivatedaccount] Investigate how to record failure modes across test cases, report on blueprint, and hand over to relevant parties: TODO
[thomir-deactivatedaccount] Find out how to mock out the google RSS service with a fake. Give feedback to devs: TODO