Power/Energy consumption testing plan discussion

Plan for installing the Power Usage Test Set (PUTS) (pronounced Putz) hardware in the lab. Preliminary planning that need to be scheduled/discussed:

1. Get the hardware into the lab and ready to use

  1A Ship the hardware to Lexington
  1B Install the hardware
      Requires:
      * rack space
      * 9 outlets
      * USB connection from test set to any accessible machine
        for debugging and reflashing firmware
  1C configure PUTS network configuration
  1D Run the PUTS API test script

  Completion criteria:

  The API test script can be run from a machine
  in the lab against the PUTS and have it pass.

2. Get the existing power tests migrated to the PUTS

  2A Figure out how to integrate the usage of the device with the current testing environment.
  2B Connect the AC current probe and Fluke meter to the PUTS,
      and run power tests there.

  Completion Criteria:

  Current power tests run successfully with the AC probe
  connected to the PUTS.

3. Test mobile devices using the PUTS

  3A. Modification of first battery device(s) to be used
  (Note: I need my modified Nexus7, a new device must be used)
  3B Set up PUTS for the new battery device(s)
      (adjust power supply output)
  3C Connect of battery device(s) to the test set
  3D Write tests for mobile devices
  3E Get power tests running regularly for mobile devices
  3F Display results reported for mobile devices

Kernel team's responsibilities in this will be limited to the following items, but they will be available to help with test set issues: 1A, 1C, 1D, 3A, 3B

Further discussion points:

A. Who owns the test cases, the QA Team doesn't have the required know-how to own the end to end testing on this instance, the test cases should still belong to the kernel team. Existing power tests are too low level for the QA team to own (they make sense in the kernel context but not necessarily in the whole system context). New ones will need to be developed if the QA Team is going to own them.

B. Which test cases are going to be run on Nexus 7 and why, it seems that running the existing desktop test cases may not make sense (hammering the memory and stressing i/o may not make as much sense has using the screen or playing a video, the right test scenarios for mobile devices need to be discussed/agreed upon.

C. Maintenance of the set up going forward.

D. Testing without the set up, i.e. just by using the battery chip, to be able to test on mobile devices without a complex set up or any modification (we can call this method "software energy consumption testing"). The set up in the lab is to be called "hardware energy consumption testing".

Documentation on how to use the battery chip: https://wiki.ubuntu.com/QATeam/AutomatedTesting/BatteryConsumption

Work items for ubuntu-13.04-month-5:
[sconklin] Complete assembly of power test set and ship to QA: DONE

Work items for ubuntu-13.04-month-6:
[ricklfowler] Sync up with sconklin for the diagnostics to verify everything's working fine: DONE
[sconklin] Run diagnostics on the lab to make sure everything went fine with the setup (need to be careful with it, it'll be very sensitive): DONE
[gema] Plan for the creation of new power test cases for the mobile devices: DONE
[gema] Look into chrome os tests as potential candidate to run and measure power consumption on (http://git.chromium.org/chromiumos/third_party/autotest.git): POSTPONED

Work items for ubuntu-13.06:
[ricklfowler] Work on setting up the equipment in the lab on the week starting 20120311: INPROGRESS
[ricklfowler] Modify devices to be used by test set: POSTPONED
[ricklfowler] Move and encorporate the power meter into the test set: POSTPONED
[ricklfowler] Ensure modified devices work with test set: POSTPONED
[pwlars] Prepare jenkins to use the new kit: TODO
[pwlars] Fix the existing jenkins power jobs to work with the new kit: TODO
[apw] nexus4: look at exposing better power data from the battery: DONE
[gema] Come up with preliminary test cases to run on phones for battery profiling: DONE
[gema] Testing of the new data exposed in the kernel on saucy image: DONE
[gema] Run premilinary test cases on the new kernel: DONE