Port checkbox-certification-server on top of plainbox
checkbox-
configurations for checkbox that define the tests to be performed for
server certification.
It has a few characteristics that make it a good candidate to start
using the new plainbox testing runner/engine. (Find a link to the
plainbox blueprint for a quick introduction):
- It's usually run only in very controlled environments
- Usually run by a trained technician who can be relied on to report and
help diagnose any problems.
- Most of the tests are automated, a few manual ones ask simple
questions that don't require complex interactions.
- At the end, it should submit the results to the certification website
and output a local XML report. (Do we want to include outputting HTML
here?)
Some components are missing for plainbox to fill this role. A user
interface and a secure way of running root-level jobs are examples.
This blueprint should result in a package that can be installed and a binary that can be run, to execute the server certification test suite, with a simple UI to ask the user for test verifications, and that can at the end submit results to the certification web site as well as output a locally-openable report, in XML or preferrably HTML.
Tackling the work needed to run these tests would carry the following
benefits:
- Creation of a prototype using the plainbox libraries, toolset and
architecture to build an actual, user-interactive application results
in a better understanding of Plainbox architecture, so developers get
the experience needed to tackle more difficult developments (such as
an eventual end-user-oriented desktop testing app based on plainbox)
- Use of a better-documented, more robust and better tested engine for
certification tests.
- End user (i.e. testers) benefit from having a more robust, simpler
tool. It's also faster and easier to adapt to their requests and
needs.
- Developers benefit from having plainbox as the testing core, since
behavior changes or improvements are easier to test and implement,
and plainbox makes test development and debugging easier.
- The work which will be potentially needed to organize and repackage
tests, even if it's only discussed during the 3-month cycle, will
enable better separation of checkbox tests and dependencies, which is
already being requested by some of our other efforts (cloud testing,
checkbox in Ubuntu server)
Topics to consider:
- urwid as possible ui?
Checkbox already abstracted the UIs, providing a simple 'API' for
UI classes to implement, this made adding new UIs potentially
simple. However, care needs to be taken with the data structures
that are sent back and forth (see a few bugs on the QT ui that
stem from assuming data is in one format (e.g. dictionary) and
then receiving something else (some missing keys, or a list)).
Also, this abstraction limited the possibilities of the UI, which
is one of checkbox's shortcomings that plainbox aims to remedy.
So while some care should be taken to make the UI code as
pluggable as possible, I think part of the emphasis should be
instad in making the plainbox libraries easy to use from whichever
UI design we choose to implement.
Blueprint information
- Status:
- Complete
- Approver:
- None
- Priority:
- Undefined
- Drafter:
- Zygmunt Krynicki
- Direction:
- Needs approval
- Assignee:
- None
- Definition:
- Approved
- Series goal:
- Proposed for trunk
- Implementation:
- Implemented
- Milestone target:
- 2013-05-23
- Started by
- Zygmunt Krynicki
- Completed by
- Zygmunt Krynicki
Whiteboard
ZK:
We need to look at how to move c-c-s on top of plainbox.
With the recent development on plainbox sru, we seem to have most of the backend logic ready. The challenge will be in putting that together to a more interactive user interface.
We also need to decide how to ensure continuity, should we have checkbox-
SP:
The link introducing plainbox: https:/
Work Items
Work items:
As a PlainBox developer I want to release PlainBox 0.3 so I can baseline all the code that was needed to perform SRU - M: DONE
As Marc Deslauriers I want PlainBox to execute jobs requiring root privileges in a secure way so I can allow ubuntu CheckBox to use PlainBox as its new core - L: TODO
As a PlainBox developer PlainBox I want to implement the secure way of running jobs as root (as agreed with the Security team) so I can run my tests without (SRU) or with just one password prompt (self-cert) - L: DONE
As a SRU Test Coordinator I want PlainBox to have the logging feature so I can debug remote systems - L: TODO
As a PlainBox developer I want to create a "CheckBox" application so I can provide an executable that has stable command line interface and production features vs PlainBox as the development tool - L: INPROGRESS
As a PlainBox developer I want to study how it's possible to reuse existing checkbox ui code so I can decide if a story to actually implement it is doable - M: TODO
As a PlainBox developer I want to reuse existing checkbox ui so I can minimize the work to be done to get certification packages using PlainBox core - L: TODO
As Ara I want a release of checkbox-