Port WI to Vala

---------------- ph7 ----------------

I have done the first version of the indicator client in Vala.
It includes the entry point, a subclass of Indicator.Object (from libindicator) and two custom gtk.menuitem widgets:
- A location menuitem, subclass of radiomenuitem, to display a selectable location on the left and its temperature on the right.
- A "conditions" menuitem, to display the current location's conditions. It is based on the document provided in bug #703815 and the omgubuntu mockup. The top part is just like in omgubuntu mockup: 3 values, the location, the condition summary and the temperature on the right (without title labels) plus the enlarged condition image on the left. From then on, follows a table like structure with arbitrary condition fields, titles on the left and values aligned nicely on the right. The widget is not selectable and takes care of the rearranging and the updating of the various fields. Ultimately the user should be able to select which condition fields will appear on the table and with what order (we should add this to preferences).
These two widgets are enough for now I think. Later on we could add more, like add a custom radar image display widget, or a lunar clock widget.

That was the easy part :) The service end needs some preparation. Namely the current python code, which exists in one executable python file (installed in /usr/bin/indicator-weather), needs refactoring. I have started working on this (got the trunk code). The various classes should go to their own files inside indicator_weather package. All except the main "indicator_weather" class which will evolve to the main service class.

I have thought the new structure of the program like this:

A) Indicator client (Vala module) in /usr/lib/indicators/4/libweather.so
 This will get loaded by the Unity panel automatically. Upon loading it tells to Indicator.ServiceManager to start our indicator dbus service in the session bus. The service manager starts the service and the indicator:
- creates a gtk dbusmenu client
- obtains a dbus proxy to our service end
- registers our custom menuitem widgets
- creates one entry in the indicators menubar (with icon, label, or both - we could use the scrolling for switching)

B) Service entry point (Python executable file) in /usr/lib/indicator-weather/indicator-weather-service.py
This gets initiated by the Indicator.ServiceManager and it:
- registers a dbusmenu
- registers a service interface for the client to connect to
- performs all the tasks that the current "indicator_weather" class performs, but instead of updating a gtk menu it updates the dbusmenu (the client gets automatically notified)
- answers all the calls the client makes via the service interface

C) Base Python module "indicator_weather" in /usr/lib/indicator-weather/indicator_weather
It contains all the existing functional classes (Location, Weather, Assistant, Preferences, etc).

D) The various data files as they are now in /usr/share/indicator-weather

E) A DBus service description file, installed in /usr/share/dbus-1/services, that points to the service executable (B)

There is one big difference notable to the end user, in comparison with the current state:
There isn't a desktop launcher. The user can't anymore start or stop the program. If it is installed, it gets loaded and it runs as long as the session runs. This is enforced by the indicator holder & loader (namely the unity panel). As you can't quit the datetime indicator you cannot quit this one. You also can't unload a loaded indicator module. And if the service end suddenly stops, for whatever reason, it gets restarted automatically by the Indicator.ServiceManager. If you think this is a problem (I don't, personally I dislike the quit button), we could only try and fake a "quit" behavior. We can hide our indicator icon and set the server on a suspended state (no weather updates). Then we could have a desktop launcher file that sends a command in
our dbus service to wake-up and resume the work.

[roignac] I don't think we do need a 'Quit' button - other indicators don't use it. If user doesn't need a weather indicator, the app should be uninstalled

---------------- ph7 ----------------

Server part is almost done.
Changes to current Python code:
- Classes are refactored in separate files inside the package
- Removed global references (log, wi). Logger instance is passed between classes.
- Constants and default settings moved to package's __init__.py
- Shared constants between server and client are in a Vala file and during the build a Python equivalent file is auto-generated.
- Dialogs are unique instances, launched from the server
- Preferences dialog usage simplified to only reading and saving settings. The service class is the sole responsible for scheduling weather updates.

CMake is used for building & installing the whole project.

The dependencies are libindicator and libdbusmenu (-glib and -gtk).
While dbusmenu provides vapi files for Vala bindings and gir&typelib for Python GIR bindings, there is nothing available for libindicator, so we have build our own. Luckily it's very easy to do this with vala and gnome introspection tools.

[roignac] Great job! Can't wait to try this!
[ph7] Now you can!

TODO:
- Make a new debian rules & control file, that uses cmake.
- During the weather updates, we need to get basic weather info for the non-selected locations, so that we can display temperature and icon
[ph7] This is done now

- Log client events to a file
[roignac] I guess, the service and indicator should use separate log files. Any thoughts on that?
[ph7] Yes absolutely separate files.

- Detect network availability and act accordingly
[roignac] We may use network-manager bindings, but not sure, whether we will benefit from this. If connection cannot be established, just show cached data and retry in 1 minute.
[ph7] Why not save system calls if we can get the detection right?

- Detect system suspend/resume and act accordingly
[roignac] there is already bug #742397

[ph7] I am hit with the missing yahoo_id bug, so I changed a bit the logic. If any source id is missing for a location, the alternative source (Google in this case) is used.

---------------- ph7 ----------------

I have migrated everything to Natty. There was a dbusmenu vala api bug that is now fixed and hope that soon it'll be released so that you can build the code. I'll post it as soon as the fix is released.

From the discussions in the related bug, is apparent that we need the forecast in main menu.
For this I've split the current "conditions" menuitem widget into two widgets:
1) one widget with the header only: Title (can be "Now" or "Today"), Subtitle (the condition text), Info (the temperature) and the icon.
We can use it to display both the current condition and today's forecast.
It is also essential that we keep a timestamp for the fetched weather.
So instead of "Now" we can display a title like "Two hours ago" if we have only cached data two hours old.

2) one widget with a table that displays arbitrary fields
This will display various fields from current condition

I'm working on a nice system to access weather fields and update the menu items.
But I need to simplify a bit the weather updates.
Currently on every refresh:
- Cached weather dict is accessed from CouchDb
- Menu is updated
- New Location object is constructed
- New Weather object is constructed
- Weather object fetches fresh data
- Weather data are saved in CouchDb
- Menu is updated

The problems I see:
- Multiple objects created/destructed
- Various weather fields are recalculated every time they are accessed
- Adding a new weather provider will complicate the code more
- Cached data display is only really needed on first time a location appears on the menu.
- Cached data might be on different units than the current ones.
- Forecast data are not supplied although they are present in the fetched report

To address this, I split the Weather class into:
1) A Weather class that provides the various data fields formatted with the current units.
Fields are calculated only once (unless the formatting parameters changed).
2) A WeatherProvider abstract class.
The subclasses of this, should fetch the data from a source, parse them and store them in raw format (float etc) and fixed units. This includes forecast data.
3) A YahooWeatherProvider class
WeatherProvider subclass that fetches data from Yahoo
4) A GoogleWeatherProvider class
WeatherProvider subclass that fetches data from Google
5) A CacheWeatherProvider class
WeatherProvider subclass that fetches stored data from CouchDb

There is only one Weather object instance in the program for every active location.
Initially is loading a CacheWeatherProvider object to load raw data from cache.
It then creates a WeatherProvider subclass according to the desired source.
Then on every refresh:
- The Provider object fetches fresh data, parses and stores them
- These raw data are saved in CouchDb
- The Weather object provides the various needed fields, based on the raw data and the formatting settings
- Menu is updated

All this is half done at the moment, by the end of the week it'll be finished.
Next step is to create a carousel like widget for selecting the location.

---------------- ph7 ----------------

I pushed new code into the branch.
It works on Natty only and it still needs debugging.
Natty (or rather unity*) of course is buggy as hell.
The bug #750575 is fixed now, but I'm hit with the terrible (and show stopper) bug #755581

Anyway, there are new models for weather fetching/accessing and for the menu.
Added new files menu.py, indicator.py, service.py, detached from server.py.
Added new subfolder "providers", for the various weather sources.

The new Weather class, provides weather report fields, organized in sections: Current conditions, Today's Forecast and Forecast for 3 next days. Yahoo only provides forecast for two days, whereas Google for four days.
Now it's very easy to add new weather sources.

The menuitems are defined statically (in MenuItem class) and support automatic mapping of a weather field into an icon/label.
So it is also very easy to add a new menuitem and map it to a weather condition field.

I've added the forecast info into the main menu and changed the location radio items into generic items. The current location's label is always centered at the top of the menu. This will be consistent with the upcoming "carousel" selection of the active location.

The current condition widget has a new more compact layout as you'll see.
Two rows only and a smaller icon:
Now (or Today or 1,2,3 hours ago): [Temperature]
    [Condition]

PS. I love this :)
---------------- ph7 ----------------

Editing to pin all TODO items I can remember at the end:

I think this port needs to land in new series (0.4) because the change is dramatic and will only support Natty and beyond.
It's unlikely that it will be ready for foggy release anyway; still there is a lot to do.
It would be better to release foggy without this (so that every Ubuntu version is covered) and start series 0.4 with this and an unstable ppa, so that more users can test it.
[roignac] I agree, there is no need to support old versions of dbusmenu. Maverick users will be bound to pure python version for some time, but this give a period to test this in unstable ppa. I'll try to compile a new weather indicator and put it in unstable ppa
[roignac] We will continue to update Cloudy version with important patches for Maverick and Lucid, but the main focus should be on Natty version

---------------- ph7 ----------------
I've updated the branch.
- Added the new location selector, which will be always visible and just show arrows when more than one location is set.
- Split Settings class into Settings and CouchBackend and updated the whole settings system.
- Added some new settings and updated the Preference dialog to present them to the user
- Added a "no data available" menu item, to appear when we don't have cached data for a new location, until we fetch new data.
- Enhanced the weather report validation & the field formatting, added barometer and visibility for Yahoo, added arrows for wind speed direction and barometer rising/falling status.
- Current condition is saved for all locations
- Notifications can be shown for all locations (it's an option in preferences)
- Various bug fixes
- Ensured proper keyboard/mouse navigation in the menu:
- Non-selectable items do not appear active, do not dismiss the menu when clicked, do not grab focus or respond to key-presses.

---------------- ph7 ----------------
Vala-port branch is merged, so I've updated the foggy and ubuntu recipe branches:
- Updated the build system for foggy/natty
- Created basic logging for the client
- Changed CouchBackend to use the new api (DesktopDatabase). Also compact is done in the beginning and the views are only created if they are missing.
- Merged some fixes from cloudy
- Created an INSTALL file and updated the README file
- Updated the condition fields (added 'Wind chill' and multiple 'Feels like')
- Added option to autohide by default the missing/invalid fields)
- Made usage of error codes on various functions
- Updated TableMenuitem to allow icons and more flexible widget positioning
- Implemented the saving of the weather data in the settings db

---------------- ph7 ----------------
Just posted to inform for the status:
The new Location/Provider/Report model is almost ready.
Uses PyGObject's dynamic introspection bindings (Gtk,Gio,GLib,GObject instead of the static gtk,gobject etc)
It also uses Gio asynchronous i/o instead of threading & urllib.
The Assistant's location search uses search-as-you-type. With the asynchronous i/o it's very smooth and fast.

------------ TODO ---------------------------
[roignac: Updated the whiteboard]
- implement the saving of the new weather model into couchdb [ph7: DONE]
[roignac: will try to reuse existing DB for foggy release]
[ph7: There are many differences between the two versions, for the moment I use weatherindicatorcs db]
ph7: I'll make use of the same database but I'll change the view names]

- implement new location add/edit dialogs and new location model. Make use of geoclue for autolocation. Move the location search code from Assistant to Location class. Make use of Location objects instead of location_details and places dict/list.
 [ph7: on progress]

- proper shutdown of the server, on session exit

- detect network availability (almost done)

- code cleanup and documentation
[ph7: let's agree to a common coding style. I'd prefer to use the standard: no-tabs/only-spaces/no-trailing-whitespace, 80 chars line limit]

- merge recent patches from parent branch (that's tricky, because the architecture changed significantly) [roignac: TODO]
[ph7: Merged almost all of the applicable ones]

- request (create patch) for proper inclusion in the indicator sort list of unity & unity2d, otherwise the indicator will appear at the far left end; I think it should appear near the time.

- request (create patch) for proper inclusion of libweather.so in indicator-applet-complete and/or indicator-applet-appmenu, otherwise the indicator will not appear in the classic gnome desktop. Alternatively create a new gnome-panel applet as a holder for our indicator.

- add apport hooks for foggy version

- use translated strings in the indicator. We need to generate new template to get the new strings.