pymon

Build JSON Configuration Schemas

Registered by Duncan McGreggor on 2011-12-14

An example of an old ZConfig for pymon is pasted below. This needs to be converted to something consumable by mongodb.

# legal values for check-by are 'services' and 'groups'
user pymon
group pymon
prefix /usr/local/pymon
pymon-app pymond
instance-name PyMonitor
check-by services
user-agent-string pymon Enterprise Monitoring (http://pymon.sf.net)
daemontools-enabled False
daemontools-service /service/pymond
log-level INFO
sendmail /usr/sbin/sendmail
mail-from <email address hidden>
smtp-username ausername
smtp-password asecret
smtp-server smtp.gmail.com
smtp-port 25

<monitor-status>
    # Here is where you tell pymon which of the defined services
    # (see below) should be monitored in its regular checks
    enable ping
    #enable ping tcp
    # Next is the section for services that will not be checked by
    # pymon.
    disable http status
    disable http content
    disable local process
    disable dns dig
</monitor-status>

<database>
connectionString sqlite:data/pymon.sqlite3
</database>

<admin>
    <config-update>
        interval 60
    </config-update>
    <backups>
        interval 60
        base-dir data
        state-dir state
        # change the next line to reflect the fact that it's a dir
        application-state application
    </backups>
    <peering>
        publish-interval 120
        check-interval 120
    </peering>
</admin>

<state-definitions>
    unknown -1
    failed 10
    error 20
    warn 30
    acknowledged 40
    recovering 50
    maintenance 60
    ok 70
    disabled 80
    escalated 90
</state-definitions>

<global-names>
    state state
    history history
    factories factories
</global-names>

<notifications>
    cut-off 3
    <types>
        enable smtp
        disable irc
        disable im
        disable rss
        disable snmp
        disable mud
        disable twitter
    </types>
    <list>
        email jojo.idiot@circus_boy.com
        email pretty@new_pet.com
    </list>
</notifications>

<web>
    port 8080
    doc-root static/web
    vhost-root vhost
</web>

<peers>
    url http://shell1.adytum.us:10001/json/localStates
    url http://shell2.adytum.us:10001/json/localStates
    url http://shell3.adytum.us:10001/json/localStates
    url http://shell5.adytum.us:10001/json/localStates
</peers>

<services>
<http-status>
<http-status-check>
    uri www.adytum.us
    enabled true
</http-status-check>
<http-status-check>
    warn-threshold 100-101, 201-206, 300-305, 401-403, 405-407, 411-416
    ok-threshold 200
    uri www2.adytum.us
    enabled true
</http-status-check>
</http-status>
<ping>
<ping-check>
    uri shell1.adytum.us
    enabled true
    escalation-enabled false
</ping-check>
<ping-check>
    uri shell2.adytum.us
    enabled true
    escalation-enabled false
    scheduled-downtime 2001.01.01 00:00:00 - 2010.01.01 00:00:00
</ping-check>
<ping-check>
    uri svn.adytum.us
    enabled false
    escalation-enabled false

    <notification-list-replace>
        email <email address hidden>
        email <email address hidden>
    </notification-list-replace>
</ping-check>
</ping>
</services>

Blueprint information

Status:: Started

Approver:: Duncan McGreggor

Priority:: High

Drafter:: None

Direction:: Approved

Assignee:: Duncan McGreggor

Definition:: Approved

Series goal:: None

Implementation:: Good progress

Milestone target:: None

Started by: Duncan McGreggor on 2011-12-15

Related branches

Related bugs

Sprints

Whiteboard

There are several ways to segment data in MongoDB:
* by database
* by collection
* by document
* by embedding in a document

So we could have a pymon database, a collection of remote services that are monitored, documents for each service that is monitored, and embedded data for lists of things that are related to an individual service.

The questions then become:
* when do we want data in a separate database?
* when do we want to split data out into separate collections?

"By doing this, certain repeating data no longer needs to be stored in every object, and an index on that key may be eliminated. More importantly for performance (depending on the problem), the data is then clustered by the grouping specified."

Collections may be sharded in MongoDB using a "...shard key, which determines how the collection is partitioned among shards. Typically (but not always) queries on a sharded collection involve the shard key as part of the query expression." So it would probably make sense to provide a field (maybe "shard_key" or "sharding_key", to avoid visual confusion with "shared key") in the monitored services collection that allows administrators to indicate how they want pymon data shardded. For instance, maybe just by host, or maybe by data center, or maybe by customer or department...

I'm thinking that we want two databases, one for all the services that are checked/monitored and one for everything else:
* pymon_config
* pymon_service_checks

pymon_service_checks could be organized by (via collections):
* service type,
* customer/organization, or
* host

Ideally, this is something that pymon admins would be able to easily decide and configure for themselves...

Work items:

[oubiwann] identify what do do with general configuration data: INPROGRESS
  * put data in pymon_config MongoDB
[oubiwann] identify what do do with "monitor-status" configuration data: INPROGRESS
  * rename to "service_type"
  * put data in pymon_config.service_type MongoDB
[oubiwann] identify what do do with "database" configuration data: INPROGRESS
  * this obviously doesn't belong in the database; probably pass as a parameter to pymon startup
  * need to figure out what do to about username/password info... maybe chmod 600 files?
[oubiwann] identify what do do with "admin" configuration data: INPROGRESS
  * rename to "app_services"
  * put data in pymon_config.app_services
[oubiwann] identify what do do with "state-definitions" configuration data: INPROGRESS
  * put in pymon.state_definitions
[oubiwann] identify what do do with "notifications" configuration data: INPROGRESS
  * put notifications -> cut-off into "app_services"
  * put notifications -> types into "pymon_config.notification_types"
  * put notifications -> list into "pymon_config.notification_list"
[oubiwann] identify what do do with "web" configuration data: INPROGRESS
  * move into "app_services"
[oubiwann] identify what do do with "agents" configuration data: INPROGRESS
  * put agents -> local-command into "app_services" as "local_agent"
  * put remove agents into pymon_config.agents
[oubiwann] identify what do do with "peers" configuration data: INPROGRESS
  * put remove agents into pymon_config.peers
[oubiwann] identify what do do with "services" configuration data: TODO
[oubiwann] as an intermediary step, convert configuration data to YAML: INPROGRESS
[oubiwann] converted YAML to JSON (programmatically): TODO

(?)

Work Items

Dependency tree

* Blueprints in grey have been implemented.

This blueprint contains Public information

Everyone can see this information.

Subscribers

No subscribers.