Logging with Heka

Registered by Éric Lemoine

This blueprint proposes a distributed and scalable logging system based on Heka.

Blueprint information

Status:
Complete
Approver:
Steven Dake
Priority:
High
Drafter:
Éric Lemoine
Direction:
Approved
Assignee:
Éric Lemoine
Definition:
Approved
Series goal:
Accepted for mitaka
Implementation:
Implemented
Milestone target:
milestone icon mitaka-3
Started by
Steven Dake
Completed by
Steven Dake

Related branches

Sprints

Whiteboard

Gerrit topic: https://review.openstack.org/#q,topic:bp/heka,n,z

Addressed by: https://review.openstack.org/270906
    [WIP] Logging with Heka spec

Gerrit topic: https://review.openstack.org/#q,topic:heka,n,z

Addressed by: https://review.openstack.org/275840
    Add a Dockerfile for Heka

Addressed by: https://review.openstack.org/275841
    Add Heka to common role

Addressed by: https://review.openstack.org/275842
    Have Heka collect the HAProxy logs

Addressed by: https://review.openstack.org/275843
    Have Heka collect the Keepalived logs

Addressed by: https://review.openstack.org/275844
    Have Heka collect the RabbitMQ logs

Addressed by: https://review.openstack.org/275845
    Add Heka decoder for OpenStack logs

Addressed by: https://review.openstack.org/275846
    Have Heka collect the Nova logs

Addressed by: https://review.openstack.org/275847
    Add Heka decoder for MariaDB logs

Addressed by: https://review.openstack.org/275848
    Have Heka collect the MariaDB logs

Addressed by: https://review.openstack.org/276667
    Add Heka log decoder infrastructure

Addressed by: https://review.openstack.org/276668
    Add Heka log decoder for RabbitMQ

Addressed by: https://review.openstack.org/276760
    Make Heka collect Glance logs

Addressed by: https://review.openstack.org/279191
    Make Heka collect kolla-toolbox logs

Addressed by: https://review.openstack.org/279192
    Add Heka log decoder for Keystone/Apache

Addressed by: https://review.openstack.org/279193
    Make Heka collect Keystone logs

Addressed by: https://review.openstack.org/279489
    Make Heka collect Neutron logs

Addressed by: https://review.openstack.org/281259
    Make Heka collect Cinder logs

Addressed by: https://review.openstack.org/281327
    Make Heka collect Heat logs

Addressed by: https://review.openstack.org/281328
    Make Heka collect Murano logs

Addressed by: https://review.openstack.org/281329
    Make Heka collect Magnum logs

Addressed by: https://review.openstack.org/279490
    Make Heka collect Swift logs

Gerrit topic: https://review.openstack.org/#q,topic:bug/1547535,n,z

Addressed by: https://review.openstack.org/283034
    Run yum clean all in Heka Dockerfile

Addressed by: https://review.openstack.org/283065
    Make cleanup-containers remove bootstrap_heka

Addressed by: https://review.openstack.org/283066
    Make cleanup-containers remove Heka volumes

Addressed by: https://review.openstack.org/283071
    Create "heka" group and add "heka" user to it

Gerrit topic: https://review.openstack.org/#q,topic:bug/1547536,n,z

Addressed by: https://review.openstack.org/283086
    Change Heka sudoers file mode bits

Addressed by: https://review.openstack.org/283118
    Fix Swift logging

Addressed by: https://review.openstack.org/283150
    Remove Rsyslog entirely

Addressed by: https://review.openstack.org/283177
    Make Heka collect Mistral logs

Addressed by: https://review.openstack.org/284074
    Update Debugging Kolla section of documentation

Addressed by: https://review.openstack.org/286876
    [WIP] Make Heka collect Manila logs

Addressed by: https://review.openstack.org/288432
    [WIP] Add cron image and playbook

I am marking this blueprint complete since it lacks a work items section and its impossible to tell if there is just feature creep with things like logrotate. I filed a bug for logrotate review here:
https://bugs.launchpad.net/kolla/+bug/1553244

Please file bugs in rc1 if you want them addressed relating to heka in the future.

Addressed by: https://review.openstack.org/294206
    Rework heka log copying

Addressed by: https://review.openstack.org/294350
    Fix 'optional' config.json values

(?)

Work Items

This blueprint contains Public information 
Everyone can see this information.

Subscribers

No subscribers.