Logging

General
1

I’m not sure what everyone else is doing for logging but at the moment I am running a copy of Graylog (I have a vmware server at home so just uploaded the OVA, but VHD’s and docker images are available)

I have then use syslog-ng to forward the messages to my graylog instance

at the moment /etc/syslog-ng/syslog-ng.conf looks like:

@version: 3.3
# @include "scl.conf"

#SYSLOGNG_OPTS="--no-caps"

source s_file { file("/usr/local/bin/homegenie/log/homegenie.log"
    follow-freq(1) flags(no-parse, validate-utf8)); };

# Define TCP syslog destination.
destination d_net {
    syslog("192.168.0.67" port(514));
};
# Tell syslog-ng to send data from source s_src to the newly defined syslog destination.
log {
    source(s_file); # Defined in the default syslog-ng configuration.
    destination(d_net);
};

I think this can be improved to send the system messages too but just playing at the moment.

I have then created some extractors which currently look like this (Still experimenting)

You can use these by clicking on import extractors (once finished i’ll add to github)

{
  "extractors": [
    {
      "title": "Homegenie_InterfacePropertyChanged",
      "extractor_type": "grok",
      "converters": [],
      "order": 0,
      "cursor_strategy": "copy",
      "source_field": "message",
      "target_field": "",
      "extractor_config": {
        "grok_pattern": "%{GREEDYDATA:Callsite}\t%{TIMESTAMP_ISO8601:TimeStamp}\t%{LOGLEVEL:Log Level}\t%{DATA:Domain}\t%{WORD:Source}\t%{DATA:Description}\t%{DATA:PropertyPath}\t%{DATA:PropertyValue}",
        "named_captures_only": false
      },
      "condition_type": "none",
      "condition_value": ""
    },
    {
      "title": "Homegenie_WebServiceGateway",
      "extractor_type": "grok",
      "converters": [],
      "order": 0,
      "cursor_strategy": "copy",
      "source_field": "message",
      "target_field": "",
      "extractor_config": {
        "grok_pattern": "%{GREEDYDATA:Callsite}\t%{TIMESTAMP_ISO8601:TimeStamp}\t%{LOGLEVEL:Log Level}\t%{DATA:Source}\t%{IP:SourceIP}\tHTTP\t%{WORD:Method}\t%{BASE10NUM:StatusCode} %{PATH:RawUrl}",
        "named_captures_only": false
      },
      "condition_type": "none",
      "condition_value": ""
    },
    {
      "title": "Homegenie_InterfaceMessages",
      "extractor_type": "grok",
      "converters": [],
      "order": 0,
      "cursor_strategy": "copy",
      "source_field": "message",
      "target_field": "",
      "extractor_config": {
        "grok_pattern": "%{GREEDYDATA:Callsite}\t%{TIMESTAMP_ISO8601:TimeStamp}\t%{LOGLEVEL:Log Level}\t%{GREEDYDATA:LogMessage}",
        "named_captures_only": false
      },
      "condition_type": "none",
      "condition_value": ""
    },
    {
      "title": "Homegenie_InterfacePropertyChanged",
      "extractor_type": "grok",
      "converters": [],
      "order": 0,
      "cursor_strategy": "copy",
      "source_field": "message",
      "target_field": "",
      "extractor_config": {
        "grok_pattern": "%{GREEDYDATA:Callsite}\t%{TIMESTAMP_ISO8601:TimeStamp}\t%{LOGLEVEL:Log Level}\t%{DATA:Domain}\t%{WORD:Source}\t%{DATA:Description}\t%{DATA:PropertyPath}\t%{DATA:PropertyValue}",
        "named_captures_only": false
      },
      "condition_type": "none",
      "condition_value": ""
    }
  ],
  "version": "2.2.0-SNAPSHOT"
}

I’ve then created a stream for Homgenie messages which looks for messages with the callsite field present - this is crude and not ideal - but I really should read the documentation…

Anyway this allows you to get visibility of logging and configure alerting:

some screenshots if I can on here:

image.png1920×1080 242 KB

image.png1424×372 25.8 KB

Just noticed some of the grok queries arent working as expected so will fix and update :slight_smile:

Again this is a work in progress to help me track down some annoying bugs.

David

2

This looks really interesting. For me not only for HG but I have been annoyed with all logs for all things I’m running on my server and how hard it is to investigate and find the problem.

Looking forward to see how you get this to work with HG. Could be useful for me in many other ways too.

3

You could also use an ELK stack (we use this at work) but I preferred the simplicity of graylog.

Can also forward router eventlogs and windows logs too… but one step at time here :smiley:

4

At work, we also use ELK (but without Logstash) writing logs directly to the Elasticsearch through Serilog sink.
But for HG I don’t use any additional logging aggregations, just plain text files.