The Memurai team • April 27, 2020

Memurai Event Log Integration

Technical articles and news about Memurai.

As a native Windows application, Memurai should work with the Windows Event Log. We received this request from several of our customers and as a result we decided to work on it. How difficult can it be, right?

TL;DR

Each row from the log file is recorded in the Event Log as “Information”. No exceptions.
The Task Category in the Event Log will capture the granularity level (debug, verbose, …).
A unique EventID is used for each distinct message.
The EventData contains: the PID, instance-name (if any), role, the parent PID (if any), and the actual message.
We are recording events using two event sources: Memurai and Memurai-Sentinel
More details on the Memurai documentation site.

Longer version

Let's explore the existing log file and the Event Log integration individually.

The existing log file

Similar to Redis™*, Memurai generates a log file where events are recorded as time passes. Each row in the log file packs plenty of metadata and a human-readable message. In the configuration file we have a flag called loglevel which establishes how much information is recorded. The four log levels are:

debug - (a lot of information, useful for development/testing)

verbose - (many rarely useful info, but not a mess like the debug level)

notice - (moderately verbose, what you want in production probably)

warning - (only very important / critical messages are logged)

One can think of the Memurai log levels as levels of granularity: more details are written into the log file for the debug level and less details for the warning level. They don't necessarily represent the severity of the events. For example:

[25428] 14 Apr 13:10:04.419 # Connection with master lost.
[428] 14 Apr 14:05:06.614 # Server started

These are both warnings, but while the first might require some action, the second just looks like a notification.

This is the way Redis works, and Memurai behaves exactly the same to ensure compatibility.

By using the Windows Event Log, the Memurai log can be reviewed on the specific machine, remotely over the network, or by most SIEM systems.

The Windows Event Log

According to the Microsoft Event Log documentation there are 6 different types of events to be logged: EVENTLOG_SUCCESS, EVENTLOG_AUDIT_FAILURE, EVENTLOG_AUDIT_SUCCESS, EVENTLOG_ERROR_TYPE, EVENTLOG_INFORMATION_TYPE, EVENTLOG_WARNING_TYPE.

Since the log file levels represent granularity and not severity, it didn't make sense to make a fixed mapping between the log file levels and the Event Log levels.

How Memurai logs to the Event Log

Each row from the log file is recorded in the Event Log as “Information”. No exceptions.
The Task Category in the Event log will capture the granularity level (debug, verbose, …).
A unique EventID is used for each distinct message.
The EventData contains: the PID, instance-name (if any), role, the parent PID (if any), and the actual message.
We are recording events using two event sources: Memurai and Memurai-Sentinel.

An image worth a 1000 words

Each row in the Event Log is recorded with the Level as Information and has a unique Event ID (i.e. 62196)

 The Task Category is mapped to the loglevel (i.e. Warning)

 The EventData contains: the PID (i.e. 884), instance-name (if any, in this case Memurai-Rocks), role (i.e. sentinel), the parent PID (if any), and the actual message (i.e. Sentinel ID is 4b3…)

The full list of log file mappings to the Event Logs is described on the Memurai documentation site.

Configuration flags for the Event Log

We added two new flags which are Memurai specific:

winlog-level - with five different options: debug, verbose, notice (default), warning and off.
instance-name - to distinguish between different instances of Memurai running on the same server. By default, it is blank.

To maintain backward compatibility with Redis, we did not make any changes to the existing loglevel flag.

This implementation gives the user plenty of flexibility. For instance, the Windows Event Log can be turned off while the log file is populated with verbose logs. Or the Windows Event Log can be set at the debug level while the log file records only warnings.

Pro tip to filter the Event Logs

If you prefer to explore only the Memurai-related events, we suggest creating a Custom View and selecting the event sources “Memurai” and “Memurai-Sentinel”.

Resource summary

Thank you Dave Jensen, @djensen47 for the detailed article Redis Log Format and Log Levels .

Mappings to the Event Logs are described the Memurai documentation site

The Microsoft Event Log documentation

Redis is a trademark of Redis Ltd. Any rights therein are reserved to Redis Ltd. Any use by Memurai is for referential purposes only and does not indicate any sponsorship, endorsement or affiliation between Redis and Memurai