Home → Supercharger KB → Functionality → Continuity Tracing

3.18. Continuity Tracing

Supercharger remediates many WEC issues, often times without the end user knowing an error has occurred.  For example, our watchdog for stalled subscriptions and stalled event logs does this.  WEC has proven itself to be very robust.  From the endpoints to the collectors WEC is very reliable but sometimes issues lie beyond the control of WEC.  One thing that has come to our attention is an issue of dropped events upstream of WEC and to the downstream consumer (think SIEM) losing events because they are down or not ingesting events as fast as they are coming to to the WEC collector - and thus the event log wraps before the SIEM consumes some events.  This is where Supercharger's Continuity Tracing feature plays a roll.

What is Supercharger's Continuity Tracing?

We inject tracer events into your WEC subscription event flow so that you can determine if your downstream consumer is receiving all of the events you are collecting.

How does it work?

Supercharger logs event ID 42031 to the Application Log on your WEC collector with log source "Supercharger Continuity Tracing".  Then for each destination log (ForwardedEvents or a Supercharger custom destination log) Supercharger creates a Supercharger managed subscription.  This subscription subscribes the local collector as the only source computer and "Supercharger Continuity Tracing" as the only event log source to collect in to the destination log.  The custom "Builtin ContinuityTracing Subscription Policy" is configured so that tracer events are forwarded immediately.  

These subscriptions will be displayed on the dashboard of the collector with the naming convention "ScContinuityTracing###".  

If the feature is disabled, Supercharger will delete the tracing subscriptions and stop logging the event.  If a destination log no longer has subscriptions targeting it then Supercharger deletes the continuity tracing subscription.  

How to use event 42031?

Event ID 42031 is logged at the correct time within a few hundredths of a second at the max. For instance, if you configure it to inject a tracer event every minute, then it will log a 42031 at the top of the minute.  Here is an example of the event:

Supercharger tracer event. 
Collector: lab-sc2-2.lab.local
TraceStart: 5/11/2024 5:34:17 AM -07:00
Sequence: 167
Scheduled: 5/11/2024 8:22:00 AM -07:00
SecondsBetweenTracers: 60
Use this event to ensure downstream pipeline components (e.g. SIEM) are keeping up with event flow and to measure lag. Sequence is a counter that re-starts at 0 with Supercharger service startup (see TraceStart above). Scheduled should be very close to Event.TimeCreated. Disable this tracing by setting ContinuityTraceLoggerDisable to true.  Adjust SecondsBetweenTracers by setting ContinuityTraceLoggerSeconds to a factor of 60 between 5 and 60.  Change EventId with ContinuityTraceLoggerEventId.Compare this event along with event ID 42032 to intake statistics from your downstream logging pipeline consumers to make sure they are keeping up with the rate of events. 

The event details are explained in the event itself.  The sequence above restarts at zero when the service is restarted.  So in the your downstream consumer (Splunk, LogRhythm, ArcSight, Qradar, etc...) analytics you need to either look for 42031 being received at every sequence (default 60 seconds) or look for gaps in the sequence number.  To help identify the sequence starting back at zero (when the service restarts or server restarts) the event content also includes a start time which is static per sequence.

You can use overrides in Superchargers settings to control and customize continuity tracing:

This page was: Helpful | Not Helpful
← Stalled Event Log Watchdog EPS Tracking →