I'm trying to come up with some more custom health queries that address a bit of a range of issues. The way I look at it, there has to be a good way to track the health of a rule, specifically by looking at the contributing log sources, and their parsing health.

I designed a relatively basic search that displays something relatively similar to what I need, except I'm not entirely sure that the data can be stitched together.

The premise I have is as follows:

Parsing, or health issues a log source will create unexpected behavior, that should, in theory, trickle down to the unparsed v. parsed event count by each log source. I'd like to show the difference in parsed v. unparsed (isunparsed,unknown event count,$whatever_works_best$ by log source.

Here's a sample of what I'd like to see:

Time/Time Range, Log Source Type, count(*), isunparsed,last_time_period

timefoo1,Log Source Bar,32000,true,y

timefoo2,Log Source Bar,2000,false,n

timefoo3,Log Source Foo,20000,false,y

timefoo4,Log Source Foo,0,true,n

OR

Time/Time Range, Log Source Type, last_count(*),diff_between_periods,uparsed_count

timefoo1,Log Source Bar,32000,30000

timefoo2,Log Source Foo,20000,0

Once this is complete, I'm wondering if I can add the unique count of rules that the log source contributes to, and potentially a separate query for a juxtaposed dashboard pane that shows Log sources and the number of rules they contribute to.

P.S. - I do have an Anomaly rule that determines when a log source's average event count is lower than n percent in t time; so I assumed that this type of search can be done via AQL.

This does essentially what I needL

SELECT LOGSOURCENAME(logsourceid) AS "LogSourceName",

DATEFORMAT(starttime,'yyyy-MM-dd HH:mm:ss') AS "Start Time",

LONG(UNIQUECOUNT(qid)) AS "Unique QIDs",

LONG(COUNT(*)) as "Total Events",

count(isunparsed) AS "Unparsed Count", 

LONG("Total Events"-"Unparsed Count") AS "Parsed Count",

LONG("Unparsed Count"*100) / "Total Events" AS "Percent Unparsed",

LONG("Parsed Count"*100) / "Total Events" AS "Percent Parsed"

FROM EVENTS 

GROUP BY logsourceid

LAST 1 HOURS