Ingest XML events

[Blaapje]

How much work would it take to ingest xml data rather than evtx files?

The XML data is generated by: https://github.com/williballenthin/EVTXtract. The tools recovers (partial) xml records from corrupted partitions.

Could you indicate if this is feasible and give some pointers towards ingesting xml data? I'd be happy to submit a PR if I get this to work.

[wagga40]

As a quick response I can say that Zircolite already handles XML formatted logs because it is the native format of sysmon for linux logs (--sysmon4linux).
I haven’t tested with logs coming from EVTXtract.

[wagga40]

Ok forgot what I've said, I took a look to the output of EVTXtract and it is a little bit more difficult than that...

[Blaapje]

It's rather interesting that both EVTXtract and evtx_dump are both maintained by @williballenthin. I will see if I can get this working.

Edit: I don't know how I came to this conclusion but this is obviously not the case.

[wagga40]

In Zircolite everything (EVTX, Auditd, Sysmon For Linux) is converted to JSON lines format.
These operations are handled by the evtxExtractor class. For each log file the Logs2JSON function is called for each line in the log file to convert it to JSON.

Since the XML output of EVTXtract is not in the "1 event per line" format, the first thing is to handle this. The easiest way to to it would be to modify EVTXtract (check here) (but I guess that's not what you want to do) and use XML to JSON conversion logic like the one used here.

If you have samples I can help...

[Blaapje]

I've made a small sample available here: https://gist.github.com/Blaapje/23e378748db737960a90c723c2dc8676

[wagga40]

Thanks, I will try this week end.

[wagga40]

Hi @Blaapje, worked on it ! I currently only handle events beginning with <event> and not the other ones (i.e <record>) since they don’t contains the fields that allows to apply sigma on them. I will push something today.

[wagga40]

You can try with this branch : https://github.com/wagga40/Zircolite/tree/evtxtract.
Please provide some feedback.