SYSLOG Parsing Deep Dive

In our last ELK blog, we set up ELK (Elasticsearch, Logstash, and Kibana) on Docker. Now we're going to transform those complex logs into building blocks that will allow you to create visualizations in Kibana.

If you haven't yet read Hands-On Practice with the ELK Stack, then you can get started by cloning the following GitHub repository to set up your ELK Stack: https://github.com/object1st/elk-stack.git

_{cd elk-stack}  

run command:

docker-compose up -d

Then, navigate to your browser at the IP address or DNS name of your VM (or local host if you are doing this locally) and on port 5601, access Kibana: http://myvm:5601

Parsing Syslog

By the end of this walkthrough, you can expect to have:

• Parsed a complex syslog message
• Created intelligent threat detection that spots a ransomware indicator automatically
• Built a dashboard to display the alert

The Challenge: Complex Log Structures

Let's start with a real-world syslog message.

Example: Veeam Malware Detection

Mar 20, 2025 @ 14:06:09.662 VBRSRV01 Veeam_MP: [origin enterpriseId="31023"] [categoryId=0 instanceId=41600 DetectionTimeUTC="03/20/2025 13:05:41" OibID="0e54d3bf-add8-48eb-9122-fad3ac1e8fb3" ActivityType="EncryptedData" UserName="TECH\user1" ObjectName="VM01" VbrHostName="vbrsrv01.tech.local" Description="Potential malware activity detected for OIB: 0e54d3bf-add8-48eb-9122-fad3ac1e8fb3 (VM01), rule: Encrypted data by user: TECH\user1."]

Why These Messages Are Important

You’ll find valuable insights in this log message—take a moment to explore it.

Veeam message:

• Potential ransomware activity in progress
• Specific user account involved (TECH\user1)
• Affected virtual machine (VM01)
• Behavioral pattern (Encrypted Data activity)

But in its current state, you can't easily:

• Track which vms are potentially affected.
• Correlate suspicious activity across time
• Generate reports on this activity

Step 1: Start Simple, Build Smart

Here's where most people go wrong: trying to parse everything at once and ending up with a configuration that does not work.

If you'd like to follow along, locate the files in the “Examples” folder of the GitHub repository. If you’ve already installed ELK in Docker from the previous blog post, simply copy those files into your setup. As you work through this blog, replace the existing files with the updated versions provided here. If you make a mistake or get lost while editing, you can always re-clone the repository to start fresh.

1. To use, delete the logstash.conf in the pipeline by replacing it with 

   logstash_version1.conf ($ cp ~/elk-stack/Examples/logstash_version1.conf ~/elk-stack/logstash/pipeline/logstash.conf)

2. Restart the logstash container ($ docker-compose restart logstash).

Why this approach works:

• Start with working input/output
• Use stdout to see exactly what Logstash receives (will display in the docker container logs)
• Raw logs still go to Elasticsearch (nothing is lost)
• Build complexity gradually without breaking things

Step 2: Test Your Setup

1. Send your sample messages to Logstash and observe the output. Here are a few methods that you can use to send a syslog message yourself.
2. Using netcat:

• Copy and paste the following command to your CLI (Veeam Syslog Message):

echo '<14>1 2025-03-20T14:06:09.662307+02:00 VBRSRV01 Veeam_MP - - [origin enterpriseId="31023"] [categoryId=0 instanceId=41600 DetectionTimeUTC="03/20/2025 13:05:41" OibID="0e54d3bf-add8-48eb-9122-fad3ac1e8fb3" ActivityType="EncryptedData" UserName="TECH\user1" ObjectName="VM01" VbrHostName="vbrsrv01.tech.local" Description="Potential malware activity detected"]' | nc -u -q0 localhost 5514

• You should see something like this when you type docker logs logstash:

This is a problem because everything Valuable is Trapped in One Giant Message Field:

“"message" =&gt; "<14>1 2025-03-20T14:06:09.662307+02:00 VBRSRV01 Veeam_MP - - [origin enterpriseId=\"31023\"] [categoryId=0 instanceId=41600 DetectionTimeUTC=\"03/20/2025 13:05:41\" OibID=\"0e54d3bf-add8-48eb-9122-fad3ac1e8fb3\" ActivityType=\"EncryptedData\" UserName=\"TECH\\user1\" ObjectName=\"VM01\" VbrHostName=\"vbrsrv01.tech.local\" Description=\"Potential malware activity detected\"]\n",”

In order to address this problem, parse the logs to make them cleaner.

Here's where things get interesting (and slightly terrifying if you're new to regex). We're going to dissect this log and make it easier to pick out bits and pieces of information.

Before writing any code, let's understand what we're working with:

Veeam Structure: [timestamp] [hostname] [application]: [origin_data] [detection_data]

Step 3: Basic Parsing

Copy the next logstash file:

1. To use, delete the logstash.conf in the pipeline by replacing it with logstash_version2.conf  ($ 

cp ~/elk-stack/Examples/logstash_version2.conf  ~/elk-stack/logstash/pipeline/logstash.conf) 

2. Restart the logstash container ($ docker-compose restart logstash)

Grok Pattern Decoded:

• %{POSINT:priority} → Captures positive integers as "priority"
• %{TIMESTAMP_ISO8601:timestamp} → Built-in pattern for ISO timestamps
• %{IPORHOST:host} → IP address or hostname
• %{GREEDYDATA:raw_message} → Everything else (we'll parse this next)

This new Grok pattern is designed to break down raw syslog messages into structured, usable data. Specifically, it separates the basic components of a syslog message (like timestamp, hostname, and log level), identified the application source generating the log, and tagged each message for further processing down the pipeline. This foundational step is what enables more advanced parsing, enrichment, and alerting logic later in the workflow.

3. Test again to see differences:

echo '<14>1 2025-03-20T14:06:09.662307+02:00 VBRSRV01 Veeam_MP - - [origin enterpriseId="31023"] [categoryId=0 instanceId=41600 DetectionTimeUTC="03/20/2025 13:05:41" OibID="0e54d3bf-add8-48eb-9122-fad3ac1e8fb3" ActivityType="EncryptedData" UserName="TECH\user1" ObjectName="VM01" VbrHostName="vbrsrv01.tech.local" Description="Potential malware activity detected"]' | nc -u -q0 localhost 5514

4. Then, check the logstash container logs ($ docker logs logstash).

The fields that we extracted:

_{priority → 0}

_{timestamp → 2025-03-20T14:06:09.662307+02:00}

_{host → ["172.19.0.1", "VBRSRV01"]}

_{application → "Veeam_MP"}

_{severity → "Emergency"}

_{severity_level → 0}

Step 4: Deeper Dive on Parsing

Now we will do even more parsing. The file is too big to print here, but it is in your “Examples” folder as logstash_version3.conf.

1. To use, delete the logstash.conf in the pipeline by replacing it with: logstash_version3.conf ($

   cp ~/elk-stack/Examples/logstash_version3.conf ~/elk-stack/logstash/pipeline/logstash.conf)

2. Restart the logstash container ($ docker-compose restart logstash).
3. At this point, test your configuration. You should see individual fields instead of one big message blob.
4. Run the same test message again, then check the logstash container logs ($ docker logs logstash).

The fields that we extracted:

_{enterprise_id → "31023"}

_{category_id → 0 (converted to integer)}

_{instance_id → 41600 (converted to integer)}

_{detection_time_utc → "03/20/2025 13:05:41"}

_{detection_timestamp → 2025-03-20T13:05:41.000Z (properly parsed timestamp)}

_{oib_id → "0e54d3bf-add8-48eb-9122-fad3ac1e8fb3" (unique alert ID)}

_{activity_type → "EncryptedData" (malware indicator)}

_{username → "TECH\\user1"}

_{object_name → "VM01" (affected VM)}

_{vbr_hostname → "vbrsrv01.tech.local"}

_{description → "Potential malware activity detected"}

Step 5: Veeam Threat Intelligence and Error Handling

Last but not least, we need to add intelligence to our system and error handling to help us debug if needed. Adding intelligence is important because it enables filtering by business impact, allows priority-based alerting, makes dashboard creation intuitive, provides actionable recommendations, and reduces alert fatigue.

Here’s how to add intelligence and handle errors:

1. To use, delete the logstash.conf in the pipeline by replacing it with: logstash_version4.conf ($ 

   cp ~/elk-stack/Examples/logstash_version4.conf ~/elk-stack/logstash/pipeline/logstash.conf)

2. Restart the logstash container ($ docker-compose restart logstash).
3. Wait for the pipeline to be running then send the test message again:

echo '<14>1 2025-03-20T14:06:09.662307+02:00 VBRSRV01 Veeam_MP - - [origin enterpriseId="31023"] [categoryId=0 instanceId=41600 DetectionTimeUTC="03/20/2025 13:05:41" OibID="0e54d3bf-add8-48eb-9122-fad3ac1e8fb3" ActivityType="EncryptedData" UserName="TECH\user1" ObjectName="VM01" VbrHostName="vbrsrv01.tech.local" Description="Potential malware activity detected"]' | nc -u -q0 localhost 5514

What new fields we extracted:

_{threat_type → "potential_ransomware" (smart threat categorization)}

_{event_category → "malware" (security classification)}

_{business_impact → "high" (risk assessment)}

_{alert_priority → "critical" (escalation priority)}

_{recommendation → "Immediate investigation required" (response guidance)}

_{event_type → "security_event" (event classification)}

Section Summary

In this section, we’ve successfully completed the following:

• Added business context to raw events
• Created intelligent severity classification
• Built threat categorization system
• Enabled priority-based alerting

Step 6: Create an Index Template

Before we start sending data into Elasticsearch, we need to optimize it for our specific use case.

1. In the GitHub repository, you will find the index_template.json file, which we can import into Kibana.

2. Let’s use a curl command to import our index_template.json from the “Examples” folder.

If you have trouble copying the commands here, you can also find them in github repository README.md file

curl -X PUT "localhost:9200/_index_template/veeam-syslog-template" \ -H "Content-Type: application/json" \ -d @index_template.json

3. You should see this message if the import was successful.

4. Next, we will create an Index Pattern (In later versions called Data Views).

• Type in the following command:

curl -X POST "localhost:5601/api/saved_objects/index-pattern/syslog-pattern" -H "Content-Type: application/json" -H "kbn-xsrf: true" -d '{"attributes":{"title":"syslog-*","timeFieldName":"@timestamp"}}'

• You could also create through the WebUI.

5. Check to see if the Index Pattern was created by taking the following actions:

• Go to Stack Management
• Click on “Index Patterns” under “Kibana”
• Here we can see our newly created Index Pattern.

6. Next, we need to create a visualization.

• Click on “Visualize Library.”
• Click on the “Lens” tile.
• From the drop-down menu, choose “Metric.”

7. In the filter field, type: threat_type: “potential_ransomware”

8. Choose “Overtime” down in the middle (this is your choice of what you want the dashboard to look like).

9. Save the Visualization by clicking on the "Save” button in the top right corner and add to a new Dashboard:

10. Change the “Name” under Metrics to “Alerts” and choose Records on the left.

• You should end up with this:

11. Save this to your Visualization Library and choose “No Dashboard” for now.

12. Next, create another visualization that we will call “Affected Machines.”

• This time, choose Table, not Metric, and use the “Top values of object_name” field and again record. Change the names of the Columns by pressing on the right side names and changing the Display name:

13. Save to your library again.

14. Now, let’s create another table visualization—this time using the “Top Values of Username” field. This can help identify which users might have been logged in during the attack. Keep in mind, this is purely for exploratory purposes and demonstrates how creatively you can leverage the data.

15. Finally create an area visualization—this time showing alerts and timelines.

16. Feel free to create more or change existing ones in order to get the feel for this. We have created some very basic visualizations just to explain the process.

17. Finally, create a Dashboard and add these visualizations.

18. Save the Dashboard with the name “Potential Ransomware.”

19. To test, let’s send another alert as if it were from Veeam but change some of the components of the message so we can see if it gets ingested properly.

20. Let’s change the username to Gustev and the VM to MainFrameSuper.

echo '<14>1 2025-03-20T14:06:09.662307+02:00 VBRSRV01 Veeam_MP - - [origin enterpriseId="31023"] [categoryId=0 instanceId=41600 DetectionTimeUTC="03/20/2025 13:05:41" OibID="0e54d3bf-add8-48eb-9122-fad3ac1e8fb3" ActivityType="EncryptedData" UserName="NASA\Gustev\" ObjectName="MainFrameSuper" VbrHostName="vbrsrv01.tech.local" Description="Potential malware activity detected"]' | nc -u -q0 localhost 5514

• As we can see, our Dashboard alerts us immediately upon receipt of the syslog message:

Conclusion

When you started this series, you faced a 347-character wall of cryptic text:

<14>1 2025-03-20T14:06:09.662307+02:00 VBRSRV01 Veeam_MP - - [origin enterpriseId="31023"] [categoryId=0 instanceId=41600 DetectionTimeUTC="03/20/2025 13:05:41" OibID="0e54d3bf-add8-48eb-9122-fad3ac1e8fb3" ActivityType="EncryptedData" UserName="TECH\user1" ObjectName="VM01" VbrHostName="vbrsrv01.tech.local" Description="Potential malware activity detected"]

Now, that same message triggers an intelligent security monitoring system that immediately identifies:

• Threat Type: Potential ransomware activity
• Priority Level: Critical business impact
• Affected User: TECH\user1 (identified as human, not service account)
• Compromised Asset: VM01 virtual machine

In this blog, we gave readers a clearer picture of what needs to happen to a raw syslog message before it can be useful in a SIEM—specifically within the ELK stack. From parsing to enrichment, we broke down the essential steps that turn noisy logs into actionable insights.

In the next and final part of our SIEM Elasticsearch mini blog series, we will briefly go over leveraging Fluentd instead of Logstash and touch on Elastic Security.