Stop Drowning in Log Files: Analyze Log Durations with DuckTrace

Are you tired of painstakingly sifting through log files, struggling to pinpoint the source of performance bottlenecks or errors? Say hello to DuckTrace, a lightweight, open-source Go tool designed to automate log analysis and calculate event durations. It's the fast and easy way to understand what's happening in your logs with clear, color-coded output.

What is DuckTrace and Why Should You Care?

DuckTrace is your new best friend for analyzing log durations. Manually calculating the time between specific events in your logs is a tedious, error-prone process. This tool automates that process, saving you valuable time and improving your debugging efficiency. Forget manual calculations, DuckTrace gives you the answers you need quickly and efficiently.

Key Benefits of Using DuckTrace for Log Analysis

• Automated Log Parsing: Define your log format once using regex and let DuckTrace handle the rest.
• Precise Duration Tracking: Get accurate duration measurements between critical events, such as backup start and completion.
• Customizable Event Definitions: Track any event you want by defining your own start and end regex patterns in the configuration file.
• Clear, Concise Output: Results are displayed in a color-coded format, making it easy to scan and identify potential issues at a glance.
• Simple Setup: Built in Go for easy installation and portability across different systems.

How DuckTrace Works: Detective Work Made Easy

DuckTrace streamlines the log analysis workflow into a few simple steps:

1. Configuration: You provide a TOML configuration file that defines your log format and the specific events you want to track.
2. Parsing: DuckTrace reads your log file, extracting timestamps and messages based on your regex pattern.
3. Matching: The tool identifies start and end events by matching your defined regexes against the parsed log data.
4. Timing: DuckTrace calculates the duration between each matched start and end event.
5. Output: Finally, it presents a color-coded summary of the calculated durations and averages directly in your terminal.

Getting Started with DuckTrace: A Step-by-Step Guide

Here's how to set up and run with DuckTrace:

1. Prerequisites: Ensure you have Go 1.20 or newer installed.

2. Clone the Repository:

   git clone https://github.com/shrsv/ducktrace.git
   cd ducktrace
   

   Copy

3. Install Dependencies:

   go mod tidy
   

   Copy

4. Build the Tool:

   go build -o ducktrace ducktrace.go
   

   Copy

5. Run DuckTrace:

   ./ducktrace --config config.toml --log sample.log
   

   Copy

Configuring DuckTrace: Unleashing the Power of TOML

The config.toml file is the heart of DuckTrace, where you define how the tool interprets your logs and identifies events. It consists of a LogFormat section that defines the universal log structure and an Events section that defines events by specifying start and end regular expressions.

Here's a breakdown of a sample config.toml:

log_level = "debug"

[LogFormat]
pattern = '^(?P<date>\d{4}-\d{2}-\d{2}) (?P<time>\d{2}:\d{2}:\d{2}) \[(?P<level>\w+)\] (?P<message>.+)$'

[Events.backup]
start_regex = 'Backup job triggered'
end_regex = 'Backup job completed'

[Events.backup_nightly]
start_regex = 'Nightly .* started'
end_regex = 'Nightly .* completed'

[Events.error_rollback]
start_regex = 'Deployment failed'
end_regex = 'Rollback completed'

Key elements:

• log_level: Set to "debug" for verbose output.
• [LogFormat].pattern: A regular expression with named capture groups (date, time, level, message) to parse each log line.
• [Events.<name>]: Defines an event, where <name> is the event's unique identifier, along with its start_regex and end_regex.

Real-World Examples: Making Log Analysis Practical

Example 1: Analyzing Backup Durations

Let's say you have a backup.log file with the following content:

2025-04-25 10:00:00 [INFO] Backup job triggered
2025-04-25 10:00:05 [INFO] Backup job completed
2025-04-25 22:00:00 [INFO] Nightly backup started
2025-04-25 22:00:10 [INFO] Nightly backup completed

Using the config.toml configuration shown above, running ./ducktrace --config config.toml --log backup.log will analyze the log file showing the duration of backup and backup_nightly events.

Example 2: Tracking Deployment Rollbacks

Consider a deploy.log file containing information about failed deployments and rollbacks:

2025-04-25 09:00:00 [ERROR] Deployment failed for app v1.2.3
2025-04-25 09:00:10 [INFO] Rollback completed for app v1.2.3
2025-04-25 10:00:00 [ERROR] Deployment failed for app v1.2.4
2025-04-25 10:00:15 [INFO] Rollback completed for app v1.2.4

You can analyze the rollback durations by updating the config.toml as shown below:

log_level = "debug"

[LogFormat]
pattern = '^(?P<date>\d{4}-\d{2}-\d{2}) (?P<time>\d{2}:\d{2}:\d{2}) \[(?P<level>\w+)\] (?P<message>.+)$'

[Events.rollback]
start_regex = 'Deployment failed for app'
end_regex = 'Rollback completed for app'

Running ./ducktrace --config config.toml --log deploy.log will output the following:

Event: rollback
- 2025-04-25 09:00:00: Deployment failed for app v1.2.3 -> 2025-04-25 09:00:10: Rollback completed for app v1.2.3 (Duration: 10s)
- 2025-04-25 10:00:00: Deployment failed for app v1.2.4 -> 2025-04-25 10:00:15: Rollback completed for app v1.2.4 (Duration: 15s)
Average duration: 12.5s

Extending DuckTrace: Tailor it to Your Needs

DuckTrace is designed to be extended and adapted to your specific log analysis needs. Here's how you can customize it:

• Add New Events: Simply add new [Events.<name>] sections to your config.toml file.
• Tweak Regular Expressions: Adjust the start_regex and end_regex to match the specific patterns in your logs.
• Change Log Formats: Modify the [LogFormat].pattern to accommodate different log structures.

Limitations and Future Enhancements

DuckTrace is a simple tool with some limitations:

• Single-Line Logs: It assumes that each event is contained within a single log line.
• Basic Pairing: It matches the first start event with the next end event, which may not be accurate for nested events (not ideal).
• Static Analysis: It processes static log files and does not support live log streams.

Future enhancements could include:

• Multi-line log support.
• Advanced event pairing algorithms for complex scenarios.
• Live log monitoring with a --watch mode.

Ready to Dive In?

DuckTrace offers a simple yet effective solution for log analysis, helping you find the root cause quickly than ever. Clone the repository, customize the configuration, and start extracting insights from your logs today! Share your feedback and contribute to the project on GitHub - your input is valuable!