Skip to content

Latest commit

 

History

History
81 lines (58 loc) · 4.68 KB

README.md

File metadata and controls

81 lines (58 loc) · 4.68 KB
Raw

TSViz is a tool for studying executions of multi-threaded systems.

  • Try the tool in your browser!

  • TSViz is described in a short paper:

    Studying multi-threaded behavior with TSViz. Matheus Nunes, Harjeet Lalh, Ashaya Sharma, Augustine Wong, Svetozar Miucin, Alexandra Fedorova, Ivan Beschastnikh. ICSE 2017, Tool demonstration.

    @inproceedings{Matheus2017,
  author    = {Nunes, Matheus and Lalh, Harjeet and Sharma, Ashaya and Wong,
               Augustine and Miucin, Svetozar and Fedorova, Alexandra and Beschastnikh, Ivan},
  title     = {{Studying Multi-Threaded Behavior with TSViz}},
  year      = {2017},
  isbn      = {9781538615898},
  publisher = {IEEE Press},
  url       = {https://doi.org/10.1109/ICSE-C.2017.9},
  doi       = {10.1109/ICSE-C.2017.9},
  booktitle = {Proceedings of the 39th International Conference on Software Engineering Companion},
  pages     = {35–38},
  series    = {ICSE-C}
}

Logs generated by multithreaded systems are complex and unwieldy. They can be huge and contain extraneous details. It can be challenging to piece together logs generated at multiple hosts. Even if these logs contain ordering information, they are often still too complex for a developer to understand. TSViz helps developers understand such logs through visualization.

TSViz is a fork of ShiViz that integrates wallclock timestamp information into the visualization.

Log Format Basics

TSViz parses the log using a user-specified regular expression. The regular expression must contain four capture groups:

  • timestamp: the physical clock timestamp of the event
  • event: the description of the event
  • host: the name of the thread, thread, or host that executed the event
  • clock: the vector clock associated with the event, in JSON format {"host": timestamp} format. The local host must be represented in the vector clock (i.e. the host specified in the host capture groups must be one of the hosts in the vector clock). The first local event for a particular host must have a clock value of "1", and from there it must increase monotonically by one.

A parser regexp must be specified, otherwise an error will occur.

The Input page has a field for setting the regular expression alongside the large text area for the actual log.

For example, to parse log entries like this one:

1456966522871394967 Exiting eviction_wait_handle.0x18988c0__wt_spin_unlock                                                                                                                                                                  
thread4 {"thread4":58}

We would use the following parser RegExp:

(?<timestamp>(\d*)) (?<event>.*)\n(?<host>\w*) (?<clock>.*)

The capture groups in TSViz is not standard regexp syntax. We recommend using a plain JS regexp tester with regular capture groups to develop expressions, and name the capture groups afterwards.

Loading from a File

When uploading a log from a file, the file must have the regular expression(s) inserted prior to the log. An input file must have the following structure:

  • The first line of the file is the log parsing regular expression.
  • The second line of the file is the multiple executions regular expression delimiter. This can be left empty if the log does not contain multiple executions.
  • The rest of the file is the log.

Additional fields

Additional capture groups can be used to capture fields such as date, IP, and priority. These fields will not be displayed in the log lines alongside the graph, but instead as a table in the sidebar when selecting an event event (unless the field's capture group is a sub-group of the event group). Field names must be alphanumeric with no spaces.

For example, to parse log entries like this one, which contains information regarding path and priority:

[0001000 voldemort.store.metadata.MetadataStore] INFO metadata init().
main {"main":1}

We could use the following parser RegExp, which contains additional fields for path and priority:

\[(?<timestamp>\d*) (?<path>\S*)\] (?<priority>(INFO|WARN)) (?<event>.*)\n(?<host>\S*) (?<clock>{.*})

How can I generate logs to use in TSViz?

You can study the execution of any system that produces logs in a format that can be parsed using the RegExp mechanism above.

We recommend DINAMITE instrumentation tool that outputs TSViz-compatible logs.