Parse Demos For Stats

[leighmacdonald]

Instead of using sourcemod plugins and log parsers, I want to parse STV demos directly. Pulling out many more stats and fixing some bugs with the current system.

The work for this is being done @ https://github.com/leighmacdonald/tf2_demostats

This project will implement a simple HTTP api that will allow the upload of a demo and return the json formatted parsed stats from it.

This api may eventually be opened up for the publics use as well, possibly paid.

[unclamped]

@leighmacdonald I'm not sure if I got your idea correctly, but could it be this what you are looking for?
https://github.com/MegaAntiCheat/demolyzer

[leighmacdonald]

@unclamped Yeah i'm aware of that project already and probably makes more sense for their project to do it in python if they are trying to (eventually...) feed it into some ml models.

Currently the project im working on that I link to does a very similar thing, implementing the parsing using the same underlying rust library, but it also exposes it over a HTTP API service so i don't really gain anything by using a python interface. Additionally i want to keep the CPU usage as low as possible as we process a lot of demos, and i want to eventually open it up for the public to use.

[Gcommer]

Additionally i want to keep the CPU usage as low as possible as we process a lot of demos, and i want to eventually open it up for the public to use.

It's not upstreamed as its still pretty hacky, but I was able to get a pretty huge (~10x) speedup on some custom tf2-demo-parser analyzers by being very careful about handling entity props: demostf/parser@master...Gcommer:parser:perf

I'd be down to contribute some profiling work if this is a pain point for you.

[leighmacdonald]

Sounds good, my rust experience is... rusty.. heh. its been almost 10 years since i've touched it so i'm having to relearn stuff as i work on this.

[Gcommer]

I've got some time to contribute again -- if you'd like I'd be up to contribute some parser work. I'd dive in myself but https://github.com/leighmacdonald/tf2_demostats is private

[leighmacdonald]

Hey @Gcommer, ive invited you to the repo if you want to take a look (I do plan to make it public eventually).

Right now its just a skeleton waiting to be filled out for the most part. A lot of the work already exists as far as parsing the stats out, but its not integrated yet. Lots of relevant stuff is commented out, though mostly originates from examples in the base parsing library. I created a very simple web service that accepts demos and will spit out some basic details in json.

The first thing to look at would be just filling out the PlayerSummary struct probably if you wanted to take a crack at it.

Ive added initial support for using it to gbans, but it just extracts the player steamid's currently, replacing the existing functionality which was just doing a ugly exec'ingthe binary directly.