Logging Extra PIDs

[jrnudd]

I've enabled some extra PIDs within Torque for various things for my Passat TDI. Those extra PIDs are obviously not part of the torque_keys.csv file. I'm hoping to find out how to log these extra PIDs within the database and view the data through the web front end.

[surfrock66]

Here's what I've found...the extra PIDs are stored as varchar values, and not float values. In get_columns.php, only float values are explicitly pulled...if you try to pull varchar values, they fail and the page doesn't load.

Additionally, column names are stored in your information_schema table...you can get them using this mysql syntax (update your db name):

use information_schema; SELECT COLUMN_NAME,COLUMN_COMMENT,DATA_TYPE FROM COLUMNS WHERE TABLE_SCHEMA='hda_torque' AND TABLE_NAME='raw_logs';

You'll see that your column IDs mean nothing, and if there's no comment, then you can't really tell which PID is what.

I'm working on 2 pieces of code:

1. Update upload_data.php to try to check fields for value type BEFORE creating the field, then defining them as float when possible
2. Adding a feature to actually edit the column name comments with a web gui. I still don't know how the hell to figure out which PID = which value.

[surfrock66]

So, I don't have a great way to do this automated, but here's the manual process:

First, open the map view in torque, and choose a session, and email the csv to yourself. Also, open the torque-viewer, choose the same session, and save the csv. Open both of them. Look for a column that you can correlate between both, and sort by that value (there's one that's like seconds to trip which is discrete AND unique, pretty solid).

From there, manually correlate the column names...the torque-emailed version actually names the comments, whereas torque-viewer uses a code. I was able to correlate about 80% of them.

Lastly, torque-viewer stores the values as "varchar(255)" values, when the code checks for floats. You can modify your columns (that will translate to float values) with syntax like the following:

ALTER TABLE raw_logs MODIFY kff1223 float NOT NULL DEFAULT 0 COMMENT 'Acceleration Sensor(Total) (g)';

Now, the PID list will include that sensor (since it's a float) and the label will be the label you set. If you want to list out ALL of your columns (showing you what the comments are, and which need to be changed from varchar -> float) use this:

show full columns from raw_logs in hda_torque;

Going forward, it'd be useful for the "create column" code in "upload_data.php" checked some sample data and created a float when possible, I've started editing that code. I didn't find a clean way to perform an after-the-fact change, it's so infrequent it may just be better to do in SQL rather than make a ui for it.

[surfrock66]

I should mention, this has a pretty huge performance impact. In get_columns.php, there's a check to count distinct values for each known column...I commented that whole thing out to make it work reasonably.

[surfrock66]

Got it working. Example: http://hda.surfrock66.com/torquetest/session.php