Introduction

‘Table Builder’ is a universal tool for converting text log files into a numeric table suitable for analysing with ‘Scanalyser’ or other tools. The utility is especially useful with mixed field types where the field values can be a mix of ASCII numeric or hexadecimal. It can also combine segmented hex bytes into a single decimal value. ‘Table Builder’ generates a table in the precise format chosen by the user with selectable field names and all values converted to numeric. Default numeric precision is to 3 decimal places and it can handle 32 bit hexadecimal.

A comprehensive filter allows specific records to be selected into the table using 2 criteria employing: ‘equal to’, ‘not equal to’, ‘greater than’, ‘less than’ and ‘select between two values’. Delimited files using space, comma or tab separation are handled automatically. Fixed length fields require the user to define the partitions by positioning a cursor.

For frequent processing of log files with the same format, the user settings can be saved in template form to remove the need of repeating field identification and the selection criteria. This results in a very speedy conversion process.

‘Table builder’ is in the form of a wizard presenting each stage in sequence with the ability to step back through the process.

Sequence

The fist task is to select the file to be used as the source for data.
Clicking on ‘Select’ in the window shown below opens a dialogue from which to select the file you wish to use.

Once the file has been opened, it will be displayed in the window below.
The text will be greyed out until the format style has been selected:

Delimited allows the delimiting character to be selected
Fixed field requires partitions to separate the fields
Template uses a saved format from a previous session

When a box has been checked (selected) the file text will turn black allowing a typical record line to be chosen by clicking on it. This is necessary in case the file consists of dissimilar record types like comments or records of a different format.

Delimited

You can use the arrow keys or the mouse to move the cursor to the delimiting character such as a space or comma. As the cursor moves across the record, the selected character is displayed in the bottom box. When this shows the delimiting character click on ‘OK’

Fixed field

You can use the arrow keys or the mouse to move the cursor to the position representing the border between 2 adjacent fields. Pressing the Enter key fixes the position which is marked with a vertical bar and the number of characters constituting the field length is displayed in the top list. Pressing the DEL key will remove the partition marker. When all the required fields have been separated, pressing ‘Done’ will save and continue. Note: it is not necessary to partition fields you are not intending to use.

If the field lengths are known, then their values can be entered directly in the field widths box. Use caution when segmenting fields as text fields can be left or right justified: E.g. “2345 CDEF”. The space separating the two fields can belong to either “2345_” or “_CDEF”.If in doubt check other records that may show “23456|CDEF” or “2345|BCDEF”. Fields not to be used can be ignored. Although they will appear initially as one large field they will be discarded by the conversion process.

The drop down button at the right hand side of the field width box displays the last 10 selections that were used. Clicking over a line automatically applies that separation format. This removes the tedium of having to duplicate a common field width format.

Template

If a previous setup has been saved as a template then it can be selected from a list of templates using the name used to save it. This will segment fields, apply field names and apply the selection criteria precisely as it was used originally.
The option to save the setup is offered by the second last frame.

Fields from the ‘typical’ record selected are displayed on the left in a column.

Defining fields:
Click on the field you want to use from the left list and drag it into the ‘Field Ref’ box. If it is a hexadecimal value then check the ‘Hex’ box. Enter a name for the field in the ‘Field name’ box then click on ‘Add’ to add it to the selected fields.
In the above example, reference 12 has been selected as the time base field and named ‘TB’ this is a numeric (N) value. Field reference 1 has been identified as a hex field (H) and named ‘PGN’ This value will be converted into decimal from its original hex value. Similarly, Field ref 2 is designated as hex and named ‘STATE’. The field currently being processed comprises 2 hex bytes ref(4,5). It is important to put these in the correct sequence starting with the most significant byte. This will be verified by its converted (decimal) value being displayed in the ‘Example’ window as other field refs are added. To clear the field ref should the order be incorrect, click the right mouse button or use the DEL key.
Click ‘Next’ to continue.

After selecting the fields you want to use, you can define a filter to select only specific records you wish to be built into the table.
All the fields will be listed on the left as in the previous screen as you may wish to use them as part of the selection criteria without necessarily including them in the table.
If you want to build all the available records into the table without filtering then click ‘No filter’ to skip to the next window.
Note: records not conforming with the format used by your ‘typical’ record will not be included in the table. I.e. comments etc.
In the above example, we only want to select records where the time base (ref 12) value is between 0.1 seconds and 2.76 seconds. The Field ref box contains the ‘12’ reference, its Type is ‘Num’ (numeric) and the criteria is selected as ‘Between’. The 2 values are separated by a comma. Pressing the ‘Prove’ button displays the subsequent translation using the original field value from the ‘typical’ record selection. If the value of the ‘typical’ record was below 0.1 and above 2.76 then the Proof box would have concluded with ‘False (exclude)’ to indicate that these records would not have been included in the table.
Note, you cannot proceed to the next stage until the ‘Prove’ button has been clicked as this fixes the selected criteria.

In this example we have added a second selection criteria.
Field reference 5 is a text character field and we have requested that only records where this field’s value is “2D” will be included.
The ‘Proof’ box has added the extra criteria and as it does indeed contain the required “2D” value it has been accepted for inclusion.
If the field has been designated as a hex type then the proof box will display both the hex and decimal values (in brackets).
You can enter segmented hex values as the ‘Field ref’ exactly as with the previous field selection window.
The 2 criteria are conditionally joined by the AND clause, alternatively OR can be used.
Click ‘Next’ to continue.

When you are happy with the filter you can proceed to the next stage by pressing the ‘Build table’ box. Progress is indicated by the bottom thermometer bar.

When the build process is complete, the new table is displayed in the window for checking. The table has not been saved at this point so you can click ‘Back’ to refine your selection if necessary.

The table can be saved to any location in Excel or DBF format. (Both if you repeat the last stage). For graphing the table with Scanalyser it is better to select the DBF option as it is more compact and operates faster.
The option to save the setup is offered by the second last frame.