Simple Approach :
Initially, positions of each packet are provided to the server for parsing.
Format of each packet in string representation : header; imei; value_1; value_2…….value_N; checksum
Example of packet received on server is as follows :
Input packet
“48;868324025308367;$GPRMC,000905.006,V,,,,,0.00,0.00,060180,,,N*48\r\n$GPGGA,000905.006,,,,,0,0,,,M,,M,,*42\r\n;52;52;52;52;52;52;52;52;48;41;……….;”
Pros:
Ease for debugging at communication level as packet is present in human readable format.
Cons:
Having no flexibility of managing different sensors(quantity and length) in individual devices.
Approach-1 :
Format of each packet in binary representation : header; imei; trip_no; version_no; type_1-length_1-value_1; type_2-length_2-value_2…….type_N-length_N-value_N; checksum
Example of packet received on server is as follows :
Input packet
“\x00\x00\x00\x41\x38\x36\x38\x33\x32\x34\x30\x32\x31\x34\x36\x31\x38\x35\x35\x00\x00\x00\x42\x00\x02\x00\x0a\x00\x14\x00\x13\x00\x12\x00\x11\x00\x10\x00\x15\x00\x16\x00\x17\x00\x18\x00\x19”
For following a packet:
4 byte header(unsigned int) | 15 byte IMEI(string representation) | 4 byte trip_number(unsigned int) | 2 byte type_no(unsigned int)let be T | 2 byte length(unsigned int)let be x | x length byte packet of type T(unsigned int) | …
65 | 868324021461855 | 66 | 2 | 10 | 20,19,18,17,16,21,22,23,24,25 | ….
Detailed trace:
“\x00\x00\x00\x41” = 65
“\x38\x36\x38\x33\x32\x34\x30\x32\x31\x34\x36\x31\x38\x35\x35” = “868324021461855”
“\x00\x00\x00\x42” = 66
“\x00\x02” = 2
“\x00\x0a” = 10
“\x00\x14\x00\x13\x00\x12\x00\x11\x00\x10\x00\x15\x00\x16\x00\x17\x00\x18\x00\x19” = 20,19,18,17,16,21,22,23,24,25
Pros:
Gives flexibility of managing different sensors(quantity and length) in individual devices.
Suitable for both static length and dynamic length values sensor fields.
Cons:
Increases the size of each packet by repeating same “Type-No” and “Length” in every packet as it will remain same for every packet of same firmware version.
Debugging at communication level requires conversion of binary to the decimal representation.
During parsing at the server end requires iterating byte by byte for fetching readings of byte-stream and it is sequential processing approach.
Approach-2 :
Format of each packet in binary representation : header; imei; trip_no; version_no; type_1-length_1; type_2-length_2…….type_N-length_N; value_1; value_2…….value_N; checksum
Pros:
During parsing at the server end requires iterating byte by byte for identifying the structure of packet till type and length ie., about 10% of the packet. And for rest 90% of the packet, we can apply the mask to fetch data from respective sensors from value. It is parallel processing approach.
Packet size is same as Approach-1.
Can apply the mask for directly fetching data of specific sensor from complete byte-stream.
Suitable for both static length and dynamic length values sensor fields.
Approach-3 :
Format of initial packet when device reboots : imei; version_no; type_1-length_1; type_2-length_2…….type_N-length_N; checksum
Format of each packet in binary representation : header; imei; trip_no; version_no; value_1; value_2…….value_N; checksum
Pros:
Individual packet size gets reduced by about 10%
Can apply the mask for directly fetching data of specific sensor from complete byte-stream.
Cons:
There is no way for identifying parsing approach if the first packet is lost on every reboot.
Not Suitable for dynamic length values sensor fields.
Approach-4 :
Format uploaded on server via “meta_file” when hex file of firmware is uploaded on server: version_no; type_1-length_1; type_2-length_2…….type_N-length_N; checksum
Format of each packet in binary representation : header; imei; trip_no; version_no; value_1; value_2…….value_N; checksum
Pros:
Individual packet is independent of the first packet and is just dependent on availability of new firmware details on server
Individual packet size gets reduced by about 10%
Can apply the mask for directly fetching data of specific sensor from complete byte-stream.
Cons:
Increases dependency of first uploading firmware code and processing formats(“meta_file”) on the server before installing to any device.
Not Suitable for dynamic length values sensor fields.
Approach-4 Modified:
Format uploaded on server via “meta_file” when hex file of firmware is uploaded on server: version_no; type_1-length_1; type_2-length_2…….type_N-length_N; checksum
Format of each packet in binary representation : header; imei; trip_no; version_no; value_1; value_2…….value_N; type_V1-length_V1-value_V1; type_V2-length_V2-value_V2; type_V3-length_V3-value_V3; checksum
Here “type_Vx-length_Vx-value_Vx” represents packet of variable length
Pros:
Individual packet is independent of the first packet and is just dependent on availability of new firmware details on server
Individual packet size gets reduced by about 10%
Can apply the mask for directly fetching data of specific sensor from complete byte-stream.
Suitable for both static length and dynamic length values sensor fields.
Cons:
Increases dependency of first uploading firmware code and processing formats(“meta_file”) on the server before installing to any device.
Refs.
https://www.ldeo.columbia.edu/res/pi/MB-System/formatdoc/gsf_spec.pdf
Example of “Approach-4 Modified”
Format uploaded on server via “meta_file” when hex file of firmware is uploaded on server: (we can choose “meta_file” format in decimal or an hex) version_no(4); type_1(1)-length_1(2); type_2(1)-length_2(2)…….type_N(1)-length_N(2); checksum
Example: “\x00\x00\x00\x41\x02\x00\x14\x03\x00\x14\x04\x00\x14”
Which represents
version_no: 65 with “\x00\x00\x00\x41”
type_1: 2 with “\x02” ie., acceleration_x
length_1: 20 with “\x00\x14”
type_2: 3 with “\x03” ie., acceleration_y
length_2: 20 with “\x00\x14
type_3: 4 with “\x04” ie., acceleration_z
length_3: 20 with “\x00\x14”
Format of each packet in binary representation : header(4); imei(15); trip_no(4); version_no(4); value_1(length_1); value_2(length_2)…….value_N(length_N); type_V1(1)-length_V1(2)-value_N(length_V1); checksum
Example:“\x00\x00\x07\xDA\x38\x36\x38\x33\x32\x34\x30\x32\x31\x34\x36\x31\x38\x35\x35\x00\x00\x0C\x8A\x00\x00\x00\x41\x00\x14\x00\x13\x00\x12\x00\x11\x00\x10\x00\x15\x00\x16\x00\x17\x00\x18\x00\x19\x00\x14\x00\x13\x00\x12\x00\x11\x00\x10\x00\x15\x00\x16\x00\x17\x00\x18\x00\x19\x00\x14\x00\x13\x00\x12\x00\x11\x00\x10\x00\x15\x00\x16\x00\x17\x00\x18\x00\x19\x05\x00\x96\x24\x47\x50\x52\x4D\x43\x2C\x31\x35\x30\x38\x35\x31\x2C\x41\x2C\x32\x38\x33\x38\x2E\x31\x39\x37\x36\x2C\x4E\x2C\x37\x37\x32\x34\x2E\x36\x38\x36\x39\x30\x30\x30\x30\x30\x30\x30\x31\x2C\x45\x2C\x33\x30\x2C\x32\x35\x30\x2E\x37\x31\x2C\x32\x36\x31\x30\x31\x36\x2C\x2C\x2C\x41\x2A\x36\x31\x0D\x0A\x24\x47\x50\x47\x47\x41\x2C\x31\x35\x30\x38\x35\x31\x2C\x32\x38\x33\x38\x2E\x31\x39\x37\x36\x2C\x4E\x2C\x37\x37\x32\x34\x2E\x36\x38\x36\x39\x30\x30\x30\x30\x30\x30\x30\x31\x2C\x45\x2C\x31\x2C\x39\x2C\x30\x2E\x38\x38\x2C\x31\x37\x39\x2E\x30\x2C\x4D\x2C\x2D\x33\x36\x2E\x31\x2C\x4D\x2C\x2C\x2A\x34\x31\x0D\x0A”
Which represents
header: “\x00\x00\x07\xDA” = 2010
imei: “\x38\x36\x38\x33\x32\x34\x30\x32\x31\x34\x36\x31\x38\x35\x35” = “868324021461855”
trip_no: “\x00\x00\x0C\x8A” = 3210
version_no: “\x00\x00\x00\x41” = 65
value_1: “\x00\x14\x00\x13\x00\x12\x00\x11\x00\x10\x00\x15\x00\x16\x00\x17\x00\x18\x00\x19” = 20,19,18,17,16,21,22,23,24,25 ie., 10 readings of acceleration_x
value_2: “\x00\x14\x00\x13\x00\x12\x00\x11\x00\x10\x00\x15\x00\x16\x00\x17\x00\x18\x00\x19” = 20,19,18,17,16,21,22,23,24,25 ie., 10 readings of acceleration_y
value_3: “\x00\x14\x00\x13\x00\x12\x00\x11\x00\x10\x00\x15\x00\x16\x00\x17\x00\x18\x00\x19” = 20,19,18,17,16,21,22,23,24,25 ie., 10 readings of acceleration_z
type_4: 5 with “\x05” ie., gps_packet
length_4: 150 with “\x00\x96”
value_4: “\x24\x47\x50\x52\x4D\x43\x2C\x31\x35\x30\x38\x35\x31\x2C\x41\x2C\x32\x38\x33\x38\x2E\x31\x39\x37\x36\x2C\x4E\x2C\x37\x37\x32\x34\x2E\x36\x38\x36\x39\x30\x30\x30\x30\x30\x30\x30\x31\x2C\x45\x2C\x33\x30\x2C\x32\x35\x30\x2E\x37\x31\x2C\x32\x36\x31\x30\x31\x36\x2C\x2C\x2C\x41\x2A\x36\x31\x0D\x0A\x24\x47\x50\x47\x47\x41\x2C\x31\x35\x30\x38\x35\x31\x2C\x32\x38\x33\x38\x2E\x31\x39\x37\x36\x2C\x4E\x2C\x37\x37\x32\x34\x2E\x36\x38\x36\x39\x30\x30\x30\x30\x30\x30\x30\x31\x2C\x45\x2C\x31\x2C\x39\x2C\x30\x2E\x38\x38\x2C\x31\x37\x39\x2E\x30\x2C\x4D\x2C\x2D\x33\x36\x2E\x31\x2C\x4D\x2C\x2C\x2A\x34\x31\x0D\x0A” = “$GPRMC,150851,A,2838.1976,N,7724.686900000001,E,30,250.71,261016,,,A*61\r\n$GPGGA,150851,2838.1976,N,7724.686900000001,E,1,9,0.88,179.0,M,-36.1,M,,*41\r\n” ie., 150 chars of gps_packet