GPS Logger III – NMEA Processing


The GPS Logger is currently able to access files and directories on a FAT16 filesystem, contained on a SD card. This post covers processing NMEA with a finite state machine and some of the problems encountered so far with the logger.

NMEA processing on a budget

In order to quickly and efficiently process the incoming NMEA strings a finite state machine (FSM) was constructed to look at each byte as it arrives and perform an action based upon the byte and the current state. By default the FSM operates in the INVALID state, waiting for a ‘$’ to arrive which indicates the start of a sentence and moving to the START state. If an unexpected byte is ever received the FSM returns to the INVALID state.

As the next 5 bytes come in they are compared against ‘G’, ‘P’ and then ‘G’, ‘G’, ‘A’ or ‘R’, ‘M’, ‘C’ with a single state for each byte. The GPS receiver is set up to only send GGA and RMC sentences which contain all the needed information. Any other values will return to the INVALID state, waiting for the next sentence to be sent. Once the sentence type has been determined the FSM switches into counting commas for the required fields and processing their contents, if any. The following code implements the FSM and data processing.

//GPS state
typedef enum {GPS_INIT = 0, GPS_SLEEP, GPS_NOLOCK, GPS_LOCK} gps_state;
static gps_state GPSState = GPS_INIT;		//Set the initial state

//NMEA processing FSM
typedef enum {GPS_INVALID = 0, GPS_START, GPS_G, GPS_GP, GPS_GPG, GPS_GPGG, GPS_GPGGA, GPS_GPR, GPS_GPRM, GPS_GPRMC, GPS_DATE, GPS_TIME, GPS_VALID, GPS_END } gps_process_state;
static gps_process_state GPSProcess = GPS_INVALID;
//////////////////////////////////////////////////////////////////////////////
//
// Function:	gps_receive()
// Description:	Receives a byte from the GPS and processes it
//				should do minimal work as will be called from a interrupt
// Parameters:	uint8	byte	a byte received from the GPS
// Returns:		void
//
void gps_receive(uint8 byte)
{
	static uint8 comma_desired = 0;
	static uint8 comma_current = 0;
	static uint8 index = 0;
	static bool	log = false;

	switch (GPSProcess)
	{
		case GPS_INVALID:
			if(byte == '$')
			{
				if(GPSState == GPS_LOCK)
				{
					log = true;
				} else {
					log = false;
				}
				index = 0;
				comma_current = 0;			//reset counters
				GPSProcess = GPS_START;		//start of a NMEA string
			} else {
				log = false;
				if(byte == 0xA0)			//SiRF Binary Protocol start command
				{
					//damnit, we are in SiRF mode, but on the right baud rate
					//resend the initialisation string
					serial_send(sirf_config, sizeof(sirf_config));
				}
			}
			break;

		case GPS_START:
			if(byte == 'G')
			{
				GPSProcess = GPS_G;			//starting to build a string
			} else {
				GPSProcess = GPS_END;	//invalid character (not supported)
			}
			break;

		case GPS_G:
			if(byte == 'P')
			{
				GPSProcess = GPS_GP;			//starting to build a string
			} else {
				GPSProcess = GPS_END;	//invalid character (not supported)
			}
			break;

		case GPS_GP:
			if(byte == 'G')
			{
				GPSProcess = GPS_GPG;			//hopefully a GPGGA sentence
			} else {
				if(byte == 'R')					//hopefully a GPRMC sentencs
				{
					GPSProcess = GPS_GPR;
				} else {
					GPSProcess = GPS_END;	//invalid character (not supported)
				}
			}
			break;

		case GPS_GPG:
			if(byte == 'G')
			{
				GPSProcess = GPS_GPGG;			//starting to build a string
			} else {
				GPSProcess = GPS_END;	//invalid character (not supported)
			}
			break;

		case GPS_GPGG:
			if(byte == 'A')
			{
				GPSProcess = GPS_GPGGA;			//starting to build a string
				comma_desired = 6;
			} else {
				GPSProcess = GPS_END;	//invalid character (not supported)
			}
			break;

		case GPS_GPGGA:
			//we now know we are on a GPGGA sentence, extract the number of satellites inuse
			//lat/long support will need to be added later for no movement sleep
			if(byte == ',')
			{
				comma_current++;
				if(comma_current == comma_desired)
				{
					index = 0;
					GPSProcess = GPS_VALID;		//decode the data validity
				}
			}
			break;

		case GPS_GPR:
			if(byte == 'M')
			{
				GPSProcess = GPS_GPRM;			//starting to build a string
			} else {
				GPSProcess = GPS_END;	//invalid character (not supported)
			}
			break;

		case GPS_GPRM:
			if(byte == 'C')
			{
				GPSProcess = GPS_GPRMC;			//starting to build a string
				comma_desired = 1;
			} else {
				GPSProcess = GPS_END;	//invalid character (not supported)
			}
			break;

		case GPS_GPRMC:
			//we now know we are on a GPRMC sentence, extract the date and time
			if(byte == ',')
			{
				comma_current++;
				if(comma_current == comma_desired)
				{
					index = 0;					//reset the string index
					GPSProcess = GPS_TIME;		//decode the time
					comma_desired = 9;			//set the field number for the date
				}
			}
			break;

		case GPS_TIME:
			//copy the time into the buffer until a comma is received or
			//the buffer is full (index == 5)
			if(byte == ',')
			{
				//end of the time field, wait for the date to arrive
				comma_current++;
				if(comma_current == comma_desired)
				{
					index = 0;					//reset the string index
					GPSProcess = GPS_DATE;		//decode the time
				}
			} else {
				if(index < GPS_TIME_LEN)
				{
					gps_time[index] = byte - '0';
					index++;
				}
				if(index == GPS_TIME_LEN)
				{
					//we managed to fill the time buffer, set the system time
					uint8 h, m, s;
					uint16 year;

					h = gps_date[0];
					h *= 10;
					h += gps_date[1];

					m = gps_date[2];
					m *= 10;
					m += gps_date[3];

					s = gps_date[4];
					s *= 10;
					s += gps_date[5];

					system_set_time(h, m, s);
					index++;
				}
			}
			break;

		case GPS_DATE:
			//copy the time into the buffer until a comma is received or
			//the buffer is full (index == 5)
			if(byte == ',')
			{
				GPSProcess = GPS_END;		//done with processing this sentence
			} else {
				if(index < GPS_DATE_LEN)
				{
					gps_date[index] = byte - '0';
					index++;
				}
				if(index == GPS_DATE_LEN)
				{
					//we managed to fill the date buffer, set the system date
					uint8 day, month;
					uint16 year;

					day = gps_date[0];
					day *= 10;
					day += gps_date[1];

					month = gps_date[2];
					month *= 10;
					month += gps_date[3];

					year = gps_date[4];
					year *= 10;
					year += gps_date[5];
					year += 2000;			//add the century

					system_set_date(day, month, year);

					index++;
				}
			}
			break;

		case GPS_VALID:
			//determine if the provided information is valid
			if(byte == ',')
			{
				if(index >= 1)
				{
					GPSState = GPS_LOCK;
					LED_Blue = 1;
				} else {
					GPSState = GPS_NOLOCK;
					LED_Blue = 0;
				}
				GPSProcess = GPS_END;			//done with processing this sentence
			} else {
				index *= 10;
				index += (byte - '0');			//add the current digit to the sat count
			}
			break;

		case GPS_END:
			if(byte == '\n')
			{
				GPSProcess = GPS_INVALID;		//we have the end of the sentence, wait for the start
			}
			break;
	}

	if(log == true)
	{
		//add the byte to the buffer
		uint8 size = (inbuffer_tail - inbuffer_head);
		size &= GPS_BUFFER_MASK;

		//there is space in the buffer, write the byte
		if(size < (GPS_BUFFER_LEN-1))
		{
			LED_Red = 0;
			inbuffer[inbuffer_tail] = byte;
			inbuffer_tail++;
			inbuffer_tail &= GPS_BUFFER_MASK;
		} else {
			LED_Red = 1;
		}
	}
}

Problems encountered so far

Data was being dropped occasionally when a sector was being written to the card and usually when a new cluster was allocated to a file. The allocation routine required a write (to write the last sector in the cluster) a read (of the FAT table) then two writes (to update both copies of the FAT). If the previous cluster number was located in a different sector of the FAT then another read and two writes were required to create the link to the newly allocated cluster. The data was being lost because the circular buffer for in incoming serial data was getting full while the SD card was being written to. As all SD cards support operation in SPI mode up to 25MHz, the spi_xmit() function was modified to ignore timer/baud rate settings and run as fast as possible. This modification mostly eliminated data loss and freed up extra processing time for the PIC. Writing an optimised assembly version of the spi_xmit() function should result in no data loss. Currently the data that is being lost consists of part of the GPRMC NMEA string, whose only unique component is the date field. This loss can be tolerated and watched for by processing software when offline.

While testing the logging capabilities it was noticed that occasionally the filesystem would get trashed, due to an overwrite of the MBR and FAT sectors. This was an intermittent fault that could occur with a single file on the card or multiple files. Initial attempts at preventing the problem resulted in blocking writes to sector 0. Dumping an image of the SD card reveiled that the FATs and directory entries were being written properly,  however new files were filled with NULLs (0x00). By copying a large file (JPEG photo) onto the card and then deleting it proved that after a certain size, the files contained old data and not new data, whereas the sectors at the start of the card contained the new data. The problem was guessed to be caused by integer wraparound, however all the code checked out. Eventually it was discovered that FileSector (the current sector on the sdcard of the file being written) had been declared as a 8-bit unsigned integer instead of 32-bits, as it was assumed it had. Declaring it as the proper size removed the problem completely.

Continue on to part IV

,

  1. No comments yet.
(will not be published)