Saturday, 15 April 2017

Embedded Product Design on Linux

An embedded product on Linux involves several stages. As an example, we use a Zigbee embedded product on Linux to discuss this.

The software components that make up an embedded product:

Dynamic UI: HTML, JQuery, Javascript, XML data, CGI process

Application and Libraries: Many applications, and third-party libraries


The figure above shows the interaction among the application processes, CGI app and Dynamic UI.

Linux Kernel and Drivers:
It maybe required to integrate new drivers into Linux kernel space, to enable the SoC on the main board.

Bootloader:
It maybe required to modify boot loader if we upgrade to new generation of flash and DRAM IC.


Real-Life Embedded Systems problems and solutions:


  • Zigbee Carrier board integration:

Problem:
Old carrier board is responding to Zigbee Ping command, and can find its IEEE address.
New carrier board is responding to Zigbee Ping command, but cannot find its IEEE address.

Reasons: Firmware is upgraded in New carrier board, so a upgraded SDK is required to work with the new firmware in New carrier board.

Debugging process: In Linux, open the serial port where the Zigbee module is connected, then set the firmware in carrier board to command mode, then send the Zigbee Ping command to the carrier board.

  • Board Graceful shutdown:

Problem:
Power-off the board immediately without proper shutdown sequence will cause linux file system corruption.

Solution: Ask MSP430 to set an interrupt to MPC8315 when the power switch is pressed. An interrupt handler is added to linux kernel (as a linux driver module) to handle MSP430 interrupt. The driver module will create a kthread to listen to the interrupt. When interrupt arrives, a user space helper function is called (by using usermode helper API).

The userspace function handles the proper shutdown sequence, such as carrying out the "init 0" command.

  • Move DRAM content forward:

Problem:
We want to copy DRAM content.

Solution:

void loader(void)
{
  long * ptr = loki_plus_dram;
  int i;
  for (i=0; ptr[i]!=0x1FFFF; i++)
  {
    *((long *)ptr[i]) = ptr[i+1];
  }
  return;
}

  • TI Zigbee chipset cc2530 UART 1 setup:

Problem:
Unable to communicate to TI cc2530's UART 1 port.

Solution:
In IAR Workbench, Project->Options, choose C/C++ compiler, set:
HAL_UART_DMA = 0
HAL_UART_ISR=2

This is a must for the source code to call hal_uart_isr() function in hal_uart.c

  • Wifi camera lose connectivity to Wifi AP:

Problem:
Wifi camera lost connection to Wifi AP after 1 day, meaning Wifi AP cannot detect the Wifi camera after 1 day.

Solution:
In /etc/hostapd.conf, set wpa_strict_rekey = 0
or set wpa_group_rekey = 0 and wpa_ptk_rekey = 0 if maintaining wpa_strict_rekey = 1

  • Glucometer timeout issue:

Problem:
The Bluetooth program on linux is not able to read all data from Glucometer, read function hang, need to power off/on Glucometer .

Solution:
Use a function called setsockopt to set socket to non-blocking with timeout.

setsockopt(sk, SOL_SOCKET, SO_RCVTIMEO, (struct timeval *)&tv, sizeof(struct timeval));
ret = read(sk, buf, sizeof(buf));
if (ret == -1)
{
  timeout!!
  read again
}

  • Add debug statement control:

Problem:
debug statements are lost if we do not have serial port to monitor them

Solution:
Define a debug macro


//DEBUG functions
#define LOG_OUTPUT stderr

#define LOG(level, ...) \
        {
                fprintf(LOG_OUTPUT, level); \
                fprintf(LOG_OUTPUT, __VA_ARGS__); \
                fflush(LOG_OUTPUT); \
        }

#define DEBUG(...)   LOG("Myglucohealth   ", __VA_ARGS__)

Actual usage:
DEBUG("Can't find adapter %s\n", adapter);

Normally the debug statements will be sent to console, but we can redirect them to anywhere with a easy command:
<process_command>  &> /mnt/storage/bluetooth/process_command.log  &
---> redirect both stdout and stderr to a file

  • TCP socket: too many open files:

Problem:
A TCP client connected to TCP server every 60 seconds and close the connection after sending a message. After 6 hours, socket call failed, saying "socket: too many open files"

Solution:
The TCP server must close the socket that is returned by accept(). And the TCP client should close the socket first, then the TCP server close it. This is due to the TIME_WAIT settings which will create waiting burden on the TCP server if the TCP server closes the socket first, and if the TCP server is a busy one.

Tips:
Use netstat to check for open socket connections
Use "ls /proc/<pid>/fd" to check for open file descriptor
Turn off SO_LINGER, call it after open the socket, eg: setsockopt(sd, SOL_SOCKET, SO_LINGER,...);

  • Simultaneous Wifi and Ethernet transmission, and data lost

Problem:
When there are simultaneous Wifi and Ethernet transmission to the DUT, and DUT does not receive data via the Wifi after a few hours.

Debugging:
Check the interrupt mask and interrupt enable register bits. On the general interrupt controller (GIC), the Wifi and ethernet bits are adjacent to each other. When data is received at the hardware interface, internet status bit is set. When inside the ISR, interrupt enable bit is disabled, and interrupt mask bit is masked out. When exiting ISR, interrupt enable bit is enabled, interrupt mask bit is cleared. If ISR interrupt enable bits is not set, something is amiss.

Solution:
The problem is due to a hardware error that ethernet interrupt bit will affect the Wifi interrupt bit. The software workaround is to enable the Wifi and Ethernet interrupt enable bits in the ISR when exiting ISR.