Sunday 24 May 2020

Begin Machine Learning as a software engineer

In this post, i am writing about how to start applying machine learning (ML) in software program as a software engineer.
The picture below shows that in ML, we build ML model using data and results. This stands in contrast to traditional programming where we use software program to do the actual computation.
After the model is built, we use it to make predictions.
For sure, we can always create our own ML model. In order to do that, we have to get a rather good understanding of ML algorithms. If we are not able or not willing to create our own ML model (such as when we want to apply ML to a practical solution), we can re-use existing models in the ML libraries.
In machine learning (ML) programming, we choose the existing model, build the model architecture (such as a classifier), feed training data to the model, and use the trained model to make decisions on newly arrived data.
The difference between algorithm and model is:
This is the algorithm of linear regression with one variable 𝑦=𝑤0+𝑤1x
This is the model after applying data and results 𝑦=(5)+(-2)x
The purpose of training a model, is to adjust the model parameters so that the model fit well with the user supplied data.
We choose keras library as a starting point. There is Sequential model in the keras library. For this article, we look at the bank customer data and decides if the customer would stop using the bank’s services. This is a classification problem. We will use the Sequential model to do the classification.
In part one, we will do data preprocessing. Firstly, we import the numpy and pandas library. Pandas is a data manipulation library.
import numpy as np
import pandas as pd
Secondly, we import the dataset using pandas.
dataset = pd.read_csv(‘Churn_Modelling.csv’)
X = dataset.iloc[:, 3:13].values
y = dataset.iloc[:, 13].values
Then, we encode the categorical data (one hot encoding).
from sklearn.preprocessing import LabelEncoder, OneHotEncoder
labelencoder_X_1 = LabelEncoder()
X[:, 1] = labelencoder_X_1.fit_transform(X[:, 1])
labelencoder_X_2 = LabelEncoder()
X[:, 2] = labelencoder_X_2.fit_transform(X[:, 2])
onehotencoder = OneHotEncoder(categorical_features = [1])
X = onehotencoder.fit_transform(X).toarray()
X = X[:, 1:]
We split the dataset into the training set and test set.
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)
After that, we apply feature scaling to the data so that data is adjusted to a particular range.
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)
In part two, we will make an artificial neural network. Firstly, we import keras library.
import keras
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout
Secondly, we use sequential model to build a ML classifier. Sequential model is a model with a sequence of layers; input, hidden, output layers.
classifier = Sequential()
We add the input layer and the first hidden layer.
classifier.add(Dense(units = 6, kernel_initializer = ‘uniform’, activation = ‘relu’, input_dim = 11))
We add the second hidden layer.
classifier.add(Dense(units = 6, kernel_initializer = ‘uniform’, activation = ‘relu’))
Lastly, we add the output layer.
classifier.add(Dense(units = 1, kernel_initializer = ‘uniform’, activation = ‘sigmoid’))
Now, compile the classifier. The adam optimizer is an adaptive moment estimator. N-th moment of random variable is the expected value of the variable to power of N. Optimizer decides how network weight will be updated.
classifier.compile(optimizer = ‘adam’, loss = ‘binary_crossentropy’, metrics = [‘accuracy’])
For binary classification, the binary_crossentropy loss function is suitable and should be used.
We fit the training set to the classifier.
classifier.fit(X_train, y_train, batch_size = 10, epochs = 100)
In part three, we use the classifier to make predictions and evaluating the model. Firstly, we feed the test data to the classifier.
y_pred = classifier.predict(X_test)
y_pred = (y_pred > 0.5)
Secondly, we use the confusion matrix to evaluate the model.
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_test, y_pred)
When i examine the cm value, it is shown as:
It means out of 2000 samples, it makes correct predictions 1595 times.
The python and CSV files can be found at:

Sunday 2 February 2020

Bug descriptions and solutions

This post is about bugs and their solutions on embedded Linux.

Bug 1
Remove internet connectivity on Gateway device. For the first time, plug in USB device with swupdate.swu, no firmware update is performed. For the second time, plug in USB device with swupdate.swu, firmware update occurs. When internet connectivity is removed, it should not do firmware update. So it is a bug.

Solution:
Check the gateway system time against x.509 certificate validity period. If system time is outside x.509 validity period, swupdate does not run. That is because swupdate software checks the x.509 certificate. The problem has nothing to do with internet connectivity.

# openssl x509 -in x509.crt -text -nount
validity: not before
              not after
# date
-- show system time

Compare the system time and x509 certificate validity

Bug 2
When using mobile App to connect to Gateway device via BLE, multiple occurrence of DBUS error message "Rejected send message" is seen in journal log. This error message should not happen.

Solution:
Using dbus-monitor, we can see the dbus system message. There is method call from Bluez to config daemon, there is a signal and method return in response to the method call. The method return is rejected by the dbus daemon and dbus error message is printed out.

The gateway sends GATT Indicate to mobile app, mobile app responds with GATT Confirm. The Indicate is used to transfer data. Confirm is the acknowledgement. When mobile app sends Confirm to gateway, Bluez receives it, it sends dbus confirm to config daemon. And the next chunk of data is sent from config daemon to Bluez. The method return is also sent out by the config daemon. The config daemon uses dbus python library.  The solution is in dbus python library. We have to check the dbus must_not_reply flag. If the flag is set, we stop the library from sending out method return.

Bug 3
If Gateway is physically connected to ethernet and has a wifi, and if ethernet link is not able to up, the gateway is not able to use wifi. The gateway should be able to auto failover to wifi.

Solution
Use network manager to check the link, if the ethernet link is not up, auto switch to wifi. The network manager config file provides checking of the link status.

Bug 4
When doing high throughput network application over ethernet , the CPU is loaded to over 90%.
This slows down the response time of other processes.

Solution:
The ethernet driver was using interrupt  driven mode, switching to NAPI , and implement TSO  so that segmentation of the packets is handled by NIC card.  This way reduces the CPU load.