Linux Information

1. Linux Auto Login

Sometimes we may want to skip the linux login prompt, for example, a remotely-powered linux server in the basement of a building. Here is how we do it on RedHat linux 7.3.

• modify /etc/inittab , comment out:
  

  1:2345:respawn:/sbin/mingetty tty1

  Add:

  1:2345:/usr/bin/openvt -f -c 1 -w -- /bin/login -f root

  -c 1 for console 1
  -- for separator
  
• modify ~/.bash_profile, add:
  

  startx

  to start the X windows
• modify ~/.xinitrc, add:
  

  gnome-session

  or

  startkde

2. Linux Programming

Linux Command Line Tools
Some useful tools are listed here.

% nm <object-files>
list symbols from object files

% readelf -d <elf-files>
display dynamic info about ELF files

% ldd <executable-files>
list shared library dependencies

% ar crv <lib-files> <object-files>
create static library

***GNU Make***
Make commands is used extensively in Linux software development.

A) Make command line syntax:
The `-s' or `--silent' flag prevents all echoing, as if all commands are started with `@'.
eg. make -s zImage
The `-f' flag specifies a name of the makefile.
eg. make -f myMakefile
We can run make -n <target>, to tell make to print out what it should have done, without actually doing it.

B) The general Makefile syntax is:

target : dependency
 rules (or commands)

The command lines start with a tab character. Blank lines and lines of just comments may appear among the command lines; they are ignored. (But beware, an apparently "blank" line that begins with a tab is not blank! It is an empty command.)
To make a particular target, we need to pass its name to make as a parameter. Without a parameter, make will try to make the first target listed in the makefile.
Make invokes a shell for executing rules, and uses a new shell for each rule. To let all script commands appear on one logical line, we must add backslash at the end of each line. The @sign will tell make not to print out each command on standard output.

C) Makefile contents:
Makefiles contain five kinds of things: explicit rules, implicit rules, variable definitions, directives, and comments.

directive:

     include filenames...

When make processes an include directive, it suspends reading of the containing makefile and reads from each listed file in turn. When that is finished, make resumes reading the makefile in which the directive appears.
If you want make to simply ignore a makefile which does not exist and cannot be remade, with no error message, use the -include directive instead of include, like this:

     -include filenames...

D) Other Makefile stuff:
Targets that do not refer to files but are just actions are called phony targets.

objects = main.o kbd.o command.o display.o \
               insert.o search.o files.o utils.o

.PHONY : clean
     clean :
             -rm $(objects)

clean:
 -rm -f *.o

The rules of making the target "clean" comes without dependency. This mean that the target is always considered out of date, and its rule is always executed. Also notice that the command starts with "-". This tells make to ignore the result of the command, and so make clean will always succeed. The `-' is discarded before the command is passed to the shell for execution.

For example,

     clean:
             -rm -f *.o

A few special macros are defined in make. Here is the summary.

foo.o : foo.c
 gcc -c $(CFLAGS) %^ -o $@

$^ is the dependency (foo.c)
$@ is the target (foo.o)

foo.o : foo.c defs.h hack.h
 gcc -c $(CFLAGS) $< -o $@

$< is the first dependency (foo.c)

.c.a:
 gcc -c $<
 ar rv $@ $*.o

$* is the name of the target without the suffix

Make has a special syntax for dealing with library files. The syntax is lib(file.o). It means that the object file file.o is stored in the library file lib.a.

$(LIBRARY): $(LIBRARY)(db_api.o)
db_api.o: db_api.c db_api.h

For a project with multiple subdirectories, a main makefile in the main directory will invoke the sub-makefiles. The syntax is like this:

(cd subdirectory;$(MAKE))

or, equivalently, this:

$(MAKE) -C subdir

The brackets ensure that it is all processed by a single shell. Since a new shell is invoked for this, the program running the make doesn't execute the cd command. Only the shell invoked to carry out the rule is in a different directory.

E) Variable substitution
To substitute a variable's value, write a dollar sign followed by the name of the variable in parentheses or braces: either `$(foo)' or `${foo}' is a valid reference to the variable foo. This special significance of `$' is why you must write `$$' to have the effect of a single dollar sign in a file name or command.
The first flavor of variable is a Recursively expanded variable. Variables of this sort are defined by lines using `=' or by the define directive.

foo = $(bar)
bar = $(ugh)
ugh = Huh?

all:@echo $(foo)

It will echo `Huh?'. `$(foo)' expands to `$(bar)' which expands to `$(ugh)' which finally expands to `Huh?'.
To avoid all the problems and inconveniences of recursively expanded variables, there is another flavor: Simply expanded variables. Simply expanded variables are defined by lines using `:='

foo := $(bar)
bar := $(ugh)
ugh := Huh?

all:@echo $(foo)

This will echo nothing, as $(foo) is undefined here.
There is another way, called a Conditional variable assignment operator, because it only has an effect if the variable is not yet defined. This statement:

  FOO ?= bar

is exactly equivalent to this:

ifeq ($(origin FOO), undefined)
  FOO = bar
endif

F) String Substitution (in Makefile):
$(patsubst pattern,replacement,text)
Finds whitespace-separated words in text that match pattern and replaces them with replacement. Here pattern may contain a `%' which acts as a wildcard, matching any number of any characters within a word. If replacement also contains a `%', the `%' is replaced by the text that matched the `%' in pattern. `%' characters in patsubst function invocations can be quoted with preceding backslashes (`\'). Whitespace between words is folded into single space characters; leading and trailing whitespace is discarded.
For example,

$(patsubst %.c,%.o,x.c.c bar.c)

produces the value `x.c.o bar.o'.
Substitution references are a simpler way to get the effect of the patsubst function:

$(var:pattern=replacement)

is equivalent to

$(patsubst pattern,replacement,$(var))

The second shorthand simplifies one of the most common uses of patsubst: replacing the suffix at the end of file names.

$(var:suffix=replacement)

is equivalent to

$(patsubst %suffix,%replacement,$(var))

For example, you might have a list of object files:

objects = foo.o bar.o baz.o

To get the list of corresponding source files, you could simply write:

$(objects:.o=.c)

instead of using the general form:

$(patsubst %.o,%.c,$(objects))

Another example in real Makefile:

SUB = server client

%.build:
 (cd $(patsubst %.build, %, $@) && $(MAKE))

%.clean:
 (cd $(patsubst %.clean, %, $@) && $(MAKE) clean)

all:  $(patsubst %, %.build, $(SUB))
clean:  $(patsubst %, %.clean, $(SUB))

G) Conditional statement (in Makefile):
Three types: if-else-fi, ifeq-else-endif, ifdef-else-endif.

@if [ -z $(DESTDIR) ]; then \
    /sbin/depmod -ae ; \
elif [ -f $(SYSTEMMAP) ]; then \
    /sbin/depmod -ae -b $(DESTDIR) -F $(SYSTEMMAP) ; \
else \
    echo "Don't forget the target system."; \
fi

ifeq ($(strip $(foo)),)
  text-if-empty
else
  text-if-not-empty
endif

Don't specify $(foo) for variable referencing if used in ifdef statement

foo = $(bar)
ifdef foo
  frobozz = yes
else
  frobozz = no
endif

Using GNU GDB and gdbserver
GDB can be used to debug programs written in C and C++. GDB distribution contains gdbserver in one of its subdirectory. We run gdbserver on target platform in order to save space, and use GDB on host to connect to the gdbserver for debugging. For more information, click here.

Compiling and generating library images

The sequences of generating a *.a (static library)
a) gcc -c xxx.c
b) ar crv libxxx.a xxx.o

The sequences of generating a *.so (dynamic library)

gcc -c -fPIC xxx.c
gcc -shared -o libxxx.so xxx.o

Files, Pipes, Sockets
Everything is represented as a file under Linux. Even hardware devices are represented by files in Linux. The low-level system calls can be used to access the files, and similarly the hardware devices, pipes, sockets.

int open(const char *path, int flags);

- open system call returns a new file deescccriptor

size_t read(int fd, void * buf, size_t nbytes);

- read system call reads up to nbytes frrommm file associated with the file descriptor fd to buf

size_t write(int fd, const void * buf, size_t nbytes);

- write system call writes up to nbytes tooo the file referenced by the file descriptor fd from the buf

int close(int fd);

- close system call closes a file descriiptttor, so that it may be reused

int ioctl(int fd, int cmd, ...);

- ioctl system call provides an interfacce for controlling the behavior of devices

off_t lseek(int fd, long int offset, int whence);

- lseek system call sets the read/write pooointer of a file descriptor
The standard library and its header file stdio.h provide an interface to low-level system calls. The library provides functions to take care of the buffering of the devices. In standard library, the equivalent to a file descriptor is called a stream, and is implemented as FILE * .

FILE * fopen(const char *filename, const char modes);

- fopen library function returns a FILE * pointer

size_t fread(void * ptr, size_t size, size_t nitems, FILE * stream);

- fread library function reads to ptr frrommm the stream, for a record of size and a count of nitems

size_t fwrite(const void * ptr, size_t size, size_t nitems, FILE * stream);

- fwrite library function writes from thhe ptr to the stream, for a record of size and a count of nitems

int fclose(FILE * stream);

- fclose library function closes the speeciiified stream

int fseek(FILE * stream, long int offset, int whence);

- fseek library function sets the positiionnn in the stream for the next read or write on that stream.
Pipe is a primitive form of inter process communications. It allows the data flow from one process to go to another process. For shell commands, it is entered as :
cmd1 | cmd2
For high-level pipe function, they will operate on file stream.

FILE * popen(const char * command, const char * open_mode);

It allows a program to invoke another program as a new process, and either pass data or receive data from it.

int pclose(FILE * stream);

It closes the file stream associated with the pipe.
At the same time, a low-lovel pipe function provides a way of passing data between two programs, without the overhead of invoking a shell to interpret the requested command.

int pipe(int fd[2]);

It is passed an array of two integer file descriptors. Any data written to fd[1] can be read back from fd[0]. Low-level system calls, read and write, are used to access the data.
A named pipe exists in the file system as a special type of file, but behaves like the unnamed pipes we discussed earlier. It is called FIFO too.

int mkfifo(const char * filename, mode_t mode);

It creates a named pipe, using absolute pathname.
We can remove the FIFO by using the rm command, or from within a program by using the unlink function.
A FIFO exists as a named file, not as an open file descriptor. It must be opened before it can be read from or written to. The open and close system calls can be used for the purpose. The read and write system calls can be used to access the FIFO after it is opened.

Using GCC __func__ macro
GCC provides three magic variables which hold the name of the current function, as a string. The first of these is __func__, which is part of the C99 standard:
The identifier __func__ is implicitly declared by the translator as if, immediately following the opening brace of each function definition, the declaration

           static const char __func__[] = "function-name";

appeared, where function-name is the name of the lexically-enclosing function. This name is the unadorned name of the function.
__FUNCTION__ is another name for __func__. Older versions of GCC recognize only this name. However, it is not standardized. For maximum portability, we recommend you use __func__, but provide a fallback definition with the preprocessor:

     #if __STDC_VERSION__ < 199901L
     # if __GNUC__ >= 2
     #  define __func__ __FUNCTION__
     # else
     #  define __func__ ""
     # endif
     #endif

In C, __PRETTY_FUNCTION__ is yet another name for __func__. However, in C++, __PRETTY_FUNCTION__ contains the type signature of the function as well as its bare name, such as "void a::sub(int)".
Click here to see the examples: exam.c and exam.h file.

Debugging
There are 8 severity levels in linux kernel, defined in <linux/kernel.h>. DEFAULT_MESSAGE_LOGLEVEL is specified in kernel/printk.c, and applied to printk with no specified priority. If priority is less than console_loglevel, the message is displayed. If both klogd and syslogd are running on the system, kernel message are appended to /var/log/messages, independent of console_loglevel.
In <linux/kernel.h>, #define console_loglevel as DEFAULT_CONSOLE_LOGLEVEL. As printk writes message to a circular buffer, it then wakes up any process that is waiting for the message. If klogd is running, it retrieves kernel message and dispatch them to syslogd, which in turn check the settings in /etc/syslog.conf.

To use syslogd:

/etc/init.d/syslog -- script file

/etc/syslog.conf -- config file

create /var/log directory.

An example syslog.conf config file:
# cat syslog.conf
*.*                      /var/log/messages

AWK 
It is an interpreted programming language for performing complex text processing tasks. It is also a simple text processing utility. It stands for the names of its authors: Aho, Weinberger & Kernighan.
simple syntax:
awk <search pattern> {<program actions>} <data filename>
eg. awk '/gold/ {print $5,$6,$7,$8}' coins.txt

full syntax:
awk 'BEGIN   {<initialization>}
 <search pattern 1> {<program actions>}
 <search pattern 2> {<program actions>}
 ...
 END   {<final actions>}'

eg. awk 'END {print NR, "conis"}' coins.txt

NR - Awk's pre-defined varaibles, stands for Number of Records
Awk regards each line of input data as composed of multiple fields, which are essentially words separated by blank spaces. A blank space is the default "field separator". To tell Awk to use another field separator, use -F.

eg. awk -F\"  -- use " as the separator

eg. awk -F\" '/REL/ {print $$2}' include/linux/version.h

SED
Stream editor

sed -e s/<reg expr>/<replace text>/{flags}

eg. sed -e '1,2 s/line/LINE/' test.txt
eg. sed -e 's/cat/dog/g' test.txt

regular expr
^ - match the beginning of the line
$ - match the end of the line
. - match any single character
* - match arbitrary many occurrences of charater
? - match 0 or 1 instance of character

Spinlock and Semaphore

Spinlocks are very small and fast, and can be used anywhere. If your task can't get the spinlock, your task keeps trying (spinning) until your task can.

Semaphore can have more than one holder at any time (the number decided at initialization time), although it is most commonly used as a single-holder lock (known as mutex). If your task can't get a semaphore, your task will put itself on the queue, and be woken up when the semaphore is released. This means the CPU will do something else while your task is waiting.

Semaphore is used for synchronization between user contexts. User Context means the kernel is executing on behalf of a particular process (ie. a system call or trap) or kernel thread. This is not to be confused with userspace. It can be interrupted by software or hardware interrupts.

Spinlock is used for synchronization between user context and interrupt context. spin_lock_bh() disables softirqs on that CPU, then grabs the lock. spin_lock_irq() is defined to disable interrupts on that cpu, then grab the lock.

Exporting Symbols

In source file:

 #define EXPORT_SYMTAB

and then after you write your function:

 EXPORT_SYMBOL(your_function_name);

In Makefile:

 export-objs := filename.o

Networking commands
Commands for device statistics:

cat /proc/net/dev
  show device statistics
cat /proc/net/snmp
  show snmp statistics

Commands for routing table:

route add -net 192.168.2.0 netmask 255.255.255.0 gw 192.168.1.1
  add gateway 192.168.1.1 to routing table for network 192.168.2.0
route del -net 192.168.2.0/24
  delete routing table entry
route -n
  show routing table status

Commands for ethernet:

ifconfig eth0 txqueuelen 10000
  increase the transmit queue of the network interface
echo 1 > /proc/sys/net/ipv4/ip_forward
  enable ip forwarding
/sbin/ethtool -K eth0 tso off
  disable TCP segment offloading
/bin/uname -r
  show kernel version

netstat -p --tcp

  show network connection for tcp protocol

Reset procedures for MIPS and ARM

MIPS IDT438

sys_reboot (kernel/sys.c)
--> machine_restart (arch/mips/kernel/reeseet.c)
    --> _machine_restart (assigned in arch/mips/rc32438/79EB438/setup.c)
       --> idt_reset (arch/mips/rc32438/79EB438/reset.c)

ARM IXP425

sys_reboot (kernel/sys.c)
--> machine_restart (arch/arm/kernel/prooceess.c)
    --> arch_reset (include/asm-arm/arch-ixp425/system.h)

3. Shell Script Programming

A. Passing parameters to shell script
As a simple example, i have a shell script which requires to take in one parameter. I name it "cp_script".

#!/bin/sh
echo "Parameter " $1

Then i call the shell script on the command line and pass it a parameter:

$./cp_script madwifi
Parameter madwifi

That's it, using $1 in shell script to take in the command line parameter.

B. If-else-fi in shell script:
[ expr ] is used to see if an expression is true.

if [ -f $dir/$file ] || [ -f $dir/$newfile ]; then
    echo "Either this filename [$file] exists"
    echo "Or this filename [$newfile] exists"
elif [ -d $dir ]; then
    echo "This dirname [$dir] exists"
else
    echo "Neither [$dir] or [$file or $newfile] exist"
fi

C. Variable declaration:
We use something as:

#!/bin/sh

TC="/sbin/tc"

$TC qdisc ls
$TC qdisc del dev eth0 root
$TC qdisc add dev eth0 root pfifo_fast

D. The getopt command
The getopt command is used to parse the command line parameters. It is made up of two parts: options and non-option parameters.
-o specifies the short option, -n is the name of the program, -- is the start of non-option parameters

#! /bin/bash

echo "param" $@
TEMP=`getopt -o nr -n oudanhodou -- "$@"`
echo "temp" $TEMP
unset TEMP


Execution:
$ ./oudanhodou -r
param -r
temp -r --

$ ./oudanhodou -r fresh
param -r fresh
temp -r -- 'fresh'

4. Linux Setup

A. Linux network setup on Redhat 9

/etc/hosts
127.0.0.1  localhost

/etc/host.conf
order host,bind

/etc/resolv.conf
nameserver  192.168.4.254

It is the IP of DNS server.

/etc/services

It contains port allocation information.

B. Change ip address

/etc/sysconfig/network-scripts/ifcfg-eth0
IPADDR=192.168.4.239
DHCP_HOSTNAME=192.168.4.251

/sbin/ifup eth0

C. Use dhcp client

execute redhat-config-network

then execute /sbin/dhclient

settings in /etc/dhclient-eth0.conf

For Dhcp server and client settings on embedded system, click on DHCP info.

D. Update linux kernel

1. create floppy boot disk
ls /lib/modules
 2.4.2-2

mkbootdisk --device /dev/fd0 2.4.2-2

2. clean old kernel config
make mrproper

3. change whatever settings you want
make menuconfig

4. update the change
make dep

5. make it
make bzImage

6. make the modules
make modules

7. install the newly made modules
make modules_install

8. update the boot loader config file with new kernel information
/etc/lilo.conf

9. read from new boot loader config file and store into boot sector
/sbin/lilo -v

E. lilo information
"/sbin/lilo -v" read the /etc/lilo.conf file, to determine what to write to MBR.
lilo.conf example

boot=/dev/hda #install lilo in the first harddisk
map=/boot/map
install=/boot/boot.b #specify new boot sector file
prompt
timeout=50 #wait for 5 sec
lba32    #describe harddisk geometry
default=linux
image=/boot/vmlinus-2.4.20-8 #specify linux kernel
 label=linux
 root=/dev/hda1  #specify root partition
image=/boot/vmlinus
 label=failsafe
 root=/dev/hda1
 initrd=/boot/initrd.img
other=/dev/hda2
 label=windows
 table=/dev/hda

F. Make ramdisk

mkdir ramdisk
cd ramdisk
dd if=/dev/zero of=initrd bs=1k count=8192
 bs is the block size
/sbin/mke2fs -vFm0 initrd 8192
mkdir mnt
sudo mount -o loop initrd mnt
cd mnt
cp -a .....
cd ..
sudo umount initrd

To do "mount -o loop" on target platform:
1. kernel got to have loopback device support enabled
2. On file system, create /dev/loop file
3. kernel got to enable ext2 support

target bootloader configured with:
bootparm1=/root=/dev/ram init=/linuxrc rw

G. Starting X windows

startx > log 2>&1
gcc -Wl,-v 2>&1 | grep "GNU ld"

H. Sudo List
In /etc/sudoers file, we can add in a list of users to specify their permissions.

User_Alias   SW_STAFF = julian, hunk, yeosv
# User privilege specification
root   ALL=(ALL) ALL
SW_STAFF  ALL=NOPASSWD: ALL
newcomer ALL=(ALL) NOPASSWD: ALL

I. Kernel Booting Sequences
This is the kernel booting sequences for ARM architecture processor. This information may apply to other architecture with minor modification.

(arch/arm/boot/compressed/head.S)
 -> setup the stack
 -> call decompressed_kernel (arch/arm/boot/compressed/misc.c)
 -> jump to decompressed code

(arch/arm/kernel/head.S)
_stext   -> start_kernel (init/main.c)     [ task 0, idle task ]
   -> setup_arch (arch/arm/kernel/setup.c)
   -> trap_init
   -> init_IRQ
   -> sched_init
   -> softirq_init
   -> time_init
   -> console_init
   -> init_modules
   -> kmem_cache_init
   -> calibrate_delay
   -> mem_init
   -> kmem_cache_sizes_init
   -> fork_init
   -> proc_caches_init
   -> vfs_caches_init
   -> ...
   -> rest_init             [ launch init kernel thread ]
    -> init (init/main.c)     [ the "init" kernel thread ]
     -> do_basic_setup
     -> prepare_namespace
     -> launch /sbin/init

include/asm-arm/arch-ixp425/memory.h
 contains PAGE_OFFSET and PHYS_OFFSET

arch/arm/boot/Makefile
 contains ZTEXTADDR and ZRELADDR