QuickWakeup Lightweight, Low Power Sensor Polling for Android

mangledcobwebΛογισμικό & κατασκευή λογ/κού

14 Δεκ 2013 (πριν από 3 χρόνια και 10 μήνες)

89 εμφανίσεις

Android Below Java

Understanding the Underlying System
Architecture, and Build Process

Michael Mitchell

mitchell@cs.fsu.edu

Department of Computer Science

Florida State University


Tallahassee, Florida 32306, USA

Overview of Topics


Terminology


Android System Architecture


Android Tools and Utilities


Compiling the Android System


Flashing firmware images


Compiling the Android kernel


Links to additional resources


Terminology (ROM)


Technically, ROM stands for Read Only Memory, which
-

as it says
-

means that you cannot write to it, it is read only
-

like a DVD.


Confusing, because manufacturers refer to the system partition as
ROM because they do not
intend

for the user to write to it.


So, even though it is technically writable, it is sometimes referred to as
ROM because of the way it is designed to be used.


Furthermore, ROM is often used as a shorthand for "ROM image".


In other words, the files that you put in the system partition are also
referred to as ROM sometimes.


Thus, you'll hear people say "flashing a ROM".



Example: You can flash a ROM onto the ROM, which isn't really
ROM, actually.

Terminology (Root)


Like ROM, root can mean several different things,
depending on the context:


'root' is the name for administrator in Android and other
Linux based systems.


'root' also means having administrator powers.


'root' is the lowest level one can go in a file system. In
other words, it is / in Linux or C:/ in Windows.


'rooting' is the process of acquiring root (or administrator)
privileges on the device.



Example: You need to root the device so that you can
have root access to the root of the file system.

Terminology cont.


Firmware

-

Programs stored in the ROM, EPROM, or
flash memory that usually control various internal
electronic devices (Hard Drives, Keyboards, Displays,
etc). Firmware is typically 'fixed' software that is not
updated in consumer devices, however it is often
updated (or 'flashed') by advanced users to fix bugs or
add features to the device.


Kernel

-

The central or core software component of
most operating systems. Its responsibilities include
managing the system's resources (the communication
between hardware and software components) and can
provide the lowest
-
level abstraction layer for resources
(especially memory, processors, and I/O devices).

Terminology cont.


Radio

-

The cellular radio on the device which needs
firmware to control it.


SPL

-

Second Program Loader, in conjunction with the
IPL comprise a device's
bootloader
. Aside from
bootstrapping Android, the
bootloader

also fulfills
various diagnostic functions. One of these functions is
the manipulation of data in the device's internal flash
ram. Depending on the SPL installed, the user might be
able to flash signed NBH files, flash
nand

images, and
more.


Note that the SPL is installed and operates independently
of the Android build that runs atop it!

Terminology cont.


Flashing

-

The process of applying a firmware image (or
ROM) to a device. It generally entails a very specific order
of steps. Failing to complete any one of these steps
properly may result in bricking the device.


Unlock

-

Most GSM devices are locked to only work with
the
sim

cards of a particular carrier. Obtaining root access
on the device usually does nothing to the
sim
-
lock. The
usual way to unlock the device to all SIM cards is to obtain
a subsidy unlock code from a carrier.


Brick

-

a device that no longer functions, generally caused
by a failed firmware or SPL update. Since the device no
longer works as intended, it is often referred to as a "brick"
or "paper
-
weight", since that is all it is good for.

Android Tools


ADB

-

Android Development Bridge. Command line
tool used to communicate with & control the device
over a USB link from a computer.


Daemon runs as root on emulator/
root’d

phone, otherwise
very limited privileges


Can send shell commands (
dd
,
ls
, mount, cat,
ps
, date,
uptime,uname

-
a, mount, etc.)


Can recursively push/pull files (logical)
adp

pull|push

<
src
>
<
dest
>


Fastboot

-

a protocol used to directly update the flash
filesystem

in Android devices from a host over USB. It
allows flashing of unsigned partition images.

Android Tools


flash_image

-

A small utility to be used on the device
that allows you to re
-
write system partitions with
image files.


NANDroid

-

A set of tools that will enable anyone who
has root on their Android device to make FULL system
backups.
NANDroid

will backup (and restore) /system,
/data, /cache, and /boot partitions.


Zipalign

-

Reduces the amount of RAM consumed
when running the application by allowing data to be
mmap'd

in; which causes all uncompressed data within
the .
apk
, such as images or raw files, to be aligned on
4
-
byte boundaries.

Android System Architecture


Android is a multiprocessing system.


Each application runs on top of the Android
Dalvik machine.


Each Dalvik machine runs on top of a Linux
process.


Each process runs in its own sandbox, which
means it can access only the resources it creates.


To access resources outside of the application
sandbox, the application needs to request
permission from the Android system.


android.Manifest.permission
.

System Architecture Overview

Linux Kernel


Android relies on Linux version 2.6.x for core


Approximately 115 patches from mainline


Provides system services, including:


security, memory management, process management,
network stack, and device driver model.


The kernel also acts as an abstraction layer
between the hardware and the rest of the
software stack.(easier access to peripheral
hardware devices)

System Libraries


On top of Linux is a set of libraries including
bionic (the Google
libc
), media support for
audio and video, graphics and a lightweight
database, which is a useful repository for
storage and sharing of application data.


Provides most of the functionality available in
the core libraries of the Java programming
language.

Android Run
-
time


Provides a consistent environment on which every
Android application runs


Each Android application runs in its own process, with
its own instance of the Dalvik Virtual Machine.


Relies on the Linux Kernel for underlying functionality
such as:


Multi
-
Threading


Low
-
level memory


management

Dalvik Virtual Machine


Dalvik VM != Java VM


Designed specifically for Android and is
optimized in two key ways.


multiple instantiation (each application has its
own private copy running in a Linux process)


register based (instead of being stack based like
most Java VMs)


Executes the Dalvik Executable (.
dex
)


optimized for minimal memory footprint.

Processes & Lifecycles


Components share a common Linux process:


by default, one process per .
apk

file.


.
apk

files are isolated and communicate with each other via
Intents or AIDL.


When the first of an application's components needs to be
run, Android starts a Linux process for it with a single
thread of execution (Main Thread).


Android tries to maintain a process for as long as possible,


eventually will need to remove old processes when memory
runs low.


To determine candidates to be killed, Android places each
process into an "importance hierarchy" based on the
components running in it and the state of those
components.

Android Device Partitions


Android devices have their internal flash memory divided into partitions.


Common Partitions


splash1
: This partition contains the very first image you see when you turn on
the device.


boot
: This partition stores the kernel. It also controls other aspects of the
operating system on the device, such as what file formats the device is able to
support.


recovery
: This partition holds the device's recovery software, which gives the
user access to special functions, such as upgrading the operating system, or
making a backup of the device's software.


system
: This is the main partition where the device's operating system is
stored. Generally speaking, this partition is write
-
protected, and only able to
be modified under special circumstances.


cache
: This partition stores temporary files. Generally, files on this partition
are not critical to the operation of the device.


data
: This partition is where all of your data is stored. If you download an app
from the Android Market, it goes here. If you change a setting, that change is
stored here. If you receive a SMS, it is stored here.

Android file systems

michael@xt2lin:~$
adb

shell

# mount

rootfs

on / type
rootfs

(
ro,relatime
)

tmpfs

on /dev type
tmpfs

(
rw,relatime,mode
=755)

devpts

on /dev/pts type
devpts

(
rw,relatime,mode
=600)

proc on /proc type proc (
rw,relatime
)

sysfs

on /sys type
sysfs

(
rw,relatime
)

none on /acct type
cgroup

(
rw,relatime,cpuacct
)

tmpfs

on /
mnt
/
asec

type
tmpfs

(
rw,relatime,mode
=755,gid=1000)

tmpfs

on /
mnt
/
obb

type
tmpfs

(
rw,relatime,mode
=755,gid=1000)

none on /dev/
cpuctl

type
cgroup

(
rw,relatime,cpu
)

/dev/block/mtdblock3 on /system type yaffs2 (
ro,relatime
)

/dev/block/mmcblk0p1 on /data type ext3
(
rw,nosuid,noatime,nodiratime,barrier
=1,nodelalloc,data=ordered)

/dev/block/mtdblock6 on /data/data type yaffs2 (
rw,nosuid,nodev,relatime
)

/dev/block/mmcblk0p2 on /cache type ext3
(
rw,nosuid,nodev,noatime,nodiratime,barrier
=1,nodelalloc,data=ordered)

Android MTD blocks

michael@xt2lin:~$
adb

shell

# cat /proc/
mtd

dev: size
erasesize

name

mtd0: 000a0000 00020000 "misc"

mtd1: 00500000 00020000 "recovery"

mtd2: 00280000 00020000 "boot"

mtd3: 0f800000 00020000 "system"

mtd4: 000a0000 00020000 "local"

mtd5: 02800000 00020000 "cache"

mtd6: 09500000 00020000 "
datadata
"

Android boot.img

+
-----------------
+

| boot header | 1 page

+
-----------------
+

| kernel


| n pages

+
-----------------
+

|
ramdisk

| m pages

+
-----------------
+

| second stage | o pages

+
-----------------
+


n = (
kernel_size

+
page_size

-

1) /
page_size


m = (
ramdisk_size

+
page_size

-

1) /
page_size


o = (
second_size

+
page_size

-

1) /
page_size



Android boot.img


A
ramdisk

is a small
filesystem

containing the core files needed to initialize the system.


It includes the critical init process, as well as
init.rc
, which is where you can set many system
-
wide properties.


Files on a typical
ramdisk
:



./
init.trout.rc





./
default.prop



./proc


./dev


./
init.rc



./init

./
init.goldfish.rc


./
sbin


./
sbin
/
adbd


./system

./data


Compile Android System


Android system is complete compilation contains
the following:


Host tools; Target system, boot, and data image files;
Target Linux kernel (compiled separately); Installable
zip file to flash in Recovery mode


Basic Process:


Install Dependencies


Acquire the Source


Copy proprietary files


Configure & Compile

Install Dependencies


For 32/64
-
bit systems:

sudo

add
-
apt
-
repository "
deb

http://archive.canonical.com/
maverick partner“

sudo

apt
-
get update

sudo

apt
-
get install
git
-
core
gnupg

flex bison
gperf

libsdl1.2
-
dev libesd0
-
dev libwxgtk2.6
-
dev
squashfs
-
tools build
-
essential zip curl libncurses5
-
dev zlib1g
-
dev sun
-
java6
-
jdk
pngcrush

schedtool



For 64
-
bit systems only :

sudo

apt
-
get install g++
-
multilib

lib32z1
-
dev lib32ncurses5
-
dev
lib32readline5
-
dev gcc
-
4.3
-
multilib g++
-
4.3
-
multilib



Get the repo tool

curl http://android.git.kernel.org/repo > ~/bin/repo

chmod

a+x

~/bin/repo

Acquire the Source Tree


Make a directory for the source tree

mkdir

~/android

cd

~/android



Use the repo tool to initialize and sync the remote/local
repositories

repo init
-
u
git://github.com/CyanogenMod/android.git
-
b
gingerbread

repo sync
-
j16



~123,694 items, requires about 3.0 GB storage

Copy Proprietary Files


While android is almost entirely open
-
source certain components remain
proprietary & closed


Mostly hardware related


Must be copied from current system builds


Navigate to your specific device

cd

~/android/device/[vendor]/[device]/

./extract
-
files.sh



Get Rom
-
manager and Google App files

cd

~/android/vendor/
cyanogen

./get
-
rommanager

./get
-
google
-
files



NOTE: If some hardware isn't working, like camcorder or FM radio, you will
need to find the updated prop blobs.

Configure & Compile


Setup the build environment

.
build/envsetup.sh



Use the ‘brunch’ utility to select build target

brunch [ target ]



Takes ~1
-

3 hours to complete


After compiled:


~172,562 items, requires about 6.6 GB storage

Compilation Results


Compiled host utilities:





~/android/out/Host/Linux
-
x86/bin



Compiled target files:





~/android/out/target/Product/[target]


ramdisk

.
img

[root file system image]


root [root file system directory]


System [master file system directory]


System .
img

[main file system image]


UserData

-

qemu

.
img

[ QEMU data image]


UserData

.
img

[image data]

Flash the system image


Make a
NANDroid

backup


Copy your .zip file from:

~/android/out/target/product/[target]/update.cm
-
XXXXX
-
signed.zip



to the root of the SD card.


Download Google Apps and place at root of the SD card. (optional,
but recommended)


Reboot into device Recovery mode


Seriously, make a
NANDroid

backup


Flash one/both of these .zip files from recovery.



NOTE: Be patient, Sometimes this may take a VERY long time!

Loading source in eclipse


Make sure you have source and your build completed
without errors!


Copy the .
classpath

file to the root folder of your source

cp ~/bin/eclipse/.
classpath

~/android/



Open Eclipse


Click the File menu » new » java project


Give a project name like
Cyanogen


Uncheck use default location


Point the location to ~/android


Keep all other defaults & Click finish

Download the kernel source


Make a directory to store the kernel tree

git

clone git://github.com/CyanogenMod/cm
-
kernel.git



38,463 files, requires about 409.4 MB



View other available branches:

git

branch



Checkout a branch:

git

checkout

b ‘
branch_name



Configure the Build


Pull the .
config

from the newest boot.img

scripts/extract
-
ikconfig

boot.img > .
config



Alternatively, retrieve a working kernel
config

from the device, and
unzip it:

adb

pull /proc/config.gz
/home/
michael
/android/kernel/cm
-
kernel/config.gz

cat config.gz |
gunzip

> .
config



Just enter to accept defaults, or customize as needed:


make ARCH=arm CROSS_COMPILE=$CCOMPILER
oldconfig

make ARCH=arm CROSS_COMPILE=$CCOMPILER
menuconfig

Compile the kernel


Make the build

make ARCH=arm CROSS_COMPILE=$CCOMPILER

j5



This step may take a while (but not nearly as long as
the system compilation


Finally, you should have a kernel stored in

~/android/kernel/cm
-
kernel/arch/arm/boot/
zImage

Compiling kernel modules


After installing a custom kernel, the
wifi

module may be unstable or unusable.


The solution is to recompile the module, linking it to the new kernel build.

cd

~/android/system/system/
wlan
/
ti
/sta_dk_4_0_4_32

KERNEL_DIR=~/android/kernel CROSS_COMPILE=$CCOMPILER ARCH=arm
make

j5



A file named
wlan.ko

will be produced in the current directory.


You must install the new module to the device.

adb

shell mount
-
o
remount,rw

/system

adb

shell cp /system/lib/modules/
wlan.ko

/system/lib/modules/
wlan.ko.backup

adb

push
wlan.ko

/system/lib/modules/
wlan.ko



Reboot the device.


If all goes well, you should be using the newly compiled '
wlan.ko
'.

Additional Info & References


http://android.git.kernel.org/


http://wiki.cyanogenmod.com/


http://wiki.cyanogenmod.com/index.php?title=B
uilding_Kernel_from_source


http://android
-
dls.com/wiki/


http://android
-
dls.com/wiki/index.php?title=HOWTO:_Unpack,_
Edit,_and_Re
-
Pack_Boot_Images


http://source.android.com/source/download.ht
ml