System Reference - The UK Mirror Service

estrapadetubacityΚινητά – Ασύρματες Τεχνολογίες

10 Δεκ 2013 (πριν από 3 χρόνια και 8 μήνες)

94 εμφανίσεις

Syste


System Reference

2013




Content
List of graphics ............................................................................................................3
List of tables .................................................................................................................4
1 Purpose of LynxSight .............................................................................................5
2 Setting up project ...................................................................................................6
2.1 Project dependences ...........................................................................................6
2.1.1 Operating system.........................................................................................6
2.1.2 Eclipse IDE .................................................................................................6
2.1.3 Other development enviroments ..................................................................7
2.2 Setting up IDE ....................................................................................................7
2.2.1 Tegra android development pack .................................................................7
2.2.2 Integrating Eclipse IDE ...............................................................................8
2.2.3 Android Development Tools .......................................................................8
2.2.4 Android NDK .............................................................................................9
2.2.5 Tesseract for Android ..................................................................................9
2.2.6 OpenCV for Android ................................................................................. 10
2.2.7 Android support v7 ................................................................................... 11
3 Design reqirements .............................................................................................. 13
4 Application workflow .......................................................................................... 14
5 Description of packages ....................................................................................... 15
6 Description of classes ........................................................................................... 16
LynxSight System reference
3
List of graphics
Pic 1 Example of Android.mk file used in project LynxSight ................................ 11

LynxSight System reference
4
List of tables
Table 1: Description of application packages ............................................................... 15
Table 2: Description of application classes .................................................................. 16
LynxSight System reference
5
1 Purpose of LynxSight
LynxSight is mobile application built for Android OS devices. It is an OCR system
designed for use by visually impaired people. LynxSight has following features:

Voice commands
Simple navigation
Text reading (text to speech)

LynxSight allows user to take picture of text anywhere he is read it aloud. Text
on Picture is recognized and higlighed what is done by decompozition module.
Application is designed in Java language using android native libraries such as
OpenCV or Tesseract.

LynxSight System reference
6
2 Setting up project
Sources and resources for project LynxSight are organized in Eclipse Android
projects. Eclipse is used because of availability and suitability of tools used in
developing of project. Setting up Eclipse IDE is capitol 2.2. Besides setting up the IDE
Android development tools ADT and NDK for native development are presented. In
this section are also presented used third-party libraries such Tesseract OCR tool and
OpenCV library to increase of officiency of manipulating with rastr-based images.
Finally, requirement of the majority of Android-based devices compatibility should be
met. Therefore Android support modul is integrated to the project to ensure smoothly
run of application on devices with older version of OS Android. Some of mentioned
development tools are integrated to Tegra Android development pack presented by
NVIDIA. Usage of this pack is highly recommended for unexperienced, but also
becouse of development efficiency experienced developers too.
2.1 Project dependences
2.1.1 Operating system
• Windows XP (32-bit), Vista (32- or 64-bit), or Windows 7 (32- or 64-bit)
• Mac OS X 10.5.8 or later (x86 only)
• Linux (tested on Ubuntu Linux, Lucid Lynx)
o GNU C Library (glibc) 2.7 or later is required.
o On Ubuntu Linux, version 8.04 or later is required.
o 64-bit distributions must be capable of running 32-bit applications.
2.1.2 Eclipse IDE

Eclipse
3.6.2 (Helios) or greater
Note: Eclipse 3.5 (Galileo) is no longer supported with the latest version of
ADT.
• Eclipse
JDT
plugin (included in most Eclipse IDE packages)

JDK 6
(JRE alone is not sufficient)

Android Development Tools plugin
(recommended)
• Not compatible with GNU Compiler for Java (gcj)

LynxSight System reference
7
2.1.3 Other development enviroments

JDK 6
(JRE alone is not sufficient)
• NDK
• For all development platforms, GNU Make 3.81 or later is required. Earlier
versions of GNU Make might work but have not been tested.
• A recent version of awk (either GNU Awk or Nawk) is also required.
• For Windows,
Cygwin
1.7 or higher is required. The NDK will not work with
Cygwin 1.5 installations.

Apache Ant
1.8 or later
• Not compatible with Gnu Compiler for Java (gcj)
Note: Some Linux distributions may include JDK 1.4 or Gnu Compiler for Java, both of
which are not supported for Android development.
2.2 Setting up IDE
There are two approaches to properly set up development enviroment for running
project LynxSight. First is download and install all development tools separatelly and
avoid installing development tools which could be useless. Second approach expects
download and installation of
Tegra android development pack
. This option si
recommended for developers less experienced in android development and also every
developer which has not configured his enviroment yet. Consider properly which
approach would you choose before started developing.
2.2.1 Tegra android development pack
Tegra android development pack includes all software tools required to develop
for Android platform. The pack is availiable to dow nload at
https://developer.nvidia.com/tegra-android-development-pack
. Before download
registration at NVIDIA developer zone and signing up is required. To be able to get the
pack you have to apply
Tegra registered developer program
. After application it is
necessary to wait until your request is approved. After installation file is run you are
forced to choose which tools you want to download and install. Select
• Eclipse IDE
• Android Software development kit (SDK including ADT)
• Android Native development kit (NDK)
• Cygwin (when using Windows OS)
• Apache Ant
LynxSight System reference
8
• Java development kit (JDK)
• OpenCV for Android platform
Use of current version of development tools is highly recommended option.
Download and installation can take even several hours according to local internet
connection and variety of selected Android SDK versions. It is recommended to install
tools for every version since Android API 7.
Notice where are tools installed and properly set system enviroment variables to
Cygwin, Android SDK, JDK, NDK and Apache ant. To set system enviroment variables
go to Control panel – System – Advanced system settings – Enviroment variables. Edit
Path variable with addition of Path of mentioned tools above if they have not been set
yet.
2.2.2 Integrating Eclipse IDE
Different status of developer means different configuration. If you are using Tegra
android development pack, your Eclipse IDE should be binded with Android SDK and
NDK. If it is not, or different approach is used, including building enviroment from
scratch or adding tools to existing Eclipse enviroment, several steps should be
performed.
To import LynxSight project to Eclipse IDE go to File – Import – Existing Projects
into Workspace, select Browse and find destination of Eclipse project LynxSight in file
system. Project appears in local Workspace. Probably it would contains errors and
additional configuration is required.
2.2.3 Android Development Tools
In Tegra version check if Android SDK is properly set. Android SDK is core of
Android Development Tools. Android development tools provide tools to compile,
debug, run and test android applications. Also Android device emulator is included. At
Windows – Preferences – Android check SDK Location which should match the
location of Android SDK in file system.
In non Tegra version when Android SDK is not present, install perform the
following steps:
1. Start Eclipse, then select Help > Install New Software.
2. Click Add, in the top-right corner.
LynxSight System reference
9
3. In the Add Repository dialog that appears, enter "ADT Plugin" for the Name and
the following URL for the Location:
https://dl-ssl.google.com/android/eclipse/
4. Click OK.
If you have trouble acquiring the plugin, try using "http" in the Location URL,
instead of "https" (https is preferred for security reasons).
5. In the Available Software dialog, select the checkbox next to Developer Tools
and click Next.
6. In the next window, you'll see a list of the tools to be downloaded. Click Next.
7. Read and accept the license agreements, then click Finish.
If you get a security warning saying that the authenticity or validity of the
software can't be established, click OK.
8. When the installation completes, restart Eclipse.
2.2.4 Android NDK
In Tegra version check if Android Native Developmen Kit (NDK) Path is properly
set. The NDK is a toolset that allows you to implement parts of your app using native-
code languages such as C and C++. At Windows – Preferences – Android - NDK check
the NDK Location which should match the location of Android NDK in file system. In
project LynxSight native modul Tesseract and OpenCV for Android are used.
In non Tegra version download Android NDK from
http://developer.android.com/tools/sdk/ndk/index.html
and unzip to the file system.
Then set the NDK path as was described above.
If project was imported incorrectly, or there is need for creating new project and
importing source and resource files manually, follow the following steps. Right click to
the project – Android tools – Add native support. Check if jni directory was generated.
2.2.5 Tesseract for Android
Project LynxSight already contains tess-two.jar library in libs directory. It also
contains required .so files liblept and libtess in libs/armeabi and libs/armeabi-v7a
folders. If you build your project using Windows you may use this compilated libraries.
However, if you build your project in another operating system you should build
Tesseract engine by yourself. Download the source or clone this
git repository
at
https://github.com/rmtheis/tess-two. This project contains tools for compiling the
LynxSight System reference
10
Tesseract, Leptonica, and JPEG libraries for use on Android. It contains an Eclipse
Android library project that provides a Java API for accessing natively-compiled
Tesseract and Leptonica APIs. Go to your Android.mk file and add the following lines
between include $(CLEAR_VARS) and include $(BUILD_SHARED_LIBRARY:
APP_PLATFORM :
= android-7

Use command line to build Tesseract 3.02 toolset for Android. Set current
directory to directory where you downloaded tess-two project. Now you are going to
build tesseract for your OS enviroment. Make sure you have set enviroment variables to
android NDK, android SDK and Apache ant. Variable --path means path to your tess-
two project.
cd <project-directory>/tess-two
ndk-build
android update project --path .
ant release
After succesfull build of tesseract project import tesseract project to your
Workspace. Then right click to the project Tesseract – Properties – Android –Check Is
library radio button. After that copy tess-two.jar library from Tesseract project build
folder to libs folder of your project. Also copy content of Tesseract libs folder to your
libs folder. It contains required native .so libraries for tesseract and leptonica support.
Then right click to your project – Android tools – Fix Project properties to make sure
tesseract tools are ready to use.
After the initial import, on a non-Windows (Linux and Mac OS) operating system
Eclipse will still show build errors for applications with native C++ code. To resolce the
issues, please do the following: Open Project Properties -> C/C++ Build, and replace
“Build command” text to "${NDKROOT}/ndk-build" (remove .cmd at the end).
2.2.6 OpenCV for Android
OpenCV is toolset for image processing. It is important element of LynxSight
project becouse of presence of Decomposition modul. OpenCV for Android toolset is
also integrated in Tegra android development pack.
Once using Tegra android development kit, OpenCV toolset project should be
installed. Find the project in NVIDIA pack and inport to the Workspace. Reference your
project to project of OpenCV or reference to it’s library. To reference to OpenCV
project right click to the project OpenCV – Properties – Android – Check Is library
radio button and then right click to your project – Properties – Android – Add and select
LynxSight System reference
11
library project OpenCV for Android. Make also sure, that OpenCV is placed in the
same directory.
Another way, which is more common is to import OpenCV .jar library to your
project. Once the OpenCV project is labeled as Library project, opencv library –
2.4.5.jar is placed in its bin folder. Copy this library to the libs folder of your project.
The same way is Tesseract for Android imported.
If you are not using Tegra android development pack, download OpenCV for
android SDK at
http://sourceforge.net/projects/opencvlibrary/files/opencv-android/
.
Unzip the pack into the particular folder and notice the path. Follow the instructions like
Tegra pack user described above.
Go to your Android.mk file and add the following lines between include
$(CLEAR_VARS) and include $(BUILD_SHARED_LIBRARY:
OPENCV_CAMERA_MODULES:
=on
OPENCV_INSTALL_MODULES:
=on
An example is shown on Pic. 1.

Pic 1 Example of Android.mk file used in project LynxSight
For more information abou setting up the OpenCV for Android toolset follow the
instructions at:
http://docs.opencv.org/doc/tutorials/introduction/android_binary_package/O4A_S
DK.html#get-the-opencv4android-sdk

2.2.7 Android support v7
Although there is a presumption, that software developed with official android
development tools should be able to run at most android based devices, the whole
system involve and older version use not to respect new features. That is why
Android
LynxSight System reference
12
support v7
is used. In project LynxSight it brings compatible action bar to all Android
versions since Android SDK version 7.
To get Android support v7 library follow the following instructions:
1. Start the Android
SDK Manager
.
2. In the SDK Manager window, scroll to the end of the Packages list, find the
Extras folder and, if necessary, expand to show its contents.
3. Select the Android Support Library item.
Note: If you're developing with Android Studio, select and install the Android
Support Repository item instead.
4. Click the Install packages... button.

Fill metadata in uses-sdk tab in your Android manifest file. MinSdkVersion field is
required.
<uses-sdk
android:minSdkVersion="7"
android:targetSdkVersion="17"
/>
After installing the particular packages in your AVD manager notice their location
and import project Android support v7 appcompat to your Workspace. Then put a
reference from your project to the Android support project. It is important to use project
reference instead of library reference becouse resource files in Android support project.
Putting a project reference process is described in OpenCV and also Tesseract section.
For troubleshooting and more details about Android support v7 setup follow
http://developer.android.com/tools/support-library/setup.html





LynxSight System reference
13
3 Design reqirements
Designed of application has to follow some requirements:
o User flexibility
o Highest market target
User flexiblity means that applications should be usable without any special
conditions. For example application should be usable without access to internet. This
requirement determines that all application components should work offline.
Highest market targeting means that application should be programmed in way
that maximizes number of its potential users. Application is therefore implemented on
Android OS for versions up to 2.1.
LynxSight System reference
14
4 Application workflow
Application workflow starts with Main menu displayed. User has choice to go to
settings menu or to Picture capture screen. Settings menu allows user to set som basic
settings sucha s voice assistance, language of OCR and text to speech and so forth.
Picture capture screen is first step to OCR. In this step camera is initialized and
camera preview is displayed. User has possiblity to change flashlight settings. Picture is
taken simply by pressing any point on preview screen. Focus is done automaticaly after
the touch.
After the Picture is taken decomposition task is fired. In this step application tries
to find logical text regions. When task is finished result screen is displayed with
highlighted regions of text. User has possibility to read selected region aloud simply by
touching on it. The touch on certain region fires OCR task which results in recognized
text from region that is read aloud by text to speech module.



Main
menu

Settings
Capture
screen

Result
screen
LynxSight System reference
15
5 Description of packages
Table 1: Description of application packages
Package Description
Sk.LynxSight This package contain basic classes of
application logic. Main activity class be
found here along with classes covering
general functionality,
Sk.LynxSight.camera This package collects classes of camera
preview functionality.
Sk.lynxsight.OCR The OCR is done using classes of t his
package. OCR requests are done using
asynchronous tasks using native code.
Sk.lynxsight.storage Classes for storing informatio n about
application state and settings are stored
here. For example SQLite Access layer.
Sk.lynxsight.magnifying Classes of camera for Magni fying glass
feature
Fejero.lsight.decomposition This package covers cla sses which brings
tools for decomposition of document taken
by digital camera
Fejero.lsight.font This package include classes whi ch bring
text size detection tools using opencv
library
LynxSight System reference
16
6 Description of classes
Table 2: Description of application classes
Package Class Description
Sk.LynxSight MainActivity MainActivity represents
general structure of
application logic. Activity
layouts and their contents,
actions and behavior is
programmed here. We use
only one activity in
LynxSight for all
submenues so far.
Logger The Logger class provides
loging functionality mostly
for debuging purpouses.
This is just stub.
Orientation Enum for display
orientation naming used in
application
ScreenDisplayed Enum for possible menus
(layout) currently displayed
on screen.
SensorListener Event listener for listening
on screen orientation
changes in three axes.
SettingsManager This is a container class
that groups and stores all
user settings data.
Utils Static class for general
purposes methods.
Sk.lynxsight.camera CameraPreview The class represe nts camera
LynxSight System reference
17
preview view displayed on
camera screen when taking
Picture. Camera settings
and preview quality is
managed in this class.
Sk.lynxsight.OCR OCRAsyncTask The Class
OCRAsyncTask. This class
provides OCR method to
get string representation of
bitmap. We use tesseract

3.01 engine encapsulated in
Java interface. It is
important to initialize
tessdata
from local
database. Tessdata
file have
to be in particular folder. In
this case it is root/tesseract
.
OCRBundle
The Class
OCRBundle. Encapsulates
language of OCR and
bitmap to be translated.
This bundle is important,
becouse asynchronus call
get only one reference.
OCRCorrection This class performs some
post processing corrections
on recognized texts in order
to make results better,
Sk.Lynxsight.storage DatabaseHandler The class is Access layer to
persistent data storage. In
current implementation it is
SQLite handler.
LynxSight System reference
18
Sk.lynxsight.magnifying MagnifyCameraPreview Interface and camera holder
for Magnifier tool. This
class has simmilarities with
CameraPreview class.
MagnifyingActivity Activity class which
activate camera-magnifying
glass tool. This activity
initialize camera preview.
Fejero.lsight.decomposition

Area On of basic elements of
Decomposition algorithm.
Contains all necessary
information about individal
text block in docment sch a
size and position in
document.
BlockFinder In this class is algorithm for
text detecting implemented.
Graphical Seed-Fill
algorithm is used
combinated with random
selection of tiles.
Decomposition Interface for LynxSight
application, which provides
call for list of detected text
blocks.
Threshold Class which provides canny
tools for binary image
output. Uses OpenCV for
android graphic library.
Tile The basic element of
decomposition algorithm.
Provides information about
LynxSight System reference
19
size, location, textual
content and calculate those
data for himself.
TileDrawer Class provides method
which draws tiles in image
view. Usefull in debug
mode.
TileManager This class cuts the bitmap
into a 2
nd
array of tiles and
analyse those tiles. Also
calculates number of rows
and cols and size of one
tile. Also creates lists of
every group of neightbour
tiles and generate Areas.
Fejero.lsight.font FontSizeDeterminer Class calcula te average
height of characters sed in
text. From this data is
appropriate size of tile
calclated.