Real time detection of pin hole on worm-eaten chestnut with 2CCD camera

The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013


Real
t
ime
d
etection

of
pin

h
ole on
worm
-
eaten c
hestnut with

2
CCD
c
amera


Soo Hyun Park
1


Soo Hee Lee
2




Seong Min Kim
3

and S
ang Ha Noh
1,4

*



1
Department of Biosystems &

Biomaterials Science and Engineering, Seoul National
University, Seoul, 151
-
921, Republic of Korea.

2
Life& Technology CO. LTD., Hwaseong
-
si Gyeonggi
-
do, 445
-
964, Republic of Korea.

3
Department of Bioindustrial Machinery Engineering, Chonbuk National

Univ
ersity
,
Jeonju, 561
-
756,

Republic of Korea.

4
Research Institute for Agriculture and Life Sciences, Seoul National University, Seoul, 151
-
921, Republic of Korea.


* Corresponding Author

Tel: +82
-
2
-
880
-
4603, Email :
noh@snu.ac.kr


Abstract


Overall q
uality of chestnut is determined mainly by size, weight, shape

and
internal
disorders

such as decay, worm
-
eaten, etc. In Korea
chestnuts having
internal disorders are
picked out

manually

on
individual

basis
by
surface
color and pin hole
s

made by worm.

The
ultimate purpose
of this study is to develop
a chestnut sorter using multi
-
channel vision
system
which can detect pin hole
s

and the color of chestnut. Primary
,

this
study

is
focused

to
detect
ion of
pin holes on chest
n
ut

surface
.
Both c
olor image and
g
ray
-
scale image in near
infrared
band

were captured simultaneously
from the chestnut sample with a
n image
acquisition

system constructed with
a
2
CCD camera
and

an algorithm to detect pin holes
were developed with
b
inary
image process
ing
of the gray scale images
.
A sorting system
which
is
consisted
of feeder, roller conveyor, a multi
-
channel camera and automatic
discharging
device

was developed and used
to

evaluate the
effectiveness of the pin
-
hole
detection and sorting
algorithm
.
Finally
,

p
in
-
hole detection and sorting rate w
ere

evaluated

great than 85 %,
show
ing

possibility of
commercialization

of the developed system with
further study on analysis of surface color

related to internal
disorder.


Keywords:
Chestnut
,
I
nternal disorder,
Machine vision,
NIR

image, On
-
line
detection
.





The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013


Introduction

Chestnut (Castanea spp.) is popular for its abundant nutrients and delicious taste in
Asia and Europe

(
6
). More than 2,000,000 ton chestnuts were produced annually, and the
consumption and production of chestnut continue to rise all over the world according to the
Food and Agriculture
Organization Statistics (
3
).

Generally, chestnut quality is measured not
o
nly by

internal
disorders such as decay, worm
-
eaten, etc
.,

which are important for consumer
acceptance
, but also

by external factors such as color, shape, size, and surface status.

Most
importantly, the external appearance usually is not altered, at least initially
, but

it may cause
the moisture loss inside, which leads internal disorders usually accelerating the anatomical
and physiological changes within the tissue

(1
)
.
On that ac
count,
p
in holes
made by worms
on chestnut hull

are

of great concern to the chestnut
producers

and
consumers

because they
indicate inside of the chestnut is
disordered

and appearance of caterpillar
in peeling process
can make customers feel revolting.
As t
ime elapses,
pin
-
hole

chestnuts start to decay easily
and eventually infect
other sound chestnuts
.

If
chestnut fleshes

are decayed,
relatively

large

dark
color

is exposed on the hull
.

In chestnut packing house, s
izing of chestnut is
usually
done by
a hole

size of drum

but pin
-
hole and dark color
chestnuts

are picked up
manually

on
individual basis.

Screening work is labor intensive and its efficiency is not so high since pin
holes are hard to find because their

sizes are small and they are located
random
ly
,

and dark
color is not apparent.

In order to improve the detecting precision of the defective chestnuts with a
commercialized sorter, a technique based on near
-
infrared (NIR) spectroscopy or machine
vision have been proposed (4).
NIR spectroscopy is a non
-
destructive measuring method with
the advantages of minimized preparation of samples, fast, cheap, and easy to ope
rate and
environment
-
friendly (8
). It has been conducted to do both quantity and quality analysis of
agricultural products.

Liu et al.

used
NIR spectra in 833
–

2500 nm from chestnuts to detect
moldy chestnut, and it performed the detection ratio over 92.8%

(5
)
.

Machine vision
systems
have been used for evaluation of color grading
,
detection of surface defects, sizing and shape
detection of fr
uits and vegetables since
the late 1970s

(4
).

Machine vision which uses
cameras

instead of human eyes in carrying out measurement and judgment,

and it

is now
widely
applied to commercialized fruits sorters.
Fang

et al. designed
a

machine vision for

real
-
ti
me chestnut rating system (
2
)

and Wang et al. proposed

a recognition method

of
worm
-
eaten chestnuts based on machine vision techni
que

(7
)
.

The ultimate purpose of
this

study is in developing an on
-
line chestnut sorting system
consisted of feeder, conveyor, machine vision, automatic discharger, etc
.
, which can be used
at chestnut packing house. However, the aim of present study is
focused

to construct a multi
-
channel mac
hine vision system, to develop an algorithm to detect pin holes on chestnut hull
and to evaluate
performance of the algorithm
on
-
line status.


Materials and methods

Samples

Chestnut samples
(Daebo species)
were obtained from on
-
line market, which were
har
vested
in October 2010
at Kong
-
ju, the southern area of Korea.
The samples for the
experiment include
20 normal fresh chestnuts and.
6
chestnuts having pin
-
holes on hull.


Imag
e acquisition

Primarily,
a

CCD camera (Model: KF
-
F2A, Hitachi, Japan)
equipped
with a filter
wheel to the camera lens
was

used to
obtain

images of normal and defective chestnuts at 8
different wave bands of 720, 740, 780, 840, 880, 900, 940, and 960nm. Those images were
analyzed in view of
detecting
possibility

of
pin

hole
s

at
differ
ent wavelength bands
.

Secondarily,
a
2
CCD

camera (JAI AD
-
080GE) which is designed as shown in Fig
.

1

was
The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013


used to capture
both

the

color and NIR images
at the same time from a chestnut sample
.
This
camera

uses

two

prisms

in the optical path
in order to divide one image to two images and
band
-
pass filters
are installed
on each spectral axis to capture
color image

through a Bayer
Filter Array on one sensor and
NIR image in the range of 750 to 900 nm

on the other
.

The
camera has a resolution of 1
024 x 768 pixels and is able to capture 30 frames per second in
full frame mode. SDK

(Software Development Kit)
was

provided that we programmed to
allow simultaneous RGB and NIR
image acquisition
.



Figure

1
.

Schematic of
a 2
CCD
camera


Lights source

Krypton lamp was used for lighting, and was investigated the
spectral

characteristic

by wavelength

using a spectro
meter

(USB4000, Ocean Optics, USA). It is determined that
lighting is effective in range between 630nm and 960nm shown as Fig.
2
.




Figure

2
.

Spectral c
haracter
istic of a krypton light source


On
-
line sorting system

A laboratory scale of on
-
line sorting system
which is consisted with feeder, roller
conveyor, image acquisition and discharging units
was constructed
to evaluate the
performance of
pin
-
hole detection and sorting algorithm

developed in this study. A linear
feeder

(Fig 3
-
A)

was designed and fabricated for
d
ispersing

and aligning
the
chestnuts
The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013


supplied in bulk status.

The chestnuts fed by linear feeder moves to roller conveyor shown in
Fig.
3
-
B
, which was designed so that eac
h chestnut
could

evolve slowly on the roller and
moves to image acquisition unit.



Fig
ure

3
.

Linear feeder (A) for aligning the chestnuts and r
oller conveyor (B) for rotating and
transferring chestnuts


The camera unit was installed near

the

discharging end of

the

roller conveyor
and
guide

strips
consisted with 8 channels
w
as

put on the surface of roller
under the camera

view
area

shown as Fig. 4
. The rol
e

of guide
strips is to keep chestnuts aligned in each
line
until
they
drop to
discharge actuator.
While

chestnut sample
s pass

the
guide strips
, they should
evolve at least one turn so that total surface is exposed to the camera
. The camera shut
ter
operates three times
while

the sample passes through the
camera
view
window
.

As the re
sult,
three

image

frames are taken
from each chestnut sample
at least
and
those were
analyzed for
detection of pin holes
.



Fig
ure

4
.

Outlook of two multi
-
channel cameras, guide lines, and
discharging area


Method for d
ischarging of defective
chestnuts

An automatic discharging unit shown in Fig. 5, which was made with 8 flat
rectangular bars and 8 solenoid actuators, was installed at the end part of the roller conveyor
right after the guide strips. Each bar accounts for each guide channel. If a chestnut
sample is
decided to be normal or defective by the image analysis, a signal is transferred to the
corresponding discharge solenoid. Whenever pin
-
hole samples are sliding on the discharging
The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013


bars, the corresponding solenoids are actuated and the bar moves up

so that flow directions
are changed (Fig. 5
-
A
). Otherwise, the actuators are not on and the bars stay down (Fig.
5
-
B
).
In other words,
the defective chestnuts are
discharged

upward

and
the normal fall through the
ordinary path. In actuating the dischargin
g bar
,

the delay time of the signal transfer from the
camera and to actuator reaction is very important in order to synchronize the sliding speed of
chestnut sample on to the bar and the timing of the bar actuation.



Figure

5
.

Bar
-
type discharging devi
ce actuated by solenoid (when the
defective

chestnut
passes,
it moves up (A) to change direction of the defective, otherwise t
he bar stays down (
B
)


Development of
pin hole
detection

algorithm

Detection software for screening worm
-
eaten and decay
ed

chestnut was developed
using
an image
acquisition

system
equipped with a 2CCD camera.
S
oftware provided by the
frame grabber SDK was used to be opera
ted on Windows XP configuration

and b
inary image
process was

applied to the NIR band images for

screen
ing

pin holes.

Three threshold vales of

90, 100, and 110 were examined

to get binary image and the ratios of the major and minor
axes of the blobs in the binary image

were computed to
reconize

if it
is

a pin hole or not.
Effect of roller conveyor speed

(0.15,

0.20 and 0.25 m/sec)

on pin hole detection was also
examined
.


Results and discussion


Features of Gray
-
scale image in NIR band

Images captured at different wave bands (Fig.
6
) indicate that pin hole on chestnut
surface is clearly visualized in the wave
length

range of 740 to 960 nm

but it was not in
visible range. Therefore, it
was

concluded that
chestnut
image
s taken in the range of 750 to
950 nm
(named as NIR band image in this study)
are
very
useful in detecting pin holes.

Pictures are not shown here

but relative brightness of the pin hole was affected by the
lightening method

and relative
position of the pin hole to the lightening
, showing
bad effect
of
specula
r

reflection.
It is remarked that location of pin hole is very random

and
that

surface
color of chestnut becomes locally dark
er than the normal
when chestnut flesh is

rotted
to
certain degree
. Pixel values of this dark area are similar to those of pin hole and affects binary
image

processing for pin hole detection.
In order
to avoid this typ
e of problem
,
morphological feature of the dark
parts

such as ratio of major and minor
axes was

adopted in
binary image processing

as described in pin
-
hole detection algorithm.


According to the preliminary results on the image c
haracteristics of the pin
hole and
decayed chest
nut, it was concluded that a 2
CCD camera as shown in Fig. 1 would be useful
The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013


for sorting the decayed and worm eaten chestnuts.
In this research
,

further study is planned to
find if those darker parts could be detected from the color image processing and related to the
decay degree of the chestnut flesh.



Fig
ure

6
.

F
ilter images of
pin hole on worm eaten
chestnut



Features of the
images taken by the
2
CCD camera

Fig
ure

7

shows
the
chestnut images captured by the
2
CCD camera while chestnuts
were being

conveyed
in between the guide plates
on roller conveyor.
It was observed that the
color and gray scale images
generated

by the both
image
sensors are well synchronized in
terms of image size and location. Therefore,
the mask generated from NIR
band image

can

be
applied to the color image

and

point processing of two images such as
their
subtraction and
multiplication

can be taken if needed
.


Real time detection and separation rate of pin
-
hole chestnut

In order to

simplify the detection algorithms, binary processing and ratio of the major
and minor axes of the blobs in the resulted binary image were adopted to the NIR band
images for detection of pin holes. It was considered as pin hole if the ratio of blob is great
er
than 0.3. Effects of threshold value and conveyor speed on detection rate were examined and
results are
shown in Table 1 and 2
. Detection accuracy of the sound is from 83 to 94% and
that of pin
-
hole chestnut is 73 to 90% under the test conditions in thi
s study. As the threshold
value increases, adverse effect is given to the sound but favorable effect to the pin
-
hole

samples. And the test results indicate that about 15% of the sound is discharged to the
defective and about 15 to 20% of the defective to t
he normal, and the secondary manual
sorting should be accompanied. If we think about absolute amount of the sound and the
The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013


defective in chestnut production, separation rate of the defective should be maximized in
order to minimize the secondary separation w
ork with the discarded.



Figure

7
.

Real time color and NIR band images captured from the chestnuts
which are
being
revolved and
transferred on roller conveyer.



Table 1. Detection and separation rate of pin
-
hole chestnut depending on threshold value

Threshold value

Sound chestnut (%)

Wormhole chestnut (%)

90

TP: 92

TN: 8

FP: 81

FN: 19

100

TP: 89

TN: 11

FP: 84

FN: 16

110

TP: 83

TN: 17

FP: 90

FN: 10

Total detection rate

TP: 88

TN: 12

FP: 85

FN: 15

Note:
True
P
ositive (TP), True
N
egative (TN), False
P
ositive (FP), and False
N
egative (FN)


Table 2. Detection and separation rate of pin
-
hole chestnut depending on roller speed

Roller transfer speed

Sound chestnut (%)

Wormhole chestnut (%)

15 cm/sec

TP: 89

TN: 11

FP: 84

FN: 16

20
cm/sec

TP: 94

TN: 6

FP: 83

FN: 17

25 cm/sec

TP: 90

TN: 10

FP: 73

FN: 27

Total detection rate

TP: 91

TN: 9

FP: 80

FN: 20

Note:
True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN)

The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013



Conclusion

Both the color and
NIR band images
could be
obtained from chestnuts with a

2
CCD
cam
e
ra sy
s
tem

and the latter

were very effective for separation of pin
-
hole chestnuts

which
are
evolved and
transferred on roller conveyor, indicating detection accuracy of about 85%
.

Further
study is required to

maximize

the detection rate of the decayed chestnuts as well as
the pin
-
hole sample by analyizing the color images.


Acknowledgements

This study was carried out with the support of ´Forest Science & Technology Projects
(Project No.
S121010L040120)´ provided by Korea Forest Service.



References

1.

Donis
-
González, I. R., Guyer, D. E., Pease, A., and Fulbright, D. W. (2012). Relation of
computerized tomography Hounsfield unit measurements and internal components of
fresh chestnuts (
Castanea

spp.
).
Postharvest Biology and Technology

64(1): 74
-
82.

2.

Fang, J. J., Liu, S. L., and Zhang, H. (2004). Real
-
time Chestnut Sorting System by
Machine Vision.

Light Industry Machinery
, 3: 92
-
93.

3.

Food and Agriculture Organization. (2011). FAOSTAT onli
ne. Food and Agriculture
Organization of the United Nations. Available at Web site: http://faostat.fao.org/ (verified
July 15, 2013).

4.

Hui, Z., Xiaoyu, L., Zhu, Z., Chenglong, W., and Yun, G. (2011). Detection of chestnut
defect based on data fusion of near
-
infrared spectroscopy and machine vision.
Transactions of the Chinese Society of Agricultural Engineering
, 2: 060.

5.

Liu, J., Li, X. Y., Li, P. W., Wang, W., Zhang, J., Zhang, R., and Liu, P. (2010).
Nondestructive detection of moldy chestnut based on near

infrared spectroscopy.
African
journal of Agricultural research
, 5(23): 3213
-
3218.

6.

Liu, L., J., Li, X., Li, P., Wang, W., Zhang, J., Zhou, W., and Zhou, Z. (2011). Non
-
destructive measurement of sugar content in chestnuts using near
-
Infrared spectroscopy
.
In Computer and Computing Technologies in Agriculture IV
, 246
-
254.

7.

Wang, C., Li, X., Wang, W., Feng, Y., Zhou, Z., and Zhan, H. (2011). Recognition of
worm
-
eaten chestnuts based on machine vision.
Mathematical and Computer Modelling
,
54(3): 888
-
894.

8.

Wu,
D., He, Y., Feng, S., & Sun, D. W. (2008). Study on infrared spectroscopy technique
for fast measurement of protein content in milk powder based on LS
-
SVM.
Journal of
Food Engineering
, 84(1):124
-
131.