The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013
Real
t
ime
d
etection
of
pin
h
ole on
worm
-
eaten c
hestnut with
2
CCD
c
amera
Soo Hyun Park
1
Soo Hee Lee
2
Seong Min Kim
3
and S
ang Ha Noh
1,4
*
1
Department of Biosystems &
Biomaterials Science and Engineering, Seoul National
University, Seoul, 151
-
921, Republic of Korea.
2
Life& Technology CO. LTD., Hwaseong
-
si Gyeonggi
-
do, 445
-
964, Republic of Korea.
3
Department of Bioindustrial Machinery Engineering, Chonbuk National
Univ
ersity
,
Jeonju, 561
-
756,
Republic of Korea.
4
Research Institute for Agriculture and Life Sciences, Seoul National University, Seoul, 151
-
921, Republic of Korea.
* Corresponding Author
Tel: +82
-
2
-
880
-
4603, Email :
noh@snu.ac.kr
Abstract
Overall q
uality of chestnut is determined mainly by size, weight, shape
and
internal
disorders
such as decay, worm
-
eaten, etc. In Korea
chestnuts having
internal disorders are
picked out
manually
on
individual
basis
by
surface
color and pin hole
s
made by worm.
The
ultimate purpose
of this study is to develop
a chestnut sorter using multi
-
channel vision
system
which can detect pin hole
s
and the color of chestnut. Primary
,
this
study
is
focused
to
detect
ion of
pin holes on chest
n
ut
surface
.
Both c
olor image and
g
ray
-
scale image in near
infrared
band
were captured simultaneously
from the chestnut sample with a
n image
acquisition
system constructed with
a
2
CCD camera
and
an algorithm to detect pin holes
were developed with
b
inary
image process
ing
of the gray scale images
.
A sorting system
which
is
consisted
of feeder, roller conveyor, a multi
-
channel camera and automatic
discharging
device
was developed and used
to
evaluate the
effectiveness of the pin
-
hole
detection and sorting
algorithm
.
Finally
,
p
in
-
hole detection and sorting rate w
ere
evaluated
great than 85 %,
show
ing
possibility of
commercialization
of the developed system with
further study on analysis of surface color
related to internal
disorder.
Keywords:
Chestnut
,
I
nternal disorder,
Machine vision,
NIR
image, On
-
line
detection
.
The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013
Introduction
Chestnut (Castanea spp.) is popular for its abundant nutrients and delicious taste in
Asia and Europe
(
6
). More than 2,000,000 ton chestnuts were produced annually, and the
consumption and production of chestnut continue to rise all over the world according to the
Food and Agriculture
Organization Statistics (
3
).
Generally, chestnut quality is measured not
o
nly by
internal
disorders such as decay, worm
-
eaten, etc
.,
which are important for consumer
acceptance
, but also
by external factors such as color, shape, size, and surface status.
Most
importantly, the external appearance usually is not altered, at least initially
, but
it may cause
the moisture loss inside, which leads internal disorders usually accelerating the anatomical
and physiological changes within the tissue
(1
)
.
On that ac
count,
p
in holes
made by worms
on chestnut hull
are
of great concern to the chestnut
producers
and
consumers
because they
indicate inside of the chestnut is
disordered
and appearance of caterpillar
in peeling process
can make customers feel revolting.
As t
ime elapses,
pin
-
hole
chestnuts start to decay easily
and eventually infect
other sound chestnuts
.
If
chestnut fleshes
are decayed,
relatively
large
dark
color
is exposed on the hull
.
In chestnut packing house, s
izing of chestnut is
usually
done by
a hole
size of drum
but pin
-
hole and dark color
chestnuts
are picked up
manually
on
individual basis.
Screening work is labor intensive and its efficiency is not so high since pin
holes are hard to find because their
sizes are small and they are located
random
ly
,
and dark
color is not apparent.
In order to improve the detecting precision of the defective chestnuts with a
commercialized sorter, a technique based on near
-
infrared (NIR) spectroscopy or machine
vision have been proposed (4).
NIR spectroscopy is a non
-
destructive measuring method with
the advantages of minimized preparation of samples, fast, cheap, and easy to ope
rate and
environment
-
friendly (8
). It has been conducted to do both quantity and quality analysis of
agricultural products.
Liu et al.
used
NIR spectra in 833
–
2500 nm from chestnuts to detect
moldy chestnut, and it performed the detection ratio over 92.8%
(5
)
.
Machine vision
systems
have been used for evaluation of color grading
,
detection of surface defects, sizing and shape
detection of fr
uits and vegetables since
the late 1970s
(4
).
Machine vision which uses
cameras
instead of human eyes in carrying out measurement and judgment,
and it
is now
widely
applied to commercialized fruits sorters.
Fang
et al. designed
a
machine vision for
real
-
ti
me chestnut rating system (
2
)
and Wang et al. proposed
a recognition method
of
worm
-
eaten chestnuts based on machine vision techni
que
(7
)
.
The ultimate purpose of
this
study is in developing an on
-
line chestnut sorting system
consisted of feeder, conveyor, machine vision, automatic discharger, etc
.
, which can be used
at chestnut packing house. However, the aim of present study is
focused
to construct a multi
-
channel mac
hine vision system, to develop an algorithm to detect pin holes on chestnut hull
and to evaluate
performance of the algorithm
on
-
line status.
Materials and methods
Samples
Chestnut samples
(Daebo species)
were obtained from on
-
line market, which were
har
vested
in October 2010
at Kong
-
ju, the southern area of Korea.
The samples for the
experiment include
20 normal fresh chestnuts and.
6
chestnuts having pin
-
holes on hull.
Imag
e acquisition
Primarily,
a
CCD camera (Model: KF
-
F2A, Hitachi, Japan)
equipped
with a filter
wheel to the camera lens
was
used to
obtain
images of normal and defective chestnuts at 8
different wave bands of 720, 740, 780, 840, 880, 900, 940, and 960nm. Those images were
analyzed in view of
detecting
possibility
of
pin
hole
s
at
differ
ent wavelength bands
.
Secondarily,
a
2
CCD
camera (JAI AD
-
080GE) which is designed as shown in Fig
.
1
was
The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013
used to capture
both
the
color and NIR images
at the same time from a chestnut sample
.
This
camera
uses
two
prisms
in the optical path
in order to divide one image to two images and
band
-
pass filters
are installed
on each spectral axis to capture
color image
through a Bayer
Filter Array on one sensor and
NIR image in the range of 750 to 900 nm
on the other
.
The
camera has a resolution of 1
024 x 768 pixels and is able to capture 30 frames per second in
full frame mode. SDK
(Software Development Kit)
was
provided that we programmed to
allow simultaneous RGB and NIR
image acquisition
.
Figure
1
.
Schematic of
a 2
CCD
camera
Lights source
Krypton lamp was used for lighting, and was investigated the
spectral
characteristic
by wavelength
using a spectro
meter
(USB4000, Ocean Optics, USA). It is determined that
lighting is effective in range between 630nm and 960nm shown as Fig.
2
.
Figure
2
.
Spectral c
haracter
istic of a krypton light source
On
-
line sorting system
A laboratory scale of on
-
line sorting system
which is consisted with feeder, roller
conveyor, image acquisition and discharging units
was constructed
to evaluate the
performance of
pin
-
hole detection and sorting algorithm
developed in this study. A linear
feeder
(Fig 3
-
A)
was designed and fabricated for
d
ispersing
and aligning
the
chestnuts
The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013
supplied in bulk status.
The chestnuts fed by linear feeder moves to roller conveyor shown in
Fig.
3
-
B
, which was designed so that eac
h chestnut
could
evolve slowly on the roller and
moves to image acquisition unit.
Fig
ure
3
.
Linear feeder (A) for aligning the chestnuts and r
oller conveyor (B) for rotating and
transferring chestnuts
The camera unit was installed near
the
discharging end of
the
roller conveyor
and
guide
strips
consisted with 8 channels
w
as
put on the surface of roller
under the camera
view
area
shown as Fig. 4
. The rol
e
of guide
strips is to keep chestnuts aligned in each
line
until
they
drop to
discharge actuator.
While
chestnut sample
s pass
the
guide strips
, they should
evolve at least one turn so that total surface is exposed to the camera
. The camera shut
ter
operates three times
while
the sample passes through the
camera
view
window
.
As the re
sult,
three
image
frames are taken
from each chestnut sample
at least
and
those were
analyzed for
detection of pin holes
.
Fig
ure
4
.
Outlook of two multi
-
channel cameras, guide lines, and
discharging area
Method for d
ischarging of defective
chestnuts
An automatic discharging unit shown in Fig. 5, which was made with 8 flat
rectangular bars and 8 solenoid actuators, was installed at the end part of the roller conveyor
right after the guide strips. Each bar accounts for each guide channel. If a chestnut
sample is
decided to be normal or defective by the image analysis, a signal is transferred to the
corresponding discharge solenoid. Whenever pin
-
hole samples are sliding on the discharging
The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013
bars, the corresponding solenoids are actuated and the bar moves up
so that flow directions
are changed (Fig. 5
-
A
). Otherwise, the actuators are not on and the bars stay down (Fig.
5
-
B
).
In other words,
the defective chestnuts are
discharged
upward
and
the normal fall through the
ordinary path. In actuating the dischargin
g bar
,
the delay time of the signal transfer from the
camera and to actuator reaction is very important in order to synchronize the sliding speed of
chestnut sample on to the bar and the timing of the bar actuation.
Figure
5
.
Bar
-
type discharging devi
ce actuated by solenoid (when the
defective
chestnut
passes,
it moves up (A) to change direction of the defective, otherwise t
he bar stays down (
B
)
Development of
pin hole
detection
algorithm
Detection software for screening worm
-
eaten and decay
ed
chestnut was developed
using
an image
acquisition
system
equipped with a 2CCD camera.
S
oftware provided by the
frame grabber SDK was used to be opera
ted on Windows XP configuration
and b
inary image
process was
applied to the NIR band images for
screen
ing
pin holes.
Three threshold vales of
90, 100, and 110 were examined
to get binary image and the ratios of the major and minor
axes of the blobs in the binary image
were computed to
reconize
if it
is
a pin hole or not.
Effect of roller conveyor speed
(0.15,
0.20 and 0.25 m/sec)
on pin hole detection was also
examined
.
Results and discussion
Features of Gray
-
scale image in NIR band
Images captured at different wave bands (Fig.
6
) indicate that pin hole on chestnut
surface is clearly visualized in the wave
length
range of 740 to 960 nm
but it was not in
visible range. Therefore, it
was
concluded that
chestnut
image
s taken in the range of 750 to
950 nm
(named as NIR band image in this study)
are
very
useful in detecting pin holes.
Pictures are not shown here
but relative brightness of the pin hole was affected by the
lightening method
and relative
position of the pin hole to the lightening
, showing
bad effect
of
specula
r
reflection.
It is remarked that location of pin hole is very random
and
that
surface
color of chestnut becomes locally dark
er than the normal
when chestnut flesh is
rotted
to
certain degree
. Pixel values of this dark area are similar to those of pin hole and affects binary
image
processing for pin hole detection.
In order
to avoid this typ
e of problem
,
morphological feature of the dark
parts
such as ratio of major and minor
axes was
adopted in
binary image processing
as described in pin
-
hole detection algorithm.
According to the preliminary results on the image c
haracteristics of the pin
hole and
decayed chest
nut, it was concluded that a 2
CCD camera as shown in Fig. 1 would be useful
The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013
for sorting the decayed and worm eaten chestnuts.
In this research
,
further study is planned to
find if those darker parts could be detected from the color image processing and related to the
decay degree of the chestnut flesh.
Fig
ure
6
.
F
ilter images of
pin hole on worm eaten
chestnut
Features of the
images taken by the
2
CCD camera
Fig
ure
7
shows
the
chestnut images captured by the
2
CCD camera while chestnuts
were being
conveyed
in between the guide plates
on roller conveyor.
It was observed that the
color and gray scale images
generated
by the both
image
sensors are well synchronized in
terms of image size and location. Therefore,
the mask generated from NIR
band image
can
be
applied to the color image
and
point processing of two images such as
their
subtraction and
multiplication
can be taken if needed
.
Real time detection and separation rate of pin
-
hole chestnut
In order to
simplify the detection algorithms, binary processing and ratio of the major
and minor axes of the blobs in the resulted binary image were adopted to the NIR band
images for detection of pin holes. It was considered as pin hole if the ratio of blob is great
er
than 0.3. Effects of threshold value and conveyor speed on detection rate were examined and
results are
shown in Table 1 and 2
. Detection accuracy of the sound is from 83 to 94% and
that of pin
-
hole chestnut is 73 to 90% under the test conditions in thi
s study. As the threshold
value increases, adverse effect is given to the sound but favorable effect to the pin
-
hole
samples. And the test results indicate that about 15% of the sound is discharged to the
defective and about 15 to 20% of the defective to t
he normal, and the secondary manual
sorting should be accompanied. If we think about absolute amount of the sound and the
The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013
defective in chestnut production, separation rate of the defective should be maximized in
order to minimize the secondary separation w
ork with the discarded.
Figure
7
.
Real time color and NIR band images captured from the chestnuts
which are
being
revolved and
transferred on roller conveyer.
Table 1. Detection and separation rate of pin
-
hole chestnut depending on threshold value
Threshold value
Sound chestnut (%)
Wormhole chestnut (%)
90
TP: 92
TN: 8
FP: 81
FN: 19
100
TP: 89
TN: 11
FP: 84
FN: 16
110
TP: 83
TN: 17
FP: 90
FN: 10
Total detection rate
TP: 88
TN: 12
FP: 85
FN: 15
Note:
True
P
ositive (TP), True
N
egative (TN), False
P
ositive (FP), and False
N
egative (FN)
Table 2. Detection and separation rate of pin
-
hole chestnut depending on roller speed
Roller transfer speed
Sound chestnut (%)
Wormhole chestnut (%)
15 cm/sec
TP: 89
TN: 11
FP: 84
FN: 16
20
cm/sec
TP: 94
TN: 6
FP: 83
FN: 17
25 cm/sec
TP: 90
TN: 10
FP: 73
FN: 27
Total detection rate
TP: 91
TN: 9
FP: 80
FN: 20
Note:
True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN)
The International Symposium on Agricul
tural and Biosystem Engineering (
ISABE
)
2013
Conclusion
Both the color and
NIR band images
could be
obtained from chestnuts with a
2
CCD
cam
e
ra sy
s
tem
and the latter
were very effective for separation of pin
-
hole chestnuts
which
are
evolved and
transferred on roller conveyor, indicating detection accuracy of about 85%
.
Further
study is required to
maximize
the detection rate of the decayed chestnuts as well as
the pin
-
hole sample by analyizing the color images.
Acknowledgements
This study was carried out with the support of ´Forest Science & Technology Projects
(Project No.
S121010L040120)´ provided by Korea Forest Service.
References
1.
Donis
-
González, I. R., Guyer, D. E., Pease, A., and Fulbright, D. W. (2012). Relation of
computerized tomography Hounsfield unit measurements and internal components of
fresh chestnuts (
Castanea
spp.
).
Postharvest Biology and Technology
64(1): 74
-
82.
2.
Fang, J. J., Liu, S. L., and Zhang, H. (2004). Real
-
time Chestnut Sorting System by
Machine Vision.
Light Industry Machinery
, 3: 92
-
93.
3.
Food and Agriculture Organization. (2011). FAOSTAT onli
ne. Food and Agriculture
Organization of the United Nations. Available at Web site: http://faostat.fao.org/ (verified
July 15, 2013).
4.
Hui, Z., Xiaoyu, L., Zhu, Z., Chenglong, W., and Yun, G. (2011). Detection of chestnut
defect based on data fusion of near
-
infrared spectroscopy and machine vision.
Transactions of the Chinese Society of Agricultural Engineering
, 2: 060.
5.
Liu, J., Li, X. Y., Li, P. W., Wang, W., Zhang, J., Zhang, R., and Liu, P. (2010).
Nondestructive detection of moldy chestnut based on near
infrared spectroscopy.
African
journal of Agricultural research
, 5(23): 3213
-
3218.
6.
Liu, L., J., Li, X., Li, P., Wang, W., Zhang, J., Zhou, W., and Zhou, Z. (2011). Non
-
destructive measurement of sugar content in chestnuts using near
-
Infrared spectroscopy
.
In Computer and Computing Technologies in Agriculture IV
, 246
-
254.
7.
Wang, C., Li, X., Wang, W., Feng, Y., Zhou, Z., and Zhan, H. (2011). Recognition of
worm
-
eaten chestnuts based on machine vision.
Mathematical and Computer Modelling
,
54(3): 888
-
894.
8.
Wu,
D., He, Y., Feng, S., & Sun, D. W. (2008). Study on infrared spectroscopy technique
for fast measurement of protein content in milk powder based on LS
-
SVM.
Journal of
Food Engineering
, 84(1):124
-
131.
Enter the password to open this PDF file:
File name:
-
File size:
-
Title:
-
Author:
-
Subject:
-
Keywords:
-
Creation Date:
-
Modification Date:
-
Creator:
-
PDF Producer:
-
PDF Version:
-
Page Count:
-
Preparing document for printing…
0%
Σχόλια 0
Συνδεθείτε για να κοινοποιήσετε σχόλιο