Machine vision on fishing vessels

builderanthologyΤεχνίτη Νοημοσύνη και Ρομποτική

19 Οκτ 2013 (πριν από 3 χρόνια και 5 μήνες)

45 εμφανίσεις

Machine vision on fishing vessels
for the identification of fish
species (Digital Observer)

Eric Davis

Committee
-


Dr. Roth (Chair)


Dr. Nance


Dr. Bueler

Overview


What is the Digital Observer?


What is my part in the Digital Observer?


What happened over the summer?


Let it end! When will my part be done?


Questions/flames/etc.

What is the Digital Observer?

What is an Observer?


Records data about what the boat is
catching(species, length, weight).


Records data if the boat or its personnel
are doing anything wrong.


Writes lots and lots of reports.

Difficulty with observers


They take up space on the boat.


They must be paid for by the boat.


Try having someone watching you work
and making notes 24 hours a day.


Every fishing vessel has got a story
about a bad observer.


High turnover in the industry.

Digital Observer to the rescue!


Won’t take up much room on the boat.
Ideally, it will just be a box and a couple
of cameras that are out of the crew’s
way.


Easier to reproduce than humans.


Completely objective data.


Creates the reports automatically.


Only records necessary data.


The Equipment

A Camera

A Computer

or

Industrial

Underwater

Vessel Monitoring Systems
abroad


Canada has a VMS system already
created by Archipelago Marine
Research, Ltd.


Record all the fishing activities, as well
as location, to videotape.


The tapes are later audited by humans
to see if the crews report was accurate.

Fish identification abroad


A cannery system has been created
that works at identifying fish.


Takes one fish at a time into a box,
takes pictures of it, and identifies the
fish.

How the Digital Observer is
different


No humans involved except for
identifying problem fish.


Provides a census of all fish, not just a
statistical sample.


Has a distinct lack of environment
control on the boat.


Can provide real
-
time reporting.

The parts of the Digital Observer.

Image Capture

Camera and computer
work together to get an
image of the fish

Segmentation

Computer determines
which segments of
picture are part of a
fish and which are not.

Morphometrics

The fish segments are
measured for
characteristics such as
length, width, and color

Identification

The measurements are
sent to a neural
network to identify the
species of the fish.

Image Capture


Written in C, adjustable in number of
frames / second.

Segmentation


The area that I have been researching.


Current product by SciFish is written in
Matlab and works pretty well for chute
images in controlled lighting.

Segmentation example

Morphometrics


Gathers data on the segmented fish


Length


Various widths at different points along the fish


Overall Hue


Eccentricity of ellipse that fits the fish.


Perimeter

Neural Network


Takes the morphometric data for each
image and tries to make a best guess
on the best picture.


Works pretty well on a limited sample
test set(about 80% correct).

An example output

Current Group Status


A prototype form of each component
has already been constructed.


Works at a little more than 80%
accuracy for limited in
-
the
-
chute test
data in the lab.

New worries


Variable lighting
-

Needs to work reliably
in variable conditions
-

cloudy, sunny,
even at night.


Over the side data(Loss of all
environment control).


Both worries have to do with
segmentation.

Over the side data


Improves counts of fish actually caught in the
lines instead of those hauled and kept on the
boat.


Lose any control of picture elements
-

no
choice of background along with little light
control. A whole extra level of difficulty.


Need a breakthrough in segmentation to
make this possible(and possibly improve
current software).

What was my part in the Digital
Observer?

My goals


Evaluation of LEGION.


Implement and evaluate the
segmentation algorithm for the Digital
Observer.


Evaluation documentation
(segmentation and project).

LEGION


An oscillator based neural network that has
shown some amazing results in difficult
segmentation problems.


Consists of a smoothing algorithm followed
by the oscillator network algorithm.


Had to evaluate its feasibility for this particular
project.

LEGION uses


Implemented against the chute data to
gain familiarity with the project and see
basic feasibility of LEGION.


If successful, would try LEGION to
approach the over
-
the
-
side problem.

LEGION results

Original image

Segmented image

Legion
-

current results.


Has really good segmentation on both
example files and on fish in the chute.


The algorithm had two problems with it
that indicate it won’t work for this
project:


Too slow(by a couple of orders of
magnitude)


Not robust enough in parameters.

Fish Segmentation
-

4 minutes

Original

No smoothing

Too much smoothing

Smoothing just right

Fish segmentation examples

What I did on my summer
vacation...

The Cruise


Went on the Kariel for a month to do a test of
the current system created by SciFish.


Also had the objective of gathering data for
where the project should go in the future
(especially over
-
the
-
side data).


While collecting data, I was to evaluate the
process and the equipment being used for
the project (cameras, lighting control, camera
positions, etc.).

The Cruise results


Found no satisfactory solution for
lighting control: daytime, night
-
time, or
underwater.


The system didn’t work quite as well as
in the lab tests. Under the best
conditions, we were probably getting
about 50% accuracy.


Gathered hours upon hours of data from
every conceivable angle for future work.

More cruise results


After the cruises I continued in my evaluation
of the equipment used to see if a change of
equipment might afford us better results.


From watching the process of gathering fish
and talking to the crew, it was realized that no
chute camera system would ever replace
observers.


Bruises. Lots of bruises.


Say that again? A chute system
won’t work?


In normal fishing
operations, the only
thing that goes through
the chute is the target
species.


To gather information
on what is really
caught, we have to
process some sort of
over
-
the
-
side data.

Three kinds of over the side data

Stabilizer Camera

Underwater Camera

Rail Camera

Over the side data


All three vantage points have their
advantages as well as disadvantages. The
segmentation is either extremely complex, or
a large amount of control needs to be exerted
on the physical conditions.


This is the angle that is going to be looked at
in Phase II of the Digital Observer project
(beyond the scope of my project).

What I still need to accomplish


More evaluation and documentation of
the segmentation algorithm used.


More evaluation and documentation of
the project as a whole and on future
promising avenues of research for
problems of this type.


Hopefully will be done by mid
-
November.

Questions?