ARTIFICIAL NEURAL NETWORKS vs LINEAR REGRESSION IN A FLUID MECHANICS AND CHEMICAL MODELLING PROBLEM: ELIMINATION OF HYDROGEN SULPHIDE IN A LAB-SCALE BIOFILTER

sciencediscussionΤεχνίτη Νοημοσύνη και Ρομποτική

20 Οκτ 2013 (πριν από 3 χρόνια και 10 μήνες)

91 εμφανίσεις

ARTIFICIAL NEURAL NETWORKS vs LINEAR REGRESSION IN A FLUID
MECHANICS AND CHEMICAL MODELLING PROBLEM: ELIMINATION
OF HYDROGEN SULPHIDE IN A LAB
-
SCALE BIOFILTER



G.Ibarra
-
Berastegi
,

A. Elias
,

R. Arias
,

A. Barona

Dpt. NI & Fluid Mechanics

Email:
gabriel.ibarra@ehu.es

D
pt.

of Chemical
&

Environmental Engineering

Engineering Faculty. University of the Basque Country. Alda Urkijo s/n. 48013 Bilbao (Spain)



Abstract


A biofilter is a biological reactor in which

a certain
pollutant is eliminated by the action of microorganisms.
In this work, the removal efficiency of a lab
-
scale biofilter
for eliminating hydrogen sulphide (H
2
S) has been
modelled. To that end, multilayer perceptron (MLP)
neural networks and multip
le linear regression (MLR)
have been used and then, results obtained with both
techniques have been compared. The biofilter has been
operating during 194 days and for modelling purposes, it
has been considered as a system in which changes in the
flow and c
oncentration of H
2
S entering the biofilter are
followed by changes in the removal efficiency of the
reactor. In all cases, to obtain true representative values
corresponding to the different equilibrium situations,
before removal efficiencies (outputs) wer
e measured, 24
hours were allowed after the H
2
S load was changed by
altering the inlet concentration and flow. The results
showed that a multilayer perceptron 2
-
2
-
1 (MLP) model
was able to explain 92% (
R
2
=0.92) of the overall
variability detected in the re
moval efficiency of the
biofilter corresponding to a wide range of operating
conditions. The MLR model yielded a value of
R
2
=0.72.
The MLP outperforms the MLR though not dramatically.
The explanation might be that the combination of a great
number of hig
hly non
-
linear mechanisms tends to
linearize the overall effect, at least to a certain extent. As
a conclusion, the use of neural networks and more
specifically, MLP models can describe the behaviour of a
biofilter more accurately than simple linear regres
sion
models.


1. Introduction

A biological reactor (also known as biofilter) is
intended to eliminate pollutants by the action of
microorganisms. The processes involved in the
degradation of pollutants are very complex and are known
to be highly non
-
linea
r. The main mechanisms responsible
for the success of the whole process are chemical and
fluid mechanics effects like flow transport, biological
degradation, adsorption and absorption .

Basically, there are two ways in which a model can be
built: it can b
e derived in a deductive manner using laws
of nature, called mechanistic modelling, or it can be
inferred from a set of data collected during a practical
experiment with the system, called black
-
box or statistical
modelling [1].

The aim of this work was to

model the performance of a
biofilter for eliminating hydrogen sulphide (H
2
S) by using
a statistical or black
-
box modelling approach. To that end,
the candidate mathematical tools considered have been
two:

1.

a particular type of neural network (MLP) and

2.

mul
tiple linear regression (MLR).

The performance of both techniques was compared at a
95% confidence level according to a group of statistical
indicators.


2. Methodology


2.1.

Data


The database consisted of 194 experimental daily
cases obtained during

three y
ears from a biofilter designed
to degradate H
2
S. These 194 valid cases included start
-
up,
sudden variations and steady state operation. In each case,
the measured variables were the concentration of H
2
S fed
into the reactor (C, in ppm
v
) and the flow (Q, in

l min
-
1
) as
inputs, and the removal efficiency obtained in the biofilter
24 hours after (E, %) as the only output. 24 hours was
considered as the typical period required by the biofilter
to reach an equilibrium

stage after the H
2
S load was
changed by alt
ering the inlet concentration and flow.

The original variable Q (inlet flow, l min
-
1
) was divided
by the volume of the reactor (7.85 l in this study) to obtain
a new variable called unit flow (Q
u
, in min
-
1
). Being C, Q
u

and E variables that do not depend
on the size of the
reactor, the conclusions could be extrapolated to future
bioreactors built at different scales but with the same
design that the one used in this study. The operation
limits ranged between 1.5 and 4.7 min
-
1

for Q
u
, 25
-
346
ppm
v

for C a
nd 37
-
100% for E.

To describe the process of H
2
S elimination inside the
biofilter, a black
-
box or statistical modelling approach
was used. To that end, the biofilter was considered as a
system with two inputs (Q
u
, C) at time T and one output
(E) measured 2
4 hours later. The chosen candidate tools
were two: ANN,
-
and more precisely the multilayer
perceptron (MLP)
-

and multiple linear regression (MLR).
Both types of models were built and then tested on the
same group of cases which were not used to build the
models and reserved from the beginning for performance
comparison purposes.

2.2. Multilayer perceptron model

Different MLP architectures were possible but all of
them would have only two inputs (C and Q
u
) in the input
layer and one single output (E) in the

output layer.
Therefore, several combinations in the number of hidden
layers (1 or 2) and in the number of nodes in these layers
were possible
[7], [8], [9], [10], [11]
.

Also, the initial random values corresponding to the
parameters (weights and bias) th
at need to be estimated
can have an influence since some initial values may be
closer than others to the minimum absolute error
[7], [8],
[9], [10], [11]
. For this work, to obtain the best neural
network, 10000 MLP’s with one or two layers, with a
number o
f nodes ranging from 1 to 6 in the hidden
layer(s) and with different initial random values were
tested
[10],[11]
. The chosen algorithms to fit the networks
were a combination of Backpropagation at the beginning
and Conjugated Gradient Descent in the last
stages of
convergence. As training progresses, the error is
calculated in the validation subset and when it reaches a
minimum after a certain number of epochs (or cycles), the
process stops
[10], [11].

The network that can be expected to perform best when
applied to the test dataset, will be the one which has the
lowest error at the point in which, during the convergence
process, the error reaches a minimum for the validation
subset. Therefore, to identify the best network it was
necessary to compare at thi
s point the errors obtained with
the 10000 networks tested. To automatize all the
calculations, the Statistica 7.0 software was used and final
results were obtained after 100 minutes of computation
time on a PC. Sigmoid activation functions were used as
tr
ansfer functions in all cases.

The best network turned out to be a MLP with a 2
-
2
-
1
architecture, that is, with 1 hidden layer of 2 nodes. In this
network, the error reached a minimum value of 0.064158
in the validation subset. This value was reached after

100
epochs using backpropagation followed by 42 epochs
using conjugated gradient descent as convergence
algorithms. The response surface relating Q
u
, C and E
corresponding to the selected network MLP 2
-
2
-
1 can be
seen in figure 1.



Figure 1. Response s
urface
relating Q
u
, C
and E corresponding to the MLP 2
-
2
-
1
network model.


The influence of Q
u

and C on E can be estimated by
means of a sensitivity analysis in which the relative
impacts on the output due to changes in the inputs are
measured [11]. The r
esults of the sensitivity analysis show
that the unit impact of Q
u

is 3.67 while the impact of C is
3.04. This means that in a proportion according to the
ratio of their relative impacts, Q
u

is more relevant than C
to explain E.


2.3. Multiple linear regr
ession model

For a classical MLR analysis only training and test
data subsets are necessary and therefore, 103+18 =121
cases were used to fit the linear equation, while the same
73 than for MLP were also now reserved for test.
Following the same black
-
box
approach, the candidate
input variables (Q
u

and C) and output (E) were the same.

To conduct the MLR analysis, stepwise regression and
tolerance filtering was used to include in the equation only
the meaningful variables. Equation 1 represents the linear
re
gression obtained with Q
u

and C as independent
variables and E as the dependent. The same 121 cases
than those used to fit the MLP were used to build equation
1 which yielded a R
2

of 0.747 with a standard error of
6.95. In equation 1, the numbers in parent
heses represent
the standard error of the coefficients which are small
(below 10% of their mean values) thus making equation 1
robust enough.


E=128.217(2.36
)
-
15.415(0.9
3
)Q
u

0.0454(
6E
-
3
)C
(1)




The relativ
e importance of the input variables in equation
1 can be measured by the increase in the overall R
2

due to
the incorporation of each input to equation 1. In this case,
Q
u

was responsible for an increase of 0.54 and C of 0.207
to reach the final value of
R
2

obtained in the fitting
process
(0.747). This indicates that according to the linear
model, the influence on the elimination percentage E
attributable to changes in C, is less than half the influence
of Q
u
.

Both models, the MLP 2
-
2
-
1 and MLR, identify Q
u

as the
most relevant input variable. This was in agreement with
the practical experimental knowledge gained during the
research period. However, in the linear model the relative
influence of Q
u

is higher than in the MLP 2
-
2
-
1 model.


3. Results

When the
predictions of both models (MLP and NLR)
are faced with the 73 cases of the test data
R
2
,
d

and
RMSE

have good values and in the two models, 100% of
the predictions (
FA2=
1) fall in the range [0.5
-
2]. However

at a 95% confidence level,
R
2

(0.92) and
d
(0.81
) are
higher in the case of MLP. The error measured by the
RMSE

(3.87) is also lower in the case of MLP. The value
of
R
2

can also be understood as the proportion of the
overall variability explained by the model. The high
fraction of the explained variabi
lity (92%) along with a
high
d

and a low value of
RMSE
, indicates that in general
the MLP 2
-
2
-
1 has a good performance and at a 95%
confidence level, it can simulate how this biofilter
eliminates SH
2
, better than the MLR model. As expected,
the value of
R
2
=0.72 obtained with the linear model
applied to the 73 test data cases is slightly lower than that
obtained with the 121 cases used to build equation 1
(
R
2
=0.747).


4. Conclusions

Models represent an important tool for developing
biofilter management and c
ontrol strategies. In this work,
two models based on neural networks and linear
regression

have been built to predict the amount of H
2
S
that can be biologically degraded within a broad range of
gas flow and concentration of the contaminant fed into the
rea
ctor. These models has been obtained using input
variables that are common to any biofilter and the results
may be used for management and control of commercial
units. The MLP model outperformed the MLR, though not
dramatically. The explanation may be that

the
combination of a great number of highly non
-
linear
mechanisms may result in the partial linearization of the
overall effe
ct. This can also be detected in

figure 1
where
the response surface corresponding to the MLP (figure 1)
is not really very far fr
om a plane (MLR model)

[12
],
[
13
], [
14
], [
15
], [1
6
]
,
[
1
7], [
1
8]
.

A

statistical approach based on neural networks, and more
precisely multilayer perceptrons, can constitute an
appropiate tool to be used for control and management
purposes involving

future b
iofilters treating H
2
S.
According to the fluid mechanics laws of similarity, f
uture
reactors of the same design, but built at different scales,
are expected to be governed by similar types of neural
networks, although the overall removal efficiency is like
ly
to be higher the higher the size of the bioreactor is.
Although the MLP 2
-
2
-
1 model has been built for a
particular type of biofilter intended to remove a specific
pollutant like
H
2
S, the same methodology can be used
with any other pollutant and bioreac
tor.

However, before obtaining the statistical model governing
the behaviour of any bioreactor, an intensive and
systematic program of data acquisition and processing is
required, covering a wide range of operating conditions.
These data will be used to
build a model describing the
future behaviour of the biofilter. The model obtained and
its associated response surface will constitute the core of
the biological reactor control and management system.


I
n future similar modelling efforts, the best operatio
nal
solution will represent a trade
-
off between the simplicity
of linear models and the higher capacity of neural
networks to simulate non
-
linear effects.

A fully operational electronic version
(MLP221MLRBiofilter.xls) of the two models described
above

ca
n be downloaded
from
ftp://ftp.ehu.es/cidirb/profs/inpibbeg/AICCSA07

at no
cost.


Acknowledgements

The authors wish to thank the Basque Government
(BERRILUR project No. IE03
-
110), the Spanish

Government (MCYT PPQ2002
-
01088 with European
FEDER funding) and the University of the Basque
Country (UPV 00149.345
-
E
-
15398/2003 project) for the
financial support to develop this research.

References


[1]
Strik, David P.B.T.B., Domnanovich, A.M., Zani,
L.,
Braun, R., Holubar, P., 2005. Prediction of trace
compounds in biogas from anaerobic digestion using the
MATLAB Neural Network Toolbox.
Environmental
Modelling & Software

20 (6)

803
-
810.

[2] Bhat, T.R., Divya Venkataramani, Ravi, V., Murty,
C.V.S. 2006
. An improved differential evolution method
for efficient parameter estimation in biofilter modeling.
Biochemical Engineering Journal

28 (2) 16
-
176.

[3]
Saho Ikemoto
, Jennings, Aaron A., Skubal, K., L.
Skubal. 2006.
Modeling hydrophobic VOC biofilter
treat
ment in the presence of nutrient stimulation and
hydrophilic VOC inhibition.

Environmental Modelling &
Software
.
21(10) 1387
-
1401.

[4]
Ok
-
Hyun Park, In
-
Gyung Jung, 2006. A model study
based on experiments on toluene removal under high load
condition in bio
filters.
Biochemical Engineering Journal.

28 (3) 269
-
274.

[5] Mitsuyo Hirai, Masatoshi Ohtake, Makoto Shoda,
1990. Removal kinetics of hydrogen sulfide, methanethiol
and dimethyl sulfide by peat biofilters.
Journal of
Fermentation and Bioengineering

70 (5)

334
-
339.

[6]
Devinny, J.S., Ramesh, J. 2005. A phenomenological
review of biofilter models.
Chemical Engineering Journal

113 (2
-
3) 187

196.

[7] Bishop, C., 1995.
Neural Networks for pattern
recognition.

Oxford University Press.

[8] Masters T., 1993.

Pract
ical Neural Network Recipes
in C++.
Academic Press.

[9]
Haykin, S., 1994.
Neural Networks:
A comprehensive
foundation.

New York. McMillan Publishing.


[10]Neuralware.
http://www.neuralware.com/sup
port_faq.jsp
. 2006.

[11] Statistica 7.0. User`s manual.
http://www.statsoft.com
. 2006.

[12] Ibarra
-
Berastegi, G. Short
-
term prediction of air
pollution levels using neural networks.
Air Pollution XIV.
pp. 23
-
32. IS
BN 1
-
84564
-
165
-
5.

WIT Press.
Southampton. UK. 2006.

[13] Maier, H.R., Dandy, G.C., 2000. Neural networks for
the prediction and forecasting of water resources
variables: a review of modelling issues and applications.
Environmental Modelling & Software

15

(
1) 101

124.


[14]
Maier,
H.R., Morgan, N., Chow, C.W.K., 2004. Use
of artificial neural networks for predicting optimal alum
doses and treated water quality parameters.
Environmental Modelling & Software

19

(5) 485

494.

[15] Elías A., Barona A., Arreguy A
., Ríos J., Aranguiz I.,
Peñas J., 2002.
Evaluation of a packing material for the
biodegradation of H
2
S and product analysis.
Process
Biochemistry
37 (8) 812
-
820.

[16] Barona A., Elías A., Arias R., Cano I., Gonzalez R.,
2004.
Biofilter response to gradual

and sudden variations
in operating conditions.
Biochemical Engineering Journal

22 (1) 25
-
31.

[17] Elias A., Ibarra
-
Berastegi G., Ormazabal J., Murgia
I., Zugazti P., 1996.
ADM: A model for water treatment
in an anaerobic biological reactor, in Developmen
t and
Application of Computer Techniques to Environmental
Studies VI. III , CMP.

WIT Press, Southampton, pp. 153
-
162.

[18] Altaf H. Wani, Branion, R. M. R. , Lau, A.K., 1998.
Effects of periods of starvation and fluctuating hydrogen
sulfide concentration

on biofilter dynamics and
performance.
Journal of Hazardous Materials

60 (3) 287
-
303.