Direct Numerical Simulations of Turbulent Nonpremixed

busyicicleMechanics

Feb 22, 2014 (3 years and 3 months ago)

98 views

Direct Numerical Simulations of Turbulent Nonpremixed
Combustion: Fundamental Insights Towards Predictive
Models

Evatt R. Hawkes, Ramanan Sankaran, James C. Sutherland,

Jacqueline H. Chen


Combustion Research Facility

Sandia National Laboratories

Livermore CA


Supported by

Division of Chemical Sciences, Geosciences, and Biosciences,

Office of Basic Energy Sciences
, DOE

SciDAC


Computing: LBNL NERSC, SNL CRF BES Opteron cluster

Computing Support: David Skinner (NERSC)

Visualization: Kwan
-
Liu Ma, Hongfeng Yu, Hiroshi Akiba UC Davis

Outline

1.
Direct Numerical Simulation
(DNS) of turbulent combustion


challenges and opportunities


2.
Sandia S3D terascale DNS
capability


3.
INCITE project


3D simulations
of a turbulent CO/H
2

jet flame

Scalar dissipation fields in DNS of a turbulent jet flame

(volume rendering by

Kwan
-
Liu Ma and Hongfeng Yu)

Turbulent combustion is a grand challenge!

Diesel Engine Autoignition, Soot Incandescence

Chuck Mueller, Sandia National Laboratories


Stiffness : wide range of
length and time scales


turbulence


flame reaction zone


Chemical complexity


large number of species
and reactions (100’s of
species, thousands of
reactions)


Multi
-
Physics complexity


multiphase (liquid spray,
gas phase, soot, surface)


thermal radiation


acoustics ...


All these are tightly
coupled

Several decades of relevant scales


Typical range of spatial scales


Scale of combustor:
10


100
cm


Energy containing eddies:
1


10
cm


Small
-
scale mixing of eddies:
0.1


10
mm


Diffusive
-
scales, flame thickness:
10


100

m



Molecular interactions, chemical reactions:
1


10
nm


Spatial and temporal dynamics
inherently
coupled


All scales are relevant

and must be resolved or
modeled


O(4)

Range

O(4)

Continuum


Terascale computing:

~3 decades in scales

(cold flow)

What is DNS?


Complete resolution of all relevant continuum scales.



Does not require any explicit sub
-
grid scale model

(or implicit
sub
-
grid scale model provided by numerical dissipation).



CPU limitations imply only a finite range of scales can be tackled


implies restrictions on Reynolds number (ratio of convective to
diffusive influences).



Usually tackle building
-
block, canonical configurations.



Contrast with CFD used in industry



large scales are handled
but
must provide a turbulence or sub
-
grid scale model.

Role of Direct Numerical Simulation (DNS)


A

tool

for

fundamental

studies

of

the

micro
-
physics

of

turbulent

reacting

flows








A

tool

for

the

development

and

validation

of

reduced

model

descriptions

used

in

macro
-
scale

simulations

of

engineering
-
level

systems


DNS

Physical

Models

Engineering
-
level

CFD codes

DNS


Physical

insight

into

chemistry

turbulence

interactions


Full

access

to

time

resolved

fields


CH4

Oxidizer

Fuel

CH3O

HO2

O

Piston Engines

S3D MPP DNS capability at Sandia


S3D code characteristics:


Solves compressible reacting Navier
-
Stokes


F90/F77, MPI, domain decomposition.


Highly scalable and portable


8
th

order finite
-
difference spatial


4
th

order explicit RK integrator


hierarchy of molecular transport models


detailed chemistry


multi
-
physics (sprays, radiation and
soot) from
SciDAC TSTC



70% parallel efficiency on 4096
processors on NERSC (weak scaling test)

S3D is a state
-
of
-
the
-
art DNS code developed with

13 years of BES sponsorship.

#
o
f
p
r
o
c
e
s
s
o
r
s
S
p
e
e
d
u
p
1
0
0
1
0
1
1
0
2
1
0
3
1
0
4
1
0
0
1
0
1
1
0
2
1
0
3
1
0
4
S
3
D
I
d
e
a
l
S3D scales up to 1000s of
processors… and beyond?


Performance improvements on Seaborg


Terascale computations


need optimizations customized to architecture


Lots of assistance from NERSC consultant David Skinner


Used Xprofiler, IPM


Scalar improvements:


used vector MASS libraries for transcendental evaluations


re
-
structured loops in legacy code (eg vectorize)


eliminated unnecessary memory allocation introduced by compiler


flops reductions: tabulate thermodynamic quantities, minimize unit
conversions, eliminate unimportant reactive species.


Parallel improvements:


removed non
-
contiguous MPI data
-
types


re
-
wrote parts of communication to decouple communication
‘directions’, removing possible blocking


removed all unnecessary barriers

= net 45% reduction in execution time!

Outline

1.
DNS of turbulent combustion


challenges and opportunities


2.
Sandia S3D terascale DNS
capability


3.
INCITE project


3D simulations
of a turbulent CO/H
2

jet flame

Scalar dissipation fields in DNS of a turbulent jet flame

(volume rendering by

Kwan
-
Liu Ma and Hongfeng Yu)

Understanding turbulence
-
chemistry
interactions in non
-
premixed flames


Fuel and Air are separate


‘non
-
premixed’


Example


aircraft gas turbine combustor


Separated for safety reasons


Molecular mixing of fuel and air is a needed for reaction to occur


Combustion depends on mixing rate (burning intensity,
emissions, extinction, flame stabilization)


Compressive strain

CO reaction rate

imaging experiment

J. H. Frank et al.

Methane

Air

Flame

INCITE project: Direct simulation of a 3D turbulent
CO/H
2
/air jet flame with detailed chemistry

Scalar dissipation rate,

100 million grid point run

x

x

y

z


Understand the dynamics of extinction and
re
-
ignition in turbulent nonpremixed flames


Find ways to parameterize local chemical
states with lower
-
dimensional manifolds


Understand the influences of differential
diffusion on combustion


Contribute to the interpretation of
experimental data


Develop and validate modeling
approaches


Understand how the details of molecular
transport and reactions can interact with
turbulent mixing.

Community data sets


How to maximize the impact of these large data
-
sets?



TNF workshop (1996
-
present): International Collaboration
of Experimental and Computational Researchers

Description of Run

-

Temporally Evolving Non
-
premixed Plane Jet Flame


Mixing,

Reaction

Mixing,

Reaction

Spanwise BC:

periodic

Streamwise BC:

periodic

DNS data
-
sets of turbulent nonpremixed
CO/H
2

flames


INCITE allocation enables extension to 3D, and hence realistic turbulence


Detailed CO/H
2

chemistry (16 d.o.f., Li et al. 2005)


Parameters selected to maximize Reynolds number, Re (largest range of scales)


~40 small calculations prior to main run (mostly, on our local cluster)



INCITE calculation:


90% completed


Re 4500


350 million grid points


2048, 3072 or 4096 Seaborg processors


(most efficient on 4096!)


3.0 million hours total


~ 10TB raw data



Plan to complement the INCITE calculation with additional runs at different Re

Non
-
premixed combustion concepts


Mixture fraction Z: the amount of fluid from the fuel stream in the
mixture



Z is a conserved (passive) scalar


(no reactive source term)



Scalar dissipation, a measure of local molecular mixing rate:




Z
Z
D




2

Volume rendering of scalar dissipation


Scalar dissipation exists in
thin, highly intermittent
layers



Initially fairly organized
structures aligned across
principal strain directions.



Later, jet breaks down and
a more turbulent, isotropic
structure exists.


Comparison with 2D simulation


2D and 3D flows are
qualitatively different …



Stanley, Sarkar et al. 1998


nonreacting 2D and 3D DNS


2D jet is dominated by a large
scale vortex dipole instability,
which does not occur in 3D


3D, more small
-
scale structures

Vorticity fields

2D

3D

Comparison with 2D simulation


In 2D, see large coherent
structures


high dissipation regions very
persistent


allows mixing with non
-
reacting
pure air and fuel streams


leads to over
-
prediction of
extinguished states



In 3D, see considerable
generation of small scale energy


high dissipation structures are more
transient


smaller structures


mixing occurs
with reacting states

Under
-


prediction

in ‘braids’

Over
-


prediction

in vortex cores

2D

3D

OH mass fractions

Stoichiometric mixture fraction

Mixing timescales


Models for molecular mixing are required in the PDF
approach to turbulent combustion (Pope 1985), a sub
-
grid
model used in engineering CFD approaches.



TNF workshop


CFD predictions are dependent on mixing
timescale choice.



Models assume that scalar mixing timescales are identical
for all scalars and determined by the turbulence timescale.


scalars with different diffusivities?


reactive scalars?

Definitions


Mechanical time
-
scale:




Scalar time
-
scale:




Time
-
scale ratio:



k
u










D
2
'
2




u
r

is assumed to be order unity in most models


r
is assumed to be the same for all scalars


r
Mixture fraction to mechanical timescale
ratio


Confirmation that mixture fraction to mechanical time scale ratio is
order unity.


Average value about 1.6, similar to values reported by
experiments, simple chemistry DNS, and used successfully in
models.


Effect of diffusivity


Smaller, more highly
diffusive species do
have faster mixing
timescales



Ratio is not as large as
the ratio of diffusivities


indicates a partial
balance of production
and dissipation exists.



Future work: compare
with models in literature.




Increasing

diffusivity

Chemistry effects on mixing?


Major species such as
CO
2

are relatively
constant while minor
radicals O, OH and HO
2

are time varying.



At late times, the
diffusivity trend does not
appear to hold for HO
2

versus O and OH.



Theory: somehow
chemistry effects are
causing these different
behaviors.

Radical production and destruction in high
dissipation regions


OH is destroyed while HO
2

is produced in high dissipation
regions

HO
2

OH

Color scale: mass fraction

Blue contours:


Dissipation of passive and reactive scalars


Blue:

Z
, Green:

OH
, Red:

HO2



Dissipation fields of Z and HO
2

are co
-
incident and aligned with
principal strain directions



OH dissipation occurs elsewhere,
more in the centre of the jet



These fundamentally different
structures are due to the different
chemical response of the species



Future work


how does this
affect the mixing timescales?


Conclusions
-

mixing timescales


New finding: detailed transport and chemistry effects can
alter the observed mixing timescales



Models may need to incorporate these effects


a poor mixing model could lead to incorrectly predicting a stable
flame when actually extinction occurs



This type of information cannot be determined any other
way at present


ambiguities in a
-
posteriori model tests


too difficult to measure


need 3D and detailed chemistry to see this


Summary


We used a state
-
of
-
the
-
art DNS capability to perform some very
challenging turbulent combustion simulations, utilizing up to
4096 IBM SP3 processors at NERSC.



INCITE Award enabled extension to 3D and correct
representation of the turbulence dynamics



3D DNS of detailed finite
-
rate chemistry effects in turbulent jets
provides new insights and data for combustion modeling.


First glimpse of results reveals mixing of reactive and differentially
diffusing scalars can be very different from conserved scalars.


More to come…!

Knowledge Discovery From Terascale
Datasets


Challenge
:


Large data size, complex physics


Lots of researchers with different
questions


flexible workflow


Post
-
processing needs to be interactive


Remote archives and slow network


Solution
:


Need interactive knowledge discovery
software


Multi
-
variate visualization


Feature extraction/tracking


Scalable transparent data sharing and
parallel I/O across platforms


100 million grid run

Scalar Dissipation

Vorticity

u




Z
Z
D




2

100 million grid run

HO
2

dissipation

OH dissipation

2
2
2
2
2
HO
HO
HO
HO
Y
Y
D





OH
OH
OH
OH
Y
Y
D




2