Parallel Linear Solvers for Petascale Reservoir

exhaustedcrumΜηχανική

24 Οκτ 2013 (πριν από 3 χρόνια και 11 μήνες)

69 εμφανίσεις

Saudi
Aramco

Technology Quest

10
-
12/10/2011

Parallel Linear Solvers for Petascale Reservoir
Simulation using Adaptive and Unstructured Grids

Matthew Jackson, Ann Muggeridge, Matthew Piggott,
Christopher Pain, Gerard Gorman and Jeff Gomes

Applied
Modelling

and Computational Group (AMCG)

Petroleum Engineering & Rock Mechanics Group (PERM)


Imperial

College
:

Leading

university



strong

in

Engineering
.




AMCG

&

PERM

within

ESE
:

Leading

modelling

and

reservoir

physics

groups



largest

university

research

group

in

the

UK

and

winner

of

the

Imperial

College

Research

Excellence

Award
.

Developed

major

models

e
.
g
.

the

Imperial

College

Ocean

Model

as

the

next

gen
.

Ocean

model

for

the

UK

and

Internationally
.




The

team
:


Prof

Matt

Jackson
:

Reservoir

characterisation

and

engineering


Prof

Ann

Muggeridge
:

Reservoir

physics,

EOR


Prof

Chris

Pain
:

Adaptive

meshes,

linear

solvers,

discretisation


Dr

Matt

Piggott
:

Geophysical

CFD


Dr

Gerard

Gorman
:

HPC,

mesh

generation


Dr

Jeff

Gomes
:

Multi
-
physics

CFD

modelling


Leading team with the
breadth of skills

Technology Quest

Development

of

computational

technologies

that

enable

detailed

multi
-
scale

simulations

of

fluid

flows

in

giant

oil

and

gas

fields
.

How we
will tackle
this
Tech Quest

We

will

develop

the

following

modular

technologies

that

may

be

integrated

into

Saudi

Aramco

simulator
:



Petascale

linear

solvers

for

multi
-
scale

reservoir

simulations

with

high

aspect

ratio

elements/cells

building

on

open
-
source

code
.




Numerical

dynamic

core

for

multiphase

flow

on

unstructured

adaptive

meshes

that

are

accurate

(exact)
.




Demonstration

through

simulations

of

fluid

flow

in

giant

oil

and

gas

fields
.


CFD Modelling:

Parallel
Anisiotropic

Mesh Adaptivity and Domain
Decomposition

Page
6

Movie

Adaptive Mesh and Petascale Solvers

CFD Modelling:

Parallel Mesh Adaptivity
-

Parallel Scaling

Adaptive Mesh and
Petascale

Solvers


Parallel

adaptive

mesh

technology



world

leading
.




Unique

high

aspect

ratio

multi
-
grid

solvers
.

Front

runner

in

GFD
.




AMCG

and

Fujitsu

are

enhancing

the

adaptive

mesh

library

as

part

of

their

Open

Petascale

Libraries

(OPL)

initiative

(MPI/
OpenMP
)
.




On
-
going

collaboration

with

Argonne

National

Lab

on

hybridizing

(MPI/
OpenMP
)

PETSc

libraries
.


Maintained

by

NAG
.




AMCG,

Fujitsu,

Dainsbury

and

EPCC
:

collaboration

on

HPC

and

mesh

adaptivity

to

scale

>

100
K

cores
:

better

able

to

exploit

memory

hierarchy
.


Numerical Dynamic Core


Development

of

novel

family

of

tetrahedral

elements
.




P
1
DGP
2

+

overlapping

elements

have

unique

abilities

to

represent

key

force

balances

on

unstructured

mesh

e
.
g
.

Darcy’s

Law

&

hydrostatic

pressure
.



Exact

force

balance

with

only

approximation

in

advection



limited

CV
-
DG

or

CV
-
FEM

and

implicit

time

stepping



Compositional

modelling
.


Technology Transfer : Training
,
Dissemination
&

Communication with Saudi
-
Aramco

Page
10



Train

PDRAs

and

PhD

students

for

the

future

e
.
g
.

for

Saudi

Aramco
.



Intensive

established

training

events

and

workshops

on

methods,

use

of

models,

how

to

develop

code

&

physics
.




Joint

workshops,

exchange

visits

and

placements
.



Software
:

Manual,

Visualization,

Management

(regression

testing,

IRC

chat,

bug

and

issue

tracking,

code

management)
.






Added value and future work

To

develop

the

following

modular

technologies

to

be

integrated

into

Saudi

Aramco

simulator
:


Hiding

the

details

of

the

hardware

that

the

model

is

run

on

e
.
g
.

GPUs
.



Predictive

modelling

(
adjoints
,

EnKF
,

reduced

order

modelling,

sensitivity

analysis,

adaptive

observations,

data

assimilation,

uncertainty)
.


Cross

fertilization

with

other

disciplines
.


Summary


Looking for long term strategic partnership in reservoir
simulators.



Will develop key components of a next
-
generation simulator.



Parallel mesh
adaptivity

will be a key aspect of a future multi
-
scale simulator.



Dynamic core that can represent physics accurately on meshes
is essential.


A Few Backup Slides

Page
15

Conservative Interpolation: Crucial for many applications, with
Adaptivity, Coupled models

‘donor’
mesh

‘target’
mesh

‘super mesh’
and mapping
from donor
mesh

Following construction of the
supermesh

we are able to perform
projection operations to achieve conservative, bounded (etc…)
interpolation schemes.
2D: Farrell et al., 2009

3D: Farrell,
Maddison
, 2009

Immersed
-
Body
Method (Fluid
-
Solid
Interaction Modelling)

Lower

frame

shows

the

domain

decomposition

which

has

been

optimised

to

balance

the

load

based

on

number

of

nodes

and

minimised

edge

cut

Mesh Adaptivity and Domain
Decomposition

Applications: Surface
-
based reservoir
modelling

Applications: Surface
-
based reservoir
modelling