Microsoft BizTalk Server End-to-End Performance Testing Guidance

triangledriprockInternet and Web Development

Aug 7, 2012 (5 years and 1 month ago)

775 views

Microsoft BizTalk Server

End
-
to
-
End Performance

Testing Guidance

Ewan Fairweather & Paolo Salvatori

BizTalk Customer Advisory Team

Microsoft



Session Objectives


Describe how we run BizTalk

performance assessments


Demonstrate tools and techniques that can
be used during a BizTalk performance
assessment


Introduce the BizTalk Server Performance
Optimization Guide

Session Agenda


Assessing application performance


Key learnings


BizTalk Server performance assessment


Methodology


Tools


Tuning the Microsoft Platform

Using LogParser To
Generate KPIs For Your
Results

Example results spreadsheet from

a recent performance lab

Session Agenda


Assessing application performance


Key learnings


BizTalk Server performance assessment


Methodology


Tools


Key Learnings

Platform Tuning


Turn off
hyperthreading


Adjust Windows Server 2003 for best
performance


Use a separate disk for PAGEFILE


Stop unnecessary services (for instance,
Print Spooler)


Defrag volumes hosting BizTalk
dbs

Key
Learnings

Platform Tuning


Disable the SynAttackProtect feature


Prevent antivirus scanning


Networking


Adjust TCP/IP parameters in the registry


Use 1
-
gigabit networks


Separate subnets for Mgmt, Data, and
Messaging


Tune NICs for performance


Memory for packet buffering


Offload processing to NIC if possible

Key
Learnings

Platform Tuning


CLR Hosting Threads


To prevent thread starvation, BizTalk needs
enough threads allocated to the .NET thread pool


Controlled by per host reg key


HKEY_LOCAL_MACHINE
\
SYSTEM
\
CurrentControlSet
\
Servi
ces
\
BTSSvc$hostname
\
CLR Hosting

DWORD entry

Default Value

Recommended Value

MaxIOThreads

20

100

MaxWorkerThreads

25

100

MinIOThreads

1

25

MinWorkerThreads

1

25

Key Learnings

BizTalk Host Configuration


Split up the sending, receiving, processing, and
tracking into multiple hosts


This technique allows each host to:


Have its own set of resources such as memory,
handles, .NET thread pool


Decreases the contention on the
MessageBox

Host
Queue tables


Avoid running too many host instances on the
same box


Key Learnings

SQL Performance


Isolate BizTalk data and transaction log files


Pre
-
allocate space to prevent
autogrowth

at run
time


Create multiple data files of the same size, one for
each available core on the SQL machine, for the
TempDb


Add
filegroups

and files to the
BizTalkMsgBoxDb


Scale up the SQL Server resources


CPU, disk, memory


Separate BizTalk Databases across servers


Multiple
MessageBox

databases


Configure DTA Purge and Archive job


Key
Learnings

Low Latency



Internal message queue


MaxReceiveInterval

parameter


500 ms default


BatchSize

parameter


Turn off global tracking


Use inline sends

Key Learnings

Orchestrations (1 of 2)


Eliminate unnecessary persistence points


Minimize the internal state/size


Use the Call Orchestration shape for performance


Asynchronous mechanisms give flexibility but require a
MessageBox

round trip


Direct Bound Ports, Self
-
Correlating Ports and Start Orchestration
shape


Eliminate logical ports and use inline sends


Prefer distinguished field over promoted properties


Turn off unnecessary tracking:


Message send and receive tracking


Begin and end tracking shape


Define a performance counter for each orchestration type

Key Learnings

Orchestrations (2 of 2)


Avoid using
XmlDocument

for processing large
messages


The memory amount required to load an
XMLDocument

is
up to 10 times the actual size of the message


Instead use one the following techniques:

// Use an
XmlReader

object to read the content of the message part

using (
XmlReader

reader =(
XmlReader
)
xlangMessage
[0].
RetrieveAs
(
typeof
(
XmlReader
)))

{


...

}

// Retrieve the content of the message part as a string

using (string message =(
XmlReader
)
xlangMessage
[0].
RetrieveAs
(
typeof
(string)))

{


...

}

//
Deserialize

the content of the message part to an instance of a custom .NET class

using (
RequestClass

request =(
XmlReader
)
xlangMessage
[0].
RetrieveAs
(
typeof
(
RequestClass
)))

{


...

}

Key Learnings

Pipelines (1 of 2)


Use
PassThruReceive

and
PassThruTransmit

pipelines whenever possible


Avoid loading the entire document into memory using
an
XmlDocument

object


Use a streaming approach instead


If you need to read the value of specific nodes inside
an XML message part, do not use the
SelectSingleNode

method exposed by the
XmlDocument

class


Use an instance of the
XPathReader

class contained in the
Microsoft.BizTalk.XPathReader.dll

assembly


Use an instance of
XPathMutatorStream

class contained in
the Microsoft.BizTalk.Streaming.dll
if you need to read and
eventually change the value of one or more elements.

Key Learnings

Pipelines (2 of 2)


Use an instance of the
VirtualStream

class
(
Microsoft.BizTalk.Streaming.dll
) when reading a large
message


Promote items to the message context only if you need
them for:


Message Routing (Orchestrations, Send Ports)


Demotion of message context properties (Send Ports)


“Write” to the context instead of “Promote”


Use the
IPipelineContext.ResourceTracker

to track and
dispose non
-
CLR resources


Instrument your source code to make your components
simple to debug


Avoid ordered delivery and transaction support

Optimizing BizTalk
Memory Usage

Streaming techniques that can be used
in pipelines

WCF Adapters


Whenever possible, use one of the following WCF
-
* Adapters.


WCF
-
NetTcp

Adapter


WCF
-
Custom Adapter +
CustomBinding

with the
binaryMessageEncoder

and
tcpTransport

Binding Elements.




















Add the
serviceThrottling

service behavior to a WCF
-
Custom Receive
Location and use the following settings:



Parameter

Default

Suggested


tcpTransport
-
>ConnectionPoolSettings
-
>maxOutboundConnectionsPerEndpoint

1


try changing to 20


tcpTransport
-
>maxPendingAccepts

1


try changing to 20


tcpTransport
-
>maxPendingConnections

10


try changing to 20


tcpTransport
-
>listenBacklog

10


try
changing to 20

Send

Side (BizTalk Send Port):

Receive Side (BizTalk Receive Location):


Parameter

Default

Suggested


tcpTransport
-
>maxPendingAccepts

1


try changing to 20


tcpTransport
-
>maxPendingConnections

10


try changing to 200


tcpTransport
-
>listenBacklog

10


try
changing to 20


Parameter

Default

Suggested


serviceThrottling
-
>maxConcurrentCalls

16


try changing to 20


serviceThrottling
-
>
maxConcurrentSessions

10


try
changing to 20


serviceThrottling
-
>maxConcurrentInstances

-


leave the default

WCF Adapter

Optimizing WCF for performance

Session Agenda


Assessing application performance


Key learnings


BizTalk Server performance assessment


Methodology


Tools


BizTalk Server
Performance
Optimization Guide

MSDN

http://msdn.microsoft.com/en
-
us/library/cc558617.aspx




Assessing
Application Performance


The usual story


We all know testing is important for every
solution


Testing is neglected


This is a problem


Solutions using BizTalk tend to be
business
critical
and tolerate little downtime


Poor testing increases the risk profile of your
project


This runs the risk of huge technical and political
problems


Minimize Project Risk


Unit testing


Functional testing


Integration testing


User acceptance testing


Disaster recovery testing/dry runs


Overload testing


Performance assessment


Effective testing
reduces your
project risk

Session Agenda


Assessing application performance


Key learnings


BizTalk Server performance assessment


Methodology


Tools


BizTalk Performance Assessment


An end to end process with a set of goals

and objectives


Occurs over a set period of time


It requires a team of individuals with the

right skill sets to “tune the stack”


Iterative knowledge transfer process


Optimize

Operate

Design & Deliver

POC Performance
Assessment

Pre
-
Live
Performance
Assessment

Post
-
Live
Performance
Assessment

Scope

Engagement Summary



Determine resource requirements


What scenarios are you testing


What technologies are involved


What adapters are needed


SAN requirements


32 bit vs. 64 bit


Other requirements


Throughput


Latency


Document all information!


Guidelines, Goals, Constraints, etc.


Staff the right people

Scope

Defining Goals


Provide objectives and success criteria for the lab


Goals should cover


Performance metrics


Throughput, Latency


Roles and Responsibilities


Deliverables


Define any constraints


Is hardware fixed?


Scale up/out BizTalk/SQL


Can third party systems be scaled out?


Web servers, MQ Series, Custom DB’s

Scope

Defining Goals (Example)

Objective


Measure the Maximum Sustainable Throughput (MST) for the given
use cases with different hardware configurations.


Test Case 1


Messaging
-
Only scenario



Throughput : 50 messages/sec sustainable



Latency: response time < 2 sec


for 90% of messages

Test Case 2


Payment service BizTalk orchestration



Throughput : 3.6 million messages within



an 8 hour period (125 messages/second)



Latency : N/A

Scope

Logical and Physical Architecture


Plan

Determining Resource Requirements

Timeline


2 weeks of preparation time


1 week per test case


1 week to document findings

SMEs required


MQSeries, SQL Server,

IIS, SAN Specialist

Hardware request


SAN Storage (HBA, Luns), Networking, Clustering

Lab space


Appropriate space, whiteboard, projector, security of room


Plan

Sample Timeline

Prepare


Detailed Solution Design


Code complete


Invest in test assets


Build/Deploy


MSBUILD, MSI,

BTSTask
, Scripts


Functional test


BizUnit
, Visual Studio


Load test


Loadgen
, Visual Studio

Building The Lab Environment

Installation and Configuration


Infrastructure


RAID10/0+1?


Databases and logs separated


Database
autogrowth


Virus scanning software


Third party systems


Non
-
default values, scale
-
out options


BizTalk


Host configuration


Message tracking


Adapter configuration

Building The Lab Environment


Performance monitoring


Deployment


MSBuild
,
BTSTask
, MSI


Functional testing


Validate automated load test


Document Solution

Performance Baseline


Latency


Throughput


Spool depth


etc

Executing the Performance Lab
Assessment High Level Flow


Run automated test


BizUnit
, VSTS,
LoadGen


Document results


KPIs, Check after every run


Tuning for performance


One modification at a time


Perform platform

modifications

Session Agenda


Assessing application performance


Key learnings


BizTalk Server performance assessment


Methodology


Tools



Helpful Tools


BizTalk BPA 1.2


PerfMon


BizTalk
LoadGen


BizUnit


Log Man


Relog


Log Parser


PAL


SQLIO



BizTalk Orchestration Profiler

Related Content


MSDN Guidance


Microsoft BizTalk Server Performance Optimization Guide
http://msdn.microsoft.com/en
-
us/library/cc558617.aspx



Microsoft BizTalk Server Operations Guide
http://msdn.microsoft.com/en
-
us/library/cc296643.aspx



BizTalk Server 2006 R2 Hyper
-
V Guide
http://msdn.microsoft.com/en
-
us/library/cc768518.aspx



BizTalk Customer Advisory Team Site


http://msdn.microsoft.com/en
-
us/biztalk/dd876758.aspx

Resources


BizTalk Server 2006: Managing a Successful Performance Lab


http://msdn2.microsoft.com/en
-
us/library/aa972201.aspx


Scaling Your Solutions


http://msdn2.microsoft.com/en
-
us/library/aa561567.aspx


Configuration Parameters that Affect Adapter Performance


http://msdn2.microsoft.com/en
-
us/library/aa561380.aspx


Persistence and the Orchestration Engine


http://msdn2.microsoft.com/en
-
us/library/aa547090.aspx


Troubleshooting
MessageBox

Latency Issues


http://msdn2.microsoft.com/en
-
us/library/aa561709.aspx


Performance Tips and Tricks


http://msdn2.microsoft.com/en
-
us/library/aa560694.aspx


Rule Engine Configuration and Tuning Parameters


http://technet.microsoft.com/en
-
us/library/aa548008.aspx


Performance Considerations When Using the Rule Engine


http://msdn2.microsoft.com/en
-
us/library/aa952929.aspx

Filegroup


Resources


BizTalk Server Database Optimization


http://msdn.microsoft.com/en
-
us/library/bb743398.aspx


Understanding Files and Filegroups


http://msdn2.microsoft.com/en
-
us/library/ms189563.aspx


How to: Add Data or Log Files to a Database (SQL Server
Management Studio)


http://msdn2.microsoft.com/en
-
us/library/ms189253.aspx


How to: Move an Existing Index to a Different Filegroup (SQL
Server Management Studio)


http://msdn2.microsoft.com/en
-
us/library/ms175905.aspx


Physical Database Files and Filegroups


http://msdn.microsoft.com/en
-
us/library/ms179316(SQL.90).aspx


Working with tempdb in SQL Server 2005


http://www.microsoft.com/technet/prodtechnol/sql/2005/workingwithtempd
b.mspx


Optimizing Server Performance Using I/O Configuration
Options


http://msdn2.microsoft.com/en
-
us/library/ms189892.aspx






MQ Adapter Resources


HOW TO: Set Up and Test the BizTalk Adapter for
MQSeries


http://support.microsoft.com/kb/821489


BizTalk Adapter for WebSphere MQ


http://msdn2.microsoft.com/en
-
us/library/aa772116.aspx

©
2008 Microsoft
Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademar
ks
and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the
dat
e of this presentation. Because Microsoft must respond to changing market conditions, it should
not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any informatio
n p
rovided after the date of this presentation.
MICROSOFT
MAKES NO WARRANTIES, EXPRESS,
IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.