Intelligent Load Balancing

boorishadamantΤεχνίτη Νοημοσύνη και Ρομποτική

29 Οκτ 2013 (πριν από 3 χρόνια και 11 μήνες)

121 εμφανίσεις

Load Balancing and
Intelligent Load Balancing

Jesús González


Escalation Engineer

Objectives


Deep understanding of load balancing architecture



Troubleshooting techniques



Identify the root cause of any LB problem

Agenda


Architecture


Data Collector


Updates
-

Examples


Performance Data Helper


Citrix and the PDH


Where can it go wrong?


Black hole effect (Load Throttling
-

Intelligent Load Balancing)


Some servers get all the connections


Full load after installing an MUI


Troubleshooting


Architecture

Architecture



Data Collector DC
-

Dynamic Store DS

LmsSS.dll
-

IMA Subsystem

MFRules.dll
-

XA counters

LMS20Rules.dll

-

System counters

PDH.dll
-

Performance Data Helper

Data Collector Updates



Data Collector is updated every 30 s



Only when the change in the load evaluator is bigger than 5%


Every 5 minutes we send a full update



Connection logon or logoff

Data Collector Updates

Server load
-

CPU Load example


Data
Collector

XenApp

Time

30s

60s

15

20

23

25

23

50

98

34

45

56

X

20

23

25

23

50

98

34

45

56

40

X

23

25

23

50

98

34

45

56

40

12

38

41

40

Server Load % :

5 minutes

Data Collector Updates

XA CPU Load != Task Manager CPU Load


CPU load



XA = 25%



Task Manager =
1%



Ctxnotif.dll

MFSrvSS.dll

LMSSS.DLL

Data Collector Updates

At Logon time
-

Default load evaluator

Data Collector

ICA Client

Dynamic Store

Server A

Server B

IMA

Server B

Server A

Server A

BIAS

WI/XML

IMA

X

Performance Data Helper


API


The performance data helper interface calls the registry
interface to retrieve performance data


Uses the PDH.DLL to access the PDH API



http://msdn2.microsoft.com/en
-
us/library/aa373083.aspx





Performance Data Helper


C++ Example

pdhStatus = PdhAddCounter (

hQuery,







"
\
\
Processor(0)
\
\
% Processor Time
",








0,







&hCounter);






YOU MUST CALL THE PERFORMANCE COUNTER BY ITS NAME

Citrix and the PDH



\
\
Processor(_Total)
\
\
% Processor Time


\
\
System
\
\
Context Switches/sec


\
\
Memory
\
\
% Committed Bytes In Use


\
\
Memory
\
\
Page Faults/sec


\
\
Memory
\
\
Pages/sec


\
\
PhysicalDisk(_Total)
\
\
Disk Bytes/sec


\
\
PhysicalDisk(_Total)
\
\
Disk Reads/sec


\
\
PhysicalDisk(_Total)
\
\
Disk Writes/sec

Performance Data Helper
-

Registry

Performance Data Helper


File system



English


Perfc009.dat


Perfh009.dat


German


Perfc007.dat


Perfh007.dat




Performance Data Helper
-

perfmon

Performance Data Helper
-

IMA

IMA

(At start up)

Citrix and the PDH

We also
provide
Performance
counters

Citrix and the PDH

HKLM
\
SYSTEM
\
CurrentControlSet
\
Services
\
IMAService
\
Performance

Where can it go wrong?

Black hole effect

Problem

Solution



At Peak logon times a
recently boot up server will
get all connections and might
become unresponsive



Cause
:
Server is unable to
update the DC


Load Throttling


Intelligent load balancing

HKLM
\
SOFTWARE
\
Citrix
\
IMA
\
LMS
\

(DWORD )

UseILB = 1

(DWORD )

ILBMultiplier = 2






Before Intelligent Load Biasing

BIAS

Current Load (0)

Max Load (10000)

0 sessions

Current Load (100)

Max Load (10000)

1 session

BIAS

Max Load (10000)

2 sessions

Current Load (200)

1 session comes

2nd session comes

Default BIAS = 10000/100 = 100

Intelligent Load Biasing

BIAS

Current Load (0)

Max Load (10000)

ILBMultiplier (2)

0 sessions

Current Load (5000)

Max Load (10000)

1 session

BIAS

Max Load (10000)

2 sessions

Current Load (7500)

ILBMultiplier (2)

1 session comes

2nd session comes

Load = [(Max Load


Current Load) / ILBMultiplier] + Load

Load Throttling
-

Low logon rate

Data Collector

ICA Client

Dynamic Store

Server A

Server B

Server B

Server A

Server Load

BIAS

0

0

0

0

(10000
-
0)/2+0 = 5000

100

(10000
-
0)/2+0 = 5000

100

Load Throttling
-

Low logon rate

Server load if Low logon rate


Load Throttling
-

Black hole effect

Data Collector

ICA Client

Dynamic Store

Server A

Server B

Server B

Server A

Server Load

BIAS

0

6000

0

0

(10000
-
0)/2+0 = 5000

(10000
-
5000)/2+5000 = 7500

(10000
-
6000)/2+6000 = 8000

(10000
-
7500)/2+7500 = 8750

Load Throttling

Server load at Peak logon times


Upgrade to W2K3 causes full load

Problem

Solution


English W2K


\
\
System
\
\
% Total Processor Time



English W2K3

\
\
Processor(_Total)
\
\
% Processor
Time


In W2K3, use

\
\
Processor(_Total)
\
\
% Processor
Time




Some servers get all the connections

Problem

Solution



1 server all the connections



Cause
:
Fail to read
performance counters
causes Load = 0


Fail to read performance
counters causes Load =
10000



Full load after installing an MUI

Problem

Solution



After installing MUI with
advance load evaluators it
results in load = 10000



Cause
:

-
>

perfc007.dat (German)

perfc009.dat (English)


HKLM
\
Software
\
Citrix
\
IMA
\
LMS
\


EnableTranslation=1 (dword)


Full load after installing an MUI


Jesús González




in a German restaurant


English Menu (
perfc009.dat)


22 .
Vegetable Soup


German Menu (
perfc007.dat)


22.
Gemüsensuppe

Troubleshooting

Troubleshooting

Data Collector DC

LmsSS.dll

LMS20Rules.dll

PDH.dll

\
\
Processor(_Total)
\
\
% Processor Time



XXXXXXXXXXXXXXXX

Full Load



Troubleshooting

Failing to read performance counters => full load


Use
procmon (filemon) while restarting IMA

Make sure the correct perfcXXX.dat file can be accessed.



Consider
rebuilding performance counters

http://support.microsoft.com/kb/300956/en
-
us



Check that
no performance counter are disabled


[HKEY_LOCAL_MACHINE
\
SYSTEM
\
CurrentControlSet
\
Services
\
PerfDisk
\
Performance]

"Disable Performance Counters"=dword:00000001



Use CDFControl to gather CDF traces

http://support.citrix.com/article/CTX111961




LMSRuleDll_Interface::CreatePDHQuery() ERROR!!!!!
rc =
-
1073738823



-
1073738823
-
> FFFFFFFFC0000BB9 (HEX)



http://msdn2.microsoft.com/en
-
us/library/aa373046.aspx


“The specified counter could not be found”



Troubleshooting

CDF Traces example

Summary


Deep understanding of load balancing



BIAS, PDH…


Understanding Intelligent load balancing



Load Throttling


Typical scenarios



Full load versus 0 load, MUI…


Troubleshooting techniques



So far we are able to resolve all the problems that arrive to EMEA tech
support with those tips

Before you leave…



Session surveys are available online at
www.citrixsynergy.com

starting Thursday, 7 October


Provide your feedback and pick up a complimentary gift card at the registration desk



Download presentations starting Friday, 15 October, from your My
Organiser Tool located in your My Synergy Microsite event account