Final Report - Nyu

overratedbeltΤεχνίτη Νοημοσύνη και Ρομποτική

25 Νοε 2013 (πριν από 3 χρόνια και 8 μήνες)

104 εμφανίσεις

Mining Approaches



We began by attempting to utilize SQL Server 2000 Analyst Services. However,
we soon realized that this version of Analyst Services did not provide the association rule
mining algorithms necessary to complete our project. SQL Service
2000 Analyst
Services only provides Microsoft's Decision Tree and Clustering Algorithms.


We then switched to Weka to analyze and mine our data. Using a small Java
program we separated the sequences into 1,280,778 ordered pairs of the form (start, next)
o
n each line. Pairs containing identical values of start and next were dropped since we
were only searching for sequential movement between categories. The data set was then
stored in Weka's ARFF file format by adding the necessary meta
-
tags. A very smal
l
sample of this file is shown here:




@relation MSNBC_WEB_DATA



@attribute start {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17}



@attribute next {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17}



@data



3,2



2,4



4,2



We then used the predictive Apriori Algorithm to fin
d sequential category
associations within our dataset.


Figure: Weka Explorer using MSNBC web data file




In addition to using Weka to analyze and mine the data, we also utilized SQL Server
2005 Analyst Services (SSAS 2005). This is the updated version
of SASS 2000 and it
did include the out
-
of
-
the
-
box algorithms that we intended to study in our project.
Specifically, we were interested in using the Microsoft Association Algorithm and the
Microsoft Sequence Clustering algorithm.


The Microsoft Associati
on Algorithm was used to find category associations from
our data with the following parameters:



Minimum Support 3% or total case,



Minimum Probability 40% of the total cases and a



Max Itemset of 3.


We also utilized the following training structure when
analyzing the data,




Case Table:

Webstats




Key:

RowID



Input:

None



Predictable:

None



Nested Table:

Webstats




Key:

PageID




Input:

PageID



Predictable:

PageID



Figure: Mining Structure for Association Algorithm



T
he Webstat table was utilized as both the case and nested table since it is where
all of our pertinent information was stored. The interesting portion of this structure is the
utilization of the PageID as the key, input, and predict; this equates to the a
lgorithm
attempting to predict associations within the data.


In addition to the Microsoft Association Algorithm, we also utilized the Microsoft
Sequence Clustering Algorithm to explore the data containing events linked by the page
sequences. The algori
thm finds the most common sequences by clustering identical
sequences together. The mining structure was as follows:



Case Table:


Webstats


Key:



RowID


Input:




Predict:



Nested Table:


Webstats


Key:



SeqID


Input:



PageID, SeqID


Predict:


PageI
D, SeqID



Figure: Mining Structure for Sequence Clustering Algorithm



In addition to these parameters, we needed to change the Content Type value from
the predicted ‘Continuous’ value to ‘Discrete’ since the Sequence Clustering Algorithm
only functions
with discrete values.



Figure: Setting PageID Content Type to Discrete


We ran the algorithm with 2 different sets of parameters:




Parameters


-

2 sets:



-

A)




CLUSTER_COUNT = 10




MAXIMUM_SEQUENCE_STATES = 64




MAXIMUM_STATES = 100




MINIMUM_SUP
PORT = 10




-

B)




CLUSTER_COUNT = 10




MAXIMUM_SEQUENCE_STATES = 20




MAXIMUM_STATES = 17




MINIMUM_SUPPORT = 1000






However, we were only able to analyze the results obtained from parameter set A
since set B generated fewer usable clusters, most
likely because of the higher minimum
support.


Results:


Weka:



Using a predictive Apriori algorithm, we were able to mine common 2 category
sequences within our data. The rules created were ordered by accuracy. Below are the
top 25 rules that were retu
rned by predictive Apriori. The top 100 rules, including run
information can be found in Appendix A.



1. next=12 78321 ==> start=1 41211

acc:(0.17342)


2. next=11 52002 ==> start=1 27260

acc:(0.17338)


3. next=2 152909 ==> start=1 73820

acc:(0.15136)


4. next=17 11007 ==> start=1 5038

acc:(0.14109)


5. start=13 40881 ==> next=14 18090

acc:(0.13941)


6. start=11 43792 ==> next=1 19014

acc:(0.13758)


7. start=14 62591 ==> next=1 26361

acc:(0.13622)


8. start=17 9115 ==> next=1 3707

acc:(0.13291)


9
. next=16 2678 ==> start=1 1049

acc:(0.12908)


10. next=14 89649 ==> start=1 35992

acc:(0.12854)


11. start=12 69285 ==> next=1 27158

acc:(0.12793)


12. next=10 51062 ==> start=1 18870

acc:(0.1256)


13. next=3 68937 ==> start=1 23795

acc:(0.12174)


14. sta
rt=2 132353 ==> next=1 45476

acc:(0.11934)


15. start=16 2062 ==> next=1 691

acc:(0.11778)


16. next=15 35749 ==> start=6 11516

acc:(0.1117)


17. next=5 28196 ==> start=1 8604

acc:(0.10839)


18. next=4 110286 ==> start=7 33018

acc:(0.10826)


19. start=3 55
634 ==> next=1 16517

acc:(0.10577)


20. next=7 124541 ==> start=6 36783

acc:(0.1038)


21. start=10 40731 ==> next=1 11944

acc:(0.10058)


22. start=5 19796 ==> next=1 5787

acc:(0.10039)


23. start=6 125220 ==> next=7 36783

acc:(0.10006)


24. start=7 115516
==> next=4 33018

acc:(0.09665)


25. next=13 34131 ==> start=14 9731

acc:(0.09643)


Given that in the dataset, 13 corresponds to MSN
-
Sports and 14 corresponds to Sports,
the rule


start=13 40881 ==> next=14 18090

acc:(0.13941)


can be read as follows: if a

user was looking at a page in the MSN
-
Sports category (start),
the rule predicted with 13.9% accuracy that they would next navigate to a page in the
Sports category (next).




start=Travel


next=Frontpage acc:(0.13291)







Microsoft Association Algori
thm:



Once the algorithm has completed analyzing the data, association rules are
produced. There are three categories that describe rules: the probability, the importance,
and the rule itself. The “probability” is simply a measure of how likely a rule i
s to occur
within the dataset. The ‘Importance’ is a much more interesting measure. This measure
expresses the usefulness of the rule. A higher value means a better rule. Simply looking
at the probability of a rule can be extremely misleading. For exam
ple, if every transaction
contains an item A, the rule B predicts that A has a probability of 1, meaning that A will
always occur. Even though the accuracy of the rule is very good, it does not relay very
much useful information, because every transaction
contains A regardless of B.



After running the algorithm, we were given rules in this form:



Figure: Rules derived from Association Algorithm


An example of a rule is described as follows:



0.596

1.019

17=Existing, 6=Existing


11 = Existing


This equa
tes to:




Exists(Travel) ^ Exists(On
-
Air)


Exists(Living)


This states, if there exists a category view of Travel and a category view of On
-
Air then
there existed a view on the Living category.



In addition to using these mining association rules, we we
re also able to utilize the
Association Dependency Network provided to us by SASS 2005. The Association
Network is a graph that shows associations found between items in a dataset. In this
graph, which was filtered to only show the strongest links betwee
n categories, we see that
the Travel category is strongly associated with the Frontpage, News, and Living category.
More specifically, we can deduce that if a user visits the Travel category, they have or
will visit the Frontpage, News, or Living category

within that session.































Figure: Association Dependency Network showing

only the strongest links.





Lastly, we analyzed our dataset using the Microsoft Sequence Clustering
Algorithm. We were able to generate a clustering di
agram shown here:



Figure: Sequence Clustering Diagram




Each cluster represents a sequence of category views from a user session. The
darker the links between the nodes, the stronger the associations are. However, due to
time constraints, we were un
able to analyze and produce any real value from this cluster
graph. With more time, we would have been able to use this graph and its findings in our
conclusions.



In addition to this clustering diagram, we were able to product a state diagram.
This is
shown below;






















Figure: State Transition Diagram




Each category in our data is represented by a “state” in this graph. The
connections represent the probabilities transitions from one category to another and the
darker these connectio
ns, the stronger the transition. Here, we can see that each category
is strongly associated with itself, which makes logical sense since a user will browse
many links within the same category. This is shown in this diagram. Yet, we do see that
a transit
ion does exist from the Travel and Living categories to the Frontpage category.


Below are the top 25 rules that were returned by Microsoft's Association Algorithm. The
entire set of rules can be found in Appendix B.


0.596

1.01893607386409

17 = Existing,

6 = Existing
-
> 11 = Existing

0.559

0.992190991675298

17 = Existing, 4 = Existing
-
> 11 = Existing

0.550

0.982542735834353

17 = Existing, 10 = Existing
-
> 11 = Existing

0.523

0.962815763106429

17 = Existing, 12 = Existing
-
> 11 = Existing

0.464

0.91614523
9289783

17 = Existing, 2 = Existing
-
> 11 = Existing

0.457

0.9066668144746

5 = Existing, 14 = Existing
-
> 11 = Existing

0.457

0.906423945206197

5 = Existing, 10 = Existing
-
> 11 = Existing

0.457

0.903454550671399

17 = Existing, 3 = Existing
-
> 11 = Existin
g

0.429

0.889013990365444

17 = Existing, 1 = Existing
-
> 11 = Existing

0.435

0.885536672518499

5 = Existing, 12 = Existing
-
> 11 = Existing

0.407

0.8752641433198

17 = Existing
-
> 11 = Existing

0.401

0.851719140763221

5 = Existing, 3 = Existing
-
> 11 = Exis
ting

0.468

0.782148116100019

8 = Existing, 4 = Existing
-
> 7 = Existing

0.458

0.759149450411111

15 = Existing, 4 = Existing
-
> 7 = Existing

0.423

0.746912716649694

9 = Existing, 14 = Existing
-
> 13 = Existing

0.402

0.699241403043302

10 = Existing, 8 = Exis
ting
-
> 7 = Existing

0.448

0.695723632191358

13 = Existing, 12 = Existing
-
> 9 = Existing

0.446

0.694049832581722

13 = Existing, 3 = Existing
-
> 9 = Existing

0.558

0.670404284230851

7 = Existing, 8 = Existing
-
> 4 = Existing

0.521

0.656927573041699

7 = Exi
sting, 2 = Existing
-
> 4 = Existing

0.525

0.639394765412092

11 = Existing, 7 = Existing
-
> 4 = Existing

0.521

0.639376216149085

13 = Existing, 2 = Existing
-
> 14 = Existing

0.516

0.628663133888778

5 = Existing, 7 = Existing
-
> 4 = Existing

0.473

0.59922601
9995095

13 = Existing, 1 = Existing
-
> 14 = Existing

0.466

0.588577756093422

10 = Existing, 7 = Existing
-
> 4 = Existing



Here is where stuff follows:






Appendix A:



Weka Predictive Apriori Results




=== Run information ===


Scheme: weka.assoc
iations.PredictiveApriori
-
N 100

Relation: MSNBC_WEB_DATA

Instances: 1280778

Attributes: 2


start


next

=== Associator model (full training set) ===



PredictiveApriori

===================



Best rules found:



1. next=1
2 78321 ==> start=1 41211 acc:(0.17342)


2. next=11 52002 ==> start=1 27260 acc:(0.17338)


3. next=2 152909 ==> start=1 73820 acc:(0.15136)


4. next=17 11007 ==> start=1 5038 acc:(0.14109)


5. start=13 40881 ==> next=14 18090 acc:(0.1394
1)


6. start=11 43792 ==> next=1 19014 acc:(0.13758)


7. start=14 62591 ==> next=1 26361 acc:(0.13622)


8. start=17 9115 ==> next=1 3707 acc:(0.13291)


9. next=16 2678 ==> start=1 1049 acc:(0.12908)


10. next=14 89649 ==> start=1 35992 a
cc:(0.12854)


11. start=12 69285 ==> next=1 27158 acc:(0.12793)


12. next=10 51062 ==> start=1 18870 acc:(0.1256)


13. next=3 68937 ==> start=1 23795 acc:(0.12174)


14. start=2 132353 ==> next=1 45476 acc:(0.11934)


15. start=16 2062 ==> next=1

691 acc:(0.11778)


16. next=15 35749 ==> start=6 11516 acc:(0.1117)


17. next=5 28196 ==> start=1 8604 acc:(0.10839)


18. next=4 110286 ==> start=7 33018 acc:(0.10826)


19. start=3 55634 ==> next=1 16517 acc:(0.10577)


20. next=7 124541 ==>

start=6 36783 acc:(0.1038)


21. start=10 40731 ==> next=1 11944 acc:(0.10058)


22. start=5 19796 ==> next=1 5787 acc:(0.10039)


23. start=6 125220 ==> next=7 36783 acc:(0.10006)


24. start=7 115516 ==> next=4 33018 acc:(0.09665)


25. next=1
3 34131 ==> start=14 9731 acc:(0.09643)


26. next=6 107869 ==> start=1 30579 acc:(0.09593)


27. next=6 107869 ==> start=7 29210 acc:(0.09551)


28. next=17 11007 ==> start=11 2933 acc:(0.09199)


29. start=7 115516 ==> next=6 29210 acc:(0.0863
4)


30. next=4 110286 ==> start=1 27130 acc:(0.08631)


31. start=4 91785 ==> next=1 21565 acc:(0.0846)


32. next=13 34131 ==> start=7 7984 acc:(0.0842)


33. start=4 91785 ==> next=7 21496 acc:(0.08411)


34. next=7 124541 ==> start=1 28117 ac
c:(0.08293)


35. start=1 340184 ==> next=2 73820 acc:(0.08279)


36. start=7 115516 ==> next=1 25214 acc:(0.08277)


37. start=8 37047 ==> next=2 8173 acc:(0.08276)


38. next=8 38335 ==> start=1 8095 acc:(0.07873)


39. start=15 31386 ==> next=6 6
575 acc:(0.07848)


40. start=13 40881 ==> next=7 8438 acc:(0.07825)


41. next=14 89649 ==> start=13 18090 acc:(0.07821)


42. start=16 2062 ==> next=14 405 acc:(0.07574)


43. next=16 2678 ==> start=14 511 acc:(0.07408)


44. next=1 244329 ==>
start=2 45476 acc:(0.07178)


45. start=10 40731 ==> next=2 7191 acc:(0.07114)


46. start=17 9115 ==> next=11 1611 acc:(0.07113)


47. next=7 124541 ==> start=4 21496 acc:(0.07111)


48. next=8 38335 ==> start=2 6565 acc:(0.07107)


49. start=8
37047 ==> next=1 6241 acc:(0.07067)


50. next=15 35749 ==> start=2 5872 acc:(0.0684)


51. next=15 35749 ==> start=1 5739 acc:(0.06669)


52. start=9 63400 ==> next=1 10065 acc:(0.06638)


53. start=9 63400 ==> next=4 9981 acc:(0.06596)


54. ne
xt=3 68937 ==> start=2 10842 acc:(0.06593)


55. start=15 31386 ==> next=2 4953 acc:(0.06575)


56. start=14 62591 ==> next=13 9731 acc:(0.06447)


57. next=9 50777 ==> start=6 7672 acc:(0.06153)


58. next=10 51062 ==> start=2 7708 acc:(0.0615)


59. start=9 63400 ==> next=7 9541 acc:(0.06137)


60. next=13 34131 ==> start=9 5071 acc:(0.06135)


61. next=9 50777 ==> start=7 7399 acc:(0.06128)


62. start=6 125220 ==> next=1 18177 acc:(0.06128)


63. next=9 50777 ==> start=4 7293 acc:(0
.06127)


64. start=3 55634 ==> next=2 7939 acc:(0.06126)


65. start=5 19796 ==> next=2 2739 acc:(0.061)


66. start=16 2062 ==> next=6 287 acc:(0.06097)


67. next=16 2678 ==> start=6 371 acc:(0.06078)


68. next=12 78321 ==> start=2 10062 acc:
(0.05955)


69. next=10 51062 ==> start=6 6445 acc:(0.05856)


70. start=12 69285 ==> next=2 8577 acc:(0.05665)


71. next=8 38335 ==> start=9 4718 acc:(0.05649)


72. start=4 91785 ==> next=2 11305 acc:(0.0561)


73. start=1 340184 ==> next=12 4121
1 acc:(0.05559)


74. next=8 38335 ==> start=6 4434 acc:(0.05558)


75. start=10 40731 ==> next=6 4608 acc:(0.05557)


76. next=1 244329 ==> start=12 27158 acc:(0.05556)


77. next=1 244329 ==> start=14 26361 acc:(0.05556)


78. start=15 31386 ==
> next=1 3514 acc:(0.05556)


79. start=5 19796 ==> next=6 2228 acc:(0.05555)


80. next=9 50777 ==> start=12 5536 acc:(0.0555)


81. start=1 340184 ==> next=14 35992 acc:(0.05508)


82. next=5 28196 ==> start=2 3019 acc:(0.05484)


83. start=16
2062 ==> next=2 225 acc:(0.0544)


84. next=8 38335 ==> start=4 4028 acc:(0.05384)


85. next=1 244329 ==> start=7 25214 acc:(0.05195)


86. next=4 110286 ==> start=2 11173 acc:(0.05188)


87. next=5 28196 ==> start=9 2807 acc:(0.05173)


88. nex
t=5 28196 ==> start=6 2776 acc:(0.05137)


89. next=14 89649 ==> start=2 8528 acc:(0.0486)


90. next=9 50777 ==> start=3 4817 acc:(0.0481)


91. next=16 2678 ==> start=5 247 acc:(0.0476)


92. start=13 40881 ==> next=9 3748 acc:(0.04459)


93. s
tart=15 31386 ==> next=10 2850 acc:(0.04444)


94. start=11 43792 ==> next=2 3992 acc:(0.04435)


95. start=5 19796 ==> next=15 1763 acc:(0.04431)


96. start=17 9115 ==> next=2 784 acc:(0.04427)


97. start=8 37047 ==> next=4 3351 acc:(0.04427)


98. start=6 125220 ==> next=15 11516 acc:(0.04415)


99. next=5 28196 ==> start=15 2487 acc:(0.04413)

100. next=4 110286 ==> start=9 9981 acc:(0.04411)
Appendix B:



Microsoft Association Results


Prob

Importance


Rules

0.596

1.01893607386409

17
= Existing, 6 = Existing
-
> 11 = Existing

0.559

0.992190991675298

17 = Existing, 4 = Existing
-
> 11 = Existing

0.550

0.982542735834353

17 = Existing, 10 = Existing
-
> 11 = Existing

0.523

0.962815763106429

17 = Existing, 12 = Existing
-
> 11 = Existing

0.464

0.916145239289783

17 = Existing, 2 = Existing
-
> 11 = Existing

0.457

0.9066668144746

5 = Existing, 14 = Existing
-
> 11 = Existing

0.457

0.906423945206197

5 = Existing, 10 = Existing
-
> 11 = Existing

0.457

0.903454550671399

17 = Existing, 3 = Existing
-
> 1
1 = Existing

0.429

0.889013990365444

17 = Existing, 1 = Existing
-
> 11 = Existing

0.435

0.885536672518499

5 = Existing, 12 = Existing
-
> 11 = Existing

0.407

0.8752641433198

17 = Existing
-
> 11 = Existing

0.401

0.851719140763221

5 = Existing, 3 = Existing
-
> 11 = Existing

0.468

0.782148116100019

8 = Existing, 4 = Existing
-
> 7 = Existing

0.458

0.759149450411111

15 = Existing, 4 = Existing
-
> 7 = Existing

0.423

0.746912716649694

9 = Existing, 14 = Existing
-
> 13 = Existing

0.402

0.699241403043302

10 = Existin
g, 8 = Existing
-
> 7 = Existing

0.448

0.695723632191358

13 = Existing, 12 = Existing
-
> 9 = Existing

0.446

0.694049832581722

13 = Existing, 3 = Existing
-
> 9 = Existing

0.558

0.670404284230851

7 = Existing, 8 = Existing
-
> 4 = Existing

0.521

0.656927573041
699

7 = Existing, 2 = Existing
-
> 4 = Existing

0.525

0.639394765412092

11 = Existing, 7 = Existing
-
> 4 = Existing

0.521

0.639376216149085

13 = Existing, 2 = Existing
-
> 14 = Existing

0.516

0.628663133888778

5 = Existing, 7 = Existing
-
> 4 = Existing

0.473

0.599226019995095

13 = Existing, 1 = Existing
-
> 14 = Existing

0.466

0.588577756093422

10 = Existing, 7 = Existing
-
> 4 = Existing

0.670

0.582074481480543

5 = Existing, 10 = Existing
-
> 2 = Existing

0.465

0.581058451122438

5 = Existing, 10 = Existing
-
> 3

= Existing

0.452

0.575528164839493

7 = Existing, 12 = Existing
-
> 4 = Existing

0.450

0.571943724660374

7 = Existing, 3 = Existing
-
> 4 = Existing

0.440

0.568904341552659

7 = Existing, 9 = Existing
-
> 4 = Existing

0.647

0.565886225871579

17 = Existing, 3 =

Existing
-
> 2 = Existing

0.794

0.565639123043752

15 = Existing, 7 = Existing
-
> 6 = Existing

0.637

0.558066333793884

17 = Existing, 10 = Existing
-
> 2 = Existing

0.405

0.556044091784983

17 = Existing, 3 = Existing
-
> 12 = Existing

0.436

0.553952550098229

5 = Existing, 12 = Existing
-
> 3 = Existing

0.627

0.553670092847154

5 = Existing, 12 = Existing
-
> 2 = Existing

0.401

0.552446201953081

7 = Existing, 1 = Existing
-
> 4 = Existing

0.622

0.54852437351911

17 = Existing, 4 = Existing
-
> 2 = Existing

0.605

0.53
5871318355219

15 = Existing, 8 = Existing
-
> 2 = Existing

0.604

0.535546679816423

17 = Existing, 12 = Existing
-
> 2 = Existing

0.604

0.534486036539933

17 = Existing, 14 = Existing
-
> 2 = Existing

0.600

0.534169696552367

5 = Existing, 14 = Existing
-
> 2 = E
xisting

0.415

0.530309807890537

17 = Existing, 12 = Existing
-
> 3 = Existing

0.593

0.530274970380233

5 = Existing, 3 = Existing
-
> 2 = Existing

0.592

0.525984677071341

17 = Existing, 6 = Existing
-
> 2 = Existing

0.574

0.520519027893504

10 = Existing, 3 = E
xisting
-
> 2 = Existing

0.575

0.518527084083734

10 = Existing, 11 = Existing
-
> 2 = Existing

0.577

0.51820358688152

5 = Existing, 11 = Existing
-
> 2 = Existing

0.576

0.517533556400128

10 = Existing, 14 = Existing
-
> 2 = Existing

0.565

0.507430572974396

5 =

Existing, 7 = Existing
-
> 2 = Existing

0.558

0.507206501207268

10 = Existing, 4 = Existing
-
> 2 = Existing

0.565

0.507015160391428

10 = Existing, 8 = Existing
-
> 2 = Existing

0.553

0.499731366513955

5 = Existing, 4 = Existing
-
> 2 = Existing

0.545

0.49617
1126456774

10 = Existing, 12 = Existing
-
> 2 = Existing

0.550

0.494062123729928

5 = Existing, 8 = Existing
-
> 2 = Existing

0.548

0.492076246623159

17 = Existing, 7 = Existing
-
> 2 = Existing

0.546

0.49190544430888

15 = Existing, 12 = Existing
-
> 2 = Existi
ng

0.537

0.48759255191906

11 = Existing, 7 = Existing
-
> 2 = Existing

0.538

0.486285385861869

15 = Existing, 4 = Existing
-
> 2 = Existing

0.529

0.484089865561407

11 = Existing, 3 = Existing
-
> 2 = Existing

0.528

0.478274884309842

5 = Existing, 15 = Existin
g
-
> 2 = Existing

0.526

0.475881422900291

15 = Existing, 14 = Existing
-
> 2 = Existing

0.648

0.474585922690253

15 = Existing, 10 = Existing
-
> 6 = Existing

0.520

0.474255030785549

10 = Existing, 7 = Existing
-
> 2 = Existing

0.646

0.471079495163187

15 = Exi
sting, 8 = Existing
-
> 6 = Existing

0.641

0.46894257911987

5 = Existing, 15 = Existing
-
> 6 = Existing

0.509

0.464828230735576

7 = Existing, 3 = Existing
-
> 2 = Existing

0.511

0.4628078375154

11 = Existing, 8 = Existing
-
> 2 = Existing

0.626

0.458333324810
928

5 = Existing, 7 = Existing
-
> 6 = Existing

0.498

0.457447252141364

11 = Existing, 12 = Existing
-
> 2 = Existing

0.497

0.455806464248229

15 = Existing, 1 = Existing
-
> 2 = Existing

0.484

0.445119907211272

11 = Existing, 4 = Existing
-
> 2 = Existing

0.48
6

0.444850910248928

11 = Existing, 14 = Existing
-
> 2 = Existing

0.486

0.444499151373064

5 = Existing, 6 = Existing
-
> 2 = Existing

0.477

0.440016825255943

5 = Existing, 1 = Existing
-
> 2 = Existing

0.481

0.437106724598724

17 = Existing, 11 = Existing
-
> 2

= Existing

0.469

0.434089193330301

12 = Existing, 3 = Existing
-
> 2 = Existing

0.460

0.433968833865412

10 = Existing, 1 = Existing
-
> 2 = Existing

0.477

0.433909743979988

15 = Existing, 11 = Existing
-
> 2 = Existing

0.473

0.432282262506959

17 = Existing,
1 = Existing
-
> 2 = Existing

0.470

0.430807772780086

7 = Existing, 12 = Existing
-
> 2 = Existing

0.472

0.429102874422352

15 = Existing, 10 = Existing
-
> 2 = Existing

0.578

0.426217858603255

10 = Existing, 7 = Existing
-
> 6 = Existing

0.441

0.41905721773216
4

3 = Existing, 1 = Existing
-
> 2 = Existing

0.459

0.417562463108212

15 = Existing, 3 = Existing
-
> 2 = Existing

0.567

0.415241549876735

15 = Existing, 11 = Existing
-
> 6 = Existing

0.815

0.412440202647036

17 = Existing, 12 = Existing
-
> 1 = Existing

0.815

0.411974749201938

17 = Existing, 10 = Existing
-
> 1 = Existing

0.450

0.411363941221313

14 = Existing, 3 = Existing
-
> 2 = Existing

0.447

0.410094421249623

11 = Existing, 6 = Existing
-
> 2 = Existing

0.451

0.409314973894458

8 = Existing, 12 = Existing
-
> 2

= Existing

0.808

0.408426638980129

17 = Existing, 14 = Existing
-
> 1 = Existing

0.423

0.403554527848069

4 = Existing, 1 = Existing
-
> 2 = Existing

0.551

0.402398845152016

5 = Existing, 10 = Existing
-
> 6 = Existing

0.433

0.397468181591547

10 = Existing, 6

= Existing
-
> 2 = Existing

0.437

0.395704490354349

8 = Existing, 3 = Existing
-
> 2 = Existing

0.769

0.392576029188612

11 = Existing, 12 = Existing
-
> 1 = Existing

0.754

0.388084059101318

11 = Existing, 2 = Existing
-
> 1 = Existing

0.766

0.386908086606112

17 = Existing, 2 = Existing
-
> 1 = Existing

0.422

0.383829351822425

17 = Existing
-
> 2 = Existing

0.420

0.382431919688479

12 = Existing, 14 = Existing
-
> 2 = Existing

0.760

0.382234679518967

17 = Existing, 4 = Existing
-
> 1 = Existing

0.759

0.3808399628317
49

17 = Existing, 7 = Existing
-
> 1 = Existing

0.415

0.378303850775411

12 = Existing, 4 = Existing
-
> 2 = Existing

0.749

0.377287878907374

10 = Existing, 14 = Existing
-
> 1 = Existing

0.413

0.375634072273449

4 = Existing, 3 = Existing
-
> 2 = Existing

0.740

0.374121081330695

11 = Existing, 14 = Existing
-
> 1 = Existing

0.745

0.37353353296156

17 = Existing, 3 = Existing
-
> 1 = Existing

0.400

0.373422120101471

11 = Existing, 1 = Existing
-
> 2 = Existing

0.511

0.371134109880431

11 = Existing, 7 = Existing
-
> 6
= Existing

0.735

0.370206401448224

10 = Existing, 11 = Existing
-
> 1 = Existing

0.409

0.368959785088052

7 = Existing, 8 = Existing
-
> 2 = Existing

0.407

0.364919480300056

15 = Existing, 7 = Existing
-
> 2 = Existing

0.503

0.362532351639045

15 = Existing, 4
= Existing
-
> 6 = Existing

0.719

0.359210212747789

17 = Existing, 11 = Existing
-
> 1 = Existing

0.497

0.359155331959239

7 = Existing, 3 = Existing
-
> 6 = Existing

0.716

0.358600754143127

11 = Existing, 7 = Existing
-
> 1 = Existing

0.481

0.356665897477577

1
5 = Existing
-
> 6 = Existing

0.497

0.356416536625476

5 = Existing, 8 = Existing
-
> 6 = Existing

0.708

0.35434179098766

7 = Existing, 12 = Existing
-
> 1 = Existing

0.492

0.352393958010775

15 = Existing, 14 = Existing
-
> 6 = Existing

0.704

0.348526911831904

17 = Existing, 6 = Existing
-
> 1 = Existing

0.701

0.347702569431084

5 = Existing, 14 = Existing
-
> 1 = Existing

0.484

0.345967240644566

15 = Existing, 12 = Existing
-
> 6 = Existing

0.683

0.343433718984109

10 = Existing, 2 = Existing
-
> 1 = Existing

0.690

0
.343068223089377

10 = Existing, 12 = Existing
-
> 1 = Existing

0.477

0.340505823232426

15 = Existing, 3 = Existing
-
> 6 = Existing

0.683

0.340059315726443

17 = Existing
-
> 1 = Existing

0.662

0.332173058785483

12 = Existing, 2 = Existing
-
> 1 = Existing

0.67
1

0.329296723256298

5 = Existing, 11 = Existing
-
> 1 = Existing

0.667

0.328587987835498

10 = Existing, 3 = Existing
-
> 1 = Existing

0.671

0.3283574436998

5 = Existing, 10 = Existing
-
> 1 = Existing

0.662

0.325495650023565

11 = Existing, 3 = Existing
-
> 1 =

Existing

0.664

0.323900059355776

5 = Existing, 12 = Existing
-
> 1 = Existing

0.459

0.322270106829386

10 = Existing, 8 = Existing
-
> 6 = Existing

0.650

0.322253832394862

7 = Existing, 2 = Existing
-
> 1 = Existing

0.457

0.321551042682991

5 = Existing, 11 =
Existing
-
> 6 = Existing

0.645

0.313326320256791

5 = Existing, 2 = Existing
-
> 1 = Existing

0.413

0.310718700829226

7 = Existing
-
> 6 = Existing

0.639

0.310630423584795

11 = Existing, 4 = Existing
-
> 1 = Existing

0.632

0.309383873324641

14 = Existing, 2 =
Existing
-
> 1 = Existing

0.440

0.306161112854438

15 = Existing, 2 = Existing
-
> 6 = Existing

0.635

0.305736520057593

10 = Existing, 7 = Existing
-
> 1 = Existing

0.619

0.294463352587422

7 = Existing, 3 = Existing
-
> 1 = Existing

0.616

0.294361519418997

12 =

Existing, 14 = Existing
-
> 1 = Existing

0.427

0.292769853676367

15 = Existing, 1 = Existing
-
> 6 = Existing

0.615

0.292388833151127

10 = Existing, 4 = Existing
-
> 1 = Existing

0.602

0.290449633894711

4 = Existing, 2 = Existing
-
> 1 = Existing

0.426

0.2895
61431493487

5 = Existing, 14 = Existing
-
> 6 = Existing

0.423

0.287429199008893

5 = Existing, 12 = Existing
-
> 6 = Existing

0.608

0.285032482131662

5 = Existing, 7 = Existing
-
> 1 = Existing

0.594

0.283744406847867

2 = Existing, 6 = Existing
-
> 1 = Existin
g

0.606

0.283609952584792

15 = Existing, 12 = Existing
-
> 1 = Existing

0.416

0.280172977286027

5 = Existing, 3 = Existing
-
> 6 = Existing

0.598

0.277115144058354

16 = Existing
-
> 1 = Existing

0.568

0.276045690873227

11 = Existing
-
> 1 = Existing

0.413

0.27
5683650588897

11 = Existing, 8 = Existing
-
> 6 = Existing

0.581

0.273790798735467

3 = Existing, 2 = Existing
-
> 1 = Existing

0.408

0.271735426112776

5 = Existing, 4 = Existing
-
> 6 = Existing

0.587

0.269273741183301

15 = Existing, 14 = Existing
-
> 1 = Exis
ting

0.579

0.267798904848947

12 = Existing, 3 = Existing
-
> 1 = Existing

0.403

0.267335956879092

10 = Existing, 11 = Existing
-
> 6 = Existing

0.570

0.259445319791528

11 = Existing, 6 = Existing
-
> 1 = Existing

0.572

0.259306662390124

5 = Existing, 3 = Exis
ting
-
> 1 = Existing

0.569

0.25824395506611

14 = Existing, 3 = Existing
-
> 1 = Existing

0.558

0.248121215448467

5 = Existing, 4 = Existing
-
> 1 = Existing

0.550

0.244784480880925

10 = Existing, 6 = Existing
-
> 1 = Existing

0.540

0.241021080684457

7 = Exist
ing, 4 = Existing
-
> 1 = Existing

0.543

0.237481355859018

15 = Existing, 2 = Existing
-
> 1 = Existing

0.519

0.229965167598536

10 = Existing
-
> 1 = Existing

0.527

0.224043387531083

5 = Existing, 6 = Existing
-
> 1 = Existing

0.524

0.222759774674969

7 = Exist
ing, 14 = Existing
-
> 1 = Existing

0.524

0.220546710855641

15 = Existing, 4 = Existing
-
> 1 = Existing

0.524

0.22032316150274

15 = Existing, 11 = Existing
-
> 1 = Existing

0.515

0.215564839261223

12 = Existing, 4 = Existing
-
> 1 = Existing

0.506

0.204460739
426567

15 = Existing, 8 = Existing
-
> 1 = Existing

0.494

0.199905285216233

5 = Existing
-
> 1 = Existing

0.491

0.193845903508462

14 = Existing, 4 = Existing
-
> 1 = Existing

0.488

0.191242452153328

12 = Existing, 6 = Existing
-
> 1 = Existing

0.488

0.18947514
2889451

5 = Existing, 15 = Existing
-
> 1 = Existing

0.483

0.184810183943221

11 = Existing, 8 = Existing
-
> 1 = Existing

0.481

0.182812712185016

5 = Existing, 8 = Existing
-
> 1 = Existing

0.479

0.18134407731502

10 = Existing, 8 = Existing
-
> 1 = Existing

0.
452

0.1720176368158

7 = Existing
-
> 1 = Existing

0.465

0.168703300778248

15 = Existing, 10 = Existing
-
> 1 = Existing

0.426

0.163016264487586

2 = Existing
-
> 1 = Existing

0.458

0.162447912780458

14 = Existing, 6 = Existing
-
> 1 = Existing

0.443

0.147290109
76805

15 = Existing, 7 = Existing
-
> 1 = Existing

0.437

0.142416011672849

4 = Existing, 3 = Existing
-
> 1 = Existing

0.438

0.142172011669273

15 = Existing, 3 = Existing
-
> 1 = Existing

0.426

0.130050191016598

8 = Existing, 12 = Existing
-
> 1 = Existing