The BizTalk Benchmark Wizard (BBW) is a tool to validate a BizTalk installation and was built by Mikael Håkansson and Ewan Fairweather. It is intended to verify your BizTalk installation and is a useful tool to prove the performance characteristics prior to deploying the first solutions. You may also want to use it before you are about to scale out your environment, to make sure you are really using all resources, before investing in additional hardware and licenses. The BBW performs load to BizTalk Server in relation to specific scenarios. During the execution of the test, counter information is collected and benchmarked against collected statistics relevant to your BizTalk Server environment.
Microsoft provides guidance on performance optimization through the performance optimization guide that provides in-depth information for optimizing the performance of a BizTalk Server.
Note
Even though the BizTalk Server 2010 Optimization Guide (http://www.microsoft.com/download/en/details.aspx?id=10855) is very useful while designing a BizTalk system, it does not provide any expected performance numbers related to specific environments.
This is where the BBW comes into the picture. After having configured the BizTalk group and applied some of the best practices by using the BPA, you can test your BizTalk configuration.
The tool is not a load or analyzing tool; it doesn't give advice or hints like the BPA. However, if the test fails, you can analyze the data using the PAL tool. You can run the tool on either a single-server or a multi-server installation. Regardless of the number of BizTalk Servers in your group, you should not run it with more than two "active" servers (http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=2290), as it will otherwise not be covered by the benchmark values.
Note
This tool should be used after the BizTalk Server has been installed and before any solutions are deployed to the environment. This will ensure that you are getting consistent and clean results from the BBW.
The latest version of the Benchmark Wizard tool can be obtained from CodePlex (http://bbw.codeplex.com/) and must be installed on the BizTalk machine. Follow the steps given next:
In a browser, such as Internet Explorer, navigate to the BBW download location (http://bbw.codeplex.com/).
On the BBW download page, download and run the self-extracting file.
Open a command prompt window and navigate to
[installation folder]\Artefacts\BizTalk. By default, the installation folder isC:\Program Files\Blogical\BizTalk Benchmark Wizard.In this folder, you will find the
InstallHosts.vbsfile. Execute it using the following parameters:UserName: The name of the user account running the service instancesPassword: The password of the user account running the service instancesReceive Host: The name of the server where you want to run the receive host instanceSend Host: The name of the server where you want to run the send host instanceProcessing Host: The name of the server where you want to run the process host instance
If you have a single box installation, your script command might look similar to the following:
cscript InstallHosts.vbs "BizTalk Application Users" "\MyUser" "MyPassword" "BtsServer1" "BtsServer1" "BtsServer1"The result will be as depicted in the following diagram:
If you have a multi-server installation, your script command might look similar to the following:
cscript InstallHosts.vbs "MyDomain\BizTalk Application Users" "MyDomain\MyUser" MyPassword" "BtsServer1" "BtsServer2" "BtsServer2"The result will be as depicted in the following diagram:
Execution of the script in a command line will result in creation of the hosts and the result of these actions is displayed in the following screenshot:
Open the BizTalk Administration Console, point to the Applications node and import the
BizTalk Benchmark Wizard.msifound in the folder[Installation folder]\Artefacts\BizTalk.Run the
BizTalk Benchmark Wizard.msion all BizTalk servers to add the assemblies to the Global Assembly Cache (GAC).Go to Start | All Programs | BizTalk Application Wizard to start the application.
When the installation of the BBW is successful, you can start it from C:\Program Files\Blogical\BizTalk Benchmark Wizard and follow the steps given next:
You will be welcomed first and then you will have to set up some prerequisites such as the BizTalk Management Database:
The user can opt to collect data for further analysis but this is optional. Collection of datasets enables the creation of a log file that contains all information of a Benchmark run. This log can be analyzed later using, for instance, the PAL tool:
Then the user gets to select one of the two scenarios—Messaging or Orchestration. Each scenario has a set of tested environments as follows:
Single server (2*Quad CPU, 4GB RAM)
1*BTS (1*Quad CPU. 4GB RAM) + 1*SQL(1*Quad CPU, 8GB RAM)
2*BTS (2*Quad CPU. 8GB RAM) + 2*SQL(2*Quad CPU, 16GB RAM)
The user can select the environment that best resembles his own environment. The next screen will display what will happen given the chosen scenario:
As a user, you will have to select where each host instance resides on each machine, if applicable. In case of a single-server installation, all the hosts are on the same machine:
One of the following steps is to configure the Indigo Service, a console application hosting service which will be called from the BizTalk Send port. You will have to either host the service on a separate server or go to the folder
[Installation folder]\Artefacts\IndigoServiceand then right-click IndigoService. In the BBW, you will have to either follow instructions and/or fill the Server name and click Test the Indigo Service . There is also the option of collecting performance counters:
As the user clicks Run Test, the tool continues to start ports and orchestrations. It will also start the Perfmon collector sets if the user has chosen to create those (refer to step 2).
As the test proceeds, the user can monitor the counter values through the gauges (Avg Received msgs/sec and Avg Processed msgs/sec). The default test duration is 30 minutes, with a warm-up of two minutes. In this case, five minutes has been chosen.
After warming up, the test will run and you will see the gauges moving certain values.
Finally, the user is presented with a result, which is either Succeeded, Acceptable, or Failed. After the test is run, you can save a report:
By completing the BizTalk Benchmark wizard installation, the following artifacts have been created:
Three hosts –
BBW_RxHost,BBW_PxHost,BBW_TxHostThree host instances
Two adapter handlers for NetTcp
One BizTalk application
Two Receive ports
Two Send hosts
One Orchestration
Hosts and instances are created through scripts and ports, orchestration is created through installing the msi files. The user can start the wizard and, at the end, run a test based on the selections made. During the test, loadgen will generate the xml messages that will be sent over NetTcp to BizTalk. The messages will be published in the MessageBox database and picked up either by the subscribing send port or orchestration. Finally, the messages will be sent to the backend web service through the WCF-NetTcp adapter.
The following section deals with KPIs in detail for certain scenarios.
The test run failed as it did not meet the KPIs, at least not all of them. Meeting the KPIs is not easy. The following is the list of KPIs for each scenario. They are based on separate machines for BizTalk and the SQL Server instance, Intel Xeon CPUs with multiple cores, x64 bit platforms and software (refer to the Test Environment table).
KPIs for the scenario of Messaging a single-message and multi-message box are stated in the following table:
|
Number of BTS |
CPU BizTalk Server |
Number of SQL Servers |
CPU SQL Server |
Msg/Sec Received |
Msg/Sec Processed |
|---|---|---|---|---|---|
|
1 |
1 Quad |
1 |
1 Quad |
160 |
200 |
|
1 |
1 Quad |
1 |
1 Quad |
280 |
350 |
|
1 |
1 Quad |
1 |
2 Quads |
390 |
490 |
|
1 |
1 Quad |
1 |
2 Quads |
560 |
700 |
|
2 |
1 Quad |
1 |
2 Quads |
620 |
770 |
|
2 |
2 Quads |
1 |
2 Quads |
730 |
910 |
|
2 |
2 Quads |
1 |
4 Quads |
780 |
980 |
KPIs for the scenario of an orchestration single-message box are stated in the following table:
|
Number of BTS |
CPU BizTalk Server |
Number of SQL Servers |
CPU SQL Server |
Msg/Sec Received |
Msg/Sec Processed |
|---|---|---|---|---|---|
|
1 |
1 Quad |
1 |
1 Quad |
110 |
140 |
|
1 |
1 Quad |
1 |
1 Quad |
170 |
210 |
|
1 |
1 Quad |
1 |
2 Quads |
190 |
240 |
|
1 |
1 Quad |
1 |
2 Quads |
220 |
270 |
|
2 |
1 Quad |
1 |
2 Quads |
230 |
290 |
|
2 |
2 Quads |
1 |
2 Quads |
260 |
320 |
|
2 |
2 Quads |
1 |
4 Quads |
300 |
370 |
Each has a Windows 2008 Enterprise x64 Edition as the operating system, with 64 bit CPUs having four cores:
|
Type |
Model |
CPU Type |
Number of CPU |
Logical Disks |
Software |
|---|---|---|---|---|---|
|
Database |
DL875 |
Intel Xeon |
8 * 2,4 Ghz |
2 x 72gb 10k* |
SQL Server 2008 SP1 |
|
BTS Host Receive |
R805 |
Intel Xeon |
2 * 2,33 Ghz |
2 x 72gb 10k SAS |
BizTalk Server 2009 |
|
BST Host Send |
R805 |
Intel Xeon |
2 * 2,33 Ghz |
2 x 72gb 10k SAS |
BizTalk Server 2009 |
|
Load |
R805 |
Intel Xeon |
2 * 2,33 Ghz |
2 x 72gb 10k SAS |
BizTalk Benchmark Wizard |
|
Backend |
R805 |
Intel Xeon |
2 * 2,33 Ghz |
2 x 72gb 10k SAS |
Indigo Service |
Following is the configuration of the test environment for Storage: EMC Clarion CX-240 (five solid state drives):
Global tracking enabled
Partitioning of the
TempDb(SQL Server system database),BizTalkDTADbandBizTalkMsgBoxDbto as many files as CPUsSeparation of the BizTalk MessageBox database into multiple file groups or files
Disabled throttling on send and processing hosts
Updated the thread settings (CLR Hosting) on all hosts (there you will find the update install path of
BBW artefacts\registry settings)