BizUnit - Framework for Automated Testing of Distributed Systems

New Post: BizUnit 3.0.1026.0 (3.0 Beta 2): XmlValidationStep.ExecuteValidation(Stream, Context) throws NullReferenceException

crestall — Tue, 01 Jul 2008 23:41:20 GMT

One more addition to this one, if you intend to use ContextLoader in the FileValidationStep, you'll need to modify the following as well in this file:

if (null != this.contextLoaderStep)
{
this.contextLoaderStep.ExecuteContextLoader(data, context);
}

To:

if (null != this.contextLoaderStep)
{
this.contextLoaderStep.ExecuteContextLoader(data, this.contextConfig, context);
}

New Post: BizUnit 3.0.1026.0 (3.0 Beta 2): XmlValidationStep.ExecuteValidation(Stream, Context) throws NullReferenceException

jmoons — Mon, 30 Jun 2008 16:50:13 GMT

Thanks, for saving me the debug time! I am using 3.0.1027.0 and that fixed it for me as well.

New Post: BizUnit 3.0.1026.0 (3.0 Beta 2): XmlValidationStep.ExecuteValidation(Stream, Context) throws NullReferenceException

SirEel — Tue, 24 Jun 2008 04:53:59 GMT

Correction, that was Build 3.0.1027.0. Same bug AFAIK.

New Post: BizUnit 3.0.1026.0 (3.0 Beta 2): XmlValidationStep.ExecuteValidation(Stream, Context) throws NullReferenceException

SirEel — Tue, 24 Jun 2008 04:48:36 GMT

I modified FileValidateStep.cs by adding two private fields on line 97:

private XmlNode validationConfig;
private XmlNode contextConfig;

and then changing these lines at 168:

validationConfig = testConfig.SelectSingleNode("ValidationStep");
contextConfig = testConfig.SelectSingleNode("ContextLoaderStep");

and then changing line 219:

if (null != this.validationStep)
{
this.validationStep.ExecuteValidation(data, this.validationConfig, context);
}

I also had to add this to XmlValidationStep.cs line 115:

if (this.xPathValidations == null)
{
this.xPathValidations = new List<Pair>();
}

Don't know if this is what Kevin had in mind, but it makes FileValidationStep work in Build 3.0.1026.0

COMMENTED ISSUE: WSDL import on SOAPHTTPRequestResponseStep

tanveerrashid — Sun, 08 Jun 2008 09:48:38 GMT

Hi,

I'm not sure if this is a bug or not but wanted confirmation. I have a schema published as a web service using the BizTalk WSE Publishing Wizard. I've created a SOAPHTTPRequestResponseStep to test this but get the error System.InvalidOperationException: Unable to import operation 'RatePropertyQuote'. ---> System.InvalidOperationException: The element 'http://www.polaris-uk.co.uk/Schema/6/PropertyOwnersSmallNBRq:PropertyOwnersSmallNBRq' is missing.

If I browse to the WSDL itself the WSDL includes the following
<wsdl:message name="RatePropertyQuoteMessageIn">
<wsdl:part name="part" xmlns:q1="http://www.polaris-uk.co.uk/Schema/6/PropertyOwnersSmallNBRq" element="q1:PropertyOwnersSmallNBRq">
<wsdl:documentation xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/">BizTalk part types: BizTalk part type (TypeName="Broker.Polaris.MsgSchemaPropertyOwnersSmallNBRq", AssemblyName="EProgramme.Schemas, Version=1.0.0.0, Culture=neutral, PublicKeyToken=b8f6ef39775924f1", TargetNamespace="http://www.polaris-uk.co.uk/Schema/6/PropertyOwnersSmallNBRq", RootName="PropertyOwnersSmallNBRq")</wsdl:documentation>
</wsdl:part>
</wsdl:message>

This mentions the element type that error says can't be found. Also in the WSDL is the following:

- <s:schema targetNamespace="http://Broker.Quotes/Imports">
<s:import schemaLocation=".\xsd\Broker.Polaris.MsgSchemaPropertyOwnersSmallNBRq_0x6a6dfb34.xsd" namespace="http://www.polaris-uk.co.uk/Schema/6/PropertyOwnersSmallNBRq" />
<s:import schemaLocation=".\xsd\Broker.Polaris.MsgSchemaPropertyOwnersSmallNBRs_0x7f98a7a8.xsd" namespace="http://www.polaris-uk.co.uk/Schema/4/PropertyOwnersSmallNBRs" />
</s:schema>
</wsdl:types>

The first of these import xsd declarations contain the actual definition of the element mentioned in the error. The question I therefore have is does BizUnit follow these import definitions to obtain the complete WSDL as it appears that it is not doing this and therefore causing the error I'm seeing. As far as I'm aware, all web services exposed by the BizTalk WSE publishing wizard declare thier WSDLs in this manner using imports and so if this is the case the BizUnit SOAPHTTPRequestResponseStep cannot be used for testing any BizTalk WSE Adapter published web services.
Comments: ** Comment from web user: tanveerrashid **

There is potentially another solution - using the Discovery Protocol.

A configuration flag could be defined which determines if the regular "wsdlImport.AddServiceDescription" should be used (for dynamically generating the proxy assembly) or if the discovery protocol should be used.

Code similar to

if (useDiscoveryProtocol)
{
DiscoveryClientProtocol discoveryProtocol = new DiscoveryClientProtocol();
discoveryProtocol.DiscoverAny(wsdlURI);
discoveryProtocol.ResolveAll();
foreach (DictionaryEntry entry in discoveryProtocol.References)
{
SchemaReference schemaRef = entry.Value as SchemaReference;

if (schemaRef != null)
wsdlImport.Schemas.Add(schemaRef.Schema);

ContractReference contractRef = entry.Value as ContractReference;

if (contractRef != null)
wsdlImport.ServiceDescriptions.Add(contractRef.Contract);
}
}
else
{
wsdlImport.AddServiceDescription(wsdl, null, null);
}

COMMENTED ISSUE: WSDL import on SOAPHTTPRequestResponseStep

tanveerrashid — Sun, 08 Jun 2008 09:43:28 GMT

There is potentially another solution - using the Discovery Protocol. A configuration flag could be defined which determines if the regular wsdlImport.AddServiceDescription is used (for dynamically generating the proxy assembly) or if the discovery protocol should be used.

Code similar to

if (useDiscoProtocol)
{
DiscoveryClientProtocol discoProtocol = new DiscoveryClientProtocol();
discoProtocol.DiscoverAny(wsdlURI);
discoProtocol.ResolveAll();
foreach (DictionaryEntry entry in discoProtocol.References)
{
SchemaReference schemaRef = entry.Value as SchemaReference;

if (schemaRef != null)
wsdlImport.Schemas.Add(schemaRef.Schema);

ContractReference contractRef = entry.Value as ContractReference;

if (contractRef != null)
wsdlImport.ServiceDescriptions.Add(contractRef.Contract);
}
}
else
{
wsdlImport.AddServiceDescription(wsdl, null, null);
}

CREATED ISSUE: TestStepBuilder.ContextLoaderStepBuilder - missing?

tanveerrashid — Sun, 08 Jun 2008 09:37:18 GMT

It seesm that TestStepBuilder is missing a definition for a ContextLoaderStepBuilder property?

This similar to the ValidationStep & ValidationStepBuilder properties.

There is a ContextLoaderStep, but no ContextLoaderStepBuilder defined.

Could we include a definition for this?

NEW POST: Property ContextLoaderStep on FileValidateStep is not optional

jsilveira — Tue, 27 May 2008 17:51:07 GMT

I have the same problem and had to recompile bizunit with the same workaround.

Is this a bug that will get fixed or is there something we don't know?

Regards

UPDATED WIKI: Home

kevinsmi — Mon, 19 May 2008 22:06:14 GMT

BizUnit

The adoption of an automated testing strategy is fundamental in reducing the risk associated with software development projects, it is key to ensuring that you deliver high quality software. Often, the overhead associated with developing automated tests is seen as excessive and a reason to not adopt automated testing.

BizUnit enables automated tests to be rapidly developed. BizUnit is a flexible and extensible declarative test framework targeted that rapidly enables the automated testing of distributed systems, for example it is widely used to test BizTalk solutions. BizUnit is fully extensible. Its approach is to enable test cases to be constructed from generic reusable test steps, test cases are defined in XML which allows them to be auto-generated and also enables the ‘fixing up’ of Url’s for different environments, e.g. test, staging and production environments. Defining test cases in XML enables test cases to be auto-generated.

BizUnit v3.0 - Beta 3 - now available, new features include a BizUnit object, Excel test case runner and a new UI automation test step library

BizUnit is a framework and as such does not have any dependency on either NUnit of VS Unit Testing, either of these make a great way to drive BizUnit test cases, though equally you could write custom code to do the same.

Test Case Format

A test case can be represented as an Xml document or using the BizUnit object model, it is made up of three stages, test setup, test execution and test cleanup, the cleanup stage is always executed (even if the main execution stage fails) and intended to leave the platform in the same state that it started.
Each stage may consist of zero or more test steps, test steps are in general autonomous, state can be flowed between them if required using the ‘context’ object that is passed to each test step by the framework.
BizUnit also has the notion of TestGroupSetup and TestGroupTearDown, these are test cases that are executed at the beginning and end of a suite of unit tests.

The diagram below illustrates the format of a test case.

In addition to test steps, BizUnit has the notion of validation steps and context loader steps. These can be thought of as sub-steps and can in general be independantly executed from any test step. For example, an MSMQ-read step might be used to read and validate both Xml and Flat File data from a queue, the same step can be used with both the RegExValidationStep and the XmlValidationStep to validate the data read.

A test step within a test case can be marked with the attribute - runConcurrently which causes subsequent test steps to be started before it has completed. In addition test steps maybe marked with the attribute - failOnError, setting it to false cause BizUnit to ignore a failure of that test step, this is particularly useful for the setup and cleanup stages of test cases.

Lets look at an Example Scenario...

BizUnit takes a black box approach to testing solutions, lets look at the BizTalk scenario below, a BizTalk solution receives a request-response message over HTTP, the message is routed to an Orchestration which, sends a message to MSMQ and another to a FILE drop, the Orchestration waits for a FILE to be received, after which the Orchestration sends the response back to the waiting HTTP client. The solution also uses BAM, writing business data to the BAM database.

In order to test this scenario, a BizUnit test case is defined that has 5 test steps:

The HttpRequestResponseStep sends the request to the two-way receive port and waits for the response. This step is executed concurrently so that the other test steps may execute whilst it waiting for the response
The MSMQReadStep waits for a message to appear on an MSMQ queue.

When it reads the message it uses the XmlValidationStep to perform schema validation and also execute a number of XPath expression to ensure the message contains the correct data

The FileValidateStep waits for a FILE to be written to a given directory

When it reads the FILE it validates the data using the RegExValidationStep validation step since the FILE picked up was a flat file format

The FileCreateStep creates a new FILE in the specified directory containing the data that the backend system would typically create. This allows the Orchestration to complete and send the response back to the waiting HttpRequestResponseStep step
Finally, DBQueryStep is used to check that all of the BAM data has been successfully written to the BAMPrimaryImportDB

Test Steps

A test step is a .NET class which implements the ITestStep interface:

public interface ITestStep
{
    void Execute(XmlNode testConfig, Context context);
}

BizUnit will create and execute the test steps as dictated by the Xml test case. The test case will list the steps that need to be excuted in each stage of the test. The example below will cause BizUnit to create the Microsoft.Services.BizTalkApplicationFramework.BizUnit.FileCreateStep. BizUnit uses the assemblyPath and typeName to load and create the type:

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.FileCreateStep">
    ...
</TestStep>

Executing Steps Concurrently

Test steps can be maked to execute concurrently by decorating them with the runConcurrently attribute:

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.HttpRequestResponseStep" runConcurrently="true">
    ...
</TestStep>

Reading Configuration from Context

BizUnit enables state to be flowed between test steps using the Context object, subsequent steps may read configuration from the Context object which was written by a previous test step, this is acheived using the takeFromCtx attribute within a steps configuration. For example:

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.HttpPostStep">
    <SourcePath>.\TestData\InDoc1.xml</SourcePath>    
    <DestinationUrl takeFromCtx="HTTPDest">http://localhost/TestFrameworkDemo/BTSHTTPReceive.dll</DestinationUrl>
    <RequestTimeout>60000</RequestTimeout>
</TestStep>

Wild Cards

BizUnit supports wild card for reading configuration, the following wild cards are supported

%DateTime% - will replace the wild card with the current date time in the format HHmmss-ddMMyyyy
%ServerName% - will replace the wild card with the name of the server BizUnit is being executed on
%Guid% - will be replaced by a new Guid

For example, for the test step configuration below:

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.FileCreateStep">
    <SourcePath>..\..\..\TestData\InDoc1.xml</SourcePath>         
    <CreationPath>..\..\..\Rec_03\TransactionId_%Guid%_%ServerName%.xml</CreationPath>
</TestStep>

CreationPath becomes "..\..\..\Rec03\TransactionId12345678-D6AB-4aa9-A772-938972E3FD51ZEUS001.xml"_

Validation Steps

BizUnit supports the notion of validation steps which may be nested within test steps which support validation. This means that an MSMQ read step may use an XML validation step or a regular expression validation step to validate the data that it receives. Validation steps need to implement the IValidationStep interface.

public interface IValidationStep
{
    void ExecuteValidation(Stream data, XmlNode validatorConfig, Context context);
}

A test step may use the Context object utilities to execute the appropriate validation step as shown below:

public void Execute(XmlNode testConfig, Context context)
{
    XmlNode validationConfig = testConfig.SelectSingleNode("ValidationStep");
    MemoryStream data = null;
 
    ...
 
    context.ExecuteValidator( data, validationConfig );
}

The test step snippet below illustrates how a validation step is embeded in a file read step:

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.FileValidateStep">
    <Timeout>3000</Timeout>
    <Directory>..\..\..\Rec_03\</Directory>
    <SearchPattern>TransactionId_*.xml</SearchPattern>
    <DeleteFile>true</DeleteFile>
			
    <ValidationStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.XmlValidationStep">
        <XmlSchemaPath>..\..\..\TestData\PurchaseOrder.xsd</XmlSchemaPath>
        <XmlSchemaNameSpace>http://SendMail.PurchaseOrder</XmlSchemaNameSpace>
        <XPathList>
            <XPathValidation query=
                "/*[local-name()='PurchaseOrder' and namespace-uri()='http://SendMail.PurchaseOrder']
                /*[local-name()='PONumber' and namespace-uri()='']">PONumber_0</XPathValidation>
        </XPathList>
    </ValidationStep>			
</TestStep>

Context Loader Steps

Similarly, BizUnit supports context loader steps, which are responsible for loading data into the BizUnit context. These types of need to implement the IContextLoaderStep interface as shown below:

public interface IContextLoaderStep
{
    void ExecuteContextLoader(Stream data, XmlNode contextConfig, Context context);
}

Getting Started

The best way to get started is to download the latest version, install it and then take a look at the SDK samples. All the source is currently included, so feel free to take a closer look at the code. All the test steps along with the framework itself are documented in the .CHM that is installed.

Finally, my apologies for the shameless plug :-), but Chapter 10 of Professional BizTalk Server 2006 book which I co-authored discusses in more detail how BizUnit may be used as an integral part of your development process, and how it may be used to drive stress and performance testing.

Acknowledgments

I'd like to thanks the follwing people who have contributed to BizUnit in some way, either by donating test steps, identifying bugs, or by providing requirements which have subsequently been implemented. I may have missed some people off, if so, it's not intensional, please drop me a mail to remind me:

Dharshana Kalahejagoda
Jon Fancey
Isaac Young
Mike Becker
Tanveer Rashid
Young Jun Hong
Dave Regan
Ian Cross
Greg Beach
Daren Jefford
Kevin Purcell
Karina Apostolides
Jon Bonnick
Brian Milburn
Rahmatullah Khan

Enjoy!

UPDATED RELEASE: 3.0.1027.0 - Beta 3

Mon, 19 May 2008 21:58:18 GMT

*BizUnit v3.0* - _Beta 3 Release_

This is a major release with some big changes:

*BizUnit Object Model* - firstly BizUnit introduces an object model to enable BizUnit tests to be created programatically. This primarily targets the ability to create test matrix's in Excel and have BizUnit run the spreadsheet directly - no need to serialise out Xml. The SDK has a sample spreadsheet and a generic spreadsheet parser.

*UI Automation Testing* - another powerful feature is a new test step library which enables UI's to be tested by BizUnit, this enables seemless integration of server side and client side test code, truely powerful :-). Special thanks to my colleague Dharshana Kalahejagoda for delivering this excellent functionality.

*Note*: this is still a beta release, not all of the tests steps have been enhanced with the new object model and there are still some know doc issues

Beta 3 fixes some know bugs in previous v3 Beta releases, such as test case false-positives when using the OM and also the default value for failOnError is consistent with the XML tests, i.e. defaults to true.

Enjoy,

*Kevin*

NEW POST: BizUnit 3.0.1026.0 (3.0 Beta 2): XmlValidationStep.ExecuteValidation(Stream, Context) throws Null...

edschweppe — Tue, 13 May 2008 20:31:10 GMT

I'm trying to run a simple proof-of-concept using BizUnit 3.0.1026, and XmlValidationStep.ExecuteValidation(Stream data, Context context) is blowing up. Apparently, this.xPathValidations is only initialized in XmlValidationStep.ExecuteValidation(Stream data, XmlNode validatorConfig, Context context), but that method isn't being called.

Here's the ValidationStep XML:

      <ValidationStep assemblyPath="" typeName="BizUnit.XmlValidationStep">
        <XmlSchemaPath>C:\play\BT_AutoTestingDemo\BT_AutoTestingDemo\Schemas\Member.xsd</XmlSchemaPath>
        <XmlSchemaNameSpace>http://BT_AutoTestingDemo.Member</XmlSchemaNameSpace>
        <XPathList>
          <XPathValidation query="//Member/FirstName">Fred</XPathValidation>
          <XPathValidation query="//Member/LastName">Foobar</XPathValidation>
        </XPathList>
      </ValidationStep>

And here's what's being written to Console.Out:

Info: Adding context property: BizUnitTestCaseName, value: GoodClient

-------------------------------------------------------------------------------
                                   S T A R T

Test: GoodClient started @ 16:25:59.449 13/05/2008 by FHP\sched01
-------------------------------------------------------------------------------
Info: Adding context property: BizUnitTestCaseStartTime, value: 5/13/2008 4:25:59 PM
Info: Adding context property: testData, value: C:\play\BT_AutoTestingDemo\DemoBizUnitTests\bin\Debug\..\..\TestData\ClientToMember_GoodClient_Input.xml
Info: Adding context property: inputPort, value: C:\play\BT_AutoTestingDemo\BT_AutoTestingDemo\Ports\IncomingClient\GoodClient_input.xml
Info: Adding context property: outputPort, value: C:\play\BT_AutoTestingDemo\BT_AutoTestingDemo\Ports\OutgoingGreeting\

Setup Test: GoodClient

Step: BizUnit.FileDeleteMultipleStep started @ 16:25:59.464 13/05/2008, failOnError = True
Info: 1 files were found matching the File Mask: "*.xml" in the directory: "C:\play\BT_AutoTestingDemo\BT_AutoTestingDemo\Ports\OutgoingGreeting\"
Info: File: "C:\play\BT_AutoTestingDemo\BT_AutoTestingDemo\Ports\OutgoingGreeting\{ACFE6A53-F40D-44B6-B6C3-F7CBBA856C4C}.xml" was successfully deleted.
Step: BizUnit.FileDeleteMultipleStep ended @ 16:25:59.464 13/05/2008

Execute Test: GoodClient

Step: BizUnit.FileCreateStep started @ 16:25:59.464 13/05/2008, failOnError = True
Info: FileCreateStep about to copy the data from File: C:\play\BT_AutoTestingDemo\DemoBizUnitTests\bin\Debug\..\..\TestData\ClientToMember_GoodClient_Input.xml to the File: C:\play\BT_AutoTestingDemo\BT_AutoTestingDemo\Ports\IncomingClient\GoodClient_input.xml
Step: BizUnit.FileCreateStep ended @ 16:25:59.480 13/05/2008

Step: BizUnit.FileValidateStep started @ 16:25:59.480 13/05/2008, failOnError = True
Info: Searching for files in: "C:\play\BT_AutoTestingDemo\BT_AutoTestingDemo\Ports\OutgoingGreeting\*.xml"
Info: 1 fies were found at : "C:\play\BT_AutoTestingDemo\BT_AutoTestingDemo\Ports\OutgoingGreeting\*.xml"
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Data: Loaded FILE: C:\play\BT_AutoTestingDemo\BT_AutoTestingDemo\Ports\OutgoingGreeting\{EF6ECC08-25F5-4416-83FD-75C5DDA93C57}.xml
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<ns0:Member xmlns:ns0="http://BT_AutoTestingDemo.Member"><FirstName>Frank</FirstName><LastName>Success</LastName><EmailAddress>frank@example.com</EmailAddress></ns0:Member>
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*******************************************************************************
Error: Exception caught!
System.NullReferenceException: Object reference not set to an instance of an object.
   at BizUnit.XmlValidationStep.ExecuteValidation(Stream data, Context context) in C:\play\BT_AutoTestingDemo\BizUnit 3.0\Src\BizUnit\ValidationSteps\XmlValidationStep.cs:line 151
   at BizUnit.FileValidateStep.Execute(Context context) in C:\play\BT_AutoTestingDemo\BizUnit 3.0\Src\BizUnit\TestSteps\FileValidateStep.cs:line 211
   at BizUnit.FileValidateStep.Execute(XmlNode testConfig, Context context) in C:\play\BT_AutoTestingDemo\BizUnit 3.0\Src\BizUnit\TestSteps\FileValidateStep.cs:line 162
   at BizUnit.BizUnitOM.BizUnitTestStepWrapper.Execute(Context ctx) in C:\play\BT_AutoTestingDemo\BizUnit 3.0\Src\BizUnit\BizUnitOM\BizUnitTestStepWrapper.cs:line 117
   at BizUnit.BizUnit.ExecuteTestStep(BizUnitTestStepWrapper stepWrapper) in C:\play\BT_AutoTestingDemo\BizUnit 3.0\Src\BizUnit\BizUnit.cs:line 1038
*******************************************************************************

Tear Down Test: GoodClient
-------------------------------------------------------------------------------
Test: GoodClient ended @ 16:26:00.105 13/05/2008
                             ****** F A I L ******

-------------------------------------------------------------------------------

Help?

NEW POST: Bug in BizUnit 3.0

edschweppe — Tue, 13 May 2008 15:09:28 GMT

kevinsmi wrote:
This is now fixed in 3.0.1026.0

Note the src is in the MSI, don't sync from codeplex, just download the .MSI

thanks.

Kevin

Ah, that's where to get the code. Thanks for the info (and thanks for the fix!)

NEW POST: Can we check Orchestrational logic with BizUnit?

kevinsmi — Sun, 11 May 2008 21:35:24 GMT

Yes, thats exactly right. In general the approach I've always taken is to vary the inputs into the Orch in order to ensure that all of the execution paths have been exercised. Using BizUnit in conjuction with the Orchestration Profiler to check your code coverage metrics: http://www.codeplex.com/BiztalkOrcProfiler. Also, you can use BizTalk to validate any side effects form Orchs and indeed the rest of the solution, for example did all of your BAM data get correctly recorded in the PrimaryImportDB.

Thanks,

Kevin

NEW POST: Can we check Orchestrational logic with BizUnit?

Data — Sun, 11 May 2008 08:33:17 GMT

Hi,

I think you have to think of the orchestration as a black box, so you have to use BizUnit to make the action (the input data to fire the orchestration) and then you have to use BizUnit again to test the reaction impact on (DB, Files, ERP, etc)

To realized that, you have to use multiple test cases.

Regards,
Firas

COMMENTED ISSUE: Global Variable

Data — Sun, 11 May 2008 07:50:38 GMT

Add the ability to save and retrieve global variables that could be used by any test case during the test session. I’m thinking of this because I think we should divide the full scenario test into many test cases to be run later by BizUnit framework in sequence. That will be easier to modify, enhance, and track the errors.

Let’s assume that we have a very complicated scenario to test, it will be better to divide that full scenario testing into many test cases, like the following:

* ST001_Initialize
* ST002_SetupEnv
* ST003_CheckBusinessOperation1
* ST004_CheckBusinessOperation2
* ST005_CheckBusinessOperation3
* ST006_CheckBusinessOperation4
* ST007_CheckBusinessOperation5
* ST008_CheckBusinessOperation6
* ST009_CheckBusinessOperation7
* ST010_Finalize
* ST011_PrintTestDuration

Let’s assume that we are using a URI like folder path in test steps across the divided test cases. So it will be better to set that global variable to be used later by other test cases.

I’m suggesting to add global variable in initialization file, in the following format.
{
<TestStep assemblyPath=""
typeName="namespace.AddGlobalVariable"
description="Set Start Time"
failOnError="False">
<GlobaleVaiable>
<add name=”Archive.FolderPath”>c:\MySystem\FileDrops\Archives</add>
<add name=”Operation.SummaryReport”>c:\MySystem\FileDrops\Summary</add>
<add name=”DatabaseConnectionStr”>ConnectionString</add>
</GlobaleVaiable>
</TestStep>
}

To be used later as:

{
<TestStep
assemblyPath="
typeName="BizUnit.BizTalkApplicationFramework.BizUnit.WaitOnFileStep">
<Path>%Archive.FolderPath%</Path> // Using Global variable
<FileFilter>*.xml</FileFilter>
<TimeOut>70000</TimeOut>
</TestStep>
}
Comments: ** Comment from web user: Data **

Hi Kevin,

Context object is great and it solves the global variable functionality, that will means that I have to maintain the test context object in .net code, where I think it will be much better to maintain the global variables using the native syntax of BizUnit (xml).

Thanks,
Firas

NEW POST: Automate LoadGen with Bizunit

kevinsmi — Thu, 08 May 2008 10:34:24 GMT

Whats the call stack? and which version of BizUnit are you using?

Kevin

UPDATED WIKI: Home

kevinsmi — Thu, 08 May 2008 08:10:00 GMT

BizUnit

The adoption of an automated testing strategy is fundamental in reducing the risk associated with software development projects, it is key to ensuring that you deliver high quality software. Often, the overhead associated with developing automated tests is seen as excessive and a reason to not adopt automated testing.

BizUnit enables automated tests to be rapidly developed. BizUnit is a flexible and extensible declarative test framework targeted that rapidly enables the automated testing of distributed systems, for example it is widely used to test BizTalk solutions. BizUnit is fully extensible. Its approach is to enable test cases to be constructed from generic reusable test steps, test cases are defined in XML which allows them to be auto-generated and also enables the ‘fixing up’ of Url’s for different environments, e.g. test, staging and production environments. Defining test cases in XML enables test cases to be auto-generated.

BizUnit v3.0 - Beta 2 - now available, new features include a BizUnit object, Excel test case runner and a new UI automation test step library

BizUnit is a framework and as such does not have any dependency on either NUnit of VS Unit Testing, either of these make a great way to drive BizUnit test cases, though equally you could write custom code to do the same.

Test Case Format

Lets look at an Example Scenario...

The HttpRequestResponseStep sends the request to the two-way receive port and waits for the response. This step is executed concurrently so that the other test steps may execute whilst it waiting for the response
The MSMQReadStep waits for a message to appear on an MSMQ queue.

When it reads the message it uses the XmlValidationStep to perform schema validation and also execute a number of XPath expression to ensure the message contains the correct data

The FileValidateStep waits for a FILE to be written to a given directory

When it reads the FILE it validates the data using the RegExValidationStep validation step since the FILE picked up was a flat file format

The FileCreateStep creates a new FILE in the specified directory containing the data that the backend system would typically create. This allows the Orchestration to complete and send the response back to the waiting HttpRequestResponseStep step
Finally, DBQueryStep is used to check that all of the BAM data has been successfully written to the BAMPrimaryImportDB

Test Steps

A test step is a .NET class which implements the ITestStep interface:

public interface ITestStep
{
    void Execute(XmlNode testConfig, Context context);
}

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.FileCreateStep">
    ...
</TestStep>

Executing Steps Concurrently

Test steps can be maked to execute concurrently by decorating them with the runConcurrently attribute:

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.HttpRequestResponseStep" runConcurrently="true">
    ...
</TestStep>

Reading Configuration from Context

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.HttpPostStep">
    <SourcePath>.\TestData\InDoc1.xml</SourcePath>    
    <DestinationUrl takeFromCtx="HTTPDest">http://localhost/TestFrameworkDemo/BTSHTTPReceive.dll</DestinationUrl>
    <RequestTimeout>60000</RequestTimeout>
</TestStep>

Wild Cards

BizUnit supports wild card for reading configuration, the following wild cards are supported

%DateTime% - will replace the wild card with the current date time in the format HHmmss-ddMMyyyy
%ServerName% - will replace the wild card with the name of the server BizUnit is being executed on
%Guid% - will be replaced by a new Guid

For example, for the test step configuration below:

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.FileCreateStep">
    <SourcePath>..\..\..\TestData\InDoc1.xml</SourcePath>         
    <CreationPath>..\..\..\Rec_03\TransactionId_%Guid%_%ServerName%.xml</CreationPath>
</TestStep>

CreationPath becomes "..\..\..\Rec03\TransactionId12345678-D6AB-4aa9-A772-938972E3FD51ZEUS001.xml"_

Validation Steps

public interface IValidationStep
{
    void ExecuteValidation(Stream data, XmlNode validatorConfig, Context context);
}

A test step may use the Context object utilities to execute the appropriate validation step as shown below:

public void Execute(XmlNode testConfig, Context context)
{
    XmlNode validationConfig = testConfig.SelectSingleNode("ValidationStep");
    MemoryStream data = null;
 
    ...
 
    context.ExecuteValidator( data, validationConfig );
}

The test step snippet below illustrates how a validation step is embeded in a file read step:

<TestStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.FileValidateStep">
    <Timeout>3000</Timeout>
    <Directory>..\..\..\Rec_03\</Directory>
    <SearchPattern>TransactionId_*.xml</SearchPattern>
    <DeleteFile>true</DeleteFile>
			
    <ValidationStep assemblyPath="" typeName="Microsoft.Services.BizTalkApplicationFramework.BizUnit.XmlValidationStep">
        <XmlSchemaPath>..\..\..\TestData\PurchaseOrder.xsd</XmlSchemaPath>
        <XmlSchemaNameSpace>http://SendMail.PurchaseOrder</XmlSchemaNameSpace>
        <XPathList>
            <XPathValidation query=
                "/*[local-name()='PurchaseOrder' and namespace-uri()='http://SendMail.PurchaseOrder']
                /*[local-name()='PONumber' and namespace-uri()='']">PONumber_0</XPathValidation>
        </XPathList>
    </ValidationStep>			
</TestStep>

Context Loader Steps

public interface IContextLoaderStep
{
    void ExecuteContextLoader(Stream data, XmlNode contextConfig, Context context);
}

Getting Started

Acknowledgments

COMMENTED ISSUE: Tests pass that should fail

kevinsmi — Wed, 07 May 2008 23:04:29 GMT

I have been migrating some tests from running against BizUnit 2 to BizUnit 3. When I run them I found some instances where tests should have failed which in actual fact passed.

The scenarios are as follows:

1. I had specified the test step name incorrectly and a type load exception was thrown and logged, but the test still passed

2. I had a custom test step which throws an exception yet the test still passes.

I have used reflector to examine the code and i think the problem is in the below code snippet in the BizUnit object.

private void ExecuteSteps(XmlNodeList steps)
{
if (null != steps)
{
foreach (XmlNode node in steps)
{
try
{
BizUnitTestStepWrapper stepWrapper = new BizUnitTestStepWrapper(node);
this.ExecuteTestStep(stepWrapper);
}
catch (Exception exception)
{
this.logger.LogException(exception);
}
}
this.FlushConcurrentQueue(true);
}
}

In this snippet you can see the exception is swallowed up in the catch block.

This is probably a significant defect as it means that anyone migrating their tests cant as they will always pass, and i assume this would also affect new tests in the same way.

Hope this helps and look forward to a patch

Mike
Comments: ** Comment from web user: kevinsmi **

The fix is in the latest version - 3.0.1026.0

thanks,

*Kevin*

COMMENTED ISSUE: Global Variable

kevinsmi — Wed, 07 May 2008 23:03:42 GMT

Hi,

Is there any reason why you can't use the BizUnit context object to flow this? This is what it is for, flowing state between test steps and test cases.

Thanks,

*Kevin*

UPDATED RELEASE: 3.0.1026.0 - Beta 2

Wed, 07 May 2008 23:01:24 GMT

*BizUnit v3.0" - *Beta 2 Release*

This is a major release with some big changes:

*BizUnit Object Model* - firstly BizUnit introduces an object model to enable BizUnit tests to be created programatically. This primarily targets the ability to create test matrix's in Excel and have BizUnit run the spreadsheet directly - no need to serialise out Xml. The SDK has a sample spreadsheet and a generic spreadsheet parser.

*UI Automation Testing* - another powerful feature is a new test step library which enables UI's to be tested by BizUnit, this enables seemless integration of server side and client side test code, truely powerful :-). Special thanks to my colleague Dharshana Kalahejagoda for delivering this excellent functionality.

*Note*: this is still a beta release, not all of the tests steps have been enhanced with the new object model and there are still some know doc issues

Beta 2 fixes some know bugs in the v3 Beta 1 release, such as test case false-positives when using the OM and also the default value for failOnError is consistent with the XML tests, i.e. defaults to true.

Enjoy,

*Kevin*