Pipeline unit test broken with the multiple doc specs

Hi All,

I just came across an issue/bug with the pipeline unit test in BizTalk 2013/r2. In my case, I’ve multiple document specification present in the  XML disassembler stage. When we write Unit test for this custom pipeline it fails with the error “System.InvalidOperationException: None of the disassembler components could recognize a data”. The error basically means that I do not have a Schema Document Specification added to the XML Disassembler for the message, but that’s not correct. I’ve the doc spec available along with the  other schema doc specs in the collection.



I’ve written unit test using Spec Flow and Winterdom BizTalk Pipeline Testing tool. The Unit test is mentioned below:

Feature: Pipelines
Scenario Outline: Test Inbound Request Pipeline
Given the pipeline has been configured with the “<TypeName>” and “<SubscriberName>”
When the inbound request pipeline is executed with the “<InputMessage>”
Then the following Context Properties should be promoted “<MessageType>”, “<SubscriberName>”
| InputMessage                     | TypeName                  | MessageType                                                                                                                       | SubscriberName |
| ManageAlertResponse.xml | SQLProviderResponse | http://schemas.microsoft.com/Sql/2008/05/TypedProcedures/dbo#ManageAlertsResponse |                |

using System;
using System.IO;
using System.Linq;
using Microsoft.BizTalk.Message.Interop;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using TechTalk.SpecFlow;
using Common.Patterns.PipelineComponent;
using Common.Utilities.BAM.Message.V1;
using Common.Utilities.Helper.Component;
using DB.Common.Pipelines;
using DB.Common.Schemas;

using Winterdom.BizTalk.PipelineTesting;

namespace DB.Common.Test.Bindings.Pipelines
public sealed class InboundRequestPipeline : BasePipelineBinding
[Given(@”the pipeline has been configured with the “”(.*)”” and “”(.*)”””)]
public void GivenThePipelineHasBeenConfiguredWithTheAnd(string typeName, string subscriberName)
ReceivePipelineWrapper pipeline = PipelineFactory.CreateReceivePipeline(typeof(RcvDBResponseMsg));

ScenarioContext.Current.Add(“Pipeline”, pipeline);

[When(@”the inbound request pipeline is executed with the “”(.*)”””)]
public void WhenTheInboundRequestPipelineIsExecutedWithThe(string inputMessageName)
var pipeline = (ReceivePipelineWrapper)ScenarioContext.Current[“Pipeline”];

string inputMessagePath = String.Format(“{0}.{1}”, MESSAGE_RESOURCE_PATH, “Input”);

using (Stream inputMessageStream = GetResourceStream(inputMessagePath, inputMessageName))
IBaseMessage inputMessage = MessageHelper.CreateFromStream(inputMessageStream);

inputMessage.Context.Promote(“InterchangeID”, “http://schemas.microsoft.com/BizTalk/2003/system-properties”, Guid.NewGuid().ToString());
inputMessage.Context.Promote(“CorrelationToken”, “http://schemas.microsoft.com/BizTalk/2003/system-properties”, Guid.NewGuid().ToString());
   MessageCollection outputMessageCollection = pipeline.Execute(inputMessage);

IBaseMessage outputMessage = outputMessageCollection.FirstOrDefault();

ScenarioContext.Current.Add(“OutputMessage”, outputMessage);

[Then(@”the following Context Properties should be promoted “”(.*)””, “”(.*)”””)]
public void ThenTheFollowingContextPropertiesShouldBePromoted(string targetMessageType, string targetSubscriberName)
var outputMessage = (IBaseMessage)ScenarioContext.Current[“OutputMessage”];

string messageType = outputMessage.Context.Read(“MessageType”, “http://schemas.microsoft.com/BizTalk/2003/system-properties”).ToString();

Assert.AreEqual(targetMessageType, messageType);

#region PrivateHelper


The test fail on the highlighted step above with the below exception :

Test Name:    TestInboundRequestPipeline_ManageAlertResponse_Xml
Test FullName:    DB.Common.Test.Features.PipelinesFeature.TestInboundRequestPipeline_ManageAlertResponse_Xml
Test Source:     : line 2147483647
Test Outcome:    Failed
Test Duration:    0:00:00.1399699

Result Message:
Test method DB.Common.Test.Features.PipelinesFeature.TestInboundRequestPipeline_ManageAlertResponse_Xml threw exception:
System.InvalidOperationException: None of the disassembler components could recognize a data
Result StackTrace:
at Microsoft.Test.BizTalk.PipelineObjects.Stage.Execute(IPipelineContext pipelineContext, IBaseMessage inputMessage)
at Microsoft.Test.BizTalk.PipelineObjects.GenericPipeline.ExecuteSubPipeline(IPipelineContext pipelineContext, IBaseMessage inputMessage, Int32 startStageIndex, Int32 endStageIndex)
at Microsoft.Test.BizTalk.PipelineObjects.ReceivePipeline.Execute(IPipelineContext pipelineContext)
at Winterdom.BizTalk.PipelineTesting.ReceivePipelineWrapper.Execute(IBaseMessage inputMessage)
at .DB.Common.Test.Bindings.Pipelines.InboundRequestPipeline.WhenTheInboundRequestPipelineIsExecutedWithThe(String inputMessageName) in c:\ShadabWS\Development\Integration\DB\Common\2.0\DB.Common.Test\Bindings\Pipelines\InboundRequestPipeline.cs:line 46
at lambda_method(Closure , IContextManager , String )
at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStepMatch(BindingMatch match, Object[] arguments)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStep(StepInstance stepInstance)
at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnAfterLastStep()
at TechTalk.SpecFlow.TestRunner.CollectScenarioErrors()
at .DB.Common.Test.Features.PipelinesFeature.ScenarioCleanup() in c:\ShadabWS\Development\Integration\DB\Common\2.0\DB.Common.Test\Features\Pipelines.feature.cs:line 0
at .DB.Common.Test.Features.PipelinesFeature.TestInboundRequestPipeline(String inputMessage, String typeName, String messageType, String subscriberName, String[] exampleTags) in c:\ShadabWS\Development\Integration\DB\Common\2.0\.DB.Common.Test\Features\Pipelines.feature:line 7
at .DB.Common.Test.Features.PipelinesFeature.TestInboundRequestPipeline_ManageAlertResponse_Xml() in c:\ShadabWS\Development\Integration\DB\Common\2.0\..Common.Test\Features\Pipelines.feature.cs:line 0

After doing some investigation, We found that if there is other Schemas Doc Specification present first in the order to the doc spec which is passed  from the unit test, the unit test component did not recognize the schema spec and fails the unit test.

For example: I’ve passed schema type “TypedProcedure_dbo“ from my unit test to the pipeline which is second in the list.


The unit test fails, because the first Schema type in the list is “EmployerSecurity”.


As soon as I move the “TypedProcedure_Dbo” up in the order, compile, Gac and run the unit test, all looks good and success.



It looks like the Pipeline Unit test only validate schema with the first spec, my question is why does the order matter in the Unit test while it does not matter at the run time? Just wondering if anyone have already experience similar issue and have a better solution?



Azure Service Bus–Connectivity issue with SB-Messaging Adapter in BizTalk

Hi All,

I just came across connectivity issue with the Azure Service Bus topics with SB-Messaging Adapter in BizTalk. The detail about the error is below. It took us sometime to figure out or resolve this connectivity issue. That is why wanted to document it so that we know the resolution.

“A message sent to adapter “SB-Messaging” on send port “Send__MEL_Topic” with URI “sb://m-dev-integration-.servicebus.windows.net/test” is suspended. Error details: System.ServiceModel.CommunicationException: Unable to connect to Service Bus using HTTP connectivity mode. —> System.ServiceModel.CommunicationException: Unable to connect upstream.

In this post I’ll demonstrate how to create Topics in Azure Service Bus and using SB-Messaging adapter to connect.

1- Create Service Bus Topic in Azure.

a- Go to Service Bus Menu


b- Create Namespace .


c- Navigate to Newly created Namespace, in my case “m-dev-integration-”.


d- Click New – Click “Topic” – Click Custom Create.


e- Add name to the new topic.


f- Click Next and Next.


g- Click on Newly created topic “Test”

h- Create Subscription.



i- Configure Topic “Test”. Click on configure, you need to add shared access policies.


Test- will be used to Send messaged to the Topic, and BizTalk is used to listen/receive messages from the Topic.

j- You need connection string and SAS information for BizTalk connectivity. Please copy the Access connection string for both Test and BizTalk.


2- Go to BizTalk Admin Console.

a- Create One-way send port to send out messages to Service Bus Topic “Test”.


b- Go to authenticate tab. Enter shared access key name – “Test” and access key from the connection string on the Topics configure page.


c- Create properties, add label to identify messages in the Service Bus/ BizTalk.


d- Send any Json message to the send port.

If you get the below error the follow the steps mention below:

“A message sent to adapter “SB-Messaging” on send port “Send_MEL_Topic” with URI “sb://m-dev-integration.servicebus.windows.net/test” is suspended. Error details: System.ServiceModel.CommunicationException: Unable to connect to Service Bus using HTTP connectivity mode. —> System.ServiceModel.CommunicationException: Unable to connect upstream. – “

Resolution to the above error is very simple. This might be issue with your Proxy settings if you are within the firewall. Check your internet explorer Lan Settings, you should be using proper proxy address and port number (please make sure you use correct port number).


or you can edit your BTS config file to include proxy setting, this setting will be used for all the users accessing the machine, otherwise in DEV vm each user need to edit the internet explorer settings.

Issue with WCF-SQL Adapter with 64-bit Host Instance

An issue reported by our BizTalk Application Support team about the orchestrations in Active state but does nothing and hang around for ages. These orchestrations was generated by the de-batching of XML node to insert individual records into the SQL Database using WCF-SQL Adapter. Initially there were 2000 odd nodes in the XML which spun 2000 orchestrations. Around 1500 got processed and inserted into the SQL Server correctly but 500 looks like they become passive instead of active status in the group hub. Luckily we can replicate this issue in out Pre-Prod environment. When we debug the orchestration found that each orchestration is waiting to receive Ack response from WCF-SQL, and these 500 wasn’t inserted into the tables.

We thought may be this issue is due to the worker/thread combination ran out, but we were wrong. I decided to change the Send port host handler to use 32-bit host instance instead of 64-bit. Luckily that did the trick and all the orchestration went to completion state.

I’m still not sure why 64-bit host instance behave abnormally? If anyone have a best solution for this issue please feel free to write back.

Note: The design of de-batching XML and spinning 1000 orchestrations to only insert records into Database is useless, we should not design such process, it will impact the performance of the BizTalk Server, instead we should use Single orchestration with Composite Operation on WCF-SQL Send port, which is mentioned in my below post.



Re-play Production Suspended messages from start

This post is to simply describe how to pull out all messages from the suspended instances in Production.
I’ve been recently involve in remediation work for my client, where they have 50000+ suspended messages due to target system failure and timeouts. Since the design initially do not cater to re-submit these suspended instances, I need to look for other alternative. We took a back up of the Message box and copied re-stored to other similar environment. The task was to pull out one specific schema message to re-play these suspended again in the production.
I’ve came up with a small VB script to actually pull out all messages for the  particular schema from suspended instances in message box.

‘ save_messages.vbs
‘ Enter cscript save_messages.vbs with no arguments from a command prompt for usage
‘ This script needs to be run under a user account that is a member of the BizTalk Administrators
‘ group. This script needs to be run on a machine that is configured with BizTalk administration
‘ tools.

dim objBtsWmiNS, objMsg, svcinsts, inst, msg, ndx, size

Dim aryHostNames()
Dim aryObjQueues()
Dim aryHostBatchSize()

Dim strKey2Instance
Dim strQuery2Msg
Dim strServiceName
Dim strMsgType
Dim strSavePath

On Error Resume Next
Dim objArgs: Set objArgs = WScript.Arguments
If ( objArgs.Count = 0 OR objArgs.Count > 4) Then
wscript.quit 0
End If

wmiQuery = “”

‘ServiceStatus = 16 – ‘Completed With Discarded Messages’ in BizTalk Server 2004
‘ServiceStatus = 32 – ‘Suspended (not resumable)’
‘ServiceStatus = 4 – ‘Suspended (resumable)’
‘ServiceClass = 64 – ‘Routing Failure Report’
‘ErrorId = “0xC0C01B4C” – is how ‘Completed With Discarded Messages’ are exposed in BizTalk Server 2006

If (objArgs(0) = “-Z” OR objArgs(0) = “-z”) Then
wmiQuery = “select * from MSBTS_serviceinstance where ServiceStatus=16 OR ErrorId=’0xC0C01B4C'”
End If

If (objArgs(0) = “-A” or objArgs(0) = “-a”) Then
wmiQuery = “select * from MSBTS_serviceinstance where ServiceStatus=4 OR ServiceStatus=32 OR ServiceStatus=16 OR ErrorId=’0xC0C01B4C’ OR ServiceClass=64”
End If

If (objArgs(0) = “-SR” or objArgs(0) = “-sr”) Then
wmiQuery = “select * from MSBTS_serviceinstance where ServiceStatus=4 and ErrorDescription like ‘%ExceptionInformation%'”
End If

If (objArgs(0) = “-SNR” or objArgs(0) = “-snr”) Then
wmiQuery = “select * from MSBTS_serviceinstance where ServiceStatus=32”
End If

If (objArgs(0) = “-DIS” or objArgs(0) = “-dis”) Then
wmiQuery = “select * from MSBTS_serviceinstance where ServiceClass=32 AND ServiceStatus=8”
‘ServiceClass = 32 ‘Isolated Adapter
‘ServiceStatus = 8 ‘Dehydrated
End If

saveMessagesBeforeTermination = True

If ( objArgs.Count > 1) Then
strSavePath = objArgs(1)
End If

If ( objArgs.Count > 2) Then
strServiceName = objArgs(2)
End If

If ( objArgs.Count > 3) Then
strMsgType = objArgs(3)
End If

If(wmiQuery = “”) Then
wscript.quit 0
End If

wscript.echo “->Connecting to BizTalk WMI namespace”
Set objBtsWmiNS = GetObject(“WinMgmts:{impersonationLevel=impersonate, (security)}\\.\root\MicrosoftBizTalkServer”)
If Err <> 0 Then
wscript.quit 0
End If

wscript.echo “->Getting BizTalk host collection”
Set hosts = objBtsWmiNS.ExecQuery(“select * from MSBTS_HostSetting”)
If Err <> 0 Then
wscript.quit 0
End If

hostCount = hosts.count

ReDim aryHostNames(hostCount – 1)
ReDim aryObjQueues(hostCount – 1)
ReDim aryHostBatchSize(hostCount – 1)

wscript.echo “->Retrieve BizTalk host names and loading host queues”
ndx = 0
For Each host in hosts
wscript.echo “Found host ” & host.Properties_(“Name”)
aryHostNames(ndx) = host.Properties_(“Name”)
Set aryObjQueues(ndx) = objBtsWmiNS.Get(“MSBTS_HostQueue.HostName=””” & aryHostNames(ndx) & “”””)
If Err <> 0 Then
wscript.quit 0
End If
ndx = ndx + 1

wscript.echo “->Getting collection of service instances”
Set svcinsts = objBtsWmiNS.ExecQuery(wmiQuery)

‘Iterate through instances and save them in host-specific arrays.

wscript.echo “->Start iterating service instances”
totalCount = 0
For Each inst in svcinsts
If (objArgs.Count = 1 Or (objArgs.Count > 1 And strServiceName = inst.Properties_(“ServiceName”) ) ) Then
wscript.echo “Found suspended instance “”” & inst.Properties_(“ServiceName”) & “”” on host ” & inst.Properties_(“HostName”)
‘Resolve host index
For hostIdx = 0 To hostCount-1
If aryHostNames(hostIdx) = inst.Properties_(“HostName”) Then
Exit For
End If

’16 is an internal service class that cannot be terminated
If 16 = inst.Properties_(“ServiceClass”) Then
wscript.echo “Skipping BizTalk internal service instances (they cannot be terminated anyway)”
’64 is a routing failure report and doesn’t have messages that can be saved
If 64 = inst.Properties_(“ServiceClass”) Or 16 = inst.Properties_(“ServiceClass”) Then
saveMessagesBeforeTermination = False
End If

errorCountSavingMessages = 0
If saveMessagesBeforeTermination Then

‘wscript.echo “Build Query”
If strMsgType > “” Then
strQuery2Msg = “select * from MSBTS_MessageInstance where ServiceInstanceID=””” & inst.Properties_(“InstanceId”) & “”” and MessageType = “”” & strMsgType & “”””
strQuery2Msg = “select * from MSBTS_MessageInstance where ServiceInstanceID=””” & inst.Properties_(“InstanceId”) & “”””
End if

‘wscript.echo “Query is:” & strQuery2Msg

Set msgInsts = objBtsWmiNS.ExecQuery(strQuery2Msg)

For Each msg in msgInsts
msg.SaveToFile strSavePath

If Err <> 0 Then
wscript.echo “Failed to save MSBTS_MessageInstance”
wscript.echo Err.Description & Err.Number
errorCountSavingMessages = errorCountSavingMessages + 1
wscript.echo “Saved message ” & msg.Properties_(“MessageInstanceID”)
End If
End If
totalCount = totalCount + 1
End If

End If

‘ Delete whatever is left
For hostIdx = 0 To hostCount-1
If aryHostBatchSize(hostIdx) > 0 Then
TerminateAccumulatedInstacesForHost hostIdx
End If

wscript.echo “SUCCESS> ” & totalCount & ” instances were found and attempted to be saved”

‘This subroutine deals with all errors using the WbemScripting object.
‘Error descriptions are returned to the user by printing to the console.
Sub CheckWMIError()

If Err <> 0 Then
On Error Resume Next
Dim strErrDesc: strErrDesc = Err.Description
Dim ErrNum: ErrNum = Err.Number
Dim WMIError : Set WMIError = CreateObject(“WbemScripting.SwbemLastError”)

If (TypeName(WMIError) = “Empty” ) Then
wscript.echo strErrDesc & ” (HRESULT: ” & Hex(ErrNum) & “).”
wscript.echo WMIError.Description & “(HRESULT: ” & Hex(ErrNum) & “).”
Set WMIError = nothing
End If

‘wscript.quit 0
End If

End Sub

Sub PrintUsage()
wscript.echo “Usage:”
wscript.echo “cscript save_messages.vbs < -Z | -A | -DIS | -SR | -SNR > SavePath [Port/Orchestration name] [MessageType]”
wscript.echo ” -Z saves all “”Zombie”” instances (e.g. completed with discarded messages)”
wscript.echo ” -A saves all suspended and zombie instances as well as all routing failure reports”
wscript.echo ” -SR saves suspended resumable instances only”
wscript.echo ” -SNR saves suspended non-resumable instances only”
wscript.echo ” -DIS saves all dehydrated ‘isolated adapter’ instances”
wscript.echo ” optionally supply the name of the orchestration or port name to filter on specific instances”
wscript.echo ” Ensure that the SavePath folder exists before running as that is where it saves the instances”
wscript.echo ” Example: cscript save_messages.vbs -SR D:\temp\Suspended PublishInvoicePaymentStatusChange http://xyz/Channel/DynamicsAX/PaymentManagement/PublishInvoicePaymentStatusChange/v1#PublishInvoicePaymentStatusChange&#8221;
End Sub

This Script can be called from the below batch file
cscript save_messages-CustomCluckHA.vbs -SR “C:\temp\Processor-SaveHASub_20151027” “Orchestration-TypName” “http://xyz/Assistance/Home/v1#HomeAssistAgreementChangedEvent&#8221;

After pulling out all the messages, we simply do the file drop in the production system. Our orchestrations cater for both one-way and two-way messaging patterns, so at the end it was easy.

Working with BizTalk and Octopus Deploy Part 2

Further to my post about deploying BizTalk solution using Octopus found here, this post will describe the substitution of variables in Octopus 3.0 release. In Octopus 2.x.x release, we had Octopus.Platform.dll which gets the variable information from the octopus database, however in octopus 3.0 this feature is removed and not included any more.

This will prevent us to use “Substitute variables in files” process template available on Octopus community. To overcome this issue, I’ve to write custom power shell script to replace variable. Yes, there is in-built substitute variable feature in octopus, apparently we cannot use it, because BizTalk deployment is slightly different from .Net.

Here is the script to substitute variables in files. You need to include all the variables in the below power shell script.

This is the custom variable replacement function”:

$TargetFile=”C:\Program Files (x86)\ABC for BizTalk\1.0\Deployment\EnvironmentSettings\SettingsFileGenerator.xml”
Function ReplaceInFile($TargetFile, [HashTable] $Values){

if ( (Test-Path $TargetFile ) -eq $false){
throw “The target file ‘$($TargetFile)’ does not exist.”

Write-Host ” — Starting custom transformation for $($TargetFile)”

$fileContent = Get-Content $TargetFile
$Values.GetEnumerator() | ForEach-Object {
Write-Host “Replacing [$($_.Key)] with [$($_.Value)]”
$fileContent = $fileContent -replace $_.Key, $_.Value

[IO.File]::WriteAllText($TargetFile, ($fileContent -join “`r`n”))

This is how it is called:

$SitesConfigFile = $TargetFile
ReplaceInFile -TargetFile $SitesConfigFile -Values @{
‘#{SsoAppUserGroup}’ = $SsoAppUserGroup;
‘#{SsoAppAdminGroup}’ = $SsoAppAdminGroup;

If someone has a better approach then please let me know.


Shadab Anwer

Using BizTalk 2013 R2 to transform an XML message element to a JSON array

Connected Pawns

I wanted to know whether the out of the box  JSON encoder pipeline component creates an JSON array  if there is only one repeating XML fragment. One of my colleagues found with BizTalk 2013 he had to add an array attribute to the XML to force the creation of a JSON array (https://connectedcircuits.wordpress.com/2014/03/27/sending-json-messages-from-a-biztalk2013-adaptor/).

I created a test schema shown below and deployed it to my run time.


I created this message and consumed it with a receive location that was configured with  passthrureceive location.

<ns0:Repeater xmlns:ns0=”http://JSONTest.Repeater”>

The XML message was then processed on a send port with a pipeline that only contained the JSON encoder pipeline component. Voila! I got the correct output shown below;

“Repeater”: {
“Repeat”: [

Thanks Microsoft this has saved me a lot of extra work that i had to do in BizTalk 2013.

View original post

Sending JSON messages from a BizTalk2013 adaptor


I had a requirement where I needed to send a message to a web API that only accepted messages as JSON.

BizTalk2013 provides a WCF-WebHttp adaptor for sending sending XML messages to a REST endpoint but not in JSON format. There are rumours this will be remedied in BizTalk2013R2, unfortunately I required a solution now.

So in the meantime I will use the the Json.NET component to convert the XML message to JSON. I also found this blog from http://blog.quicklearn.com/2013/09/06/biztalk-server-2013-support-for-restful-services-part-45/ to convert XML messages to JSON using the Json.net in a custom pipeline component which I ended up using with some mods.

When using this pipeline remember to set the Outbound HTTP Headers as shown below:


This all worked a treat until during the unit testing phase I created a sample XML document with only one repeating element.This caused the following error from the Web API.

Can not deserialize instance…

View original post 614 more words