Sabbatical

I'm taking a sabbatical next year at Estrellas en la calle in Cochabamba, Bolivia, thus I won't post anything here in 2010.
- Christoph

One year later, I'm back. Thanks for continue posting!

Directly Create an EDIFACT Interchange

If you want to create the EDIFACT interchange directly instead of using the built-in functionality of BizTalk, you can create your own schema and send it the EdiSendPipeline. When you use BizTalk's built-in batching functionality, you can see which schema BizTalk uses to prepare the interchange. As far as I see, you can build your custom interchange schema this way:

Import...

• Microsoft.BizTalk.Edi.BaseArtefacts.EdifactServiceSchema.dll (with namespace "http://schemas.microsoft.com/Edi/EdifactServiceSchema")
• the EDI schema, e.g. EFACT_D06A_INVOIC_2.0

Create...

• root node "EdifactInterchangeXml"
• Below root, Create attribut @DelimiterSetSerializedData and set fixed value to 39:-1:-1:43:58:63:42:44:-1 (if this is how UNA should look like).
• Below root, create groups UNA, UNB, UNZ and reference the types from the EdifactServiceSchema
• Below root, create group "TransactionSetGroup" and subgroup "TransactionSet"
• Below "TransactionSet", create a group that references the EDIFACT schema
• Below "TransactionSet", create attribute @TransactionSetActivityId
• Below "TransactionSet", create attribute @DocType and set fixed value to http://schemas.microsoft.com/BizTalk/EDI/EDIFACT/2006#EFACT_D06A_INVOIC_2.0

The result looks like this:

Weired Error "This is often an indication that other memory is corrupt." executing pipeline component

After migrating our biztalk app to 2009 we received a confusing error message everytime we dropped a file into a file receive location with a pipeline containing a bunch of pipeline components:

"There was a failure executing the receive pipeline:
...
Reason: Attempted to read or write protected memory. This is often an indication that other memory is corrupt"

After along irritating searching we finally debugged the pipeline component and find the root cause:
The call of

message.Context.Read(propertyName, propertyNamespace);

within an pipeline component must not be done with a "Null" value of "propertyNamespace". Obviously there is no internal check for this and the following call within the unsafe code leads into a Null reference call and the error message above.

Problems getting the current assembly in VS2008/BTS2009

BizTalk 2009 seems to copy self-developed BizTalk assemblies into hidden system folders. If you run into the problem that your assemblies are not updated (although you deploy them in Biztalk, add them to the GAC, or save them to the pipeline components folder), it might be that BizTalk still references an old version of the assembly that is in a system folder.

In that case, find and delete the old version of the assembly and try again.

Sending E-Mails with multiple attachments with BizTalk 2006

If you want to send an email with several attachments using the SMTP Adapter of BizTalk, you have to set the following context properties:

• SMTP properties of the message: Subject, From, ...
• MIME properties of each bodyparts: FileName and IsMIMEEncoded
  

The latter could look like this:
inmsg.BodyPart.PartProperties.Write("FileName", "http://schemas.microsoft.com/BizTalk/2003/mime-properties", "file.html");
inmsg.BodyPart.PartProperties.Write("IsMIMEEncoded", "http://schemas.microsoft.com/BizTalk/2003/mime-properties", false);

Errors exist for one or more children

After changing an orchestration only little, the build process sometimes throws an exception that says "Erros exist for one or more children". This message seems a bit random, since I couldn't find a real problem, like a missing assembly, definition, or port.

In my case, it helped to recreate the construct shapes from the orchestration by deleting the shape, creating a new one, and adding the code of the deleted one. After that, the error disappeared.

PS: You can even find the error message also inside the orchestration file. It might help to watch the changes inside the code, while you modify the orchestration in the orchestration designer. This way, you get a feeling of how BizTalk creates the code from the designer, and you can also observe when the error disappears.
 
3 years later... If you open the orchestration code, for example in an XML editor, and remove everything below row #endif // __DESIGNER_DATA. Then close file and rebuild project. BizTalk will now recreate the implementation code and compiles without that problem.

Is the Convoy Pattern Orchestration getting slow?

In our team, we discussed whether our orchestrations that implement the convoy pattern are getting slower while processing message by message. A small test showed that this is not true, even if you call a sub-orchestration for each message. The time line moved like this in our test:



There is just one exception where processing needs longer, but the mean duration is not increasing.

The part '%messagepart%' of message '%multipart message%' contained a null value at the end of the construct block.

BizTalk throws the following error when one tries to create a multipart message with a message part that was just created from a map in the same construct shape:

Shape name: Send_1
ShapeId: 11910fdc-f4a1-4391-b09d-6a9bd0275ff8
Exception thrown from: segment 1, progress 11
Inner exception: The part '%messagepart%' of message '%multipart message%' contained a null value at the end of the construct block.

Exception type: NullPartException
Source: Microsoft.XLANGs.Engine
Target Site: Void CheckNonNull()
The following is a stack trace that identifies the location where the exception occured

at Microsoft.XLANGs.Core.Part.CheckNonNull()

Somehow, BizTalk looses the value of the message part that was NOT mapped.

A work-around for this is to create a second construct shape. One just creates the message part in a mapping, and the second shape puts the message parts together.

Thanks, Christian!

If the Pipeline Component doesn't appear in the Pipeline Editor toolbox

If your BizTalk pipeline component doesn't show up in the toolbox of the pipeline editor, you should check these things:

• Is the GUID of the pipeline component the same in AssemblyInfo and in the class itself (it appears twice there)?
  
  
• Is the right assembly name used when instanciating an object of ResourceManager?
  
  
• Are you using the same namespace outside the class, in the properties, and in the ResourceManager?
  
  
• Is the assembly copied to %biztalk installation%\pipeline components?
  
  
• Are you looking for the right component type (encoder, decoder, ...) in the pipeline editor?
  

Archiving Table Entries

If you want to backup table entries into a second table and delete them in the master table, you can use the folling SQL:

DELETE FROM masterTable
OUTPUT DELETED.* INTO archiveTable
FROM deletionTable
WHERE masterTable.ID=deletionTable.ID

while deletionTable contains the IDs of the entries, which should be deleted.

When using this, all entries of the masterTable, which are joined to the IDs in the deletionTable are copied to the archiveTable and then deleted.

Thanks, Ronny!

EDIFACT in BizTalk 2009

In April, I talked at a meeting of the Swedish BizTalk User Group on EDIFACT and BizTalk. If you are interested, check my presentation on EDIFACT in BizTalk 2009.

Sending Big IDocs with the WCF-Custom-Adapter

In MSDN, there's a good explanation how to connect BizTalk and SAP with the WCF-Custom-Adapter. However, if you want to send big IDocs, there are some boundaries:

If you use the action Idoc, you could get an exception like: "Failed to allocate a managed memory buffer of 123982965 bytes. The amount of available memory may be low."

If you use SendIdoc instead, you get an exception in the receive pipeline: "Attempt to pass a delimited field greater than the maximum field length of 50000000."

As soon as I find something out, I post it here.

Stream validates EDI code

When using the EDISendPipeline in BTS2006 R2, that might be interesting:
My colleague Christian and me just found out that the EDI stream is validated by the stream reader while parsing the stream! So, when you work on the EDI stream which is produced by the EDI assembler, you get exceptions like "Error in serialization" if your EDI file is invalid. Quite clever how the programmers solved the validation of EDIFACT since it validates the file while returning it to the next step in the Send Pipeline.

Addition 2009-10-21:
Because of this, the EDI stream can't be read twice in following self-build pipeline components and BodyPart.Data.CanSeek is set the false.

Aggregating Values from a Hierarchy

In a session about the SQL Server 2008 datatype HierarchyID, the question came up whether you can easily ask question such as: "If I store things (and sup-things and so on) with a number in a hierachy, can I sum up the numbers of a thing, including the subthings?"
The answer is, yes, you can, and here is how it goes. The following code shows a table which hierarchically stores websites and their clickrate.

-- create hierarchical table
create table websiteCalls
(
path hierarchyid,
filename nvarchar(100),
clickCount int
)

-- insert root node "index"
insert into websiteCalls values (hierarchyid::GetRoot(), 'index.html', 1000)
go

-- insert a level 1 entry "products"
declare @index hierarchyid
select @index = path from websiteCalls where filename='index.html'
insert into websiteCalls values (@index.GetDescendant(null, null), 'products.html', 200)

-- insert two more level 1 entries "customers" and "impressum"
declare @products hierarchyid
select @products = path from websiteCalls where filename='products.html'
insert into websiteCalls values (@index.GetDescendant(@products, null), 'customers.html', 100)
insert into websiteCalls values (@index.GetDescendant(null, @products), 'impressum.html', 10)

-- insert a "product1" on level 2
insert into websiteCalls values (@products.GetDescendant(null, null), 'product1.html', 300)
declare @product1 hierarchyid
select @product1 = path from websiteCalls where filename='product1.html'

-- insert two more level 2 entries
insert into websiteCalls values (@products.GetDescendant(@product1, null), 'product2.html', 50)
insert into websiteCalls values (@products.GetDescendant(null, @product1), 'product0.html', 100)
go

-- get all websites and their clickrates
select path.ToString(), filename, clickCount from websiteCalls

-- get the sum of clickrate number of all product-pages:
declare @products hierarchyid
select @products = path from websiteCalls where filename='products.html'
select SUM(clickcount) from websiteCalls where path.IsDescendantOf(@products) = 1 and path<>@products

Developing BizTalk with Professional Edition or Team Edition*

If you have to choose a Visual Studio 2005 edition to develop BizTalk 2006 in combination with Team Foundation Server, you should keep in mind that the Team Edition has all the features of the Professional Edition and some more (quoted from Microsoft):

- Team Foundation Server CAL includfor accessing Team Foundation Server
- Team Foundation Server Workgroup Edition included when purchasing Visual Studio Team System Editions with an MSDN Premium Subscription
- 64-Bit Debugging (IA64) enables you to debug 64-bit applications running on Windows 64-bit computers remotely
- Code Analysis Tools such as Code Analysis Check-in Policy (which enforces developers to run a code analysis before a check-in of code)
- Code Profiling Tools like Hot Path (that helps finding bottle necks inside the code) and a Stand-Alone Profiler
- Code Coverage to measure the effectiveness of tests on a line-by-line or even a block-by-block basis
- Item templates to run Performance tests

That means, that you can connect to the Team Foundation Server from Visual Studio if you just buy the CAL-licence which includes the Team Explorer. However, you won't get the benefits of the Team Edition then.

*I'm using the term "Team Edition" as a short form for the Development Edition of Visual Studio Team System 2008.

Managing hierarchical data in SQL Server 2008

There are roughly three different approaches to store and retrieve hierarchical data in SQL Server:

Parent-Child-Approach with Recursive TCE:
You can save the ID of a node's parent in an extra “parent column”:

Then you connect the referenced items with recursive Common Table Expressions (CTE).

HierarchyID data type:
Or you save the node’s hierarchy location with the new data type HierarchyID, which stores a path to a node and, thus, reveals the where in a tree the row occurs. A sample table looks like this:

The hierarchy is directly visible and, unlike in the simple parent-child-pattern, you have now the HierarchyID methods to work against the tree structure.

XML data type:
The third option is to represent the hierarchy in a XML document and store the instance in an xml column:

Doing this, you can retrieve information from the tree using XQuery.

This is meant to be a teaser... ;-) I'm going to write more about this the next days.

How does the SAP adapter retrieve the list of IDocs?

At a customer, we had the problem that the SAP adapter (building on SAP .net Connector v1.0.2) didn't retrieve the IDoc that we wanted to call:


To analyse how the SAP adapter retrieves this list of IDocs, we traced the RFC communication between SAP and BizTalk (transaction ST05*) and found out that the SAP .net Connector uses the BAPIs RFC_SYSTEM_INFO and IDOCTYPES_FOR_MESTYPE_READ to get a list of IDocs which are released:


Using transaction SE37, we (thanks, Rüdiger!) could analyse the BAPI IDOCTYPES_FOR_MESTYPE_READ and see that it uses the table EDIMSG to get the IDocs. And their we saw that the BAPI doesn't call for "message types" instead of "basic types". And, guess what, the message type had a different name then the basic type (which makes perfectly sense) and, thus, couldn't be found.

* Tipp: When logging to SAP to trace, you have to use the same user that you use inside the Send Port / Receive Location.

How to solve Problems when accessing an IDoc with the SAP Adapter

If you can't see an IDoc when you try to create a schema in Visual Studio using the SAP adapter (building on SAP .net connector 1.0.2), the problem is usually that the IDocs hasn't been published ("freigegeben"). You can check this here:

we60 - Check whether your IDoc is known in a specific SAP Release (we often had the problem that segments of an IDoc weren't known in a release)

we30 - You can check here whether all segements are published. Click on the segment, then "segment editor". There's a checkbox whether this segment is published.

we20 - If you can access the IDoc but can't send data to it, the IDoc might miss a partner agreement ("Partnervereinbarung").

Creating Functoids inside the XML Code of the Mapping File

I had to create a BizTalk mapping with 246 Logical Existence-Functoids, which always checked whether a source field existed and then enabled or disabled the source field. You can save quite a lot of time (and I guess manual mistakes), if you use an text editor to create the XML manually to the transformation file (the format is called BTM and it's XML).

So, in case, your mapping looks like this, and you need many more functoids of that kind...


1) Open the transformation in some text editor.

2) Save the code of a new created functoid, for example:
<functoid functoidid="236" x-cell="59" y-cell="60" fid="317" name="Logical String" label=""><input-parameters></functoid>

3) Paste the code to a new file and put tabs into each line so that the code looks like this:

<functoid functoidid="[tab]236[tab]" x-cell="59" y-cell="[tab]60[tab]" fid="317" name="Logical String" label=""><input-parameters></functoid>

4) Now, copy the content to Excel and copy the lines so that the number, which should be pasted to extra columns, are increased-

5) Copy the content to a text editor, delete the tabs again and paste the code into the transformation file.

Ops! You should take care that the text editor that opens the BTM file externally doesn't change the unicode of the file.