Blog

Biztalk Developer Yorkshire, Leeds, Manchester, Sheffield, York

Tue, 28 May 2013 20:47:03 GMT

If you require Biztalk expertise on a consultancy basis in your organisation in the Yorkshire region, please do not hesitate to contact me.

I am a qualified MCTS (Microsoft Certified Technology Specialist) on Biztalk 2010 with numerous years experience of developing integration solutions.

Serving Leeds, Wakefield, York, Sheffield, Barnsley, Bradford, Manchester, Huddersfield.

BizTalk 2006-2010, XSLT, C#, WCF Services, Windows Services.
Design and build of software using appropriate toolsets, following the software life cycle and IT Design/Development procedures.
Delivering solution design and code from functional specifications. You will perform unit and integration testing to system specification.
Extensive experience of BizTalk and solid experience of creating technical design solutions and functional requirement documents.
Full development life cycle
Architectural consulting
Development of Orchestrations, Schemas, Maps, Pipelines
Development of Custom pipeline components, adapters etc.
Performance tuning including high throughput and low latency
Production reviews
High availability & Disaster recovery
BizTalk Installation and Clustering

Services provided:

Assessment, analysis, architecture, development, test, operate: EDI, EAI, B2B, B2C, ESB, SOA, BPM
proof of concept and prototyping
Back-end integration
Tools for development, deployment, monitoring, acceleration
CRM: MS Dynamics CRM, Siebel
Middleware optimization: on premise, hybrid, cloud
Mission critical services: high-available application integration
Microsoft BizTalk Server expertise
Training

Please email myself, Bill Winder contact@net-aspect.com.

Hemmed in a Closed Cirque of Our Own Creating

Sun, 19 May 2013 20:25:35 GMT

The vast majority of this blog is all about technology, but occasionally I like to drop in some interesting quotes.

Today's is one from Dorothy Rowe's "What Should I Believe", from the chapter "Hemmed in a Closed Cirque of Our Own Creating".

"We each live in our own individual prison. Freedom consists of chossing, consciously, the prison in which we live, and making our prison large and variable, with flexible walls, a spacoius house with wide doors and windows open to heaven and earth. Unfortunately, so many of reject freedom, for freedom means insecurity, openness to the unknown. Instead we build our prison as an impregnable fortress, enclosing a meagre, empty space, and shutting out the myriad possibilities of heaven and earth. Morever, we regard our fortress not as a something we have chosen, but as an absoluate and immutable fact of of reality. Trapped and helpless, we cannot be happy."

The burying of our talents

Fri, 08 Mar 2013 20:41:17 GMT

When we are children we discover in ourselves talents, powers, that we know are truly an essential part of ourselves and, being so, can become the source of the greatest meaning we can find in life. We feel impelled to explore our world through our powers, but most of use are stopped from doing this. Poverty, rigid education, the demands that we fill certain roles in society block us so that sometimes we have to forget the possibilities we once glimpsed, or, not forgetting, suffer punishment like the servant in the parable who buried his talent. When, so Jesus said, his lord discovered this, he commanded that the servant be cast "out into the darkness: there shall be weeping and gnashing of teeth". The gap between what we know we could have achieved and what we have achieved is usually filled with much weeping and gnashing of teeth."

from Dorothy Rowe (What Should I Believe?)

Creating New Annotation/Note Record in Dynamics CRM, Late Binding

Wed, 20 Feb 2013 22:22:34 GMT

There's a minimum amount of data you'll need to creation a new annotation/note record in Dynamics CRM. Below is a sample of how you might do it with late binding.

using (proxy)
{

// create a new note for the incident in the notes entity
Entity note = new Entity("annotation");
note["notetext"] = "notes text notes text";
note["subject"] = "Note";
note["isdocument"] = false;
Guid annotationId = Guid.NewGuid();
note["annotationid"] = annotationId;
note["objectid"] = new EntityReference("incident",new Guid("yourguid"));
proxy.Create(note);

}

Microsoft Dynamics WCF Web Services - Early vs Late Binding Comparison

Wed, 06 Feb 2013 20:48:42 GMT

There are two options when programming against the Dynamics SDK - early binding and late binding, as outlined in the Microsoft SDK documentation (see the two links below).

Use the Early Bound Entity Classes in Code

Contact contact = New Contact();
contact.EMailAddress1 = "marykay@contoso.com";

See Use the Early Bound Entity Classes in Code

Use the Late Bound Entity Classes in Code

Entity account = new Entity("account");
account["name"] = "Fourth Coffee";
_accountId = _orgService.Create(account);

See Use the Late Bound Entity Class in Code

Comparison of the two methods

The key difference between early and late binding involves type conversion. While early binding provides compile-time checking of all types so that no implicit casts occur, late binding checks types only when the object is created or an action is performed on the type. The Entity class requires types to be explicitly specified to prevent implicit casts.

Advantages/Disadvantages of Early Binding

The advantages to using early-bound entity classes is that all type references are checked at compile time. The compiled executable contains the code necessary to invoke the types’ properties, methods, and events. This results in the detection of issues more quickly since you will find them when you compile rather than run your application.

Early binding comes with a slight performance degradation however (although there is not much difference), according to Microsoft's Dynamics best practices page. If your entities are already defined at code time and slight performance degradation is acceptable, you should use the early-bound types that you can generate by using the CrmSvcUtil tool.

Obviously if the entities are being changed frequently at design stage are you are developing against early bound classes, then you will keep having to update your early bound types using the CrmSvcUtil tool.

Advantages/Disadvantages of Late Binding

Use the Entity class when your code must work on entities and attributes that are not known at the time the code is written. You may need to use this approach if the CRM entities are being designed in tandem with when you are developing your own applications that utilizes them.

With respect to performance view point late binding is marginally more preferable option, as per the Microsoft best practices page for developing with Dynamics: "If your custom code works with thousands of entity records, use of the Entity class results in slightly better performance than the early-bound entity types".

The principle disadvantages of using late binding is that you cannot verify entity and attribute names at compile time.

Conclusion

Conclusion is you might want to try both, you want to know the advtanges and disadvantages of both first hand, and you can only do that through your own experience.

Comments?

Please add any comments if you have any experiences with using one or the other to add to the debate.

Linq Cheat Sheet Examples

Mon, 04 Feb 2013 17:02:05 GMT

A few sample using query syntax. Here's a page on the Microsoft site that you might actually find more useful.

Filtering

var col = from o in Orders where o.CustomerID == 12 select o;

Ordering

var col = from o in orders orderby o.Cost ascending select o;

var col9 = from o in orders orderby o.CustomerID, o.Cost descending select o;

Joining

Example 1

var col = from c in customers join o in orders on c.CustomerID equals o.CustomerID

select new {c.CustomerID, c.Name, o.OrderID, o.Cost};

Example 2

NorthwindEntities NWEntities = new NorthwindEntities();

var categoryProducts =

from c in NWEntities.Categories

join p in NWEntities.Products

on c.CategoryID equals p.CategoryID

into productsByCategory

select new {

c.CategoryName,

productCount = productsByCategory.Count()

};

foreach (var cp in categoryProducts)

{

Console.WriteLine("There are {0} products in category {1}",

cp.productCount, cp.CategoryName);

}

NWEntities.Dispose();

Grouping

var OrderCounts = from o in orders group o by o.CustomerID into g

select new {CustomerID = g.Key,TotalOrders = g.Count()};

another...

var list = from d in db.Fee

group d.FeeDate by new {d.ReceivedFrom, d.FeeDate} into g

select g;

Group with a count

var topRankedShares = from sd in shareContext.ShareDatas

where sd.RecordDate == rankingDate

group sd by

new { sd.RecordDate, sd.Code } into groupings

select new {

recordDate = groupings.Key.RecordDate,

code = groupings.Key.Code,

postsCount = groupings.Count()

};

// insert the rows into the table

foreach (var countRow in topRankedShares)

{

Console.Write(countRow.code + ":" + countRow.postsCount);

}

Group with Count and OrderBy

(from p in DataContext.Hits

where (p.Date >= minDate) && (p.Date < maxDate)

group p by p.Page into g

select new { Page = g.Key, Number = g.Count() }).OrderByDescending(x => x.Number).Take(10);

Another grouping

db.Users.SelectMany(

x=> x.Projects

.Select(i=>

new

{

User = x.Name,

Project = i.Name,

WorkTime = (int?)i.UserWorkTimes.Sum(t=>t.WorkTime)

})

);

Paging

//select top 3

var col = (from o in orders where o.CustomerID == 84 select o).Take(3);

//skip first 2 and return the 2 after

var col3 = (from o in orders where o.CustomerID == 84 orderby o.Cost select o).Skip(2).Take(2);

Element Operators

//throws exception if no elements

var cust = (from c in customers where c.CustomerID == 84 select c).Single();

//returns null if no elements

var cust = (from c in customers where c.CustomerID == 84 select c).SingleOrDefault();

//returns a new customer instance if no elements

var cust = (from c in customers where c.CustomerID == 85 select c).DefaultIfEmpty(new Customer()).Single();

//First, Last and ElementAt used in same way

var cust4 = (from o in orders where o.CustomerID == 84 orderby o.Cost select o).Last();

//returns 0 if no elements

var i = (from c in customers where c.CustomerID == 85 select c.CustomerID).SingleOrDefault();

Random Finds

var vendors = GetWamVendorInfo().AsEnumerable()

.Where(x => x.vendor_name != null && x.vendor_id != null)

.Select(new {x.vendor_id, x.vendor_name})

.Take(2)

Using DebugView with Biztalk

Tue, 29 Jan 2013 13:06:21 GMT

You can output traces from within a Biztalk orchestration using a product called Microsoft DebugView. Here's what you need to do:

Download a copy of debugview and put in on your Biztalk runttime server/machine. You can download it from here.

In your orchestration use Expression shapes where you want to output some text to debugview and use the following syntax in your expression shapes:

System.Diagnostics.Trace.WriteLine("This message will appear in debug view.");
Build your Biztalk solution and deploy it to Biztalk, restart the host instances if needed.
According to this article it says you must build in debug mode not release mode, but I found it not be the case (see this article)
Launch Debugview. Ensure that the following are selected - under Capture menu check that CaptureWin32, Capture Global Win32 and Capture Events are all ticked.
Drop your message into Biztalk to initiate the orchestration - you should see any messages that you have written out from the orchestration appear in debug view.

Removing tempuri.org references in your WCF WSDL

Thu, 24 Jan 2013 13:30:38 GMT

When you create a WCF project, Visual Studio does not assign namespace and therefore libraries WCF automatically assign the value of http://tempuri.org. You're trying to remove the "tempuri" references from a wsdl file for a service you've created in WCF but having trouble? Personally I don't like these tempuri's hanging around on the WCF services I create, it just looks messy and incomplete. Deploying a service with this namespace is a bad practice and changing it at a later time may become a nightmare because all proxies must be regenerated. It seems clear, therefore, that the assignment of a meaningful namespace is very important.

Please note that the resolution for this problem comes from this Italian language web site, however I have done my own translation for English speaking folk, but all attribute goes to this web site.

But then how do you remove tempuri.org?

To delete just assign our namespace, and WCF there are three distinct assignments namespace.

The ServiceContract, typically the service interface:

    [ServiceContract(Namespace = "http://www.yournamespace.com/xxxxx")]
    public interface IYourService
    {...}

On deploying the service, by applying a behavior type ServiceBehavior:

    [ServiceBehavior(Namespace = "http://www.yournamespace.com/xxxxx")]
    public class YourService : IYourService
    { ... }

Endpoint must assign the namespace of Binding

Write a Service Behavior to replace the namespace on the fly

Unfortunately you will still however find tempuri.org in your WSDL. Therefore you have to take extra custom steps

using System;
using System.Collections.Generic;
using System.ServiceModel.Description;
using System.ServiceModel;
using System.Collections.ObjectModel;
using System.ServiceModel.Channels;

namespace YourProjectNamespace
{
    [AttributeUsage(AttributeTargets.Class)]
    public class BindingNamespaceBehavior : Attribute, IServiceBehavior
    {
         private string bindingNamespace { get; set; }

         public BindingNamespaceBehavior(string bindingNamespace)
        {
            this.bindingNamespace = bindingNamespace;
        }

        public string BindingNamespace
        {
            get
            {
                return this.bindingNamespace;
            }
        }

        public void AddBindingParameters(ServiceDescription serviceDescription,
            ServiceHostBase serviceHostBase,
            Collection<ServiceEndpoint> endpoints,
            BindingParameterCollection bindingParameters)
        {
        }

        public void ApplyDispatchBehavior(ServiceDescription serviceDescription,
            ServiceHostBase serviceHostBase)
        {
        }

        public void Validate(ServiceDescription serviceDescription, System.ServiceModel.ServiceHostBase serviceHostBase)
        {
            if (serviceHostBase == null)
            {
                throw new ArgumentNullException("serviceHostBase");
            }

            foreach (var endpoint in serviceHostBase.Description.Endpoints)
            {
                endpoint.Binding.Namespace = bindingNamespace;
            }
        }
    }
}

Without this, it is sufficient to apply the attribute on the implementation of the service (without touching the configuration in any

way) that thus becomes:

    [ServiceBehavior(Namespace = "http://www.yournamespace.com/xxxxx")]
    [BindingNamespaceBehavior(bindingNamespace = "http://www.yournamespace.com/xxxxx")]
    public class YourService : IYourService
    { ... }

Then simply re-download the metadata with svcutil to verify that tempuri.org is finally gone.

WCF Endpoints, Bindings, Contracts and Behaviours

Thu, 24 Jan 2013 09:38:35 GMT

ENDPOINT

Specifies an actual running instance of the service. A service endpoint has an address a binding and a contract.

BINDING

Specifies how the endpoint communicates with the world including things like TCP, HTTP, encoding (text, binary) and security requirements (SSL, SOAP message security).

CONTRACT

What the service actually does ie what operations are valid. eg what the endpoint communicates, it is essentially a collection of messages organized in operations that have message exchange pattersn (MEPs) such as one-way, duplex, request-reply.

CONTRACT DESCRIPTION

Represents a WCF contract.

SERVICE BEHAVIOUR

How the end point interacts with clients get security, concurrency, cacheing, logging etc.

OPERATION BEHAVIOUR

Similar to service behaviour but only gets applied when a specific operation is run.

See: WCF Archicture overview

http://msdn.microsoft.com/en-us/library/aa480210.aspx

See: Endpoints: Addresses, Bindings and Contracts

http://msdn.microsoft.com/en-us/library/ms733107.aspx

WCF Sections in the Web.config file

<system.ServiceModel>

<bindings>
   
   
      <binding>
      
      </binding>
   </bindings>

<behaviors>
   
      <behavior>
      
      </behavior>
   </behaviors>

</system.ServiceModel>

Configure services using configuration files

http://msdn.microsoft.com/en-us/library/ms733932.aspx

Biztalk Development Best Practices: Directory of Sites

Wed, 16 Jan 2013 14:52:03 GMT

I was going to write an article about the best practices in developing with Biztalk. I am quite new to the platform so the purpose of this would have been to imprint some development methodologies in my mind going forwards.

However, I felt that I was a little inexperienced at this stage to do this, and secondly I have found alot of other best practice articles on the internet. This article then, is a list of the Biztalk best practice web articles I have found, and it serves as much as anything a bookmark page for my own use - you might find it a useful page also.

Articles

Videos

Orchestrations

Microsoft Technet - BizTalk Server 2010 - Orchestration Best Practices

Schemas

Jason Agostoni - BizTalk Schema Inheritance Practices / Examples

Pipelines

Deployment

MSDN - Best Practices for Deploying a BizTalk Application

Instrumentation

Instrumentation Best Practices for High Performance BizTalk Solutions

Installation

Instructions for Installing Biztalk Server 2010

Maintenance

I will keep updating this list as I find new and useful articles.

There was a failure executing the receive pipeline

Tue, 15 Jan 2013 11:16:54 GMT

Problem

There was a failure executing the receive pipeline: "xxx" Source: "Unknown " Receive Port: "ReceivePort1" URI: "/httpreceive/BTSHTTPReceive.dll" Reason: Failed to get pipeline: xxx. Please verify that the pipeline strong name is correct and that the pipeline assembly is in the GAC.

I was using a custom pipeline with just some standard pipeline component on it (ie flat file disassembler), with messages coming from a HTTP adapter, and I kept getting the above message.

I double checked that the pipeline artefact was visible in both Biztalk admin console aswell as the GAC (C:\Windows\Microsoft.NET\assembly\GAC_MSIL), and they were. Despite repeatedly redeploying and trying different things, I still got the same message.

Solution

In the end the solution was that the IIS application pool that the HTTP adapter was receiving messages on was using .net 2 - I changed this to .net 4 and it all worked fine. I guess this is because the GAC location for .net 2 and .net 4 are completely different so .net 2 process would not be able to locate a .net 4 assembly.

If this doesn't solve your particular problem there are some other steps you could take.

Check that the assembly is definitely in the GAC - under C:\Windows\Microsoft.NET\assembly\GAC_MSIL
If you are deploying on dev from Visual Studio check that "Install to Global Assembly Cache" is set to true on the project properties. Also check that Redeploy is set to true
Restart the host instances to pickup the new artefacts Ensure there are multiple copies or version of the same assembly in Biztalk or the GAC.
Check the assemblies are in the correct Biztalk application and that any dependent artefacts that might be in other Biztalk project are set as Reference in the Biztalk application's properties
Check the event log for any additional errors that may be occurring before the references one
Use Fusion Log Viewer to tell you what is wrong. It is a very simply GUI application and you can copy it from you machine onto the target if you need to (just an exe, very easy). Turn on Log Failures and it will soon tell you exactly which dependency is not found
Remove the application completely, then double check that everything is removed from both the GAC and the Biztalk management console (by going Biztalk admin console > Applications > All artefacts). Then deploy everything

Integrating Sharepoint 2010 and Dynamics 2011

Fri, 11 Jan 2013 10:29:22 GMT

Just so you know when I say "Integrating Sharepoint 2010 and Dynamics 2011", I mean displaying data from Dynamics CRM entities in a Sharepoint 2010 front end (rather than displaying a Sharepoint document library from the Dynamics interface which is also possible), and enabling those entities to be edited, deleted, added to from Sharepoint.

There are a number of means of achieving this, I have tried to assess my own opinion and experiences of them below...

List web part

There is a custom list web part which was initially aimed at Dynamics 4.0, which you could use. Personally I decided not to bother with this or investigate it in any depth, since whilst it may have worked I considered it "legacy" and it may only provided view functionality. I think it uses the asmx web service provided by Dynamics CRM, rather than the more up to date WCF web service.

Write a custom web part that queries the Dynamics CRM WCF Web Service

You could write your own custom web part which queries the Dyanmics WCF web service - using the WCF service you can do everything you need to such as add records, get an entity or entities, update and delete.

You would have to use .net 3.5 of course since Sharepoint runs on this version of the framework - this limits you in that you cannot use the Dynamics SDK which is built on .net 4.0 - you must use the pure WCF service. The Dynamics SDK makes things easier and quicker to program against the WCF service, so you may run into the occasional issue when developing and looking up some of the more complex CRM datatypes such as lookups and option sets etc.

Using the CRM WCF service seems to be a pretty viable option, I did some initial investigation with this myself - view entities, adding and updating. I personally had a few issues with getting "option set" values and some other issues so I decided after that to investigate using BCS instead as a possibility. I did not try out the RESTful web service that CRM provides, though you might find this of use.

Write a WCF Proxy Service (in .net 4) and write a custom web part that queries this

If you have some issues, as I did, with querying the Dynamics WCF service using .net 3.5 (without the Dynamics SDK), then one option could be to write a .net 4 proxy WCF service. If you write it in .net 4 you can take full advantage of the CRM SDK and the ease that it provides in getting entities. You could then write you custom web part to make queries against your proxy service. The proxy service would do most of the hard work in getting the data, thus minimizing the amount of code you have to write in your custom web part.

Using Business Connectivity Services

BCS is a part of Sharepoint that enables you to connect lists and forms to external data such as SQL databases etc. Unfortunately, probably because of the complexity of the Dynamics data types and structures I guess, BCS will not work out of the box with Dynamics data. You cannot just point BCS at a Dynamics WCF service and expect it to work, it won't. Fortunately the folks at Microsoft have provided an open source custom connector project that enables you to do just this.

Download the source code and white paper for the BCS service here.

NB It is well worth looking at the two videos at the bottom of this post first though which explain some of the options available to you and provide an overview of integration between Sharepoint and CRM, with a particular focus on the BCS method and looking at the customer connector provided.

Advantages

One of the main advantages of using the BCS method is that once it is setup it is fairly quick to link entities to Sharepoint lists, and provide the ability to list data, update and delete existing and add new records to entities. Therefore if you have lots of entities to expose via Sharepoint this is going to be the quickest method, since it would take a bit longer to program custom web parts to perform CRUD operations.

Disadvantages

I found the BCS method best worked when you wanted to view/update a singe entity at a time - when I had to had functionality that allowed a form to make changes to multiple entities at once then I had to write custom C# code and web controls to do this. Displaying CRM data via BCS in a list appears to be aimed at just displaying and updating one entity table at a time - if there is more complex functionality then you need to start writing your own custom web parts to do it (see the Extending section below).

Setting Up

Setting it up is the most fiddly bit, I had some initial issues but if you follow the instructions in the white paper provided in the link above you should be ok. Note that while the white paper talks about integrating with Dynamics CRM online, it also works with on premise – however the connector asks you to enter a “device name and id” as part of the configuration but these are only valid for Dynamics Online, not on premise, so you can just enter any value in these (it does insist a value to be entered!).

The connector facility provides a WCF proxy service that sits between Sharepoint and the CRM data (a .net 4 project so takes advantages of the Dynamics CRM SDK). There is also a BCS connector facility (a .net 3.5 project) that runs in Sharepoint and talks to the proxy service to get the data from Dynamics and present it to Sharepoint.

Exending

One good thing about the connector facility and proxy service provided is that it is open source and you can get under the bonnet and make your own changes – I found I needed to do quite a bit of this for certain additional functionality I required. Also if you want to develop your own custom web parts, you could additionally program them against this .net 4 proxy service provided to get the entities etc rather than directly to the CRM WCF web service – personally I found this the easier method of doing my custom web part programming since the proxy service provides all the main operations required ie get entity, get entities, add, edit, delete.

Presentation by Girish Raja on Channel9.

View the full link here.

Presentation by Chris Auld on Channel9.

Similar sort of presentation to the above but both worth a look.

View the full link here.

Polling a Web Service/WCF from Biztalk

Wed, 09 Jan 2013 09:25:37 GMT

One of the requirements on a project I was working on was to use Biztalk to poll a web service using it's WCF adapters on a regular basis eg every few minutes. I set off looking at the possibilities of how to develop this only to find that the Biztalk WCF receive adapters to not have configuration to allow you to poll at intervals. BizTalk does nothing unless somebody "knocks on the door", so out-of-the-box BizTalk cannot call a WCF Service on a scheduled basic (the WCF http Receive Adapter actually works the other way around, somebody has to call "our" Service for something to happen). BizTalk is not a job scheduler, but this sort of workflow does comes up a lot when organizations already use BizTalk. so the best bet was to find a reliable (and resilient, monitored, logged, etc.) means of triggering the process, telling BizTalk when it is time to call the WCF service.

To achieve the requirement, there needed to be some sort of schedule mechanism in BizTalk (SQL Receive or the 3rd party Schedule Adapter) that starts a Socilit-Response Send Port that calls the Service and returns the response.

The were the options:

Scheduled Task Adapter

Use the Schedule Task Adapter (http://biztalkscheduledtask.codeplex.com/) to receive a message that an orchestration subscribes to.

Windows Scheduled Task

Run a scheduled task (under the Windows scheduler, but you could use any scheduling tool) that creates a "control message" and drops it in a folder. BizTalk watches the folder, and when it sees the control message it calls the web service. This allows for very flexible scheduling. You could poll the web service every other Wednesday at 3:00 PM if you want to. You can user Powershell to write a file or to a message queue that BizTalk sucks in to kick off its process

Write a Windows Service

Create a basic windows service which runs at scheduled time and drops a message into BizTalk and Biztalk picks up.

Third-party job scheduling software (particularly if already in use; even SQL Server can do this for you).

SQL Adapter

The SQL adapter can be configured to poll a stored procedure every few minutes, then creata an orchestraion instance on any received message. If the stored proc always returns a message, you can then get the orchestration to call the web service then process the message.

Singleton Orchestration

Have a singleton orchestration that calls the web service, then delays 10 minutes, in a while-true loop. Use a control message to start the orchestration.

Build a Custom Adapter

More difficult - either: Build a web service polling adapter using the LOB adapter SDK, or build a web service polling adapter using the BizTalk Adapter framework. Building an adapter is not an easy task, but it's easyer with the LOB SDK than the BizTalk Adapter framework.

Windows Server AppFabric and Windows Workflow Foundation: Brief Introduction

Wed, 02 Jan 2013 21:55:08 GMT

Windows Server AppFabric is a set of extensions on top of Windows Server improving the application infrastructure allowing to build, scale and manage Web and composite applications. AppFabric comes with the following capabilities:

Enhanced hosting and management: AppFabric enhances IIS Web Server features with the possibility to monitor and track WCF services. It provides the capability to monitor activities of hosted WCF and WF servers. Additionally it provides a way to save the state of hosted running workflows making it easing to create long-running workflow processes.
Caching: adds a distributed, in-memory object cache to Windows Server making it easier to scale out services.

Windows Workflow Foundation

Windows Workflow Foundation (WF) is a technology that allows to describe a business process as a workflow comprised of a discrete series of steps that involves people and software. It allows any .NET application to implement maintainable long-running processes and encapsulate blocks of functionality into Common Language Runtime object. This approach enables the enterprise developer to create more maintainable applications thus improving both the stability of the application as decrease the development effort. The latest version of WF supports the following workflow types:

Sequential workflows: allows for the execution of activities in a pre-defined pattern. This type of workflow is mostly used in situations where no human intervention occurs and a structured set of steps can be executed.
State machine workflows: these are behavior based workflows where the activities change from state based on incoming events. This type of workflow is suited best for situations where human interaction occurs.

Windows WF are supported as durable (state full) and non-durable (stateless) services on premise.

Biztalk vs Workflow Foundation 4

Sat, 29 Dec 2012 21:56:06 GMT

Architects and developers have to choose carefully in which technology to invest. With this in mind, for some recent developments I was looking at putting together an architecture for a service bus and assessing which technology would be the best to use - Biztalk or Workflow Foundation/WCF.

The service bus I was to develop would need to handle a number of different message patterns for example low latency request-responses, as well as picking up data from file and ftp locations and just sending them straight to other systems without responses back (ie fire-and-forget). Both of these message patterns could be easily handle by a pure Biztalk approach, although I considered that Biztalk is not known for low latency request-responses - whilst it can fulfill these rapid requests-responses there are time lags caused within Biztalk as it records messages to its message database etc for persistence, so other methods such as WCF/Workflow would provde to be quicker.

I started to consider a more hybrid approach - ie utilizing a combination of both Biztalk and WF/WCF and using the right tool for the particular job, however I realised I needed some sort of framework for assessing which tool would be best for the different parts of the project.

Project Technical Requirements

Here is an idea of the functionality the service bus I was working on required:

Request-responses to 3rd party systems for data, message persistence and retrys not required - perhaps an ideal candidate for WF/WCF
Regular flat file and xml data drops - data would need to be picked up and routed to systems. An ideal candidate would be Biztalk as it provides lots of adapters (FTP, file etc) which would make retrieving the data easy, whereas WF only really allows for web services
Message transformations - xml would need to be transformed into formats suitable for recipient systems. Again Biztalk is great at this, but this could equally be done in WF if it is sending the data via web services
Message persistence - alot of the messages we didn't want to lose - Biztalk provides message persistence, retries etc out-of-the-box, but also WF/Appfabric provides a persistence facility of its own

An overview of the technologies

Biztalk

BizTalk is used for system integration. for example, you run a factory, you may have a ERP system. when you order product parts from other factories, your system need to communicate with other systems. Now, it is time to use BizTalk. BizTalk can connect different system together, map message schemas. route messages(Orchestration). BizTalk Orchestration is a message driven workflow that can coorinate different systems work together to get things done. A brief summary of features:

Publish / Subscribe model (messagebox)
Adapters (Legacy application access, non based WCF ones)
Configurable retry logic with send and receive of messages
Flat file parsing
Debatching of messages
Business Activity Monitoring (BAM)
Visual mapper (BizTalk mapper). Transformation defined graphically and performed in XSLT
Business Rules Engine (BRE)
Consuming External Web Services – Reference WS and contract is generated
Debugging – Can be done from Visual Studio but you need to use and read the compiler

Workflow Foundation

WF4 is a workflow platform you can use WF4 to develop any system that involve human interaction, long running feature. However, WF4 doesn't provde any adatper like ftp, file that can connect different system together. WF4 does't provide mapper and schema editor either. Simply put. WF4 is a platform by which you can create orchestration kind workflow in any system installed .net 4.0.

The new things in the WCF4, WF4 and AppFabric space that where previously unique to BizTalk:

Host out of the Box: Scalable, reliable, load balancedmonitorable
Durability and persistence
Suspend / resume functionality
Extensive set of tools provided for IT pros to administer and track execution
Health monitoring
Failed-message management
Message tracking + archiving and purging
Long running and atomic (ACID) transactions with compensation mechanisms
Consuming External Web Services – Need to write some code and we’re tightly coupled
Transformations - .NET Code based
Debugging – More natural in Visual Studio then BizTalk

Appfabric is a enhanced WCF/WF4 service host, a place where you can host your WF and WCF application. it is kind of a plug-in for IIS7/7.5. You will gain a lot of benefit by using Appfabric. if you don't use Appfabric. IIS7 Installed with .NET 4.0 can also serve as a WF4 service host.

A Decision Matrix

When deciding on which technology to use, a decision matrix whereby the most features wins is not always the best way to go. You can end up with an apples-to-oranges comparison, if those features do not relate to the funtionality that you are targeting. That being said, there is definitely overlap in what the two products provide so I considered it would be useful to have a look at the two technologies and compare the features available.

	Biztalk	Workflow foundation
Latency	BizTalk processes are considered heavyweight and it is usually difficult to achieve very low latency with BizTalk.	WF workflows hosted in IIS do usually provide lower latencies than BizTalk equivalent
Persistence	Provides real messaging reliability. includes a SQL Server database that, among other things, provide persistance to ensure resilience in the face or hardware failures or software crash	The WF runtime provides services and features important when writing application logic such persistence of state, bookmarking and resumption of business logic
Performance	Streaming for large messages throttling happens automatically.	WF does not provide, out of the box, a streaming XSLT transform engine, suitable for processing large messages without exhausting all available server resources.
Customisable Business Rules	Rule engine is still the most powerfull BizTalk’s Rule Engine (BRE) is a complex and powerful Rules Engine that has many advanced features compared to the WF Rules Engine. A few of these are: An external Rules Composer application Support for forward-chaining Support for Vocabulary, which is a way to abstract the rules into domain specific (business)language. Support for multiple fact types: XML, .NET Objects and SQL Database structures Support for multiple facts going into and coming out of the BRE A testing tool within the Rules Composer Support for versioning Rules A database rules store	The WF Rules Engine has: A Rules Editor that resides within Visual Studio Support for forward chaining as well as options for defining how dependencies are defined Supports .NET Objects as Facts Supports one fact going into the Rules Engine and one fact coming out No support for versioning Rules Compiled (within the workflow’s assembly) rule store (although rules can be stored in external rule stores if you build the support yourself)
Mapping	BizTalk natively supports, with enhanced designers to boot, the ability to produce schemas and maps. undoubtedly a good thing in BizTalk - given that you can map either visually or directly in XSLT, you can quickly transform message formats on Ports or Orchestrations	you did needed a full BizTalk license for consuming a BizTalk Mapper Activity which seems to be a limiting factor in its adoption install and use the BizTalk runtime, even just for the ability to run maps inside a workflow, you have to have a BizTalk license
Adapters	Leave the physical technology used as an afterthought - i.e. you can implement your design at a logical level and woun't get 'bogged down' in any specific technology (MQ, WCF, SQL, FTP etc). Switching from say at file to MQ Series is as simple as changing the port configuration	You can, currently, use the BizTalk LOB Adapters with WF (once again with a BizTalk license).
Operational	BizTalk has the following operational benefits: Operational Management - Tracking / tracing, management of Suspended messages, SCOM integration packs, etc Scalability / Clustering / Failover capabilities, adapter retries, automated throttling, etc Business Monitoring and Eventing - BAM	Windows Server AppFabric provides a set of Windows Server extensions that act as an infrastructure for managing WF/WCF operations.
Future	Going forward, I anticipate that we will see more and more BizTalk functionality ported over to the IIS/AppFabric world	People have been predicting the end of the Biztalk product for a number of years now (I recently read a 5 year old article discussing this). However Microsoft will soon be releasing 2013 edition so it will be here, and supported for a long time to come. Biztalk is still the most comprehensive integration solution available.

Useful Related Reading

Code Project - WF vs Biztalk
Comparison between Windows Workflow Foundation and Biz Talk Server
Applied Architecture Patterns on the Microsoft Platform
Useful book that provides an architectural methodology for choosing Microsoft application platform technologies to meet the requirements of your solution.
Hosting Comparison
Whether or Not to Use Biztalk Server or AppFabric to Host .NET 4 WCF/WF Services
Cloud Casts on Biztalk and AppFabric
Lots of Interesting Web Casts on both Biztalk and AppFabric
Microsoft Technet - BizTalk + WF/WCF, Better Together
BizTalk vs. WF/WCF Smack Down
An assessment of the technologies but on previous versions.
BizTalk Server 2006 or WF?
Choosing the right workflow tool for your project - again on older versionfs of Biztalk and WF.
Using the BizTalk Adapter Pack and AppFabric Connect in a Workflow Service
Using the New BizTalk Mapper Shape in a Windows Workflow Service
A Developer's Introduction to Windows Workflow Foundation (WF) in .NET 4
Managing Workflow Services with Windows Server AppFabric

Training Resources for Biztalk Developers

Tue, 18 Dec 2012 15:49:04 GMT

This is just a useful list of learning resources for any new or experienced Biztalk developers. It includes virtual labs, Microsoft documentation for all recent versions, useful video tutorials and web casts, forums, blogs, white papers and courses - just about everything you will need!

Training Resources for BizTalk Administrators

Stock Market/Share Sentiment Analysis Software

Thu, 06 Dec 2012 23:22:43 GMT

I have been working on a side interest project lately called Sentiman, which is going to analysis stock market sentiment.

The web site for the site has been set up and should be launching in early in 2013.

If this is something that interests you, please follow the twitter account on the web site!

File receive port - files not being picked up Biztalk

Wed, 24 Oct 2012 14:47:49 GMT

So you are dropping files to a file receive location but they are just not going anywhere. Here are a few things to check

The host instance associated with file adapter receive is running. Make sure receive location is enabled.
The user associated with host instance has sufficient rights to the file location. The File receive adapter also does not pick up files for which it has insufficient rights and files that cannot be exclusively locked. To correct these problems, verify the security settings of the receive location and ensure that any files present are not being held by other processes. Ensure that the service account for the BizTalk Host that picks up the file must have proper permissions
The file is not read only
The correct file path is specified in the receive location.
The correct filter for file extention is applied in receive location configuration i.e if it set as *.txt, then this will not pick xml files.
Do you see any info in event log?

A few other things to try

The file which does not get copied, Try to cut And paste again in the same folder and see if it makes them to picked by BizTalk. Instead of copying an existing file, try creating a new file and dropping that in
Try creating a new folder for receive location and put those files which does not gets copied.
Try out adding another receive port at a different location and see if the files are picked up from there
Check which host instance the receive port runs under, try out running your receive port under a different host instance. Sometimes people configure all receives ports to run in one host instance, and all send ports, for instance to run in another.
Have you created a new host for the adapter - make sure you've also created a host instance for it too

Also check the application you have written, have a look at any elements of it that might cause a problem eg the schemas, pipelines etc. I recently did a flat file schema for instance that wasn't correct, it meant that the file didn't disappear from the receive location and actually caused my test version of Biztalk to seize up! Check also that the application is not in some sort of erroneous cycle that is picking up and dropping files incorrectly to the same location or message box.

If you have any other suggestions to check please post them...

Failed to configure with error message [Exception of type 'System.EnterpriseServices.TransactionProxyException' was thrown.]

Fri, 19 Oct 2012 15:38:45 GMT

You may get this error when configuring Biztalk 2010 Group section in a distributed environment with Biztalk on one server and SQL server on another.

There are two things you need to check first, which is that:

Configure Microsoft Distributed Transaction Coordinator (MS DTC) is configured.
Network COM+ access considerations is enabled

as per the Biztalk 2010 installation instructions on this page. These are essential parts of the Biztalk installation.

There are also some useful diagnostic tools on that page, in particular performing a DTCPing, and using the DTCTester. These will help you diagnose connectivity issues between your servers.

Plus on these pages there is important information on configuring your Windows Firewall correctly if you have this enabled - these can cause the same error message to occur.

Issues when working in a virtualised environment.

If you are installing Biztalk in a virtual environment where the machines have been cloned then you may find that even after doing all the above you get the same error message.

A very useful tool for debugging DTC is the DTCPing package, as mentioned above, I recommend testing with those to identify any issues. When you try DTCPing you may find the following warning in the DTCPing log file after performing tests between your Biztalk and SQL servers:

WARNING: the CID values for both test machines are the same while this problem won’t stop DTCping test, MSDTC will fail for this …

If you then check the registry on both machines under the HKEY_CLASSES_ROOT\CID key, there should be a group of keys there - if they have identical keys under there in both the Biztalk server and the SQL Server there then this could be the source of your problem as they need to be unique, since the CID values must differ between the servers.

You can fix this by reinstalling MSDTC on either the Biztalk server or the SQL server, I chose to do it on Biztalk.

From the command prompt

msdtc -uninstall

Reboot your machine, then...

msdtc -install
net start msdtc

I noticed by the way that after doing this my Enterprise SSO configuration had to be first unconfigured and then reconfigured before I could proceed.

Biztalk Configuration Error for Enterprise SSO - Failed to add the user 'domain\myuseraccount' to the domain group

Fri, 19 Oct 2012 13:43:10 GMT

I had a problem with configuring Biztalk 2010 in a distributed environment which I thought I would share. The problem was with assigning the SSO Administrators group when configuring the Windows accounts for Enterprise SSO. Since I was using a distributed environment I had to replace the local group name with the domain group name.

When I did so I got the warning icon with the detail:

"Failed to add the user 'domain\myuseraccount' to the domain group 'domain\SSO Administrators'. To add members to domain groups you must have sufficient permissions in the domain."

The error message puzzles me because I don’t know why the group name could not be found. I found it when I browsed for it, so this error message must be misleading. Also it appears to be wanting to add my own user account to that group.

There were not many posts about this on the internet however I found the following being suggested (none of which worked for myself):

One suggested to delete the existing entry and manually type in the account name - DON'T click on the ellipses (...).
Strangely another post suggested - make sure you click on the ellipses when selecting the SSO Adminstrators group!
There are blogs that say you need to re-register SSOSQL.DLL that is located in C:\Program Files\Common Files\Enterprise Single Sign-On (and also in the sub-directory \Win32). I tried that but it didn't work.
In addition to the above I also read about the same issue occuring in Biztalk 2006, for which a patch was provided by Microsoft so it was a known issue even then, but obviously I couldn't use this as it was for a previous version.

Reason

I believe the reason for this occuring is that, according to Microsoft Biztalk Installation guide, it states that:

The user running the BizTalk Server configuration must belong to the following user groups:

To the Administrators group on the local computer

To the System Administrators group on the SQL Server computer
To the domain group used for the BizTalk Server Administrators group
And to the domain group used for the SSO Administrators group

I suggest that this may be occurring because the last bullet point hasn't been implemented.

Workaround solution?

In the end I had to opt for a workaround:

I temporarily added the SSO Administrators group to the Administrator group on the server itself and to the remote desktop users group. I then logged on to the machine using the SSO Admin domain user account which was already a part of that group. I proceeded to run the Biztalk Configuration wizard under that account. Under these circumstances the SSO would install, since the configuration wizard, did not try to add the logged on user account (SSO admin user) to the SSO Administrators group, because it was already a member of this group.

Ok - its not an ideal workaround but this was the only method that appeared to work, if you have your own suggested method please comment.