http://msdn.microsoft.com/en-us/library/dd334420.aspx

Agent-Design Patterns for Building Distributed Service Bus Applications

7 out of 9 rated this helpful - Rate this topic

Danny Garber
Microsoft Corporation

November 2008

Applies to:
   Microsoft .NET Framework 3.5
   Microsoft BizTalk Server 2006 R2
   Microsoft Enterprise Service Bus Guidance
   Microsoft Managed Services Engine
   Windows Communication Foundation
   Windows Presentation Foundation

Summary: This article discusses architectural patterns in distributed systems, DSB agents, technology-realizing patterns, Itinerary and Ticket Forward traveling agent-design patterns, Message Inspector in WCF, and behaviors in WCF. (18 printed pages)

Contents

Introduction
Distributed Service Bus (DSB) Agents
Agent Itinerary and Ticket Forward Design Patterns
Conclusion
References

Introduction

Over the years, distributed-systems architecture has shown rapid evolution by applying and combining different architectural patterns and styles. In recent years, decoupling interfaces and implementation, scalable hosting models, and service orientation were the prominent tenets of building distributed systems. Distributed systems have grown from independent enterprise applications to Internet-connected networks of managed services, hosted either on-premise or in clouds.

The cloud-computing paradigm has the potential to increase the efficiency and effectiveness of the development of complex and customizable distributed systems. In spite of this, the greater flexibility that is offered by cloud computing comes at additional costs. Designing and implementing distributed code systems is more complex than systems that are based on the traditional client-server paradigm, as complete mobility of cooperating applications forms large-scale, loosely coupled, and complex distributed systems. Furthermore, cloud-services system development is not yet fully supported by sound technologies or methodological background.

As the technologies that support design, development, and deployment of distributed applications continue to advance, it is more important than ever that architects and developers establish stable libraries of architectural patterns, based on recurrent experiences with development of such distributed solutions.

Agent-design patterns are a recognized means for promoting the development of distributed enterprise services. Several agent-design patterns already have been proposed, such as an Itinerary pattern within Microsoft Enterprise Service Bus Guidance [ESBG] and a Ticket Forward pattern that is implemented in Microsoft Managed Services Engine [MSE].

This article aims to go further and deeper in the explanation of agent-design patterns by identifying the real-world forces and contexts of the problems that gave rise to the distributed design patterns and corresponding technology realizations.

Simply speaking, agent-design patterns can make your distributed applications more flexible, understandable, and reusable, which is probably why you were interested in distributed-computing technology in the first place. In this article, we report on two agent-design patterns that have been harvested from real-world Microsoft Consulting Services engagements to support building a distributed service bus (DSB) agent.

Distributed Service Bus (DSB) Agents

DSB addresses cross-enterprise service-collaboration scenarios. Organizations that attempt to implement such scenarios with a traditional Web services composition/enterprise service bus (ESB) approach are facing tremendous challenges when it comes to spanning business-process workflows across the enterprise boundaries. Firewalls, network-deployment DMZ zones, and software-product boundaries are some of the many obstacles that enterprise architects must overcome.

DSB agents are an emerging technology that makes it much easier to design, implement, and maintain distributed systems. A DSB agent is not bound to the system in which it begins execution; it has the unique ability to transport itself from one system in a network to another. This ability to travel allows a DSB agent to move to a system that contains an object with which the agent wants to interact, and then to take advantage of being in the same host or network as that object.

Figure 1. A distributed service bus (DSB) ecosystem

Figure 1 provides an extremely high-level overview of the DSB ecosystem. First, the DSB is ubiquitous, and it connects all systems and servers. There will be many composite applications in which some elements implement an itinerary of their own "ESB" and others implement a "main" itinerary on a DSB. Multi-organization composite applications are one obvious example that would deploy elements onto a DSB in the cloud. Another possibility is single-organization composite applications that have a centralized DSB managing distributed ESBs. You will find that DSB agents reduce network traffic, provide an effective means of overcoming network latency, and—perhaps more importantly—through their ability to operate asynchronously and autonomously of the process that created them, help you to construct more robust and fault-tolerant applications.

However, as in other emerging technologies, we are all pioneers, repeatedly inventing and reinventing sometimes smart but perhaps more often not-so-smart solutions to recurrent problems. During the early work on consulting engagements, Microsoft developers recognized a number of recurrent patterns in the design of distributed applications. Several of these patterns were given intuitively meaningful names, such as Itinerary and Message Routing patterns. They were implemented in Enterprise Service Bus Guidance [ESBG] and included in the Managed Services Engine [MSE]. These early patterns were found to be highly successfully for jump-starting users who were new to the distributed-services paradigm.

This experience tells us that it is important to identify the elements of good and reusable designs for distributed-service applications, and to start formalizing people's experience with these designs. This is the role of agent-design patterns. The concept originated with software engineers and researchers in the object-oriented community, and has been recognized as one of the more significant innovations in the object-oriented field.

Agent Itinerary and Ticket Forward Design Patterns

The focus of this article is a new set of agent-design traveling patterns for creating DSB applications. We present Itinerary and Ticket Forward patterns, which, when combined, can help to build very effective, reusable DSB architectures. The patterns that we have discovered so far can be divided conceptually into three classes: traveling, task, and interaction [ADP]. This classification scheme makes it easier to understand the domain and application of each pattern, distinguish different patterns, and discover new patterns.

Table 1. Traveling patterns [ADP]

Itinerary	Objectifies agent itineraries and routing among destinations
Forward	Provides a way for a host to forward newly arrived agents automatically to another host
Ticket	Objectifies a destination address, and encapsulates the quality of service and permissions that are needed to dispatch an agent to a host address and execute it there

 

The agent-design traveling patterns deal with various aspects of managing the movements of DSB agents, such as routing and quality of service. These patterns allow us to enforce encapsulation of mobility management that enhances reuse and simplifies agent design.

The Distributed Itinerary pattern is one example of a traveling pattern that is concerned with routing among multiple destinations, as shown in Figure 2.

Figure 2. Distributed Itinerary Pattern

The agent receives an itinerary on the source agency, indicating the sequence of agencies it should visit. Once in an agency, the agent executes its task locally and then continues on its itinerary. After visiting the last agency, the agent goes back to its source agency. An itinerary maintains a list of destinations, it defines a routing scheme, handles special cases such as what to do if a destination does not exist, and always knows where to go next. Objectifying the itinerary allows one to save it and reuse it later—in much the same way that one saves URLs as bookmarks.

The Ticket pattern is an enriched version of a URL that embodies requirements concerning quality of service, permissions, and other data. For example, it may include time-out information for a dispatching an agent to a remote data center in our customer scenario. It can also contain the security and auditing metadata that is required by the regulations and controlled (injected) by the execution policies. Thus, instead of naively trying to dispatch to a disconnected host forever, the agent now has the necessary information to make reasonable decisions while traveling.

Being an autonomous mobile entity, an agent is capable of navigating itself independently to multiple hosts. Specifically, it should be able to handle exceptions such as unknown hosts while trying to dispatch itself to new destinations or make a composite tour (for example, return to destinations it has visited already). It might even have to modify its itinerary dynamically. Consequently, it is probably preferable to separate the handling of navigation from the behavior and message handling of the agent, thus promoting modularity of every part. The Distributed Itinerary pattern lets you do so.

The behavior of agent-design traveling patterns is often prescribed by a set of tasks represented in an itinerary. We have used a variety of agent itinerary styles, ranging from simple, sequential expressions to complex, finite state-based representations. A classic example of the Itinerary pattern can be found in various implementations of the enterprise service bus [ESB]. However, it cannot solve alone the cross-enterprise service-collaboration scenario requirements. An Itinerary pattern implementation that uses Microsoft ESB Guidance (as well as the ESB solutions of other competitor) is limited to a perimeter network of the Microsoft BizTalk Server deployment environment, on which ESB Guidance is heavily dependent.

Fortunately, in tandem with the Ticket Forward traveling pattern, the Itinerary-Ticket Pattern makes it possible to implement DSB cross-enterprise service-collaboration scenarios that are almost transparent of the number of "hops" between various connections that can be found across the boundaries of multi-organization composite applications.

The remainder of this article explores several technical concepts of building a DSB by using Microsoft technologies that are widely available today.

Scenario Description

During the last year, we worked with a medical-insurance provider. This company insures millions of customers, and its insurance-claim application must interact with customers, providers, third-party agencies, and others. This example is a composite of a number of real-world insurance projects. This company decided to embark on a service-oriented architecture (SOA) project as a way to modernize its information technology, while leveraging existing legacy data centers and Web services that are deployed nationwide. A main data center is connected in a cross-star network to several satellite data centers, as well as to the medical claim management agencies of the third-party partner.

This medical-insurance provider decided to embrace all of its data-center services and line-of-business (LOB) applications into a net of workflow processes, utilizing the Itinerary pattern, which is controlled dynamically by business rules and run-time policies and SLAs.

Key Functional and Nonfunctional Characteristics

The overall project key functional requirements that Microsoft and the insurance company used to define the architecture included:

• Leverage legacy data-center systems, while creating new services on the platform of choice for that system.
• Support multiple communications protocols (WCF, SOAP, REST, and so on).
• Support SOA interactions securely with outside organizations.
• Operate in an environment that has several geographically dispersed data centers.
• Implement multiple itinerary workflows, some of which are nested in each other.
• Provide the ability for real-time monitoring and routing of transactions, based upon desired level of service.

Patterns

The use of traveling agents can bring the following advantages to applications and their users:

• Reduction of network traffic, as interactions can be carried out locally, independent of network latency
• Asynchronous and decentralized execution, which allows the user to disconnect from the network when agents are performing a task
• The ability to detect changes in the execution environment and react autonomously, which simplifies the development of distributed systems that are more robust and fault-tolerant

One might argue that these advantages can be obtained when using other paradigms. However, despite this, the great advantage of traveling agents is to combine these advantages in a single and more abstract paradigm. We have selected the Itinerary-Ticket pattern for this customer scenario, which is described earlier in the article.

The design and implementation of an itinerary can be a complex, time intensive task, particularly in B2B and cross-enterprise architectures, where resources and task execution occur in a distributed network. Often times, you need more sophisticated means of controlling process flow. More complex business processes might involve multiple paths of execution to happen in parallel. These multiple paths can be handled through a combination of process itineraries and special services by placing a fan-out service within an itinerary that splits the process into two parallel subprocesses. To address this issue, we have developed a set of reusable, flexible agent design patterns that enable the rapid development of complex agent itineraries. These patterns have reduced our time to develop distributed applications by up to 40 percent.

Figure 3 illustrates these patterns:

Figure 3. The Itinerary-Ticket patterns in distributed ESB implementation

Performing a simple join operation using itineraries means waiting for specific messages to arrive at a service. In this simple case, one can use an aggregator service along with some special design patterns that might require coding techniques.

These patterns can be implemented by using a hybrid SOA architecture that consists of Microsoft BizTalk Server 2006 R2, MSE, and ESB Guidance components. The need for legacy integration, service orchestration, and dynamic itinerary routing drives the architecture decisions in the use of this technology set.

The system-context diagram in Figure 4 illustrates the interaction of these components:

Figure 4. The insurance-company DSB conceptual architecture

The collaboration between the participants in the Itinerary-Ticket pattern is as follows (see Figure 5):

1. An outside agency submits a request to a main data center.
2. The Main Data Center Service Gateway, implemented in MSE, receives the outside request (GetData) and forwards it to the corresponding published virtual Web service.
3. During the service routing, implemented by using the WCF Service Behavior technique, MSE (in order to obtain the itinerary path for the requested service call) calls the Resolver service, which could be implemented in any of the common service-info containers: UDDI, SQL, or BRE.
4. MSE also can apply the run-time execution dynamic policies (such as for user authentication and authorization, itinerary path injection, and so on) while the service request is still in the MSE domain.
5. Finally, MSE invokes the corresponding ESB agent, which is subscribed to the itinerary destination of that service request.
6. ESB dynamically transform the requesting message to the canonical internal format and creates ESB Slave agents, each one with its prescribed itinerary path. These agents transfer the message to the destination that is listed in the itinerary and receive the message response.
7. Thus, an ESB Slave agent might call the legacy LOB application via a WCF service call, MQ Series adapter, or SOAP adapter, depending on the implemented transport of the LOB application.
8. Upon receiving the response from the LOB application, ESB Master Agent advances the itinerary path and forwards the message agent to the next itinerary destination (CallOtherDatacenter).
9. Because one of the itinerary steps that includes execution of business workflow resides in a remote data center, the nested ESB itinerary is invoked via an MSE intermediary, as Figure 4 shows.
10. The internal process of the remote data center continues until all of the destinations that are listed in the itinerary path are reached and executed.
11. As soon as the itinerary path is completed, the resulting message is returned to the URL of the main data center that is specified in the requested message metadata.

Figure 5. Collaboration in the Itinerary-Ticket pattern for the DSB implementation of the insurance company

Building a WPF Agency Client Application

As mentioned earlier, the insurance company receives eligibility claims from its subsidiary agencies, which are deployed nationwide in various member benefits and services offices. The code for submitting claims and displaying results can be written in any end-user format (Windows Forms, Web pages, Microsoft Office SharePoint, and so on) that is appropriate to the agency. However, for this particular customer, we could just create a nice, rich Windows Presentation Foundation (WPF) client application, as Figure 6 shows:

Figure 6. The WPF client application of the claim agency

WPF is a great technology for creating compelling user interfaces, but that does not mean that you do not have to take into account the responsiveness of your application. The simple fact is, no matter what type of long-running processes are involved—whether getting large results from a database, making asynchronous Web service calls, or any number of other potentially intensive operations—making your application more responsive is guaranteed to make your users much happier in the long run.

Fortunately, to make user experience seamless when calling a long-running itinerary processing workflow, the claim-agency client application leverages a WPF BackgroundWorker threading model. The BackgroundWorker component works well with WPF, because it allows the processing of Web service calls in a separate thread underneath using the AsyncOperationManager class that provides asynchronous behavior according to the Event-Based Asynchronous Pattern.

First, we must initialize the BackgroundWorker component and set up its events:

private BackgroundWorker mBackgroundServiceRequestWorker = new BackgroundWorker();  public MainPage()  {                    ... //other code                 // Set up the Background Worker Events              mBackgroundServiceRequestWorker.DoWork += new DoWorkEventHandler(mBackgroundServiceRequestWorker_DoWork);              mBackgroundServiceRequestWorker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(mBackgroundServiceRequestWorker_RunWorkerCompleted);  }     

When the user is ready to submit a claim by pressing the Submit button on the WPF window form, the only thing that you must do to start a new thread is to call the RunWorkerAsync method of BackgroundWorker, passing the appropriate parameters—in our case, the list of all employee claims:

private void Submit_Click(object sender, RoutedEventArgs e)  {  ArrayList empList = new ArrayList();  ... // collect the list of employees    //Start Eligibility Process Remote service in the background thread  mBackgroundServiceRequestWorker.RunWorkerAsync(empList);  }     

A new background thread is started, and its first event, "DoWork," is fired following the callback method call that is associated with this event. This is the place in which the background thread invokes a long-running Web service call that is presented by its proxy:

void mBackgroundServiceRequestWorker_DoWork(object sender, DoWorkEventArgs e)  {                  ArrayList empList = (ArrayList)e.Argument;                  //Perform WCF Web service call                  using (svcEligibilityServices.svc_EligibilityServicesSoapClient client = new Contoso.Insurance.ClaimAgencyApp.svcEligibilityServices.svc_EligibilityServicesSoapClient())                  {                      //Open channel                      if (client.State != System.ServiceModel.CommunicationState.Opened)                          client.Open();                        //Submit Response                      string response = client.RequestEligibility(mHeaderInfo,                                         (svcEligibilityServices.ConsumerFileConsumer[])empList.ToArray(typeof(svcEligibilityServices.ConsumerFileConsumer)),                                              out mEligibilityResponse.Status,                                              out mEligibilityResponse.Message);                      client.Close();                      e.Result = mHeaderInfo;                  }  }     

Note how the DoWorkEventArgs event argument parameter is cast into the submitted ArrayList that contains the list of all employee claims.

As soon as the asynchronous service call is completed, the corresponding event call back is fired; there, you can check the status of the requested call and take appropriate action:

void mBackgroundServiceRequestWorker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)   {              if (e.Cancelled)              {              //Do something about it!              }              else if (e.Error != null)              {              //Do something about it!              }              else              {        //Everything is looking good!              }  }     

Applying Run-Time ESB Itinerary Policy

Up to this point, we have just set up a client application that collects the data and submits claims via a dedicated Web service proxy using WCF binding protocol. Nothing has been said yet about the role that MSE plays in this scenario. Besides its service-virtualization capability, MSE offers a lot of other functionality that sometimes is overlooked by MSE consumers and developers.

One such powerful feature can be found in MSE: the ability to enforce a run-time policy execution on the operation of the specific Web service. In the DSB implementation of our insurance company, we must inject the ESB itinerary path into an incoming message before it has been routed down the chain of the centralized main service bus. By using the MSE run-time policy container, where you can define your own policy (see Figure 7), and through the policy assignment, we can define our own policy that will be executed each time that a message is received by the MSE Dispatch component.

Figure 7. The run-time policy definition in the MSE console

The ESB Itinerary Injector run-time policy that has been defined in the policy container then can be assigned to the Web services operation that is used to receive all incoming eligibility claims from the member services agencies, as Figure 8 shows:

Figure 8. The run-time policy assignment in the MSE console

WCF Message Inspector Implementation

The run-time policy execution is available via WCF Behavior Extensions and WCF Message Inspector implementation. A MSE Policy insertion allows us to develop a WCF Message Inspector for an individual operation of the service, which makes the WCF programming even more flexible and exciting. WCF Behavior offers the following three interfaces: IServiceBehavior, IEndpointBehavior, and IOperationBehavior. As you can see, these three interfaces allow developers to extend the behavior of services, endpoints, and operations. Often, the behavior is responsible for configuring "Inspectors" that then will be injected into the processing pipeline. While behaviors are the means by which you change the behavior of WCF, they often are not the ones that do the actual work.

Inspectors are how you actually change the processing pipeline of WCF. By creating an IDispatchMessageInspector, you can preprocess messages as they enter and exit the WCF pipeline:

public class ItineraryInjector : IEndpointBehavior, IDispatchMessageInspector, IClientMessageInspector  {          #region IEndpointBehavior Members          //Insert here implementation of the interface members           ...          

Đăng ký: Đăng Nhận xét (Atom)