http://msdn.microsoft.com/en-us/library/bb833024.aspx

Chapter 3: Workflow and Process

5 out of 7 rated this helpful - Rate this topic

"The art of progress is to preserve order amid change

and to preserve change amid order."

- Alfred North Whitehead

Mathematician, philosopher (1861-1947)

Contents

Reader ROI
Acknowledgements
Understanding Workflow
A Workflow Manifesto
Understanding the Relationship between BizTalk Server and WF
Conclusion
SOA Case Study: Dollar Thrifty Automotive Group
References

Reader ROI

Readers of this Chapter will build upon the concepts introduced in previous chapters, specifically focusing on the Workflow and Process architectural capability.

Figure 1. Recurring Architectural Capabilities

The Workflow and Process architectural capability focuses on the concept of workflow and how workflow can be used to orchestrate and aggregate granular services into larger constructs that we call processes.

Topics discussed in this Chapter include:

• Explaining the concepts of workflow
• Clarifying the relationship of BPM, SOA, orchestration and choreography to workflow
• Describing the capabilities necessary for supporting workflow
• The benefits of adopting a model-based approach to workflow

Acknowledgements

This Chapter consists of work from Dave Green (concepts, semantics, value and capabilities, Windows Workflow Foundation) and John Evdemon (workflow concepts, terms and manifesto).

Understanding Workflow

What is Workflow?

The concept of workflow is not new. Workflow technologies first emerged in the mid-1970s with simple office automation prototypes at Xerox Parc and the University of Pennsylvania's Wharton School of Business. Interest in workflow and office automation began to wane in the early 1990s until the book Reengineering the Corporation reignited interest in workflow and business processes. The reengineering trend of the 1990s gave us several books and methodologies for process analysis – unfortunately the technologies such as CASE and their ilk were immature and required significant manual intervention, exposing projects and executive stakeholders to significant levels of risk. Even worse, other "workflow products" were thinly disguised efforts to sell hardware such as scanners, printers, and other peripherals. Clearly the term "workflow" was being abused to take advantage of confusion in the market

So what is workflow? Workflow is fundamentally about the organization of work. It is a set of activities that coordinate people and / or software. Communicating this organization to humans and automated processes is the value-add that workflow provides to our solutions. Workflows are fractal. This means a workflow may consist of other workflows (each of which may consist of aggregated services). The workflow model encourages reuse and agility, leading to more flexible business processes.

Workflow Terminology

As stated above, workflow is about the organization of work. This is a very broad definition and may be interpreted in a number of ways. To avoid confusion we will identify several terms that are commonly associated with the concept of workflow:

• Business Process Management (BPM): BPM is a business philosophy that views processes as a set of competitive assets to be managed. The processes that business seeks to manage are largely composed upon and executed by a SOA infrastructure. SOA and BPM have several objectives in common:

o        Both are focused on making the organization more agile. SOA focuses on providing a loosely coupled infrastructure for service enablement while BPM focuses on effectively designing and managing loosely coupled business processes (which are aggregations of granular services).

o        Both seek to make the business proactive instead of reactive. A proactive business is able to spot market patterns, trends and competitive threats before they can negatively impact the organization. Simply identifying a threat or opportunity is not enough – the organization must also be able to proactively modify its existing processes to take advantage of the opportunity (or avoid the threat). Achieving this level of agility requires loosely-coupled business processes atop by an SOA infrastructure capable of supporting rapid reconfigurations to meet the needs of the organization.

o        Both are focused on process-centric instead of functional capabilities. Processes can no longer be constrained by a limited set functional capabilities – an SOA infrastructure is a fabric of configurable, composable services that can be assembled/disassembled or reconfigured to meet the shifting business needs of the organization.

• Orchestration: Workflows and orchestrations are effectively the same thing since each is designed to compose and execute a series of tasks. Orchestrations require a "conductor" that is always in charge of the execution. The conductor is typically manifested as an integration server (such as BizTalk Server) which monitors execution, raises events, creates execution logs and performs various other duties to ensure the process executes as expected.
• Choreography: Choreography is a set of peer to peer relationships between individual participants. There is no "conductor" in choreography. Participants interact with one another based upon a set of agreed-upon principles or contracts. For example, a Supplier might enter into an agreement with a Retailer to ship a specific set of goods and services within a given timeframe. This means the Supplier and Retailer must agree upon business events, business documents, service level agreements (SLAs) and penalties if the SLAs are not met. The Supplier may in turn enter into another agreement with a Manufacturer to supply the raw materials that the Supplier needs to meet the Retailer's demands. A choreography is usually implemented by "splitting" it up into multiple orchestrations (one for each participant involved in the choreography).

Discussions about workflow frequently mention one of the terms explained above. Orchestration is the only term that is comparable to the concept of workflow.

Why Workflow?

The problems which a workflow approach has been applied often display three characteristics: The key business value delivered is coordination, for instance, in organizing multiple contributions to the preparation of a quote or driving a document review. Each instance of the business process concerned is of long duration, measured often in days, weeks, or months, rather than minutes. The business process has human participants, who usually contribute most of the work product.

However, only a small proportion of the business problems with these characteristics are solved using a workflow approach. Most commonly, the business process is not recorded as machine-readable information at all. Rather, each of the humans participating in the business process interacts with business systems that are not aware of the semantics of the process as a whole, such as a customer information system, and with other human participants through content-neutral communications channels such as e-mail. Each human participant uses a mental model of their part in the overall business process to determine their behavior.

Three key benefits that a workflow model can bring are insight, monitoring, and optimization. A set of related workflow models can be used to gain insight into the flow of work through an organization. For monitoring, knowing which individuals are contributing work to which business process is very useful when trying to understand costs and workloads. For optimization, having a model of the work being undertaken, and being able to use the model to interpret behavior, together make it possible to reason about how to optimize the business process.

The concept of a model-driven approach is attractive it is not new – developers have been using UML and other modeling techniques for many years prior to the maturation of workflow development tools. Workflow provides either a sequential or state machine model that is used to develop, manage and run the workflow itself. Code-only approaches may take advantage of models to lesser success – for example we can parse C# code into an abstract CodeDOM model – but the usefulness of such a model is limited.

A Workflow Model

Given these compelling benefits, why haven't workflow models been used more widely? The most likely answer is that the cost of using them has been too high. These costs include product costs, that is, the direct cost of purchasing a workflow product; integration costs, where processes modeled as workflows need to be integrated as part of a larger business system; and standardization costs, where it is difficult for a large organization to standardize on a single workflow technology. Variations in workflow products also mean that skills and model portability are issues.

Let's look at the possibility of addressing these blocking issues by building applications on a workflow platform that is low cost, ubiquitous, uniform, and easily integrated in applications. To be clear, the idea is not to replace workflow products. Rather, the hypothesis is that it is useful to factor out support for some core workflow concepts into a platform on which both workflow products and other applications can be built (see Figure 2, below).

A workflow is a model, which means it is a machine-readable description of business behavior that is not code. The meaning and benefits of this concept in the context of the value of a workflow platform will be discussed later.

A workflow model describes an organization of work units. For instance, suppose that a document review process specifies that Joe writes the document and then Fred reviews it. Here, the work units are first writing and second reviewing the document, and the organization is that one task must follow the other. This concept is not a radical idea. Code that makes successive calls to two subroutines is a valid example of the concept. The interest lies rather in the forms that this organization takes.

To test the workflow platform hypothesis, we will consider a range of real-world applications and explore the characteristics that a workflow platform should have if it is to prove useful.

A document review process takes as an input parameter a set of [reviewer, role] pairs that describe which humans are involved in the workflow in which roles. Possible values for the role are required, optional, final approver, and owner. The review process then proceeds until all reviewers have performed their assigned roles and notifies the owner of the outcome.

Here, the work items are the document reviews organized by the review process. There are three interesting characteristics to call out, namely, multiple points of interaction, human and automated activity, and the need to handle dynamic change.

Workflow Contracts

The workflow has multiple points of interaction, or contracts. First, there is a contract with a reviewer. This contract involves asking a reviewer to review a document, accepting the verdict and any review comments, and also telling a reviewer that his or her input is no longer required (if the review is canceled, or perhaps if enough reviewers have voted yes). The contract might also allow a reviewer to delegate a review. Then there is a second contract with the final approver, which is a specialization of the reviewer contract. Third, there is a contract with the owner of the review that allows the owner to cancel the review and be notified of the outcome of the review. Finally, there is a contract with the initiator of the review process, who instantiates the review and supplies the required parameters.

It is typical of workflows that they connect multiple parties through a variety of contracts (see Figure 3). The document review workflow is essentially a coordinator, initiated through one contract that is coordinating a variety of participants through one or more additional contracts.

The document review workflow drives human activity. However, it might also drive automated activities, such as storing versions of the document in a repository as the review progresses. From the point of view of the workflow, there is no essential difference. A workflow can be thought of as communicating, in general, with services through contracts. One special case of a service is another workflow. Another special case is a human. In many ways, a human is the original asynchronous service: one never knows when or if it is going to respond.

A characteristic of this type of workflow is that the participants will ask for changes to the workflow as it executes. For example, a reviewer might delegate a review task to a colleague or share the work involved in a review task with a subordinate.

There are two ways of addressing this requirement. One is to build an understanding of all the possible changes into the workflow. Then, a delegation request becomes just another function of the contract between the workflow and the reviewer. The other possibility is to see change as something separate from the workflow, where change is implemented as an external function that changes the workflow model. In this approach, the result of delegation is a new workflow model identical to one in which the review task was assigned to the delegate from the beginning.

Requesting an additional approval step would add a new approval task to the workflow model, which might well have contained no approval steps at all in its original form. The workflow no longer has to anticipate all possible modifications; at the most it will be concerned with restricting the areas of the model that are subject to change.

Both approaches are useful. Building understanding into a workflow is simple to model and understand. Generalizing operations is more complex to model, but more powerful and agile.

In an extreme but interesting case of the latter approach, the workflow begins execution with little or no content, and the required behavior is added dynamically by the participants in the workflow. Here, the available operations for modifying the workflow become a vocabulary that a user can use to construct the desired behavior as the workflow progresses.

Problem-Resolution Collaboration

To look at a specific example of a problem-resolution collaboration application, consider an inventory shortfall. An assembly line is making a gadget, and the computer indicated that there were enough widgets in stock for the purpose. However, when the stockroom manager went to fetch the widgets for delivery to the assembly line, a shortfall of 10 widgets was discovered.

Collaboration among the stockroom manager, the customer's account manager, the supplies department, and the production manager is required to resolve the problem. Each role in the collaboration may take characteristic actions. The supplies department could order more widgets, perhaps using a different supplier or paying an existing supplier more money for faster delivery. The account manager could go to the customer and request deferred delivery or split the delivery into two parts and bear the extra shipping cost. The production manager could divert assembled gadgets from an order for another customer. The stockroom manager could search his or her physical stock in an attempt to find the missing widgets. Any given action might be performed multiple times.

One obvious constraint is that the collaboration is not completed until the shortfall is resolved by some combination of the previous actions. There will often also be business constraints. For instance, there might be a rule that says deferral of delivery to gold customers is never permitted. Also, the actions will affect each other. For instance, there might be a policy that states that total added cost from corrective action may not exceed 5 percent of original factory cost. Thus, placing an order for accelerated supplies at a higher price might prevent a shipment from being split.

Figure 2. A monolithic workflow stack

In this case the work items are the actions that the various participants can take as they seek to resolve the inventory shortfall. The organization, however, is not the same as that required in document review. The participants are not dictated to; instead, they choose which actions to perform and when to perform them. However, these choices are constrained by the organization of the workflow, which has two aspects: 1) The actions are focused on achieving a goal; in this case, resolving the inventory shortfall. A bounded collaboration space is created when the problem resolution starts, and is not closed until the goal has been reached. 2) The participants are not free to perform arbitrary actions. Instead, the available actions are determined by the role the participant is performing and the state of the collaboration. The set of actions available is determined by policies related to the goal and global policies such as the restriction on shorting gold customers. The actions available vary as the collaboration progresses.

Figure 3. A contract diagram for the document review application

The experience of the participant is no longer that of performing assigned tasks. Instead, a participant queries for the actions currently available to him or her, performs none or one of these actions, and then repeats the cycle.

The main new requirement here, therefore, is for a form of organization of work items that is essentially data state and goal driven. There is also a requirement to support a query/act-style of contract with a workflow participant.

Scripted Operations

Scripted operations are simply a set of operations that are composed using a script. An example might be a desktop tool that allows a user to define and execute a series of common tasks, such as copying files and annotating them.

It would be unusual to consider using a typical workflow product for this purpose. However, it does fit the workflow platform pattern of a set of work units organized by a model. In this case the model is a sequence, perhaps with support for looping and conditional execution. Therefore, if a workflow platform were sufficiently low cost and ubiquitous, it would be possible to consider applying it to this sort of problem. Would doing so add any value?

One feature of scripted operations that is not addressed by their typical implementations today is the question of data flow. It is common for the data required by one operation to be the output of some previous operation, but this information is not typically modeled in the script. Thus, a user assembling tasks using a desktop tool might not be told when creating a script that the prerequisite data for a task hasn't been supplied, and would only discover the error when running the script. A workflow model that can describe these data dependencies would add clear value for script authors.

One approach is simply to include data flow constructs in the workflow model. It is highly arguable that the basic workflow model needs to include basic structural features such as sequences, conditions, and loops; but it is not clear that data flow is sufficiently universal to be represented by first-class elements of the model.

An alternative approach is to layer support for data flow on top of an extensible, basic workflow. A workflow model that can be enriched with abstractions appropriate to a variety of problem domains fits well with the notion of a workflow platform. This approach avoids both the complexity created by including in the base model a large variety of semantic constructs specialized for different problems and also the limitations imposed by limiting the workflow model to a fixed set of constructs.

Now let's look at a guided user application. One example is an interactive voice response (IVR) system, and another is a call center system guiding telephone operators through support or sales scripts. The essence of these applications is to guide users through the series of operations needed to achieve their goal. The organization of these operations is typically used to drive the presentation to the user, whether this is generated speech or a set of enabled and disabled command buttons on a form.

A characteristic of this type of application is that the workflow is the most frequently changed part of the application. Also, the business sponsors of the system are often heavily involved in specifying the changes, making it important to provide a way for IT staff and business personnel to communicate clearly and efficiently about the changes. A workflow model that expresses the core business purpose of the application, stripped of any irrelevant technical material, is an effective way to achieve this communication.

These applications also require flexibility within the workflow structure. In an IVR application the user will typically be heavily constrained, moving through a hierarchically structured set of menus. However, there will also be escape commands—for example, "return to root menu" or "skip out of current subtree."

A call center application will have more flexibility than an IVR application, changing the options offered to the user in response to the state of an order or in response to the input from a customer, such as skipping sales prompts if the customer starts to react negatively.

This sort of application requires support for a mix of organizations of work items, combining sequences, loops, and conditions with jumps from one state to another, and also the kind of data-driven behavior seen in problem-resolution collaboration.

Rule and Policy

As discussed previously, one way in which the workflow approach can deliver value is by isolating the focus of change in an application. Often, this focus is on the way in which the work items are structured, but in some applications the focus of change is on expressions tied to a relatively slow-changing structure.

An example of this focus is an insurance policy quotation system, where a set of frequently changing calculations is used to drive decision making in the quotation process. The requirement is for the workflow to model these expressions, which has two key benefits: First, the testing and deployment costs are much lower than those that would typically be incurred if the expressions were written as code, since the model provides a strong sandbox restricting the scope of possible changes. Second, the changes can be made by personnel who understand the business significance of the expressions but do not have the skills to understand the technical code in which expressions written as code would inevitably need to be embedded.

The Model-View-Controller (MVC) pattern often is used to wire a UI to an underlying object model (see Figure 4). The model represents the behavior of the system, independent of any particular UI representation. The controller is a part of the UI layer that is used to map the events generated by the UI into the method invocations required to drive the model. The UI itself is thus not polluted by any assumptions about the underlying model.

The workflows considered so far, viewed from this standpoint, all fall into the category of Models in the MVC sense. However, the controller can also be seen as a workflow. The work items it organizes are the methods provided by Model objects. The controller also interacts with the UI and the model through well-defined contracts. A model of this kind is often termed a page flow.

As with scripted operations, page flow would not today be implemented using a typical workflow product. There are two reasons to consider building a page flow using a workflow platform. First, a model readily can be represented visually, helping developers and analysts to express and communicate the required behavior. Second, if the page flow is frequently changing, then the abstraction of the page flow as a model improves agility.

There are two main requirements if this problem is to be addressed using a workflow platform. The workflow runtime must be lightweight, since a page flow may be running within a small application on a desktop, and the contracts supported must include the event-based contract characteristic of UIs, as well as the object-method contracts exposed by the Model.

Now let's look at a test record/replay application example. The intent of this final example is to test the limits of the applicability of the workflow platform hypothesis.

Figure 4. An MVC application

The application here is a tool for testing applications built as a set of services. The tool uses an interception mechanism to record all the interaction between services that occur during manual performance of a test case for the application. This recording can then be replayed. During replay, externally sourced messages are generated without manual intervention, and messages between the set of services that make up the application are checked for sequence and content against the original recording.

The workflow is the test case, organizing the work units that are the participating services. The workflow is both active, in that it simulates the behavior of externally sourced messages, and passive, in that it monitors the interactions between services.

A unique feature of this application is that the workflow is written, not by a developer or a user, but by a program, as part of the act of recording a test case. Workflow model creation must be fully programmable. There are also requirements for extensibility and dynamic update.

Extensibility is required because the structural semantics are rich. For instance, just because two messages arrived at a service one after the other in the recording, there is no necessary implication that this order needs to be preserved in a replay. If there is no causal dependency between the messages, then a replay that reverses the order of the messages is correct. So the semantics of sequence in the model used to record the test cases need to include a notion of causality, which is not likely to be a feature of the core workflow model of sequence.

Dynamic update is required because human interaction with the model will occur during replay. Discrepancies discovered during replay between recorded and observed behavior are popped up to a tester. If the discrepancy is because a message includes a timestamp that varies from run to run, then the tester would update the model to mark the field "don't care." If the discrepancy occurs in a regression test because the software has changed, then the tester might approve the change and update the test to expect the new behavior in all subsequent runs.

Workflow Platform Value

A workflow platform does not, by definition, have the full set of features offered by today's typical workflow products. Rather, the workflow platform considered here focuses on supporting the concept of a workflow as a model of the organization of work items. We have seen that this idea of an organization of work items is indeed applicable across a broad range of applications, but the fact that a workflow platform can be used doesn't mean that it should be used.

Figure 5. The document review implementation schematic

Two questions must be asked: What additional value is derived from the workflow platform approach? And, is this approach practical? This value of the workflow platform approach must come from the expression of the organization of work as a model, which we'll discuss later. Let's summarize the characteristics that a practical and effective workflow platform must display.

To demonstrate how a model differs from code, this code snippet is a valid workflow by the definition used here, that is, an organization of work units.

public void HandleLoanRequest (string customerID, Application app)
    {
    if (CheckCredit(customerId, app.Amount))
        {
        MakeOffer (customerId, app);
        }
    }

And, in a sense, it is a model. It is possible to parse this code and build a CodeDOM tree that represents it.

However, the semantics of the resulting model are so general as to be opaque. It is possible to tell that the code contains function invocations, but it isn't too easy to distinguish a function that represents the invocation of a work item from a function that converts integers to strings. A workflow model explicitly distinguishes these ideas. Typically, a specialized model element is used to represent the invocation of a work item, and conversion functions cannot be expressed directly in the model at all. A workflow model, then, is one in which its graph is built from elements that are meaningful in the workflow domain. The semantic richness of such a model can be exploited in several ways.

Visualization. Visual representation of the model—typically in graphical form—is useful for the developer, during both development and maintenance, and also for workflow users who want to know why they have been assigned a given task or the IT operations worker who wants to understand what a misbehaving application should be doing.

Insight. The workflow model is amenable to programmatic access for a variety of purposes. An example is static analysis to determine dependencies and flow of work across a set of cooperating workflows or using the model to drive a simulation that predicts the workloads that will be generated by a new version of a process.

Expressiveness. The specialization of the workflow model to the workflow domain means that characteristic problems can be expressed more quickly and compactly. It is a domain-specific language (DSL), specialized to support characteristic problems. Consider a document review process where three positive votes out of five reviews mean that the document is good, and any outstanding reviews can be canceled. This process is quite difficult to code, but a workflow model can supply out-of-the-box constructions that address such problems.

More Semantic Exploitation

As we have seen in the scripted operations application discussion, extending the workflow model to further specialize the out-of-the-box model language is a very powerful technique for delivered additional value. An example is the creation of a language intended for end users, as in the document review conducted using an improvised definition of the review process that was discussed previously.

Execution. The specialization of the model makes it possible to add run-time support for common problems. A good example is long-running state. Of the applications discussed here, management of long-running state is required for the document review process, problem-resolution collaboration, and guided user applications. The workflow platform runtime can solve such difficult problems once, using simple expressive model elements to control a common capability and freeing up the developer to focus on the business problem.

Monitoring. The existence of a model makes it possible to produce an event stream with a meaningful semantic without any additional developer effort. Of the applications described here, this event stream is useful in the document review, problem-resolution collaboration, test record/replay, and guided user applications. The event stream can be used to monitor instances of workflows or build aggregate views of the state of a large number of workflow instances. The standardization of the event stream makes it much easier to build such aggregate views across workflows that were developed independently of each other.

Another powerful idea is the presentation of errors using a business semantic. Often, a technical failure such as the nondelivery of a message leads to escalation to a technical expert because the significance of the failure is unclear without specialist investigation. If the error can be mapped to a workflow model—so that it is clear that the error concerns a noncritical change notification, for instance—then escalation can be restricted to cases where it is necessary.

Composition. If an application is factored into work units, then these work units, with their well-understood interfaces, can be reused by other workflows. Workflows themselves also define work units that can also be used by other workflows.

Customization. Suppose that an ISV ships a workflow, which is customized by a VAR, and then again by a customer. Reapplying these customizations when the ISV ships a new base version is a challenging maintenance problem. The use of a shared, well-understood model for the workflow makes the consequent three-way merges much more tractable. Customization and composition together enable ecosystems where definitions of work and flow become shared or traded artifacts.

Manipulation. As we have seen in the discussions of the document review and test record/replay applications, there are often requirements to invent or modify workflows on the fly. This modification cannot be done securely if changing code is required. Using a model makes possible dynamic manipulation that is both controllable and comprehensible.

These benefits make a compelling list, and it demonstrates clearly that the description of an organization of work items as a model has a lot to offer.

Platform Characteristics

There must be support for basic structural concepts like sequences, conditions, and loops. However, there also needs to be support for data-driven approaches to deal with the less-structured organizations that appear in applications like problem-resolution collaboration and guided user.

It is also important to allow new semantic elements to be added to create rich, specialized languages such as the data flow-aware composition in scripted operations. Adding new semantic elements might go so far as to require the redefinition of such fundamental ideas as sequence—for example, in the test record/replay application.

The workflow must also be able to communicate in a rich variety of ways. Workflows respond to UI events, drive different types of services (human, programmatic, other workflows), and support queries over the current state of their contracts—for instance, when determining the actions available to an actor in a problem-resolution collaboration application.

If the workflow platform is to be used in all the applications where it adds value, such as MVC, then it must be lightweight. Equally, it needs to address the scale and performance requirements implied by applications such as document review.

In addition, the workflow model itself must be fully programmable, which includes model creation—such as in the test record/replay application—and dynamic model update to support unanticipated change, as in both the document review and test record/replay applications.

Now let's look at the realization of these required characteristics in the Windows Workflow Foundation (WF).

WF implements the idea of workflow as an organization of work items, abstracted away from the related ideas with which it has been coupled in traditional workflow products. The abstractions fall under three main categories: design and visualization, hosting, and semantics.

Design and visualization. A workflow in WF is a tree of work items (called activities). This tree can be manipulated directly as an object model. A designer is provided, but its use is not mandated. It is possible to create new designers specialized to particular user communities or to particular organizations of work items. It is also possible to specialize the provided designer, which can be used not only within Visual Studio but from within an arbitrary hosting application.

Hosting. The WF runtime is sufficiently lightweight to be hosted in a client context such as a controller in a rich-client application shell. It is also performant enough to scale when embedded in a server host, such as the SharePoint Server delivered by Office 2007. The WF runtime's expectations of its host are abstracted as provider interfaces for services such as threading, transactions, persistence, and communications. Useful provider implementations are supplied out of the box, but they may be substituted as required.

Semantics. Different problems respond to different model semantics. WF supports three main styles of workflow out of the box: flow, state machine, and data driven. Flow is optimal for applications where the workflow is in control such as the scripted operations example. State machine is best when the workflow is driven by external events, as in the MVC or guided user applications. A data-driven approach is suited to applications where actions depend on state, as in problem-resolution collaboration.

These semantics can be extended by building custom activities to create a domain-specific vocabulary for use in any of these styles. However, since the structure of a workflow is itself expressed as a set of activities, the same approach can be used to define new styles, and entirely novel semantics, if required.

A Common Workflow Runtime

The focus of the WF runtime is to deliver facilities required by any workflow, and therefore avoid the need to re-implement them time and again in different applications, but without compromising the flexibility of the workflow abstraction. These common facilities fall into four main categories: activity scheduling, transactions and long-running state, exceptions and compensation, and communications. Let's look at each in more detail.

Activity scheduling. The WF runtime defines an activity protocol that all work items implement. This protocol defines the basic activity life cycle (initialized, executing, and closed) and the additional states needed to handle exceptions (faulted, canceling, and compensating). This definition enables the WF runtime to provide work scheduling for all workflows.

Transactions and long-running state. The WF runtime supports the execution of ACID transactions. These transactions are particularly useful for maintaining consistency across workflow state and external state such as application and message state. However, ACID transactions are not suitable for managing long-running state because of their resource and locking implications. The WF runtime implements a broader checkpoint-and-recovery mechanism to handle long-running state. From this point of view, ACID transactions become units of execution within a larger framework. The developer needs not do any work to get the benefit of WF's support for long-running state, as it is default behavior. However, if more detailed control is required, a set of simple model elements are supplied for the purpose.

Exceptions and compensation. The familiar idea of throw-try-catch exceptions is supported by the WF runtime and represented in the out-of-the-box workflow model. However, the WF runtime also supports a broader view of fault handling that includes the idea of compensation for successfully completed transactional units.

Communications. As we have seen, workflows need to communicate in a variety of ways, which is reflected in the WF, that supports communication through .NET method, event interfaces, and Web service interfaces. Support for Windows Communication Framework will also be made available in the future. Thus, WF does indeed realize the workflow-platform approach proposed here.

Figure 5 illustrates the high-level implementation schematic of the document review application and how all of the foregoing comes together. An implementation uses SharePoint as the workflow host. The WF runtime uses the default scheduling service provided out of the box with WF. However, the default persistence and communications services are replaced with implementations specialized for the SharePoint host. The persistence service stores long-running workflow state in the SharePoint database, and the communications service makes the rich-user interaction facilities of SharePoint available to the workflow. Both of these services are in fact delivered out of the box with Microsoft Office 2007.

Three sorts of activities are used to define the document review workflow itself. First, out-of-the-box WF activities are used to provide structural elements such as If-Else and While. Second, activities provided as part of Office 2007 are used to access the user communication services of SharePoint. Third, custom activities are used to implement organization-specific semantics for forwarding and delegation in a standard and reusable way. The WF designer is used as a means to define the workflow and also provide diagrammatic representations of the state of a document review workflow instance to the workflow owner.

Attacking the Problems

In summary, the workflow platform supports an abstraction of the ideas that have made workflow products an attractive attack on business problems. It does not replace today's workflow products, however. Rather, it factors them into platform and superstructure.

The workflow platform embodies two key ideas: a workflow is an organization of work units, and a workflow is a model, that is, a machine-readable description other than code. These ideas are valuable in a broad range of applications, both within and beyond the problem domain addressed by typical workflow products. Such a workflow platform is most useful if it is low cost and ubiquitous.

The principal benefits delivered arise from the expression of an organization of work items as a model, which has several advantages over a representation in code:

• Transparency. The business purposes of the system are clear, allowing users and IT staff to communicate effectively about the desired behavior and IT staff coming onto the project to get up to speed quickly.
• Isolation of change. The areas of the application most likely to change are expressed as workflow rather than code. By isolating the rapidly moving parts of the application, changes can be made more reliably.
• Agility. The bottom line of all these benefits is business agility. If business users can understand the system, developers can get up to speed quickly, and the risks associated with change are minimized. Then the system may be termed agile.

A broadly useful workflow platform must have these characteristics: define a core workflow model as a standard that is extensible and fully programmable at design time and runtime, be able to communicate in a rich variety of ways, be lightweight and embeddable, and be able to scale and perform well in high-volume environments. WF is a product that displays all of these characteristics. As a component of .NET 3.0 and a part of the Windows platform, WF is also low cost and ubiquitous.

A Workflow Manifesto

The "workflow manifesto" is a set of statements designed to help you consider the flexibility, value and benefits that workflow adds to your solutions architecture. The emergence of a flexible, extensible framework like WF enables the implementation of workflow-oriented solutions in ways that may have been impossible in the past. The "workflow manifesto" was designed to help clarify some of the opportunities for applying and using workflow in our solution architectures:

• Workflow is everywhere
• Workflow is expressive
• Workflow is fluid
• Workflow is inclusive
• Workflow is transparent

While each of these statements provides a significantly different perspective of workflow, they all share two specific benefits in common: agility and abstraction. Before reviewing the manifesto statements in greater detail let's better understand the reason these two benefits are shared in such a broad fashion.

Agility

Agility is a key concept for both IT and business – a flexible and highly extensible solutions architecture is much more likely to meet the rapidly changing needs of the organization. Because they are model based, workflows are more suited for change than a pure-code approach. For example, workflows typically enable developers to quickly and easily add new steps to a given workflow instance at runtime. Performing a similar function in a code-only approach requires developers to understand how to use a reflection API and manipulate the low-level internals of a given set of classes. This approach exposes the solution to risk and significantly limits the pool of maintenance developers qualified to work on it. Workflows also provide support for business rules that can be modified and re-deployed at run-time, impacting the behavior of one or more workflows and instances that may utilize the associated business rules.

Note: While workflows are better suited for change than a pure code approach, workflows are not necessarily designed to replace all code. Workflows are nothing more than a set of specialized classes generated by the workflow modeling environment.

Abstraction

Abstraction is another key concept for IT and business, but for slightly different reasons. From an IT perspective, abstraction can be used to hide complexity, potentially reducing both development time and long-term maintenance costs. From a business perspective, the maintenance costs are attractive while the workflow model makes it easier for non-technical personnel to understand the objective of a given workflow without having to review the code used to implement the solution. The scope of the workflow model can also be established by a business analyst prior to handing it off to a developer for implementation.

The concept of a model-driven approach is not new – developers have been using UML and other modeling techniques for many years prior to the maturation of workflow development tools. Workflow provides both sequential and state machine models that can be used to develop, manage and run the workflow itself. Developers usually prefer to work with the text (code) representation while a business analyst may well be overwhelmed by the detail of the model itself. Most workflow models also enable the model itself to be modified to meet the needs of the organization. For example, WF's presentation surface can be "skinned" to change the look and feel of the modeling surface prior to embedding within your application.

The Workflow Manifesto

The remainder of this Chapter examines the Workflow Manifesto statements in greater detail. Several examples of each topic are also included.

Workflow is Everywhere

Workflow doesn't just live in the datacenter on integration servers. Any sequence of steps or activities can be represented as a workflow. These steps may be handled by automated processes in a system-to-system (S2S) workflow, a human–to-human (H2H) workflow, or a fractal model that touches upon both (discussed below). Workflow can be used to orchestrate services across applications or within a single application, brokering user events or exposing workflow capabilities to an end-user. Workflow may also be used within dedicated devices. For example, WF-based solutions can easily be deployed to x86-based XP-embedded devices such as a point-of-sale (POS) terminal or a bank ATM.

There are several scenarios which may benefit from a workflow-oriented approach. The two scenarios we will examine are User Interface (UI) flows and creating and Web services.

UI Flows

A workflow model can serve as the controller within a Model-View-Controller (MVC) pattern. The MVC pattern provides a separation of objects into one of three categories — models for maintaining data, views for displaying all or a portion of the data, and controllers for handling events that may impact the model or views. There are three UI flow examples worth considering:

• WinForms: A workflow can act as the event broker for common user events such as button clicks, data selection, and others. The state of a given workflow instance can also be used to configure the user interface. For example, Figure 6 shows how a workflow state is used to enable or disable buttons on a WinForm based on user interaction with a given workflow instance. (The code for this example is available in the WF Samples folder after downloading and installing the Windows SDK for .NET 3.0.)

Figure 6. Workflow instances drive the WinForms user interface

• Composite UI Application Block (CAB): CAB is an application block that provides a programming model for building smart client applications based upon the Composite Pattern. The Composite Pattern is applied to UI by combining multiple components to create complex user interfaces. The individual UI components can be independently developed, tested, and deployed (conceptually similar to SharePoint's Web Parts model). The original implementation of CAB was developed before the advent of WF and required a significant amount of code for event brokering and management between the components that make up the interface (see Figure 7 for an overview of CAB). CAB is being re-architected to include WF for event brokering and UI component management (see the area highlighted in Figure 4).

Figure 7. The Composite UI Application Block

• Pageflow – A page flow is a directory of Web application files that work together to implement a UI experience. A server-side workflow implementation is used to control the order in which pages are served to the user, possibly changing the user experience on the fly based upon events raised by the current page. Navigation logic remains within the workflow, effectively isolating navigation from presentation. While there is no "official" way to implement Pageflow using WF, the WF product team has made an example available at http://wf.netfx3.com/files/folders/sample_applications/entry10996.aspx. The example illustrates how one can implement a generic navigation framework with WF that is capable of supporting UI technologies like ASP.NET and WPF.

Web services

There are two approaches in which we can consider the relationship of workflows and Web services. A workflow can be used to orchestrate multiple Web services, potentially repurposing the return values from one Web service as input values to another. Using workflows to invoke Web services simplifies the process of working with asynchronous services since the invocation and correlation requirements can be handled by the workflow itself. We can also use workflows to build Web services, publishing the workflow as a Web service. The model is fractal since we can use workflows to both create and consume (or orchestrate) Web services. Since most services were developed and designed to operate autonomously, bringing them together into a new service composition (workflow) requires that several challenges must be met. Figure 8 illustrates some of the most common challenges facing service orchestration.

Figure 8. Capabilities for Service Composition

Most integration servers (such as BizTalk Server) support the capabilities in Figure 8, relieving developers from having to write, test and maintain "plumbing code" that provides little value to the organization. Figure 9 illustrates how a workflow can be both exposed as a service and orchestrate services.

Figure 9. Workflow and Web services

In .NET 3.0, WF includes three out of the box (OOB) activities for consuming or publishing simple ASMX Web services:

• The InvokeWebService activity enables a workflow to consume a simple ASMX Web service. Adding an InvokeWebService activity to your workflow triggers the Add Web Reference wizard, enabling you to enter the URI and proxy name of the service to be invoked. The MethodName dropdown property enables you to select which method on a given endpoint should be invoked. The InvokeWebService activity also includes optional parameter properties to support One-Way or Request-Response Message Exchange Patterns (MEPs).
• A WebServiceReceive activity is typically added as the first element of a workflow to be published as a Web service. WebServiceReceive binds the activation of a workflow instance to a Web service invocation. The interface definition is automatically generated based upon the value of the WebServiceReceive activity's InterfaceName property. The InterfaceName property is a simple interface that defines the method names and signatures to be exposed by the Web service. The WebServiceReceive activity also includes optional parameter properties to support One-Way or Request-Response Message Exchange Patterns (MEPs). A workflow with a WebServiceReceive activity is published as an ASMX Web service by simply right-clicking on the project containing the workflow and selecting "Publish Workflows as Services…"
• A WebServiceResponse activity may be added to a workflow to be published as a Web service with a Request-Response MEP (there is no WebServiceResponse in a One-Way MEP). WebServiceResponse includes a ReceiveActitivityID property which is used to associate the response message with a previously defined WebServiceReceive activity.

The NET 3.0 WF out-of-box activities do not provide support for Windows Communication Foundation (WCF). WCF and WF are more closely aligned In NET 3.5, enabling you to more easily expose workflows as WCF services and orchestrate WCF services.

Using WF to consume or create Web services provides a far richer set of options than a "pure-code" approach. In this section we illustrated how we can use workflows within user interfaces and how we can use workflows to create and consume Web services. Additional scenarios could include implementing a business logic layer within your applications, supporting long-running processes and exposing a set of configurable business rules that may be modified at runtime. Any sequence of steps or events can be represented as a workflow.

Workflow is Expressive

Workflows can do virtually anything that a "pure-code" approach can do, at a higher level of abstraction. This doesn't necessarily mean you should replace all of your code with workflows. What it does mean is that you can use a workflow-oriented approach to raise the level of abstraction in your solutions, potentially raising your level of productivity and reducing long term maintenance and ownership costs. Let's examine these assertions in greater detail.

The model is the workflow

Since the model is a direct representation of the actual workflow there is no loss of fidelity in the implementation of the model. This is a major change from older model-based technologies such as CASE and some UML-based development tools. These tools suffered from the fact that their modeling environments were far more expressive than their code generation capabilities. This meant that the models were, in fact, pictures because they were unable to generate the code necessary for implementing the solution. Moving between a workflow model and the underlying implementation code is a lossless process – the model and the code are different views of the same workflow.

The model also makes communicating and maintaining the workflow much easier than a "pure-code" approach. The model becomes part of the documentation of the workflow and can be shared with (or created by) a business analyst to illustrate the objective of the workflow. The model can also be used by maintenance programmers to understand and maintain the workflow without having to read through large sections of opaque, possibly undocumented code.

Which Workflow Model to use when?

Workflow helps us realize the fundamental difference between a picture and a model. A workflow typically includes some sort of model used to both develop and execute the workflow. With WF we have two models available – a sequential model and a state machine. Sequential models flow from top to bottom in a simple sequential fashion while a state machine model is used to trap various events that may arise during the execution of the workflow. If we review the state machine model in Figure 2 we see there are several states that a Purchase Order moves through while being processed (e.g. OrderOpened, OrderProcessed, OrderCompleted, and so on.). Each state may include one or more activities that fire based upon the context of the currently executing instance.

WF supports two models for designing your workflows – Sequential and State Machine. The model you select should be determined by your solution requirements.

• Sequential models are most effective when the workflow is in control, dictating both the activities and the order in which they are to be performed. Sequential models are most likely to be used in system-to-system (S2S) workflows. For example, an Order Validation process consists of a well-defined series of steps and operates on a highly structured set of data (e.g. a Purchase Order). An Order Validation process may be best defined using a sequential workflow model.
• State machine models are most effective when the workflow is reacting to events that are raised outside of the workflow itself. The workflow supplies a list of events that may occur while the work is being performed. The order in which these events initially occur is not controlled by the workflow – the workflow only needs to know what can be done next based upon the current state. State machine models are most likely to be used in human-to-human (H2H) workflows. For example, in a document routing and review process, a reviewer may optionally decide to make revisions to the document or route the document back to the original authors for a complete rewrite. Modeling these types of actions in a sequential model can be quite challenging.

There is a third approach that some people call "data driven." A "data driven" approach uses a set of business rules to react to data being processed by the workflow (business rules are created within WF using the Policy activity). A "data driven" approach can be used with both sequential and state machine workflow models. Your solution may choose to use a combination of these workflow models. For example, a given state within a state machine may encapsulate a sequential workflow. The final choice is up to you and should ultimately be influenced by your solution requirements.

Workflow is Fluid

Because they are model-based, workflows can be modified, interrogated or otherwise manipulated at design time or run time. That is, both users and analysts participate in both the design and updates to the workflow. Items within the workflow can change frequently, such as the order of operations; method calls; and services, data, or applications to be integrated. For example in a mortgage solution, rates can change based on the market, the loan type, the lendee's history, or several other reasons. Changes may need to flow from design to deployment in a matter of minutes. This requires a different approach than classical compiled and distributed applications. Some workflow systems use a runtime repository of business applications and rules that require updating on a frequent basis. This approach enables workflow systems to rapidly respond to market and organizational conditions. Workflow flexibility can be maximized by judicious use of business rules and orchestrations.

Business Rules

One approach to fluidity is using business rules. Business rules govern conduct of business processes. Business processes tend to stay the same over time but the business rules used by these processes are highly volatile. Separating rules from application code enables users to invoke or change rules at run time, resulting in greater workflow flexibility. Business rules are best used for:

• Discrete, point-in-time evaluations and calculations.
• Large no of permutations to encode in a control flow.
• Fact-based inferencing where control flow cannot be predefined.

Orchestrations

Orchestrations are similar to business rules, but more formal. You would use them in place of business rules where formal workflows require:

• Long-running semantics
• Transactions/ Compensations
• Messaging

Orchestrations are also used when there is a recognized control-flow that needs to be rigorously managed for performance or scale and visibility and tracking of the state of the orchestration is critical.

Business rules and orchestrations both enable workflows to be modified faster and better deal with changes in the organization. This flexibility lowers the costs typically associated with development, maintenance and ownership of a given process.

Workflow is Inclusive

Automation in a system-to-system (S2S) environment is fairly straightforward – processes tend to be fixed with a finite number of variations. The data processed within a S2S environment tends to be structured, representing well-known business messages such as a Purchase Order. Users, however, work in an ad-hoc manner and may use unstructured or semi-structured data. For example, a user may receive an email at home, convert it to a document, enter related data into a remote tracking system, and communicate these events by leaving someone a voice mail message. Computer systems are much more suited for structured, long-running asynchronous problems with limited flexibility. Workflow models can be used to describe with both human-to-human (H2H) and system-to-system (S2S) interactions. We can also use these models to describe system-to-human (S2H) and human-to-system (H2S) interactions. Translating such models into an executable representation can be challenging and usually results in a task-notification (S2H) and task-completion-response (H2S) interaction pattern.

Workflow is Transparent

Humans can be brought into process design right from the start by the abstraction and model-driven design workflow offers. User interfaces like SharePoint Designer let an experienced end user build or modify workflows quickly and easily. A workflow model makes the workflow easier to understand without having to review and maintain opaque blocks of code. Tools such as WFPad (http://www.schmidt6.com/blogfiles/WF/WFPad.exe) can be used to display the workflow model in one windows and the workflow code in another. This enables experienced users and developers to view or maintain a workflow using either a model or the code. As stated we stated earlier, the workflow is the model. The workflow model offers:

• state management across service interactions
• service aggregation and orchestration
• transparent implementation of services
• visibility into the state of the service
• configurable runtime behavior of: tracking, persistence, transactions, threading, and so on.

Workflow transparency is important because state persistence is a critical requirement for long running processes. Processes may last for days, weeks, months or even years. Processes may need to be started, held, and restarted by humans or runtime services – sometimes with different timeframes with no loss of fidelity.

Understanding the Relationship between BizTalk Server and WF

In this Chapter we made numerous references to Windows Workflow Foundation (WF). When WF was first announced many people incorrectly assumed WF was a replacement for BizTalk Server (BTS). WF and BTS are complementary technologies designed to serve two very different needs:

• BTS is a licensed product designed to implement workflow ("orchestrations") across disparate applications.
• WF is a developer framework designed to expose workflow capabilities within your application. There are no fees or licensing restrictions associated with using or deploying WF.

Use BizTalk Server if…

• You need to implement system-to-system (S2S) workflows across disparate applications or platforms.
• You need an enterprise-strength Business Process Management (BPM) suite that enables complex transformations, support for popular application/wire-level protocols and integration with line-of-business (LOB) systems like SAP, PeopleSoft, Oracle, and JD Edwards.
• You need to interface with a human-to-human (H2H) workflow.
• You need to take advantage of Business Activity Monitoring (BAM).
• You need to map authentication information between Windows and non-Windows systems (Single Sign-On (SSO)).
• You need to set up and manage trading partners in a B2B scenario.
• You need a complete set of tools for managing the infrastructure and scalability of your solutions architecture.

 Use Windows Workflow Foundation if…

• You need to expose workflow capabilities to end users through your application.
• Your application only needs a message or event broker.
• You need to build a Human to Human (H2H) workflow capable of interfacing with a System to System (S2S) workflow. (SharePoint 2007 includes several predefined Human Workflows built with WF).
• You only need workflow or simple business rules and are not interested in the other features that BizTalk Server provides.

Conclusion

In this Chapter we examined the concept of workflow and discussed other terms and concepts frequently associated with workflow. Workflow is the organization of work using activities that represent both people and software. Models represent this organization, providing a higher level of abstraction and flexibility to our solutions. This chapter also clarified the relationship of workflow-oriented capabilities on the Microsoft platform (Windows Workflow Foundation and BizTalk Server).

Chapter 4 provides a more detailed discussion of the Data recurring architectural capability.

SOA Case Study: Dollar Thrifty Automotive Group

Dollar Thrifty Automotive Group provides car rentals through two brands: Dollar Rent A Car and Thrifty Car Rental. Approximately 8,300 Dollar Thrifty employees serve travelers from company headquarters in Tulsa, Oklahoma. With an ever-expanding network of North American and international corporate-owned and franchised locations in 70 countries, the Dollar or Thrifty brands operate in virtually all of the top U.S. airport markets.

As Dollar Thrifty grew and expanded, and competition in the rental car business accelerated, company decision makers became aware of issues with their mainframe-based car rental system. Dollar Thrifty wanted a car rental system that would enhance customer experience and help the company be more agile in the face of competition, now and in the future.

Developers at Dollar Thrifty took advantage of Windows Workflow Foundation (WF) to build validations in a smart client application. WF provides the programming model, engine, and tools for quickly building workflow-enabled applications. Because they used WF, developers didn't have to write the workflow or validation logic from scratch. Developers were able to create a flexible interface that lets agents work the way they want to work yet still satisfies all the business rules.

Figure 10. Dollar Thrifty Architectural Overview

The entire Case Study is available online at http://www.microsoft.com/casestudies/casestudy.aspx?casestudyid=1000003883

See other SOA case studies at http://www.microsoft.com/casestudies/search.aspx?Keywords=SOA

References

·         "Building Applications on a Workflow Platform" by Dave Green, Architecture Journal, April 2006. Available at http://msdn2.microsoft.com/en-us/library/bb245670.aspx