http://msdn.microsoft.com/en-us/library/bb220798.aspx

Alchemy Technical Overview

1 out of 1 rated this helpful - Rate this topic

Mark Baciak
Architecture Strategy
Microsoft Corporation

December 2006
Revised February 2007

Applies to:
Microsoft .NET Framework

Summary: Background into Microsoft's first Service-Oriented Infrastructure (SOI), Alchemy, a production-quality enterprise environment for the development, deployment, and management of services. (15 printed pages)

Introduction
Background
Technical Overview
Project Overview
Lessons Learned
Conclusion
More Information

Introduction

Problem Statement

Microsoft's enterprise services needed the support of a true enterprise architecture. Incompatible development standards, lack of change-management processes, limited operations management, and insufficient automation were problems in need of an enterprise, production-ready solution.

Solution

Microsoft's Information Technology group (Microsoft IT) developed the Alchemy infrastructure, upon which an enterprise reference service architecture could be constructed. The initial architecture integrated business data from four core Microsoft line-of-business applications. When fully deployed, Alchemy managed Microsoft IT's 15 critical line-of-business application services and data.

Benefits

Alchemy:

Reduced development times from months to weeks.
Cut development and infrastructure costs.
Accelerated service adoption and improved service management.
Increased organizational agility and flexibility.

Background

When Microsoft IT started using services many years ago, they lacked tools, standards, and products to make services enterprise class and production ready. To meet that need, they funded a research-and-development project code-named Alchemy to produce the infrastructure to support a Service-Oriented Architecture (SOA). This paper introduces the Alchemy work as a reference architecture elucidating the power of SOA.

The basic services standards (SOAP, WSDL, XSD, and UDDI) provide the functionality necessary to build and connect distributed applications. However, by themselves, the basic standards are not sufficient to support a production-quality enterprise environment for the development, deployment, and management of services. A Service-Oriented Infrastructure (SOI) meets these needs and is the platform for a Service-Oriented Architecture (SOA). Microsoft IT developed the Alchemy Common Platform (ACP) over several years as the internal SOI. The Microsoft IT Alchemy implementation is presented here explicitly for reference-architecture purposes only.

With ACP, common tools and frameworks enable rapid development of standards-based services. Microsoft IT found that a typical service, which normally could be built and deployed in four to six months using only basic services standards, could be built and deployed in four to six weeks on the ACP.

Microsoft IT decided that an enterprise-class SOA based on services should address these issues. ACP added the full suite of deployment, management, and support features needed by line-of-business developers, operations staff, and line-of-business application providers.

Future Options: ACP was implemented through Web services; and, although the Web service programming model was optimal at the time of development, it should be noted that the Windows Communication Foundation (WCF) will offer additional programming models as part of Microsoft .NET Framework 3.0. Support for MSMQ, COM+, and Web Services Enhancements (WSE) will be included.

Lack of Common Development Standards

Microsoft IT's early experience developing and deploying services was a common one: Each application support group created and deployed services for its own particular application. Microsoft IT found that when they relied on the basic services standards, there were too many options available for developers to perform user and process authentication, digital signing and encryption of service requests and responses, formatting of parameter data, and handling error conditions. This resulted in incompatible security solutions, duplicated development effort, inconsistent management processes, and unpredictable service levels. The result: isolated line-of-business service "silos" that prevented Microsoft IT from realizing the full benefit of services.

No Common Deployment and Change-Management Processes

The early deployments of services at Microsoft were made on an application-by-application basis. This created a services environment that was difficult to manage from both operations and change-management perspectives.

It was not easy for a group of developers to take a service from development through testing to production. The development, testing, and deployment processes placed a heavy burden on the server infrastructure and personnel who manually deployed and configured the services. Additional servers were required, which resulted in increased hardware and operations-management costs. Furthermore, manual installation and configuration of application software produced inconsistencies that were difficult to detect, correct, and manage.

No Infrastructure for Operations Management

It is essential that an enterprise computing environment include an integrated approach for operations management and enforcement of service-level agreements (SLAs), including the need to support monitoring, logging, analyzing, and reporting on all service transactions (service requests and responses) and issuing alerts for out-of-bound SLA conditions.

Without an adequate infrastructure for service-operations management, Microsoft IT found that it was impossible for service providers to offer predictable service levels.

Lack of Automation to Support Service Subscriptions and Service-Level Agreements (SLAs)

Microsoft IT's experience deploying services for consumption by multiple development projects revealed additional requirements for service management. One of these requirements is the need for a formal process by which service providers are notified of an application developer's intent to subscribe to a service. Subscribing to a service establishes the relationship between service consumer and service provider. The subscription process enables service providers to receive notification of a consumer's intent to use a particular service, and for the appropriate SLA parameters to be established between the two entities. The subscription process requires an exchange of consumer and provider contact information, so that two-way notification of pending changes, outages, and out-of-range SLA alerts can occur.

Technical Overview

To address these issues, Microsoft IT developed and deployed the Alchemy Common Platform (ACP). ACP provided a secure, consistent, enterprise-wide infrastructure for developing, deploying, and managing services across a diverse range of Microsoft business applications.

The Alchemy Common Platform:

Created a comprehensive services-management solution using Microsoft technologies.
Provided a reusable, agile, and global services framework that supported all Microsoft line-of-business applications.
Made it easier for development teams to build and consume services in a line-of-business application environment.
Reduced the tendency for developers to create stand-alone services that use different technology solutions for authentication, digital signing and encryption, formatting of parameter data, and handling of error conditions.
Established a single comprehensive repository for services in Microsoft that enabled the categorization, discovery, and reuse of those services.
Reduced development and deployment complexities, as well as overhead costs when providing operational support for services.

Technically, ACP was implemented as a protocol-agnostic framework for services development and management across the enterprise. ACP:

Secures XML SOAP services using a common platform.
Supports .NET Remoting.
Establishes a single comprehensive repository of services.
Enables discovery and categorization of services.
Abstracts complexity from development and operational support.

ACP reduced the time and effort required to develop, test, and deploy new services. Based on Microsoft's IT experience with several projects, a services implementation that would typically require four to six months of effort was reduced to four to six weeks using ACP.

ACP Illustrated

ACP built upon the enterprise infrastructure, adding repository, runtime, and analytic functions. It is important to note that this paper is intended as an overview to the Alchemy reference project; three more papers will go into more detail on the repository, runtime, and analytic components. It is hoped that this paper will stimulate questions as to how these components have worked together; additional papers will address these questions.

Figure 1. Logical view of Alchemy architecture

Figure 2. Functional view of Alchemy architecture (Click on the picture for a larger image)

Infrastructure

Microsoft already had an extensive array of infrastructure components upon which Alchemy could build.

Functional capability	Description
Identity	There are three main identity stores from which the Alchemy Repository gets data: Active Directory (AD), Microsoft Passport, and an X.509 Certificate Authority.
Notifications	The Alchemy system integrates into Microsoft Operations Manager (MOM), SQL Server Notifications, and Exchange to deliver relevant messages to the needed recipients.
Asset Management	The prime store in which all data is gathered for physical and software machine data is System Management Server (SMS).
Portfolio Management	Portfolio data is captured in a custom-written tool called MS Apps.
Configuration Management	Custom .NET code and service that provides the ability to verify configuration versions and automatically sync to the latest configuration when changes occur.

Repository

Functional capability	Description
Security Token Service	A custom-written STS that verifies authorization, versions, and registration details.
Configuration Service	Manages all updates to the Repository.
Service Registry	Universal Description, Discovery, and Integration (UDDI), and Repository.

Runtime

Functional capability	Description
Local Call Store	A custom-written store that asynchronously captures data around all calls made to a service. This has the ability to run as an NT Service, inside the consumer's application space, or as a service on a shared host.
Service Interceptor	An extension of the SOAP class that allows for call interception without having to put in explicit calls from the consumer or provider.
Service Manager	A .NET assembly that manages the call throughout the request and response phases.
Dynamics	Custom-written code that dynamically determines the endpoint, transport options, and timing on a per-call basis. This functional capability is also charged with the critical service manager that ensures communication to all required Alchemy services, such as the Local Call Store and STS.

Analytics

Functional capability	Description
Global Call Service	Receives and analyzes all calls made by an Alchemy consumer or provider.
Global Call Store	Stores all calls made by an Alchemy consumer or provider into a transactional system for 12 to 24 hours.
Data Warehouse	Takes calls stored in the transactional system and applies extended data for easy query, analysis, and non-repudiation needs.
Enterprise Reporting	An internal tool named MS Insight that is used to provide analysis of SQL Server cubes through a Web client interface integrated with Microsoft Office Web Application (OWA) technology.

Provider Scenario

Prerequisites

Service applications must be ACP-enabled and defined as enterprise entities, if they are to be discovered and used throughout the enterprise as "managed service providers." Service providers expose capabilities through interfaces. Managed service providers are monitored services that execute transactions in compliance with Service-Level Agreements (SLAs).

ACP-Enabling the Application

The application provider first creates the service application with all of its associated Web methods.

The provider ACP-enables the application by performing several steps within the development environment:

Adds ACP-required project references (a set of DLLs)
Updates the application configuration file with ACP entries
Inserts ACP start code, to initialize the application
Adjusts IIS virtual directory settings
Adds ACP-enabled verification

The provider then deploys the service on a Web server that is accessible from the enterprise.

Defining the Application for the Enterprise

At Microsoft, applications are managed in a centralized governance portfolio: the Microsoft Application Portfolio System (MS Apps). An application provider must create an entry for each application whose services are to be published to ACP. Once the MS Apps entry is created and the application provider's alias is added to the application's contact list, the application's services can be published to ACP by the provider.

Publishing the Service Application

The application provider manages each service application by publishing it through the ACP Management User Interface. The ACP Management User Interface is a central console for administering all of the applications registered for management within ACP.

Figure 3. Service published to ACP by way of ACP Management User Interface (Click on the picture for a larger image)

The configurable properties of a managed service are controlled from this user interface. The publisher creates an ACP service entity and sets its properties through a set of wizards in the user interface. Publishing entails the following steps.

New Entity

The application provider first creates a new, unique entity: the metadata defining a managed service. The entity has several primary properties, including display name, namespace, and transaction logging type.

SLA, Role, and Transport

Every entity has default behavior properties. Its Service-Level Agreement (SLA) specifies service response and time-out limits that set the consumer's minimum performance requirements for the service. Violations of the SLA trigger MOM rules and notifications.

Roles set access defaults. Transport options set default packet-transmission behaviors for compression, encryption, and signature.

Entity Bindings

Bindings are individual services. Each service URL of the entity must be specified, along with its expiration schedules and search-criteria keywords. Once the bindings are specified, each binding gets its own SLA. The binding SLA sets its response and time-out limits, applicable deployment stage (for example, test or production stages), and selection priority.

This completes ACP publication of the application and its service bindings.

Roles

Once the application is published, users must be matched to the application, if they are to access the application's appropriate services. ACP administrators use the ACP Management User Interface to accomplish the match, by creating role definitions.

New Role

The administrator defines a new role with unique name and description. The administrator assigns individual user aliases, or entire Microsoft subsidiary aliases, to the role.

Include/Exclude Policies

The administrator then associates the role to an application with a policy. An Include Policy grants the role members access to the application's services. An Exclude Policy denies access.

Identity-Provider Synchronization

Each application maintains user identities within its own identity provider. Alchemy supports Microsoft Passport, Active Directory/NTLM, X.509 Certificate Server, and custom identity providers. The Alchemy runtime validates authentication credentials from these identity providers. After a user is authenticated, the Repository authorizes the user and applies policies to the user by role.

For details, please see "SOI on MSDN," under More Information, at the end of this paper. Alchemy Repository function is detailed in the "Alchemy Repository" white paper. The Alchemy system architecture is illustrated as the "SOI Reference Architecture: Alchemy System" poster. Runtime processing is illustrated as the "SOI Reference Architecture: Alchemy Runtime" poster.

Method-Level Overrides

Each published application has default properties, and each service binding can override some application properties with properties of its own, such as SLA parameters. This increases the granularity of management control over the application. Moreover, individual Web methods of each service can also override some service properties, for even finer management control.

Each Web method can specify its own transport options, to compress, encrypt, or sign responses from that particular Web method. Also, each Web method can define its own policy options, to include or exclude certain roles and users.

This fine-grained management control enables application providers to expose and transport the data of individual Web methods with the most appropriate level of security.

Consumer Subscription

Once a service application is published in ACP, the developers of applications consuming the service must subscribe to it. Subscribing to a service establishes the relationship between service consumer and service provider. The subscription process enables service providers to receive notification of a consumer's intent to use a particular service, and for the appropriate SLA parameters to be established between the two entities. The subscription process requires an exchange of consumer and provider human-contact information, so that two-way notification of pending changes, outages, and out-of-range SLA alerts can occur.

A consumer, for general definition, is any type of application that uses a proxy to communicate to a service. Currently, the internal Alchemy technology supports NT Services, Web Clients, Win Form Clients, and other .NET services as consumer types. The reason for the distinction stems from the differences in how the hosts affect the running of the Alchemy runtime. For example, if the application happens to be a Web application that is hosted by IIS 6.0, there are certain threading concerns to take into consideration, as IIS does its own thread management that will conflict with the standard threading code that a Win Form application would provide.

Prerequisites

The consuming application must be configured to use a published ACP-enabled service.

Using an ACP-Enabled Service in an Application

The application provider must modify the application to be a consumer of the ACP-enabled service by performing several steps within the development environment:

Add a Web reference, by selecting the service from a services list.
Modify the Web reference class to enable an ACP service proxy.
Make calls to the service using the ACP proxy.

The application can now consume the published, ACP-enabled service.

Subscription Management

When the consuming application makes a call to the ACP-enabled service, ACP manages the call according to the SLA parameters that apply to the call. Several different SLAs can apply. SLA types are prioritized, with higher priority SLAs overriding lower ones. Here is a list of the SLA types, in order of increasing priority:

The universal principal (user) SLA—An SLA associated with the user initiating the request.
The consumer (client application) SLA—An SLA associated with the client application.
The published application (entity) SLA—An SLA associated with the service application.
The published service-binding SLA—An SLA associated with a particular service binding, or physical implementation.
Any method-level override SLA settings–An SLA associated with a particular Web method.

Priority of the SLAs is correlated to specificity: More specific SLAs override the more general ones. ACP applies a Dynamic SLA filter to these prioritized SLAs, to determine the appropriate SLA settings for each service call. These dynamic SLA settings will manage the transaction's:

Response time (maximum time allowed for server response).
Stage (server location—for example, staging or production).
Priority (call-response priority).
Time out (non-response time limit).
Keywords (for locating the correct service binding).

This default, native routing can be overridden. Both the developer and the administrator can override Dynamic SLA to force a particular SLA for every call. The two override options are Client Name and Proxy Override:

Client Name

An administrator can supply a Client Name. When a Client Name is supplied, the Client Name SLA will be used, along with any Client Name keywords. The net effect is that a specific application will always point to a particular stage—such as Test—regardless of any other SLAs derived for the call.

Proxy Override

Similarly, a developer can supply a Proxy Override. A programmer can set the stage, priority, and keywords of the proxy call in order to pick a service binding "on the fly"—as, for example, by supplying geographic keywords to poll multiple geographic instances of an information service.

SLA Compliance

ACP feeds transaction performance data to Microsoft Operations Manager (MOM). MOM evaluates each transaction to determine if its SLA parameters (for example, response time) fall within the transaction's applicable SLA. If the transaction falls outside SLA parameters, MOM generates reports and alerts for management.

Call Management

ACP service transactions are automatically logged, enabling monitoring, analysis, reporting, and alerting capabilities required for operations management and enforcement of SLAs.

Exceptions and Tracing

ACP logs and inspects each exception, before reporting the exception message and the name of the exception type back to the originating proxy or service. When ACP handles an exception, an exception class is thrown from within the failing code. This informs the ACP Runtime that an error has occurred, allowing proper logging and interrogation of the exception.

ACP has features to help diagnose the cause of failures in applications using the ACP. One ACP diagnostic feature is tracing. Tracing records data for:

Inbound requests.
Outbound requests.
Startup.

Tracing logs the exceptions and instrumentation details associated with these events, writing them to an Event Log, and then extracts them to the Repository for notifications and analysis.

Root-Cause Analysis

ACP performs root-cause analysis of SLA violations and transaction exceptions, to provide proactive notification of the cause of failure.

SLA Violations

ACP monitors all SLA conditions, such as transaction response times and time outs. The GTS analyzes each transaction for SLA violations, comparing conditions against the SLA in effect on each transaction. When SLA violations pass a predefined threshold, the GTS sends notifications to the application providers and consumers, and can also send an extract to MOM for further action.

Transaction Exceptions

When a transaction exception occurs, ACP extracts the SOAP message envelope, the failing method, and the exception message, writing all to the Event Log. ACP passes this data to the GTS, where rules determine the appropriate data to be sent in notifications to application providers and consumers. GTS also forwards the extracted data to a data warehouse for OLAP analysis.

Notifications

When ACP encounters any entity failure, such as a transaction exception, it sends detailed notifications to the provider and/or consumer of that entity. ACP sends notifications via e-mail or Instant Message.

For the provider, ACP parses the transaction for the exception messages. It locates the provider contact from within its Repository, then composes and sends a notification to the provider, listing the messages.

ACP also queries the Repository for affected applications and consumers. It locates the consumer contacts, and sends notifications of the entity failure to all consumers.

OLAP Metrics

ACP maintains an OLAP data cube for transaction metrics reporting. ACP populates the cube as follows:

The ACP Staging data warehouse pulls transactions from the Global Transaction Store database and ACP domain data from the Repository.
A batch job populates the ACP Metrics data mart with data extracted from the Staging data warehouse.
A data cube exposes the Metrics transaction data for reporting.

ACP transaction parameters are represented in the data cube as OLAP dimensions. An OLAP dimension is a concept hierarchy: High concepts give abstract data representation with low detail, while lower concepts give detailed data. The transaction data itself constitutes a facts table to feed the data cube.

When users navigate OLAP dimensions in OLAP analysis, they drill down through the dimensional concept hierarchy to the level of detailed representation required. A cube viewer, Microsoft Insight, simplifies OLAP analysis by providing managed (or predefined) OLAP reports on ACP transactions.

For details, please see "SOI on MSDN," under More Information, at the end of this paper. The Alchemy analytic engine is detailed in the "Alchemy Analytics" white paper.

Microsoft Insight displays OLAP reports in pivot tables.

Figure 4. Microsoft Insight ACP Metrics report (Click on the picture for a larger image)

Versioning

The ACP system was designed with security in mind, especially around access-rights management, and provides mechanisms to ensure call-level access to particular resources. This is accomplished through policies configured on the central server and distributed to consumer clients. Checks are made at both the proxy and service, to ensure that all principals involved in the transaction have the required access credentials and adhere to the policy established for the specific service.

ACP Runtime compares security-policy versions on the server providing the service, and the consumer calling it, to ensure that the latest policy is always in effect on every transaction. To verify policy version, the Security Token Service (STS) periodically compares the locally cached consumer entity configuration version to the master version stored in the Repository.

For example, if consumer makes a call to the service when its policy is out of date, the STS will pull the master version from the Repository and send the new policy to the consumer. The consumer will cache this policy, and use it on this transaction and future transactions.

Conversely, if the server's policy is out of date, ACP will halt all transaction processing, refresh the server policy, and resume transaction processing with the new policy.

For details, please see "SOI on MSDN," under More Information, at the end of this paper. Alchemy Repository versioning is detailed in the "Alchemy Repository" white paper.

Consumer Scenario

Once a Microsoft development team has published a service application to ACP, another team can register its desire to subscribe to that application, and use its services in their own application.

Microsoft IT included an automated subscription workflow process in ACP, so that developers could register their intent to use an existing service. The subscription process includes the exchange of contact information that informs an ACP services provider about the consumers of their services and the applicable set of service-level parameters. Both providers and consumers have the option of receiving an e-mail alert or an Instant Message when an ACP service operates outside of its SLA.

Microsoft IT found that reducing the complexity that developers experienced in discovering and subscribing to services was a key factor in encouraging broad adoption of enterprise-wide services.

Discovering the Service

There are four ways for developers to discover ACP-enabled services: through discovery file, UDDI registry, Visual Studio reference, and the Alchemy Management User Interface. Usage and recommendations for each discovery method are discussed in the following subsections.

Discovery File

The simplest discovery method is direct file browsing. Developers can discover an ACP-enabled service directly by browsing the URL of the discovery file (.asmx file) of the service. While simple, this discovery method does not scale to the enterprise, because enterprise developers need a policy-based distribution method for service discovery. UDDI was the first discovery method to address this need.

UDDI

The Universal Description, Discovery, and Integration (UDDI) protocol is a member of the standards that make up the services stack. UDDI provides a basic method for publishing and discovering the services of a Service-Oriented Infrastructure (SOI).

The UDDI registry provides a policy-based distribution mechanism for services. ACP supports UDDI version 2.0. In ACP, all entities are published through a public UDDI registry to the Alchemy Repository. Once published, these entities are available to UDDI discovery by way of the Microsoft UDDI Business Registry.

The UDDI registry serves a purpose similar to that of a telephone directory. Just as a telephone directory provides a simple index of names and numbers, organized into broad categories (for example, residential and business), the UDDI registry organizes services into broad business categories. A UDDI registry does not itself provide detailed metadata on every managed property of a service. The ACP service taxonomy extends the UDDI registry with this functionality.

Alchemy Management User Interface

The ACP service taxonomy provides far more information about a service than can be provided by UDDI, and thereby makes the service much easier to find. The Alchemy Management User Interface allows developers to discover an ACP-enabled service by searching on any property of that service. For example, developers might search on both a service's name and also its exact version, to ensure they use only the latest version of the service. Searchable properties include application name, service name, version number, stage, SLAs, security options, and many other properties maintained through the Alchemy Management User Interface and supported by the ACP taxonomy.

UDDI by Way of Visual Studio

Visual Studio supports UDDI discovery from within the application project. Once the developer has discovered a published ACP-enabled service in the Alchemy Management User Interface, the developer adds a Web reference to the project. The developer then specifies the Web server's discovery file and selects a service from the list of available services. Finally, the developer provides a namespace to reference the service within the project.

Integrating with the Service

Developers can interact with the ACP Runtime from within a Service method, to perform a number of useful modifications to the standard ACP behavior.

Server-Side Programming:

Instrumentation Data

Developers can add instrumentation to their methods, to monitor or measure the level of the method's performance and diagnose errors. Developers can build their own instrumentation data-collection mechanism to facilitate proper diagnosis of how their application is working.

Server-Side Security

The ACP Runtime only provides call-level access control. Data-level access is still the responsibility of the application. With this in mind, the ACP gives the developer inbound service extended information about the accounts requesting or sending information, so that the developer can make the most informed decision when controlling service access. Developers can protect specific blocks of code by requiring appropriate permissions. They can also place security information into the metadata of their code.

Server-Side Exception Handling

Developers can throw a native or user-defined exception class from within the failing code, to inform ACP Runtime of exceptions, and facilitate proper logging and interrogation of the exception.

Client-Side Programming:

Proxy SLA Override

The ACP Runtime normally will calculate the best endpoint, based upon available information gathered from the Alchemy Repository about all the principals involved in the transaction. Developers can override this default programmatically, so that during runtime the code can set the active stage, priority, and keywords for the ACP Runtime to find the most appropriate binding. By giving developers this level of control, they can select multiple endpoints by just changing a property, such as keywords or priority.

Exception Handling

The decision to handle ACP exceptions from within the client code means direct interaction from the main exception classes that the ACP Runtime can throw during standard execution. The placement of the exception handling is entirely up to the application, and there are no true rules about doing so. In general, most applications will put proxy calls in a try-catch block, and then add multiple catches for the expected throw types.

Enterprise Scenario

Alchemy Common Platform overcomes some critical enterprise limitations to service architectures. The limitations arise from line-of-business segmentation and governance risks.

Line-of-Business Segmentation

Many enterprise line-of-business development teams deploy services with limited discovery options and highly restricted exposure.

UDDI registry discovery is restricted to a simple list of business categories. Developers can search for services by business name, service name, discovery URL and a few other general categories. UDDI provides no facility for granular search.

For line-of-business security, developers limit exposure by placing access-control lists (ACLs) on the services. An ACL-secured UDDI registry cannot serve the full enterprise, because each service is exposed only to the line-of-business segment that has ACL rights.

Need for a Global Repository

ACP addresses the need for enterprise-wide exposure and flexible security with a global repository. The Alchemy Repository stores metadata for all ACP-enabled services in the enterprise, globally, and this metadata facilitates discovery. Developers can search for ACP-enabled services by UDDI business categories, and also by granular ACP entity properties. The Alchemy Repository exposes granular service properties to the entire enterprise, to increase the visibility and use of ACP-enabled services, while maintaining ACP's robust security policies on all services. This helps service developers overstep the limiting boundaries of line-of-business segmentation.

For details, please see "SOI on MSDN," under More Information, at the end of this paper. Alchemy Repository function and metadata are detailed in the "Alchemy Repository" white paper.

Governance Risks

It is necessary to adhere to service standards and specifications, when building a robust yet flexible SOI. The risks inherent in enterprise systems must be mitigated through governance. Enterprise architecture governance groups enforce standards for service development. Governance can be implemented in the form of templates, methodologies, regulatory compliance rules, software-development life-cycle (SDLC) management procedures, strategic initiatives, and other best-practices directives.

The application of governance to services is made more complicated when an enterprise has varying needs for control in different line-of-business segments. Some segments might have a need for only light control. If a segment's services serve up public data without SLA commitments, the need for control might be light. At the other extreme, if a segment's services deal with sensitive data (for example, financial or human-resource data) or perform under high-performance SLA commitments, the need for control can be very heavy. In these environments, governance must be exercised on the full SDLC. Every published service must undergo:

Business justification, to ensure the enterprise value and correctness of the service proposal.
SDLC process management, to enforce best practices on development and deployment of the services.
Monitoring and audit, to verify the enforcement of SLAs on service transactions and to report and analyze exceptions that arise in those transactions.

Microsoft IT's ACP Governance

Microsoft built the ACP to address many facets of the enterprise-governance challenge. Microsoft IT built ACP under the direction of an internal services governance body that included representatives from the central IT group and key line-of-business application providers. These providers deployed and managed all services under ACP. They exercised particular governance on each application, entity, service, and method published within ACP.

Need to Leverage Enterprise Infrastructure

Enterprises must integrate services into their existing infrastructures, with minimum cost and complexity. Microsoft IT integrated several enterprise infrastructure systems into ACP. ACP has leveraged the following infrastructure systems:

Service development—Visual Studio 2005
Service framework and runtime—.NET Framework 1.1 and 2.0, WS-*
Automated e-mail and Instant Message notification—Exchange Server 2005, MOM 2005
UDDI service registry—UDDI 2.0
User/consumer principal authentication—Active Directory, X.509, Microsoft Passport
Service metadata, transactional data and OLAP reporting—SQL Server 2005
Application portfolio management—MS Apps
Enterprise asset management—Systems Management Server 2003

ACP's deep integration into enterprise infrastructure systems maximized its enterprise value, while minimizing its deployment cost and complexity.

Project Overview

Over a period of more than four years, Microsoft IT gained valuable experience using services to integrate line-of-business application services and data. Microsoft IT leveraged ACP to combine data from multiple line-of-business applications. The goal was to enable easy integration of business data from four core line-of-business applications at Microsoft:

Siebel: customer relationship management
Clarify: product service request management
MS Sales: sales revenue reporting
World-Wide Sales and Marketing Database: management of e-mail newsletter subscriptions, event invitations, and diverse marketing campaigns

ACP-enabled services supported read and update access to several additional business applications. Microsoft client- and server-based applications used ACP-enabled services as the Microsoft IT standard for accessing line-of-business application services and data.

Timelines

With ACP, common tools and frameworks enabled rapid development of standards-based services. Microsoft IT found that a typical service, which normally could be built and deployed in four to six months using only basic services standards, could be built and deployed in four to six weeks on ACP.

Budget

By reducing service development and deployment times from months to weeks, ACP helped Microsoft IT realize significant cost savings. In one specific project, Microsoft IT was able to reduce its custom development costs by almost 20 percent.

By co-hosting multiple services on a single server, ACP reduced the need for server hardware and software, with a corresponding reduction in the overall number of staff required to manage server operations. In one project, one full-time Microsoft IT operations staff member had previously been required, on average, for each of 15 line-of-business applications. When those applications were migrated to ACP, only two staff members were needed to manage the entire enterprise. This efficiency increase, therefore, reduced operations staff from 15 to 2.

Impact

ACP provided both Microsoft IT and Microsoft business divisions with significant benefits, based on the deployment of a consistent services development and management framework:

Seamless and easy integration of services from existing line-of-business applications
Cost savings from deploying fewer server hardware and software resources
Accelerated adoption and reuse of XML services
Proactive management of service-level agreements for services

Seamless and Easy Integration

Organizations must be more agile and responsive than ever before, and Microsoft is no exception. Companies require an application architecture that enables the rapid implementation of services that can be quickly recombined to create new customer management, sales, marketing, and product-development solutions.

Reduced Server Deployment

Microsoft IT's early deployments of services required a separate server environment to be deployed, to provide basic service support for each line-of-business application. ACP made it possible for Microsoft IT to deploy and manage several services on a single server, as well as small, medium, and large server-farm configurations.

From a services-management perspective, all services deployed at Microsoft are supported by a single network of eight ACP back-end servers. To provide the same level of redundancy and performance on an application-by-application basis would require 80 servers, a tenfold increase.

Fewer resources and better management practices increased uptime for business users, while reducing costs.

Accelerated Adoption and Reuse of Services

Automated subscription processes for managing services reduced costs and encouraged fast adoption and reuse of common business services. At Microsoft, automation reduced the process time from one week to one day for subscribing to a service.

Proactive Management of Service-Level Agreements

ACP enabled proactive operations management and practical enforcement of service-level agreements by providing standard monitoring, logging, analysis, reporting, and alerting functions. These provided service providers and consumers with the tools needed to monitor, measure, and manage out-of-bounds service levels, such as poor response times or "server-down" situations. These capabilities are mandatory for the deployment and management of a production-quality, enterprise-wide platform for services.

Lessons Learned

Manage Services

Microsoft IT found that without proactive management, services cannot scale to a production-quality, enterprise-wide platform. Unpredictable service levels are a near certainty when services encounter enterprise loads. The Alchemy Common Platform (ACP) gave Microsoft IT the granular management needed to ensure service levels in the enterprise.

Adopt a Service-Oriented Architecture

In a Service-Oriented Architecture, applications are structured as interacting and cooperating services. Microsoft IT found that by adopting ACP, they were able to respond to business changes more quickly and with more predictable results. Services make it easier to connect systems. Abstraction also gave Microsoft IT the ability to change underlying systems without changing the end-user/systems experience.

ACP helped Microsoft steer clear of monolithic applications that are resistant to change, difficult to integrate with other applications, and prone to duplicate and incompatible functionality.

Use Readily Available Tools and Frameworks

Readily available tools and frameworks enable rapid development of standards-based services. Microsoft IT has leveraged Microsoft Visual Studio .NET, Microsoft .NET Framework, and the Web Services Enhancements (WSE) for Microsoft .NET to deliver ACP very rapidly.

Line-of-Business Application Providers Should Own Their Services

Within the context of the strategy, architecture and guidelines established by the services-governance body enable each line-of-business application provider to build and own services for their own application. In most situations, providers are the most knowledgeable of specific line-of-business application functionality, and ready to reuse line-of-business data through services.

Deploy and manage all services using a common, production-quality services-management solution.

Conclusion

The Alchemy Common Platform (ACP) enabled Microsoft IT to deliver a more agile service-oriented architecture. Fifteen groups have built, deployed, and consumed services supported by ACP. Business capabilities increased as development, integration, and operations management costs decreased. ACP enabled new functionality leveraging core IT systems to be brought online more rapidly than would otherwise have been possible.

ACP adheres to the services standards and specifications implemented in the Microsoft .NET Framework and the Web Services Enhancements for Microsoft .NET. For this reason, Microsoft IT was able to build and deploy ACP with current technologies and, at the same time, position itself to interoperate with future services standards-based environments.

More Information

For links to Alchemy and SOI white papers, posters, customer implementations, presentations, and video feeds, please refer to the SOI architecture center, SOI on MSDN.

Thứ Sáu, 5 tháng 7, 2013