Book Image

Do more with SOA Integration: Best of Packt

By : Arun Poduval, Doug Todd, Harish Gaur, Jeremy Bolie, Kevin Geminiuc, Lawrence Pravin, Markus Zirn, Matjaz B. Juric, Michael Cardella, Praveen Ramachandran, Sean Carey, Stany Blanvalet, The Hoa Nguyen, Yves Coene, Frank Jennings, Poornachandra Sarang, Ramesh Loganathan, Guido Schmutz, Peter Welkenbach, Daniel Liebhart, David Salter, Antony Reynolds, Matt Wright, Marcel Krizevnik, Tom Laszewski, Jason Williamson, Todd Biske, Jerry Thomas
Book Image

Do more with SOA Integration: Best of Packt

By: Arun Poduval, Doug Todd, Harish Gaur, Jeremy Bolie, Kevin Geminiuc, Lawrence Pravin, Markus Zirn, Matjaz B. Juric, Michael Cardella, Praveen Ramachandran, Sean Carey, Stany Blanvalet, The Hoa Nguyen, Yves Coene, Frank Jennings, Poornachandra Sarang, Ramesh Loganathan, Guido Schmutz, Peter Welkenbach, Daniel Liebhart, David Salter, Antony Reynolds, Matt Wright, Marcel Krizevnik, Tom Laszewski, Jason Williamson, Todd Biske, Jerry Thomas

Overview of this book

<p>Service Oriented Architecture (SOA) remains a buzzword in the business and IT community, largely because the ability to react quickly is of utmost importance. SOA can be the key solution to this. The challenge lies in the tricky task of integrating all the applications in a business through a Service Oriented Architecture, and &ldquo;Do more with SOA Integration: Best of Packt&rdquo; will help you do just that with content from a total of eight separate Packt books. <br /><br />&ldquo;Do more with SOA Integration: Best of Packt&rdquo; will help you learn SOA integration from scratch. It will help you demystify the concept of SOA integration, understand basic integration technologies and best practices, and get started with SOA Governance. &ldquo;Do more with SOA Integration: Best of Packt&rdquo; draws from eight separate titles from Packt&rsquo;s existing collection of excellent SOA books:</p> <ol> <li>BPEL cookbook</li> <li>SOA Approach to Integration</li> <li>Service Oriented Architecture: An Integration Blueprint</li> <li>Building SOA-Based Composite Applications Using NetBeans IDE 6</li> <li>Oracle SOA Suite Developer's Guide</li> <li>WS-BPEL 2.0 for SOA Composite Applications with Oracle SOA Suite 11g</li> <li>Oracle Modernization Solutions</li> <li>SOA Governance</li> </ol> <p><br />The chapters in &ldquo;Do more with SOA Integration: Best of Packt&rdquo; help you to learn from the best SOA integration content in no less than eight separate Packt books. The book begins with a refresher of SOA and the various types of integration available, and then delves deeper into integration best practices with XML, binding components and web services from Packt books like &ldquo;Oracle SOA Suite Developer's Guide &ldquo; and &ldquo;BPEL Cookbook&rdquo;. Along the way you&rsquo;ll also learn from a number of real world scenarios. By the end of &ldquo;Do more with SOA Integration: Best of Packt&rdquo; you will be equipped with knowledge from a wide variety of Packt books and will have learnt from a range of practical approaches to really get to grips with SOA integration.<br /><br />Chapter listings with corresponding titles:</p> <ul> <li><strong>Preface</strong> - Dismantling SOA Hype: A Real-World Perspective (BPEL cookbook)</li> <li><strong>Chapter 1</strong> - Basic Principles: Types of integration (Service Oriented Architecture: An Integration Blueprint)</li> <li><strong>Chapter 2</strong> - Integration Architecture, Principles, and Patterns (SOA Approach to Integration)</li> <li><strong>Chapter 3</strong> - Base Technologies: Basic technologies needed for SOA integration (Service Oriented Architecture: An Integration Blueprint)</li> <li><strong>Chapter 4</strong> - Best Practices for Using XML for Integration (SOA Approach to Integration)</li> <li><strong>Chapter 5</strong> - Extending Enterprise Application Integration (BPEL cookbook)</li> <li><strong>Chapter 6</strong> - Service-Oriented ERP Integration (BPEL cookbook)</li> <li><strong>Chapter 7</strong> - Service Engines (Building SOA-Based Composite Applications Using NetBeans IDE 6) </li> <li><strong>Chapter 8</strong> - Binding Components (Building SOA-Based Composite Applications Using NetBeans IDE 6) </li> <li><strong>Chapter 9</strong> - SOA and Web Services Approach for Integration (SOA Approach to Integration)</li> <li><strong>Chapter 10</strong> - Service- and Process-Oriented Approach to Integration Using Web Services (SOA Approach to Integration)</li> <li><strong>Chapter 11</strong> - Loosely-coupling Services (Oracle SOA Suite Developer's Guide)</li> <li><strong>Chapter 12</strong> &ndash; Integrating BPEL with BPMN using BPM Suite (WS-BPEL 2.0 for SOA Composite Applications with Oracle SOA Suite 11g) </li> <li><strong>Chapter 13</strong> - SOA Integration&mdash;Functional View, Implementation, and Architecture (Oracle Modernization Solutions)</li> <li><strong>Chapter 14</strong> &ndash; SOA Integration&mdash;Scenario in Detail (Oracle Modernization Solutions)</li> <li><strong>Appendix</strong>: Bonus chapter - Establishing SOA Governance at Your Organization (SOA Governance)</li> </ul>
Table of Contents (20 chapters)
Do more with SOA Integration: Best of Packt
Credits
About the Contributors
www.PacktPub.com
Preface

People


There are many different people that can play a role in your SOA governance efforts. This section will go over the various roles and their responsibilities within SOA governance, whether it be in establishing policies, or in the processes associated with education, communication, measurement, or enforcement. It will also call out how these various roles can be organized and the various engagement models that can be utilized.

Solution Architect

The solution architect is the person responsible for the technical leadership and decisions on a project or program. Being the technical decision maker, this person is responsible for the project's compliance with the technical design-time policies. This includes both the solution's architecture, for which the person likely has direct responsibilities, as well as the solution's design, for which the person will be working with the developers and other project staff. Typically, the solution architect has a larger role in ensuring that services are built the right way. In ensuring that the right services get built, the solution architect must work closely with the lead analyst on the project.

The solution architect is also responsible for ensuring that the appropriate run-time governance policies are established. If the project delivers a service consumer, the solution architect must work with the lead analyst to determine the expected usage for all services. If the project delivers a service provider, the solution architect must determine the resource consumption associated with a service invocation.

It is important to remember that the solution architect is a role. Your organization may not have a formal job title of solution architect, but someone will play this role on each and every project.

Business Analyst

The business analyst is the counterpart to the solution architect on a project, focusing on the functional aspects of the solution. They frequently perform requirements gathering activities, create use case diagrams, and may also be responsible for business process modeling. A more senior business analyst will have a significant amount of domain knowledge and may have key relationships within the organization outside of IT.

Within the SOA governance effort, business analysts play a key role in determining the appropriate functional boundaries for services. If you are able to move beyond the boundaries of projects and pursue the development of business architecture models, business analysts may be a significant part of the effort. If your enterprise architecture organization is focused on technology architecture, an effort to establish business architecture will require the domain knowledge of your most senior business analysts.

Technical Lead/Domain Architect

The technical lead and/or application architect is typically the job title associated with individuals that act as solution architects. Frequently, however, these individuals also have domain architecture responsibilities. That is, they may have oversight over several projects within a particular domain, or a role in establishing reference architectures within a particular domain outside of the context of any particular project.

Clearly, these individuals have a role in the many of the governance policies discussed earlier in the book. If an organization chooses to establish a SOA Center of Excellence, it is very likely that the domain architects will play a significant role. The domain architecture provides context for all of the projects that are executed within that domain. This can include business process analysis or business architecture development, as discussed earlier. It may also include guidance for particular technologies that are only used within that domain, such as ERP modules.

A particular challenge for this role is that these individuals frequently come from a development background, rather than an analyst background. The differences in architecture from domain to domain are typically not in the technologies utilized, but the functional domains involved. If these individuals do not have the proper functional knowledge of their business domains, they need to work closely with the key analysts in their organizations.

Finally, the technical leads must have clear and open communication with each other. It is likely that the current organizational structure does not reflect the boundaries between service consumers and service providers, meaning that there is significant redundancy across those domains. In the early stages of SOA adoption, these domain architects must work together to provide assistance to the IT managers in determining appropriate ownership.

Enterprise Architect/Technology Architect

The enterprise architect plays a very similar role to that of the domain architect. The key difference is that enterprise architects normally deal within domains that are applicable to the entire enterprise. This tends to include all of the pure technology domains, as well as security. Recently, enterprise architecture is expanding into the business architecture space, being seen as a key to aligning IT with the rest of the company. Organizations that are embracing business architecture will likely include domain architects as part of their enterprise architecture team.

In terms of SOA governance, the enterprise architect, like the domain architect, is typically in an oversight position, rather than being an active project team member. Enterprise architects are frequently responsible for strategies, roadmaps, reference architectures, and patterns that project teams are expected to implement and utilize. Therefore, enterprise architects are clearly well-positioned to provide governance. A reference architecture or design pattern is nothing more than a collection of policies that ensure solutions are built the appropriate way.

Enterprise architects that are more focused on technology architecture are responsible for the infrastructure strategy. Earlier in the book, a number of alternatives were presented for implementing policy-driven infrastructure. If enterprise architecture handles technology architecture, then it is their responsibility to set the direction in this space. This also includes ensuring that application teams use the infrastructure properly. This is an especially important concern in the SOA space primarily due to the emergence of the Enterprise Service Bus. No two ESBs are alike, as products marketed as an ESB come from EAI (Enterprise Application Integration) technology, MOM (Message-Oriented Middleware) technology, network appliances, and even products that are built from scratch. It is up to the enterprise architect to provide clarity on how an ESB should be utilized, if even at all. One of the questions that SOA governance must address is how to ensure that services are built the right way; clearly, the policies that ensure this behavior are the domain of the enterprise architect.

Enterprise architects that have business architecture responsibilities have an even larger role in the SOA governance efforts. The second major question is how to ensure the right services get built, and the key is business architecture and the creation of appropriate business domain models, possibly including business process models or business capability maps. The enterprise architect that is focused on business architecture has a role in performing the analysis and incorporating the business strategy into these business domain models that ultimately will be used in the project definition process to ensure the right services get built.

Information Architect

The information architect is a role that sometimes may get overlooked in SOA governance. A key challenge in adopting SOA is striving for consistent representations of information on service messages. Frequently, information architects may not be involved in this discussion because the conversation is typically between developers or solution architects. As a result, the representation of the information may be rooted in the processing models of the service consumers and service provider. This is neither good nor bad, but it does not take into

account any storage model of the information, or any work that may be done by an information governance council in trying to define a canonical model or data dictionary.

In order for an organization to strive for consistency in the representation of information, it is important to include information architecture in the SOA governance efforts for service design, specifically in providing context for defining the schemas of the service messages.

Security Architect

The security architect is the role associated with ensuring that the technology solutions of the company take appropriate measures to protect its sensitive information, whether for protection of intellectual property, or for privacy protection related to the information maintained by the company. It is very likely that security architects are already involved with governance, whether it is part of broad IT governance related to Sarbanes-Oxley, or more exclusively focused on security policies.

One key area of focus for the security architect with regards to SOA governance is the role of identity in service interactions. Run-time governance techniques, such as traffic shaping/request throttling, are completely dependent on having identity of the consumer. As the degree of interactions between services increases, determining exactly what identity should be passed through these interactions is a challenge, and one that will require the guidance of your security architects.

IT Manager

The role of the typical IT manager is to manage personnel, manage work, and manage budget. This role is absolutely critical to the success of the SOA, because the changes that can be achieved have the most impact on IT management. Developers will still be writing code, but managers will have their entire world changed. IT managers that previously had complete control over their applications now may be dependent on other IT managers, or have other IT managers dependent on them. While most organizations are already dealing with this when it comes to infrastructure dependencies, many are not dealing with application-to-application dependencies.

If we look significantly forward, a successful SOA adoption can lead to significant organizational changes. IT managers may be reluctant to embrace this change as they may feel that their power is being taken away from them. Upper management tends to remain stable, and the people working in the trenches will still be doing analysis, project management, or development, but middle management has no such guarantees.

In order to mitigate this risk, it is important that IT management, specifically the middle managers, take an active role in the early decisions around service ownership.

Service Manager/Owner

The service manager, also known as the service owner, is a new role to most organizations, although if your organization has adopted ITIL and ITSM, you may have some familiarity with the concepts, but focused on services provided by IT operations. The service manager is the person responsible for managing the relationships with service consumers, scheduling and executing the upcoming releases of the service, and establishing relationships and service contracts with new consumers.

The service manager plays a key role in ensuring proper run-time governance is in place. Ultimately, the service manager, as the name suggests, is responsible for the service that is provided to the consumers, and the last thing needed is to have a service outage. The service manager must ensure that all details of the service contract have been established and that the infrastructure has been properly configured to enforce the policies within the service contract.

Initially, service managers may come from IT, but it is entirely possible that service managers may also be placed outside of IT, especially where services are exposed to external partners.

Platform Manager

The platform manager is responsible for the technology platform where services (and consumers, for internal consumers) are hosted. For most organizations that do not have the role of service manager defined, it falls to the operations team, whether an individual engineer or team or their associated manager to ensure that no outages occur in production.

Even when an organization does have a service manager role, the platform manager is still responsible for the underlying platforms, effectively providing a hosting service for the solutions that are deployed on them, whether those solutions represent service consumers or service providers. An individual service manager is primarily concerned about their service, and may not be watching the entire platform. Furthermore, for a service that involves several systems in its implementation, multiple platform managers may be involved, while the service manager is still responsible for the end-to-end performance of the entire service implementation across all systems, as illustrated by the following diagram:

In this diagram, Service A is comprised of components that run on an orchestration platform (BPEL engine), an application server, a mainframe, and an ERP system. The manager for Service A must be concerned with the end-to-end performance across the platforms. Service B is comprised of a Web Service hosted on the application server platform and an ERP component. The manager for Service B is concerned with these two components. In contrast, each of these platforms has a platform manager that is concerned with the overall performance of the platform and each component

running on it. The manager of the application server platform is concerned about the performance and capacity of the application server platform and must watch the performance of the Java EE component of Service A, and the web service of Service B, but is otherwise unconcerned about the remaining components that constitute Service A and Service B.

The platform manager is involved with establishing the entry criteria in order for a component to be deployed on a platform in a production environment. For example, if it is determined that a service did not use the appropriate instrumentation framework to allow for operational monitoring, it should not be allowed to go into production, since it can put the entire platform at risk.

Other Stakeholders

Besides the individuals already mentioned, there are other stakeholders that will play a role in your SOA governance efforts. While the majority of the technical governance issues are covered by the existing roles that have been identified, there are additional roles that may be involved with the non-technical governance. These individuals are likely already involved with your normal IT governance efforts, such as members of review boards that approve capital expenditures and project requests. Rather than forming additional committees designed to be involved in the governance process, it is better to ensure that the desired behaviors and goals for SOA are factored into the existing governance process.

Organizing Your People

There are many different organizational approaches that can be leveraged for your SOA governance efforts. SOA adoption is an enterprise initiative, however, which creates a challenge. Of the roles mentioned, only the Enterprise Architect, the Information Architect, and the Security Architect normally operate at the enterprise level. The remaining roles are either focused on project activities, on segments of technology, or on segments of the business. At the same time, many Enterprise Architectures are heavily focused on the underlying technology platforms, and may not have the appropriate functional knowledge to contribute to defining the functional boundaries associated with SOA adoption. As was shown in the Advasco story, both the technology architecture and the functional architecture are key aspects of SOA. Information architects are similar to enterprise architects in that they have broad visibility, but in a narrow area; in this case, the data systems of the enterprise. A similar story holds true for security architects.

Given the plethora of roles available, but the lack of the right role, it is important that the organization determine an approach to ensuring a successful adoption of SOA. There are a variety of different techniques that can be used.

Enterprise Architecture Driven

In this approach, the responsibility for SOA adoption lies solely within the enterprise architecture team. This can be a very good fit for many of the project governance concerns, and even some of the run-time governance concerns, since Enterprise Architecture is frequently tasked with driving strategic technologies into the enterprise. If your Enterprise Architecture team is also responsible for business architecture, then it is in an even stronger position, as it can also handle many of the pre-project governance concerns. The risk, however, is that technology is just a small part of SOA. The greater challenge is the cultural changes both in the way projects are defined and the way they are built. This can be a much more difficult task for Enterprise Architecture, since they are not normally in the management hierarchy with the people who will be most impacted by SOA adoption.

Two key relationships that can be a good litmus test before embracing an Enterprise Architecture driven SOA effort is the relationship with key IT managers, including the ability to influence organizational issues, and the relationship with key business leaders, especially those responsible for business strategy and architecture. If the Enterprise Architecture team is responsible for business architecture, chances are even better that they can drive the effort.

Another factor is the current role of Enterprise Architecture in the educational and governance processes that already exist in the enterprise. If the organization is used to taking direction and learning from Enterprise Architecture, this is a good thing. If the enterprise architects already play a role in the governance process for software development, whether through formal reviews or through less formal project engagements, again, it is not a big stretch to have them include enforcement of some of the SOA policies in their existing activities.

One risk associated with the Enterprise Architecture driven approach is the number of resources that can be dedicated to the effort. While one or more members of the EA group may be dedicated (or partially dedicated) to the effort, they need support from members of other organizations to participate in the effort. Another risk is in the area of run-time governance. An enterprise architect is normally not involved with the run-time management of production solutions. As a result, they are not as well-positioned to provide guidance on the right way to manage the services. Given the strong technology backgrounds of many enterprise architects, however, they usually have excellent relationships with the operational staff that allows for appropriate influence.

Center of Excellence/Competency Center

In this approach, the responsibility for SOA adoption is given to a cross-functional team of people. This has advantages, since it can draw from all areas of the organization and can include as many of the roles associated with SOA governance as desired. There are two factors that can have a large influence in the success of a Center of Excellence: the composition of the team and the engagement model with the organization.

The composition of the team is the first decision that must be made by the organization. As with any virtual team, a common mistake is to choose people who have time to participate rather than choosing people who should be participating. Just as with the Enterprise Architecture driven approach, it is important that the people chosen for the Center of Excellence have the ability to influence the rest of the organization. Choose people that will struggle trying to influence others, and there's a good chance that the entire SOA effort will struggle as well.

The second factor is the engagement model with the rest of the organization. While a permanent organization like Enterprise Architecture may already have a defined engagement model with projects, a new virtual team does not. There are many nuances to the Center of Excellence that must be examined. First, what is the team's purpose? Is it to communicate a message about SOA and educate the rest of the organization? If this is the only focus, there is a risk that the effort will become disconnected from the projects that are realizing the organization's SOA. If the Center of Excellence tries to remain connected by participating on projects, how will that occur? At a minimum, the Center of Excellence can act as a reviewer for projects. A second option is to act as a staffing center for the projects. This is a common approach for Competency Centers, where a goal of the group is to train other staff members by working alongside them. This works well for the more technical areas of SOA, such as teaching staff how to work with XML or Web Services, but it may not be as practical for the non-technical aspects. A third approach is to act as an outsourcing center for service development. This approach has benefits in that the development of key services can be separated from the projects that identified the need, allowing them to be developed in a way that benefits a broader audience. The inherent risk with this approach is that the Center of Excellence may not have the domain knowledge of the functional areas necessary to develop the services properly. Once again, it is likely that a team can be formed with the necessary technical skills, but it is far more difficult to staff a team with the necessary functional knowledge. Furthermore, it is not possible for a small, centralized team to own all services in the organization. Eventually, this approach will not scale. Your goal must be to establish service development and service integration as a standard capability of the entire organization, not just one team.

A successful approach for a Center of Excellence is to initially focus heavily on communication and education, along with providing some amount of resources that assist in staffing initial projects. Rather than have work outsourced to them, they provide staff to projects that can train other people, ensure compliance of the SOA governance policies within those projects, and also bring back feedback to the Center of Excellence on policies that encountered resistance or were problematic to implement. Over time, the need to be a staffing center will go away, but the need for SOA governance will not. The Center of Excellence must still be a source of policies, an arbitrator when policies are conflicting or confusing, a source of consulting for decision guidance, continued education and communication on the SOA efforts, and finally, the group that is measuring the efforts of the organization against the desired behavior and making whatever adjustments are necessary. Eventually, if the organization achieves the desired behavior, the need for the Center of Excellence may go away, as the behavior that they helped encourage has now become second nature for the organization.

Review Boards

Another approach to SOA governance focuses exclusively on the enforcement of the policies that have been outlined through the use of review boards. Many organizations already have formal review processes in place; therefore it makes some sense to extend the policies enforced by the existing review boards.

For example, it is very likely that your organization already has a review board that approves projects. This review board may currently focus on the financial side of the proposed projects in making their approval decisions. That review board can now factor in the business domain models to determine the architectural fit of the proposed project in addition to the financials of the proposal. This may require adding members to the approval process that are familiar with the architectural side of the equation, or it may require educating the existing members on the importance of the domain models.

The same thing holds true for architectural reviews and design reviews. The challenge for these reviews is the opposite of the project approval review. In that review, we needed to augment the process and optionally the people to take more technical issues into account. In the case of architectural and design reviews, the review board likely already covers many of technical issues, but is not properly positioned to handle organizational issues or scoping issues that may arise from applying the policies to the projects.

The biggest challenge with this approach to SOA governance, however, is that there are significant gaps. First, who is responsible for establishing policies in the first place? More often than not, review boards are responsible for enforcing policies but don't normally set them. At best, they may make decisions on the fly where policies don't exist, but even then, the reasons for the decision are unlikely to turn into official policy. In order to address this gap, some other authority in the organization must establish the policies that are enforced by the review boards. This will likely be an existing organization like Enterprise Architecture or a cross-functional group like a Center of Excellence.

Common Challenges

Regardless of the organizational approach chosen, there are several challenges that must be factored into your decisions around your organizational approach. The first is the risk of focusing too much on enforcement, and not enough on education on what the desired behavior is. When people don't understand the reasons why policies exist, they are likely to resist them when they see them as getting in the way of their responsibilities. When educated on the reason for the policies and the desired outcome that they are intended to achieve, they are more likely to be compliant with the policies. As stated earlier, the easiest way to achieve compliance is to make compliance the path of least resistance.

The second challenge is in picking governance processes that are inconsistent with the existing culture of the organization. If your organization is very consensus driven, an enforcement process that places extensive power in the hands of a few, such as with a review board, can be very risky. A strong focus on education, allowing key members to influence and persuade others in the process that establishes consensus may be easier for the organization to digest. If your organization is accustomed to a command-and-control structure, then yielding power to a few individuals and giving them authority to stop projects in their tracks may be possible. This is not to say, however, that matching the existing culture is always a good thing. Sometimes, the existing culture may be the biggest problem. Remember, SOA governance is about achieving a desired behavior in your SOA adoption efforts, and it is very likely that the desired behavior is different from what you are doing today. If the culture needs dramatic change, then dramatic changes in governance may be required. Frequently, however, this may require a change in leadership. It is typically very difficult for existing leadership to make dramatic changes in the behavior of an organization.

The next challenge is remaining relevant. A Center of Excellence is not the only approach that must be concerned about the engagement model with the rest of the organization. Enterprise Architecture is commonly at risk of becoming an ivory tower if they do not remain engaged with project activities. A review board is at even more risk, since they normally may not even meet with each other except when reviews are occurring. The team establishing policies must remain engaged with the teams that are expected to be compliant with those policies. An approach that strives to educate and mentor first, and then reviews and accepts/rejects second is more likely to be accepted, and more likely to have a higher compliance rate.

The final challenge is in matching the needs of all areas of the organization. While smaller companies may be able to have a single approach to SOA, larger companies with multiple lines of business may have different desired behaviors for each line of business. For example, if a company is focused on rapid growth in one line of business, but cost containment in another line of business, embracing policies that require company-wide standardization may help the second line of business cut their costs, but may severely restrict the ability of the first line of business to grow rapidly. Preferably, these goals are captured in the business domain models, however, they might not always be, or more likely, the domain models may not exist until further down the SOA journey. SOA must be business driven, and the behaviors, policies, and processes all must be reflective of those business goals.