[{"id":"ch01","contentType":"text","title":"Introducing the Concepts and Terminology","content":"

Chapter 1. Introducing the Concepts and Terminology

This chapter gives us an overview of the concepts and terminology that apply when implementing integrations with Oracle Integration Cloud Service. It explains the components a general integration consists of, with a step-by-step approach.

When we talk about integration, we mean the act of bringing components together into one single system. In the context of IT, we refer to a process that stitches together different subsystems, so that the data contained in each system becomes part of a larger single system that can share data more quickly and easily.

Oracle Integration Cloud Service is a solution that enables you to simplify integrations between cloud applications, and between cloud and on-premises applications. It helps you create connections to well-known and less-known SaaS and PaaS applications, using the available cloud adapters, publish or subscribe to the Messaging Cloud Service, or use industry standards such as SOAP, REST, FTP, File, and JMS. Most of these technologies will be explained in more detail later. Integration Cloud Service (ICS) provides enterprise-grade connectivity regardless of the application you are connecting to or where they are hosted.

The concepts and terminology can be categorized into three major areas:

Connections describe the inbound and outbound applications that we are integrating with
Integrations describe how information is shared between applications
Transformations and lookups describe how to interact with the data

We can engage with Oracle Cloud Services and especially with ICS by going to http://cloud.oracle.com/integration. Here we can try the service for free, which we can use when going through this book.

","url":"https://subscription.packtpub.com/book/data/9781786460721/1","metaDescription":"","titleSlug":"introducing-the-concepts-and-terminology"},{"id":"ch01lvl1sec8","contentType":"text","title":"Typical workflow and steps to execute","content":"

Typical workflow and steps to execute

Before we dive deep into the major three areas, let's first take a look at the typical workflow when creating integrations with Oracle Integration Cloud Service. Since ICS is a cloud service, you only need to open a browser and enter the URL of your Cloud instance, for example: https://instancex-domainy.integration.us2.oraclecloud.com/ics.

We can sign into Oracle Integration Cloud Service by entering our credentials. Just like any Oracle Cloud Service users can be provisioned after subscribing to a service. After logging in we are welcomed by the home page:

The home page gives an overview of all major functionalities that ICS has to offer. On this page we can easily navigate to each of these functions or to the help pages to learn the details. Besides the home page, all the functions are part of the Designer Portal. We use the Designer Portal to create the five pillars of ICS: Integrations, Connections, Lookups, Packages, Agents and Adapters. We will discuss the pillars in the chapters to come, but we specifically address the agents in Chapter 11, Calling an On-Premises API and adapters in Chapter 13, Where Can I Go From Here?:

Let's investigate the most important pillars. Each integration starts with a blank canvas:

An integration always consists of a Trigger (source) and an Invoke (target). A Trigger means the connection where the integration receives the message from. An Invoke means the connection where the integration sends the message to. These two connections are the first two objectives before creating an integration.

In the following figure, both Trigger and Invoke connections use a SOAP connector. Just simply drag and drop the connection to use from the Connections panel onto the drop zone:

When integrating two applications with each other, it is likely that the data structure which the Trigger and Invoke applications understand is different. The next objective is to map the data between the two applications:

It depends on the type of connection pattern which mappings you can create. For example, when dealing with an asynchronous/one-way operation you only have a request mapping. When dealing with a synchronous operation you have both request and response mappings. The only time you can create a fault mapping is when both trigger and invoke connections define faults. For instance, in the preceding case where both WSDLs define a business fault in their specification.

For point-to-point integrations these are the objectives to reach. But if you are dealing with more complex integrations a typical workflow can consist of a few more objectives.

For instance, if the data received from the Trigger needs to be enriched (that is, locating and adding additional data based on data included in the message) before it can be sent to the Invoke. The next objective would be to add a call to an enrichment service. This enrichment service can be a different connector from your trigger or invoke:

An enrichment service can easily be added with a simple drag and drop of the connection. Another objective can be to route to a different target based on the source data:

All of these objectives are going to be discussed in detail, but first let's explore the concepts and terminology behind them.

","url":"https://subscription.packtpub.com/book/data/9781786460721/1/ch01lvl1sec8/typical-workflow-and-steps-to-execute","metaDescription":"","titleSlug":"typical-workflow-and-steps-to-execute"},{"id":"ch01lvl1sec9","contentType":"text","title":"Connections define our integration points","content":"

Connections define our integration points

It all starts with creating connections. A connection defines the application you want to integrate with. If an application has a public API then ICS can integrate with it. For example, a well-known or lesser-known SaaS application, a public SOAP or REST API for weather or flight information, a custom application using the Messaging Service, or an on-premises Enterprise Resource Planning (ERP) application.

Oracle Integration Cloud Service comes with a large set of out-of-the-box Cloud adapters, to provide easy access to these applications. The amount of available adapters is constantly growing. Most of these adapters are built by Oracle, but through the marketplace it is also possible for customers and partners to build their own adapters.

Each connection can be used for inbound and outbound communication. The majority of available adapters support both ways. A connection commonly describes the type of application, the location of the API definition or endpoint, and the credentials needed to connect securely with the application.

Connections can be divided into four categories: SaaS adapters, Technology adapters, Social adapters, and on-premises adapters:

SaaS adapters

Oracle Integration Cloud Service offers a large collection of adapters to connect to SaaS applications natively. Software as a Service (SaaS), also called on-demand software, is software that is offered as a hosted service. SaaS applications are typically accessed by users using a browser, but most offer API's to access and modify the data or to send events to the SaaS application to perform a task. For the most popular SaaS vendors, Oracle supplies Cloud adapters that can be used by Integration Cloud Service. New adapters are released on monthly cycles. The SaaS adapters can also be developed by customers, partners, and even you.

Most SaaS applications that Oracle offers as the vendor have their own adapter in Integration Cloud Service, such as the ERP, HCM, and the Sales Cloud:

Besides that, ICS supports solutions such as Salesforce, Eloqua, and NetSuite out-of-the-box.

What's the difference with native APIs?

Because the SaaS application offers their API, you might wonder why a special adapter is necessary. The adapters offer a much more simplified experience through a powerful wizard. For example, the Oracle RightNow and Salesforce adapters support the automatic provisioning of Business Objects in the wizard. These adapters also handle security and provide standard error handling capabilities.

In Chapter 4, Integrations between SaaS Applications, we will integrate applications with some of these SaaS applications.

Technology adapters

Not all applications we see on a daily basis are SaaS applications with prebuilt adapters. Industry standards such as SOAP and REST are used by the majority of APIs. SOAP is mostly used for system-to-system integrations, whereas the lightweight REST protocol is used to provide access to mobile applications. For both protocols Oracle Integration Cloud Service provides an adapter.

SOAP adapter

Originally an acronym for Simple Object Access Protocol, SOAP is an industry standard protocol originated around 2000. This specification is used in the implementation of web services and describes exchanging structured information. The SOAP protocol uses XML as the markup language for its message format. SOAP itself is not a transport protocol, but relies on application layer protocols, such as HTTP and JMS.

Web services that are built to communicate using the SOAP protocol use the Web Service Description Language (WSDL). This is an XML-based interface and describes the functionality a web service offers. The acronym WSDL also describes the physical definition file. There are two versions of the WSDL, 1.1 and 2.0, but version 1.1 is still the most commonly used.

The WSDL structure consists of five building blocks; types, messages, porttypes, bindings, and services:

The first three describe the abstract definition and are separated from the latter two that describe the concrete use, allowing the reuse in multiple transports. Where concrete means that a specific instance of a service is referenced (meaning that you have a URI).

Types are nothing more than placeholders that describe the data. An embedded or external referenced XML Schema definition is used to describe the message structure.

Messages are an abstraction of the request and/or response messages used by an operation. The information needed to perform the operation is described by the message. It typically refers to an element in the embedded or referenced XML Schema definition.

PortTypes or Interfaces define a web service, with a set of operations it can perform and direct which messages are used to perform the operation. An operation can only have a request message (one-way), a response message (call-back), or both request and response message (synchronous).

Bindings describe the first part of a concrete WSDL. It specifies the interface with its operations and binds a porttype to a specific protocol (typically SOAP over HTTP).

Services expose a set of bindings to the web-based protocols. The port or endpoint typically represent where consumers can reach the web service:

SOAP itself, also defines a small XML Envelope, which allows XML messages to be carried from one place to another without any system having to inspect the content. Compare it to sending a parcel with a courier-the courier only needs to see the information written on the box, not what is in it!

In Oracle Integration Cloud Service you can create connections based on these WSDLs. When the SOAP adapter is used in ICS you get a wizard that lets you pick one of the available operations (or select it for you if you only have one).

REST adapter

Originally an acronym for Representational State Transfer, REST is a software architectural style also introduced in 2000. It consists of a coordinated set of architectural constraints within distributed systems. The REST architectural style introduces certain architectural properties such as performance, scalability, simplicity, addressability, portability, and reliability. Because it is a style, there are some variations going around.

Web services or APIs that apply REST are called RESTful APIs. They are simply a collection of URIs, HTTP-based calls that use JavaScript Object Notation (JSON) or XML to transmit data objects, many of which will contain relational links. JSON is a human-readable text format consisting of attribute/value pairs.

RESTful APIs are usually defined with the following aspects:

The principal of addressability is covered by the URIs, which has a base URI, such as http://www.example.com/restfulapi/, for all its resources.
Each resource has its own address, also known as an URI. A resource exposes a unique piece of information that the server can provide.
For sending the data objects, an Internet media type, often JSON, is used.
The API uses standard HTTP methods, for example, GET, PUT, POST, or DELETE.
Reference stat and reference-related resources use hypertext links.

RESTful APIs use common resource naming. When deciding which resources are available within your system, name the resources as nouns as opposed to verbs or actions. They should refer to a thing instead of an action. The name and structure convey meaning to those consuming the API.

Resource examples

In this example, we use our Flight API hosted on Apiary (https://apiary.io/).

The base URI for this API is: http://icsflightapi.apiary-mock.com/

To insert (create) an airline in our flight system, we can use:

POST http://icsflightapi.apiary-mock.com/airlines

To retrieve the details of the Airline with ICAO Identifier KLM, we can use the following:

GET http://icsflightapi.apiary-mock.com/airlines/KL

The same URI would be used for PUT and DELETE, to update and delete, respectively.

What about creating a new destination an airline travels to? One option is to POST to the resource URI http://icsflightapi.apiary-mock.com/destinations, but it's arguably outside the context of an airline.

Because you want to create a destination for a flight schedule, the context should be on the schedule. It can be argued that the option to POST a message to the URI http://icsflightapi.apiary-mock.com/airlines/KL/destinations better clarifies the resource. Now you know that the destination is added to the airline.

With this in mind, there is no limit on the depth of the URIs hierarchy as long as it is in the context of the parent resource.

In Oracle Integration Cloud Service you can create connections based on the base URI. When the REST adapter is used in ICS you get a wizard that lets you create the resource that you want to expose. Only one resource can be implemented per integration.

In Chapter 2, Integrating Our First Two Applications, both SOAP(inbound) and REST(outbound) adapters are used for our first integration.

FTP adapter

Besides web service standards of SOAP and REST, there is also a technology adapter for FTP. Originally an acronym for File Transfer Protocol, FTP is a protocol used to rapidly transfer files across servers originated around 1985. The FTP adapter enables you to transfer files from a source or to a target FTP server in an integration in ICS.

With this adapter you can transfer (write) files to any server that is publicly accessible through the Internet. Files can be written in either binary or ASCII format. The adapter enables you to create integrations, which will read a file from a source FTP and write it to a target FTP server. In this scenario, the integration also supports scheduling, which enables you to define the time and frequency the transfer occurs.

The adapter supports some welcome features, such as the possibility to natively translate file content and to encrypt and decrypt outbound files using Pretty Good Privacy (PGP) cryptography. With the first feature you can translate a file with comma-separated values to XML.

The adapter not only supports plain FTP, but also FTP over SSL and secure FTP (SFTP). FTP over SSL requires the upload of a Certificate store. SFTP requires an optional host key to ensure that you connect to the correct SFTP server and secures that your connection is not compromised.

We will use the FTP adapter when managing file transfers in Chapter 9, Managed File Transfers with Scheduling.

On-premises adapters

Of course, not all of our applications run in the cloud, for most of us it is still rather new. Most of our mission critical systems run on-premises. Oracle Integration Cloud Service provides adapters and supporting software to create a Hybrid Cloud solution.

A Hybrid Cloud is a cloud computing environment, which is a combination of on-premises, private (third-party), and public cloud services. Between the platforms we usually find an Orchestration layer. For example, an enterprise has an on-premises finance system to host critical and sensitive workloads, but want to expose this system for third-party users.

Integration Cloud Service provides adapters and the supporting software to simplify integration between cloud and on-premises applications in a secure and scalable way.

The supported adapters include technology adapters, for example, Database, File, and JMS, an adapter for Oracle E-Business Suite, Oracle Siebel and SAP, and so on.

For example, with the database adapter you can call a stored procedure in your on-premises database or execute a pure SQL statement.

The File Adapter enables file transfers between two servers that cannot talk directly with each other. Java Message Service (JMS) enables integrations with existing JEE applications.

Adapters do not indicate if it is for on-premises use only, or if it can be used with an on-premises endpoint. When creating a new connection based on the adapter it will ask for an agent to assign to the connection.

ICS includes two agents; the Connectivity and the Execution agent. An agent is a piece of software running on-premises and puts a secure bridge between the Oracle Cloud and on-premises. We will shortly describe both agents, but have dedicated Chapter 11, Calling an On-Premises API to explain them in more detail.

What is the Connectivity Agent?

The Agent is basically a gateway between cloud and on-premises, and it eliminates common security and complexity issues previously associated with integrating on-premises applications from outside the firewall. The agent can connect with on-premises applications, such as the database or ERP application, using the existing JCA adapter framework. To understand this concept we first look at the agent's architecture.

Architecture Guidelines

The Agent is developed with a few architectural guidelines in mind. The most important guideline is that it shouldn't be required to open inbound ports to communicate with on-premises applications. This means that there isn't a need to create firewall rules to provide access. Because of this no open ports can be abused.

The second guideline describes that it is not required to expose a private SOAP-based web service using a (reverse) proxy, for example, API Gateway or Oracle HTTP Server (OHS). The third describes that no on-premises assets have to be installed in the DeMilitarized Zone (DMZ). The agent is installed in the local network where the backend systems are accessible.

The fourth guideline describes that it is not required to have an existing J2EE container to deploy the agent on. The fifth and last guideline describes that it is not required to have IT personnel monitor the on-premises component. With this agent the monitoring of the component is part of monitoring UI within ICS.

Architecture

The Agent consists of two components, a Cloud Agent installed on ICS and a Client Agent installed at on-premises. The Messaging Cloud is used by the Agent for its message exchange and only allows connections established from the Oracle Cloud.

It disallows explicit inbound connections for other parties. The agent uses the existing JCA adapter framework to invoke on-premises endpoints, for example, Database, file, and ERP (Siebel/SAP):

Oracle Integration Cloud Service supports multiple agents for load distribution and high availability. For example, it is possible to group multiple agents, but place each agent on a different local host/machine. Agents can be grouped on a functional, process, or organization level.

The Connectivity Agent can be downloaded from ICS and installed on demand on-premises. What you get at the end is a fully installed WebLogic server with a domain and managed server running the necessary Agent clients and JCA adapters.

Message Exchange Patterns

A Message Exchange Pattern (MEP) describes the pattern of messages required by a communication protocol for the exchange of messages between nodes. A MEP defines the sequence of messages, specifying the order, direction, and cardinality of those messages in a service call or service operation.

Two main MEPs are synchronous (request-response) and asynchronous (fire and forget).

The agent conforms to a few message exchange patterns when communicating with on-premises applications from the cloud:

Synchronous request from cloud to on-premise to retrieve data (for example, getting the status of an order from EBS in real time)
Cloud events triggers Async message exchange with on-premises (for example, creation of an incident in RightNow causes creation of service requests in EBS)
On-Premises events triggers Async message exchange with the cloud (for example, service request updates event result in Async synchronization with RightNow)
Synchronized data extracts between on-premises and Cloud applications (for example, EBS-based customer data synchronized with CRM)

What is the Execution Agent?

This Agent is a fully fledged Integration Cloud Service that you can install on-premises. When you subscribe to ICS, you also have the option to install an on-premises version in your local environment (that is, DMZ). This enables you to use ICS as a proxy server that sits between your mission critical systems protected by a firewall and the cloud version. More on this agent in Chapter 11, Calling an On-Premises API.

Noticeable differences

After installing the on-premises version of ICS you can create users and assign roles to these users. This is done in the provided Users page of the on-premises ICS. This page is not available in the Cloud version.

You also have access to the WebLogic Console, Service Console, and Fusion Middleware Control. This means that you can take a peak in the deployed applications, the Service Bus resources, and in the log files. When something goes wrong you can debug the problem without the help of Oracle Support. This is not possible in the Cloud version.

Restrictions between connectivity and execution agent

Another difference or even a restriction between the Cloud and the on-premises version is that you aren't able to configure Connectivity Agents. With this restriction in place, the adapters for connecting to the Oracle Database, MySQL Database, and Oracle SAP are not supported.

There is a default Agent group available in the Cloud version of ICS. All installed Execution Agents will be registered under this group, which restricts assigning connections to a specific Execution Agent:

We will explore both the Connectivity and Execution Agent in Chapter 11, Calling an On-Premises API.

Social and productivity adapters

Oracle Integration Cloud Service comes with a dozen of other kinds of adapters. These applications can also be categorized as SaaS applications; they're not in a traditional sense that is, enterprise applications, but offer Internet services. I'm talking about services where people can be social or collaborate with each other.

Integration Cloud Service supports social apps such as Facebook, LinkedIn, and Twitter, to post a status update. The supported productivity apps include Google, Microsoft Calendar and Mail, Google Task, MailChimp, and SurveyMonkey; and this list is updated on a monthly cycle:

If you're looking for a full reference of all the supported adapters, please take a look at the documentation which can be found at: https://docs.oracle.com/cloud/latest/intcs_gs/ICSUG/GUID-8BE53B5C-6436-4F81-AD20-78ECE5589BA9.htm.

","url":"https://subscription.packtpub.com/book/data/9781786460721/1/ch01lvl1sec9/connections-define-our-integration-points","metaDescription":"","titleSlug":"connections-define-our-integration-points"},{"id":"ch01lvl1sec10","contentType":"text","title":"Integrations","content":"

Integrations

In Oracle Integration Cloud Service, the core function is creating integrations between applications. Integrations use the created connections to connect to our applications. Integrations define how information is shared between these applications.

Integrations can be created from scratch, but can also be imported from an existing environment (that is, using an exported integration). As explained earlier, an integration always has a source and a target connection. The source and the target define the type of integration pattern that is implemented.

There are four types of integrations; point-to-point (that is, data mapping), publish messages or subscribe to messages, content-based routing, and Orchestration:

For these types of integration you can start with a blank canvas, but Oracle Integration Cloud Service provides templates to quick start your integration.

Point-to-point integration

A point-to-point integration, also known as a Basic Map Data integration in ICS, is the least complex integration pattern. We use a point-to-point integration when we have to send a message to a single receiver (that is, a 1:1 relationship). For example, a customer is updated in the HCM Cloud, which sends out an automatic message to the ERP, to also update the customer details. Another example is that a third-party channel calls an API, which returns flight data from an on-premises database.

This integration pattern is the most common pattern that advertises Oracle Integration Cloud Service. You create an integration and use your defined connections for the Source and Target. You also define data-mappings between the Source and Target and vice versa.

As mentioned previously, the source and target operation that is implemented, determines the kind of integration capabilities. A one-way source can only send data to a one-way target, and a synchronous source can only send data to a synchronous target. This integration does not support mixing one-way with synchronous operations:

Although this pattern supports a more complex integration flow, Integration Cloud Service provides the possibility to enrich the data, by calling another service between the source and the target or vice versa. For example, the target requires more data for its input than the source provides.

By using enrichment services you are able to invoke a service operation that returns the required data. The data from the source and the data from the enrichment service can then be used for the request mapping to the target:

In a one-way integration the source data can be enriched by calling an additional service:

In a synchronous integration both sources as target data can be enriched by calling additional services.

Publish-subscribe integration

A publish-subscribe integration, in ICS separated in two separate integrations, implements a messaging pattern where messages are not directly sent to a specific receiver. The senders of messages are called publishers and the receivers of messages are called subscribers.

Topology

Publishers send messages, which are then published to an intermediary message broker or event bus without knowledge of which subscribers, if any, are interested in the message. On the other hand, subscribers register subscriptions on messages they are interested in, without knowledge of which publishers, if any, are sending these messages. The event bus performs a store and forward to route messages from publishers to subscribers:

This pattern generally uses the message queue paradigm and provides greater scalability and a more dynamic network topology. Most messaging systems support this pattern in their API using Java Message Service (JMS).

The publish-subscribe pattern in ICS can be implemented using its own messaging service, by using the Oracle Messaging Cloud Service (OMCS) adapter or the JMS adapter for on-premises connectivity. All implementations use JMS as its underlying protocol.

The ICS Messaging Service also uses the OMCS for its delivery, but these messages can't be accessed outside of ICS.

Advantages of pub-sub

The two major advantages of using this kind of pattern are scalability and loose coupling.

Scalability

This integration pattern provides the opportunity for better scalability of the message flow. Messages can be handled in parallel instead of being processed one after each other. If the amount of messages succeeds the amount the systems can process, a new system or a subscriber can be added. A subscriber can also choose to temporarily stop receiving messages to process already received messages.

Loosely coupled

When using the publish-subscribe pattern, publishers and subscribers are loosely coupled and generally don't know of each other's existence. Both can operate normally regardless of the state of the other. In a point-to-point integration, the systems are tightly coupled and can't process messages if one of the services is not running. When using ICS it is possible to even decouple the location of the publishers and subscribers by using OMCS or the on-premises JMS adapter.

Disadvantages of pub-sub

The advantage of loosely coupling also introduces side effects, which can lead to serious problems. We should also consider issues that can occur during the delivery of messages.

Inflexibility of decoupling

Because the publishers and subscribers do not need to know of each other's existence, it is important that the data is well defined. This can lead to inflexibility of the integration, because in order to change the data structure you need to notify the subscribers. This can make it difficult for the publishers to refactor their code. In ICS this should not be a problem because it can be fixed by versioning the integration and its data structure.

Issues with message delivery

Because a pub-sub system is decoupled, it should be carefully designed to assure message delivery. If it is important for the publisher to know a message is delivered to the subscriber, the receiving system can send a confirmation back to the publisher. If, for example, a message can not be delivered, it is important to log the error and notify someone to resend the failed messages. When designing an integration, keep in mind that delivery management is important to implement.

Content-based routing

An integration with content-based routing is a more advanced version of the point-to-point integration. Content-based routing essentially means that the message is routed based on a value in the message payload, rather than an explicitly specified destination:

A use case for this type of routing is the possibility to retrieve data from a different application based on the country code.

Topology

Typically, a point-to-point integration receives a message and sends the message to an endpoint. This endpoint identifies the service or client that will eventually process the message. However, what if the data is not available (anymore) at that destination? For example, what if the data specific to the callers country is available at a different site and the service cannot return the requested data? Is it also possible that one destination can only process specific types of messages and no longer support the original functions.

A solution for this is content-based routing. A content-based routing is built on two components: services and routers. Services are the consumers of the messages, and like all the other integrations they decide on which messages they are interested in.

Routers, usually one per integration, route messages. When a message is received the message is inspected and a set of rules are applied to determine the destination interested in the message content. With this pattern we can provide a high degree of flexibility and can easily adapt to changes, for example, adding a destination. The sender also does not need to know everything about where the message is going to end up.

Each destination can have its own implementation, which means the router also needs to transform the message where necessary.

Simple example architecture

The following diagram illustrates a simple example of the content-based routing architecture. It shows how a message is sent to the endpoint of Service A. Service A receives the message and the Router routes the message to Service B or Service C, based on the content of the message:

With Integration Cloud Service it is as easy as adding a filter on the request operation of the source connection. We can define multiple rules in an if-then-else construction and in every rule we can filter on multiple fields within the request message.

Advantages of content-based routing

The two major advantages for using this kind of pattern are efficiency and sophisticated decisions.

Very efficient

Content-based routing is very efficient. This is because the decision to route the message to one consumer or the other is kept away from the provider. The decision is based on the provider's request. There is also no risk of more systems than necessary consuming the message (when compared to the pub/sub integration), because we route to dedicated consumers.

Sophisticated routing decisions

When designing a content-based router it can become highly sophisticated. We can have multiple routing rules where one rule can filter on multiple fields of the request message. Content-based routing is easier to incorporate into a process pipeline more often than not.

Disadvantages of content-based routing

The disadvantages arise when the number of consumers grows.

Additional consumers

When we introduce an additional consumer it also means changing the router (compared to a pub-sub integration, which requires no change). We also need to add an extra routing decision.

Sequential processing

Content-based routing runs in a sequential process compared to Orchestration where we can execute in a parallel framework. Every rule is evaluated one after the other, which increases the execution runtime.

Orchestration for complex integrations

The usage of Orchestration comes into the picture when we discuss integrations in the context of service-oriented architecture. Service Orchestration is usually an automated process where two or more applications and/or services are integrated.

Topology

We have discussed point-to-point integrations and have seen that in many use cases this pattern fulfills our requirements. However, point-to-point integrations can lead to complex dependencies, if we need to integrate multiple application and/or services. The downside of this is that it is hard to manage, maintain, and possibly monitor.

An integration that implements Orchestration provides an approach that aggregates services into application processes. Orchestration has the capabilities for message routing, transformation, keeping the process state for reliability, and security through policies. The most important capability of Orchestration is centralized management, for example, resources and monitoring.

Orchestration, in a sense, is a controller, which deals with the coordination of (a)synchronous interactions and controls the flow. Usually Business Process Execution Language (BPEL) is used to write the code that is executed. This is also the case with Integration Cloud Service; however, a visual representation is used to define the process.

Practical example

To get a practical sense of service Orchestration, let us take a look at an example of a mortgage application. A mortgage broker wants to request a mortgage on behalf of a customer. The application uses API that calls an HTTP endpoint, which sends the initial request to the orchestrator through Service A.

The orchestrator enriches the customer data by calling Service B. Based on the age of the customer, Service C or Service D is called to find special mortgage quotes, but Service E is called for everyone. Service E retrieves credit scores belonging to that customer. The data from both service calls are merged and sent to Service F to return the best quote for that customer:

Advantages of Orchestration

The two major advantages of using this kind of pattern are loose coupling and enabling automation.

Loosely coupled

Application and business services can be designed to be process-agnostic and reusable. The business process is responsible for the management and coordination of state, which frees composite services from a number of design constraints. Additionally, the logic of the business process is centralized in one location, instead of being distributed across and embedded within multiple services.

Enables automation

In day-to-day operations, Orchestration enables us to automate processes such as insurance claims that come from medical offices. This is one process where human approval can be removed by programming parameters to accept claims. Orchestration can also automate error handling; for example, a message can be retried automatically when an endpoint is unavailable or a notification can be sent to a human when it needs to be recovered manually.

Disadvantages of Orchestration

There is one major disadvantage and that is debugging. There can be complex processes, which have multiple service interactions, nested loops, and so on. In these situations, it's not easy to debug your process if something goes wrong, because we need to know what message initiated the flow. We will touch on this topic in Chapter 10, Advanced Orchestration with Branching and Asynchronous Flows.

","url":"https://subscription.packtpub.com/book/data/9781786460721/1/ch01lvl1sec10/integrations","metaDescription":"","titleSlug":"integrations"},{"id":"ch01lvl1sec11","contentType":"text","title":"Transformation and lookups","content":"

Transformation and lookups

When talking about integrations between application and/or services we can't escape the fact that messages need to be transformed. Most of the time the applications and/or services do not talk the same language (that is, message structure or even data types, for example, milliseconds from epoch versus date time). Besides transforming the structure, we sometimes need to convert values (that is, domain value mapping).

Transformations

Oracle Integration Cloud Service uses XML as its message format for its data objects and messages. To transform XML-based messages from one structure to another there are two main open standards that can be used to manipulate data; XQuery (more info at https://www.w3.org/TR/xquery/) and XSLT (https://www.w3.org/TR/xslt).

XQuery

XQuery is like the name suggests, a query language, but besides that it is also a functional programming language. It queries and transforms collections of data, both structured as unstructured. It can transform messages to XML, but also text and other data formats, for example, JSON and HTML.

Using XQuery we can extract and manipulate XML documents or any source that can be viewed in XML, such as office documents or database schemas. It has built-in support for XPath expressions. With XPath we can address specific nodes within the XML document. XQuery is a SQL-like query language that supplements XPath and uses FLWOR expressions for performing joins. A FLWOR expression is named after the five parts it is constructed of: FOR, LET, WHERE, ORDER BY, and RETURN.

XQuery has the capability to transform, and it also allows us to create new XML documents. Where normally the elements and attributes are known in advance, it can use expressions to construct dynamic nodes, including conditional expressions, list expressions, quantified expressions, and so on. A simple example of XQuery is shown as follows:

<html><body> 
  LET $book := doc("bookstore.xml")/book 
  FOR $ch in $book/chapter 
  WHERE $book/chapter/num < 10 
  ORDER BY $ch/pagecount DESC 
  RETURN <h2>{ $ch/title }</h2> 
</body></html>

XSLT

Originally an acronym for Extensible Stylesheet Language Transformations, XSLT is as the name suggests, a language for transforming XML documents and is basically a style sheet to transform XML documents into other XML documents, or other data formats such as (X)HTML, XSL Formatting Objects (such as generating PDFs), RSS, and non XML (such as CSV):

XSLT processing takes an XML source document and a XSLT style sheet, and produces a new output document. The XSLT style sheet contains static XML that is used to construct the output structure. It uses XPath expressions, just like XQuery, to get data from the XML source. The XSLT language includes logic (if-else) and operators (for-each) to process the XML source.

An XSLT style sheet is an XML document containing the root node <xsl:stylesheet>, which is declared by the xsl prefix and is mandatory. An XSLT style sheet contains one or more <xsl:template> elements and other XML elements defining transformation rules:

<?xml version="1.0" encoding="UTF-8"?> 
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"> 
  <xsl:template match="/"> 
    <html> 
      <body> 
        <h2>Book Collection</h2> 
        <table border="1"> 
          <tr> 
            <th>Title</th><th>Author</th> 
          </tr> 
          <xsl:for-each select="bookstore/book"> 
            <tr> 
              <td><xsl:value-of select="title"/></td> 
              <td><xsl:value-of select="author"/></td> 
            </tr> 
          </xsl:for-each> 
        </table> 
      </body> 
    </html> 
  </xsl:template> 
</xsl:stylesheet>

Integration Cloud Service currently only supports XSLT, because it is a transformation/mapping language, which can easily be made visually. Let's discuss XSLT in more detail.

XPath expressions

XSLT, like XQuery, uses XPath expressions to address specific nodes within the source document and perform a calculation. XSLT can use a range of functions that XPath provides to further augment itself.

Originally an acronym for XML Path Language, XPath is used for selecting nodes from an XML document. It models XML as a tree of nodes. XPath is named after its use of a path notation for navigating through the hierarchical structure of an XML document. XPath uses a compact, non-XML syntax to form expressions for use in Uniform Resource Identifier (URI) and XML attribute values.

XPath models an XML document as a tree of nodes. The following figure depicts the different types of nodes, as seen by XPath, in an XML document:

The elements in the preceding diagram are:

A document root, which is a virtual document root that is the parent of the entire XML document containing the XML declaration, the root element and its children, comments, and processing instructions at the beginning and the end of the document.
A root element (for example, the <bookstores> element).
Element nodes (for example, the <bookstore>, <store_name>, and <store_url> elements). The root element is also an element node.
Attribute nodes (for example, the num="1" node).
Text nodes (for example, the nodes containing the text, Packt Publishing).

In the XPath model, the document root is not the same as the root element node. XPath defines a way to compute a string value for each type of node, some of which have names. Because XPath fully supports the XML Namespaces Recommendation, the node name is modeled as a pair known as the expanded name, which consists of a local part and a namespace URL, which may be null.

XPath allows various kinds of expressions to be nested with full generality (that is, using wildcards). It is a strongly typed language, such that the operands of various expressions, operators, and functions must conform to designated types. XPath is a case-sensitive expression language.

XPath Examples

The following are some examples of XPath expressions:

Get the store_name node of the bookstore identified with number 1:

/bookstores/bookstore[@num="1"]/store_name

Get the text value of the store_name node of the last bookstore:

/bookstores/bookstore[last()]/store_name/text()

Check if a bookstore exists in the source document (prefixing // means a wildcard search):

count(//bookstore[store_name="Packt Publishing"]) = 1

XLST constructs

With XSLT it is possible to create really dynamic output documents using XSLT constructs together with XPath expressions. XLST construct are specific elements that are defined by the XSLT standard. XSLT defines the following construct elements: output, template, value-of, for-each, if, and choose.

The xsl:output element

The output element specifies the output format of the result tree and it must be a direct child of the style sheet root element. The element has a few attributes. The method attribute defines which processor needs to be used, such as xml, html, or text, and the media-type attribute defines the target type, for example, application/json:

<xsl:output method="html" media-type="text/xml"/>

The xsl:template element

The template element defines a template rule that matches a specific node in the source document using an XPath pattern value. The output expression contains the formatting instructions to produce a result tree:

<xsl:template match="pattern"> 
  output-expressions 
</xsl:template>

The match pattern determines the context node within the template. The most common match is on the root element of the source document tree:

<xsl:template match="/"> 
  A simple text string 
</xsl:template>

A template can also be given a name and called by its name within another template by passing a parameter:

<xsl:template name="minValue"> 
  <xsl:param name="values"/> 
  ... 
</xsl:template> 
<xsl:template match="/"> 
  <xsl:call-template name="minValue"> 
    <xsl:with-param name="values" 
                    select="bookstore/book/year"/> 
  </xsl:call-template> 
</xsl:template>

The xsl:value-of element

The value-of element is used to insert the text value of the expression. This element defines a select attribute, which contains an expression and is used inside a template element:

<xsl:template match="name"> 
  Name: 
  <xsl:value-of select="name"/> 
</xsl:template>

The xsl:for-each element

The for-each element is used to loop over node-sets. This element defines a select attribute, which instructs the XSLT processor to loop over the node set returned by the given XPath expression:

<xsl:template match="/"> 
  <xsl:for-each select="catalog/book"> 
    <p><xsl:value-of select="title"/></p> 
  </xsl:for-each> 
</xsl:template>

Inside the for-each element, the current node in the set is the context. The position() function returns the index in the loop, that is, the iteration counter. It is also possible to sort the order in which the nodes are looped over using the xsl:sort element. It defines a select attribute that contains an expression whose value is sorted on:

<xsl:for-each select="catalog/book">
<xsl:sort select="year" data-type="number" order="descending"/>
  <p>
    <xsl:value-of select="title"/>
  </p>
</xsl:for-each>

To apply multiple criteria, we can use several xsl:sort elements after each other.

The xsl:if element

Sometimes a section of the XSLT tree should only be processed under certain conditions. With the if element we can build that conditionally. The if element defines a test attribute, which contains an XPath expression that should evaluate to a Boolean value:

<xsl:for-each select="catalog/book"> 
  <xsl:if test="year = 2016" > 
    <p><xsl:value-of select="title"/></p> 
  </xsl:if> 
</xsl:for-each>

There is no else statement, but for this XSLT defines the xsl:choose element.

The xsl:choose element

XSLT supports multiple mutually exclusive branches with the choose element. The xsl:choose element contains xsl:when tags that define parallel branches, one of which is executed. The xsl:otherwise tag can be used to define a default branch.

We use the choose element for if-the-else constructions:

<xsl:for-each select="catalog/book"> 
  <xsl:choose> 
    <xsl:when test="year = 2016" > 
      <xsl:text>New Arrivals</xsl:text> 
    </xsl:when> 
    <xsl:when test="year < 2015" > 
      <xsl:text>Our Classics</xsl:text> 
    </xsl:when> 
    <xsl:otherwise> 
      <xsl:text>Top Picks</xsl:text> 
    </xsl:otherwise> 
  </xsl:choose> 
</xsl:for-each>

When implementing integrations with ICS we will use XSLT for creating mappings between source and target connections. The major advantage of ICS is that we can build these mappings with a visual editor.

Lookups

A lookup, also known as Domain Value Maps (DVM), associates values used by one application for a specific field to the values used by other applications for the same field.

They enable us to map from one vocabulary used in a given domain to another vocabulary used in a different domain. For example, one domain may represent a country with a long name (Netherlands), while another domain may represent a country with a short name (NL). In such cases, you can directly map the values by using domain value maps.

Simple example:

CountryCode	CountryName
NL	Netherlands
UK	United Kingdom
USA	United States

A lookup also supports qualifiers. A mapping may not be valid unless qualified with additional information. For example, a DVM containing a code-to-name mapping may have multiple mappings from ABE to Aberdeen because Aberdeen is a city in both UK and the USA. Therefore, this mapping requires a qualifier (UK or USA) to qualify when the mapping becomes valid.

Qualifier example:

Country (qualifier)	CityCode	CityName
United Kingdom	ABE	Aberdeen
United States	ABE	Aberdeen
Netherlands	AMS	Amsterdam

Qualifier order support

We can also specify multiple qualifiers for a lookup using a qualifier order. Using this, ICS can find the best match during lookup at runtime. Hierarchical lookups are supported. If you specify a qualifier value during a lookup and no exact match is found, then the lookup mechanism tries to find the next match using the following qualifier. It proceeds until a match is found, or until no match is found even with all qualifiers set to an empty value.

One-to-Many Mapping Support

We can map one value to multiple values in a domain value map. For example, DVM for subscription payments can contain mapping to three values such as discount percentage, discount period, and subscription period.

","url":"https://subscription.packtpub.com/book/data/9781786460721/1/ch01lvl1sec11/transformation-and-lookups","metaDescription":"","titleSlug":"transformation-and-lookups"},{"id":"ch01lvl1sec12","contentType":"text","title":"Summary","content":"

Summary

In this chapter, we addressed the concepts and terminology surrounding Oracle Integration Cloud Service and standards such as. XML, XSLT and XQuery used by ICS. With ICS we can create integration between cloud services and between cloud and on-premises applications. An integration consists of one trigger and one invoke called connections, and it can call multiple enrichment services. Between the trigger, enrichment services, and invoke, ICS uses XSLT mappings to transform the message structure.

We looked at the ideas and terminology around how ICS connects to the applications it can integrate with. ICS comes with a large set of out-of-the-box Cloud adapters to connect to these applications, and in upcoming chapters we will explore these connections in depth.

Integrations use the created connections to connect to our applications. Integrations define how information is shared between these applications, for example, exposed operation, message structure, and so on. We discussed the four types of integrations ICS supports and its advantages and disadvantages.

When integrating applications and/or services we can't escape the fact that messages need to be transformed, because they don't talk the same language (that is, message structure or even data types, for example, milliseconds from epoch versus date time). ICS uses the open standard XSLT for manipulating data. We discussed the language and its structure. Besides transforming the data we sometimes need to convert values (that is, domain value mapping). ICS supports lookups that we can use to convert a value provided by the source to a format the target understands.

In the next chapter, we will explain the steps to create a basic integration between two cloud services based on an SOAP and a REST interface.

","url":"https://subscription.packtpub.com/book/data/9781786460721/1/ch01lvl1sec12/summary","metaDescription":"","titleSlug":"summary"}]

Implementing Oracle Integration Cloud Service

Implementing Oracle Integration Cloud Service

Overview of this book

Related Content you might be interested in

Current Title:

Implementing Oracle Integration Cloud Service

Implementing Oracle API Platform Cloud Service

WSO2 Developer's Guide

Credits