Purpose and Scope

The purpose of Application Integration Management is to model application integrations and infrastructure with just enough detail to support Business - IT Transparency & Governance as well as driving significant change.

Support Business - IT Transparency & Governance

  • Depict interconnections between applications to show the complexity in the IT landscape and how capabilities are realized.

  • Provide a foundation for data lineage showing information types that are passed through particular Interfaces, including where the data originates, how it moved from point A to point B, and where it exists today.

  • IT governance and compliance often require understanding which integration types and protocols are used; and the ability to analyze protocol types, data types, integration volumes, and maintenance costs of those integrations.

Drive Significant Change

  • Interconnections between applications show dependencies - in terms of data and Interfaces - these may need adjustment when an application is changed.

  • Show who is responsible for building integration infrastructure in addition to the applications themselves. Because many organizations have considerable development expenses related to integration infrastructure that connects diverse applications.

Supporting business and driving change define the scope of the Application Integration Management use case. There can be many other goals for integration documentation - such as detailed design information about the integration patterns being used. However, that level of detail is beyond the scope of this use case guide.

What the AIM Use Case Guide Includes

  • A foundation for data lineage but does not include a full data lineage or data governance guide.

  • The current guide will show how information entities can be referenced from particular Application Interfaces.

However, it will not include

  • Detail how information models themselves should be structured in Ardoq. This is the subject of a future use case guide.

Integration Modeling Approaches

Ardoq’s recommends the approach of modeling integrations as an explicit connection to an Application Interface. Here we will detail the different approach’s rationale and structure while comparing it with other approaches.

Ardoq modeling integrations

Recommended

Ardoq Integration Approach

Integration as an explicit connection to an Application Interface

  • Represent Integration concepts with the richest and most extendable source of information and ability to connect them to other components such as Teams and Data.

  • Provides full insight into Integration and traceability up to Business consequences.

More Simplistic Approaches

Integration as an abstract component type

  • Represent Integration concepts with richer information and connect them to other components such as Teams and Data.

  • Provides partial insight into Integrations and Business Consequences.

Integration as a reference between Applications

  • Represent simple dependencies between Applications and simple dataflows between them.

  • Supports simple impact analysis with limited insight into the Business consequences of Integrations.

Integration as an Explicit Connection to an Application Interface

Our recommended approach is to model integrations as explicit references to Interfaces offered by an Application or Application Integration Infrastructure.

What this Means Technically

In this approach, a child component - an Interface - is created for an application. This Interface component can then be used to connect other component types such as data provided by the Interface and the development team responsible for it.

Why we Recommend this Approach

We recommend this approach over more limited approaches following our Simplicity modeling principle. The meta-model should be as simple as possible. When modeling at the application level it is most useful to model the details of how integrations are implemented - through connections to Interfaces rather than abstract concepts that hide those details.

Example of Integration Connections

The model view below shows that

  • Informatica Power Center provides the Interface Product D Service Logistics.

  • Interface is consumed by Legacy Product D Service Application.

Ardoq integration connections

These concepts represent both integrations between Applications and Integration infrastructure that often exists between those Applications - for example messaging infrastructure or integration middleware. There are a multitude of different types of middleware and integration techniques. Following our metamodeling principle to limit the number of metamodel types, our approach is to represent all connection points as an Interface.

Example of Interface Model

Ardoq interface model

For instance, in the above example the Kafka messaging infrastructure provides topics that are also modeled as an Interface.

Using the same component type makes it easier to create reusable Gremlin queries and filters. Additional information such as the protocol being used (e.g., ftp, jdbc, rest/http) is captured as a field on the Interface rather than a separate component type.

Modeling

Our approach is to promote models that have enough fidelity to be useful without becoming exceedingly complex. One consequence of this modeling principle is that not all integration situations fit perfectly into the model.

For example, in our recommendation FTP Integrations are modeled as interfaces on an application. This is not precise because FTP connections are to servers rather than applications. But we choose to do it this way to maintain the usefulness of a common approach for visualizations and queries.

Ardoq ftp integrations

The consequence of this approach is that models may contain a level of detail useful in some situations but that is unnecessary for other stakeholders. For example, people may be interested in seeing connections between Applications but are not interested in Interfaces. Our recommendation is to always model at the simplest level of detail possible for the targeted information. You can easily hide these levels of detail when you are providing information for a different set of stakeholders. For instance, the Interfaces can be hidden as shown in the following model view.

Ardoq hide interfaces

Interface Child Components

Users can choose if they want to represent all Interfaces offered by an application as multiple, separate Interface child components or as a single Interface component to represent all integration points.

Using a single Interface child component will improve maintainability but limit the amount of detail and the ability to identify different data types that are available on each connection.

Ardoq interface child component

Ardoq active directory

Connects to References

Integration references between components are done using the Connects To reference type, the source of the reference represents the component that initiates the integration. Connection types in software and enterprise architecture modeling notations usually represent either invocations or dataflows. Our approach focuses on representing invocations. This is to be consistent with abstraction types in our modeling. Integrations between applications are invocations that pass data types through Interfaces.

It can also be useful to reason about dataflows between applications or between process steps that are realized using applications. To support this situation we have modeled a field attribute on the reference to represent whether the integration is to read, write, or read+write data. This allows you to model both invocations and dataflows. This is depicted in the model above that shows integrations between Sales applications and Kafka.

Our model also connects an Application Interface to a Data Entity that it provides - as shown above. The name of that Data Entity is also copied, using a calculated field, to the reference so that Integration models can be simplified.

For instance, in the model below

  • The connection between applications and Active Directory Federation Services passes Identity information

  • This can be seen on the reference rather than explicitly showing the Data Entity component connected to the Application Interface.

Ardoq active directory

Scope

The scope of Integration references is also limited to invocations from the consumer to the provider of the Interface. We have decided not to include information about detailed integration solution design in the current scope. For instance, we have not modeled support for invocation patterns (e.g., request-response, event, request-only, synch v asynch, etc). Similarly, we have not modeled support for data versus command message - or other types of detail that might represent different integration patterns. Ardoq users can extend the solution to include those as needed but we have not included them in the initial scope.

More Simplistic Approaches to Integration Modeling

Integration as a Reference between Applications

The most simplistic approach to modeling Integrations is to provide a reference between Applications.

Ardoq ServiceNow

Limitations of this approach

The Integration as a reference approach is not detailed enough to meet the purposes explained at the beginning - for example it cannot support links to other component types such as Data Entities and maintenance teams. However, it may be enough for situations where you just want to show simple dependencies between applications.

The purposes listed above identify multiple types of information that need to be associated with integrations, from maintenance teams to protocols and data types. This requires that integration information is captured as a specific component type and is not only a reference between two applications.

Integration as an Abstract Component Type

This approach represents an integration as an explicit, separate component type. The approach is explained in detail in an existing knowledge-base article on the subject.

Ardoq abstract component type

What this means technically

Representing integrations in this way as a separate component type allows you to capture more information than the “Integration as a reference” approach and achieve more of the earlier mentioned outcomes because you can connect the integration to other component types such as Data Entities

Limitations of this approach

This approach represents integrations as an abstraction hiding some of the important implementation details.

For example, often it is important to document the integration infrastructure and the license and maintenance costs associated with it. While people often talk about the team responsible for “maintaining a particular integration,” they usually mean the team responsible for maintaining the integration middleware solutions that provide those integrations. It is difficult to model and provide insight into integration infrastructure when these details are hidden in an abstract Integration component type.

Integration as an abstract component is often premised on the understanding that integrations are implemented using middleware between applications. This approach to integration was prevalent in the industry for many years (decades) and is still evident in many IT landscapes. However, individual applications and microservices are increasingly responsible for providing their own sets of rich APIs rather than relying on integration middleware for complicated data and protocol mapping. Detailed integration information based on service-APIs provided by Applications is difficult to represent in this approach.

Summary

This article has detailed Ardoq’s recommended approach to integration modeling and our reasoning behind that approach. It also lists other approaches and their limitations.


CHANGELOG

A list of changes made to the article are listed below:

November 23rd, 2021

  • First version of the article was published.

Did this answer your question?