Introduction:

With integrations representing an increasingly more critical part of enterprise architecture, and also a more likely cause of service disruption, there is a need to lift this part of the architecture documentation to the forefront. This model focuses on creating documentation that is easy to maintain via Ardoq surveys and that will provide answers to the most important business-critical questions. (This model does not on the other hand cover "data flow path mapping".)

The model can be expanded to cover more integration attributes, as well as dependencies between integrations and third parties (Who built or is responsible for maintaining the integration?), processes (which process are dependent on this integration?), integration software (e.g. MuleSoft or IBM Integration Bus) or infrastructure (Where is this integration deployed?) to name a few.

This model is easily instantiated through a single survey that both catalogs integrations with their own set of attributes and maps dependencies to applications, databases, data objects and data flows.

You should consider whether your integration catalog should reside in the same workspace as the application catalog or whether it deserves a workspace of its own. If your integration- and application catalogs consist largely of microservices, it may be difficult to tell them apart and therefore easiest to bunch them together. However, it might nevertheless be different teams owning the two, in which case mapping applications and integrations in two separate workspaces might be preferable after all in terms of enforcing ownership and permissions.

Questions to be answered:

  1. What are my integrations? Where in the life cycle are they? Who is the technical owner for each of them? Which Data Flow are they each a part of?
  2. What are my integration properties?
  3. I plan to replace system "X". Which integrations will need to be updated? Which applications might be affected down the line?
  4. What does a given data flow look like end-to-end (visual representation)?

Model:

Component types:

  • Integration
  • Application
  • Database
  • Data Object
  • Data Flow

Reference types:

  • Data Flow (Application/Database/Integration -> Integration & Integration -> Application/Database/Integration)
  • Data Input (Data Object -> Integration)
  • Data Output (Integration -> Data Object)
  • Integration Enables Data Flow (Integration -> Data Flow)

Field types:

  • Applicable to the reference types Integration Source and Integration Target:
    Integration Content Format
    Integration Frequency
    Integration Protocol
    Integration Mode
  • Applicable to the component type Integration:
    Life Cycle Status
    Technical Owner

Pro-tip:

  • If you don't have your master data mapped out, you can reference your integrations to the level of abstraction that suits you. The model works equally well, irrespective of whether you've referenced your integrations to data objects, information objects or even subsets of data relevant to each data flow.

Answering the questions:

What are my integrations? Where in the life cycle are they? Who is the technical owner for each of them? Which Data Flow are they each a part of?

An employee has left the company. Which integrations was she/he responsible for? An integration is failing. Who will fix it?

There are a couple of advantages of having integrations modelled as components in a workspace of their own.

The first is that your integrations become first-class citizens. This also allows you to pull a complete list of all your integrations simply by opening the workspace without applying any filters and selecting the Table-view. The Table-view will also display all of your data sources, data targets and data objects in a single grid, in addition to the life cycle status and technical owner.

What are my key integration properties?

The second advantage of having integrations modelled as components is that it allows you to capture when the integration is connected to more than two applications, as the integration component can have multiple incoming and outgoing references.

If an integration pulls data from two different sources, the way in which this happens might not be the same for both sources. For instance, the content format might be .json for one source and .xml for the other. The integration protocol might be SOAP for one and REST for the other. The frequency might be hourly for one and daily for the other. Therefore these types of properties must be captured on the references between the integration-component and the integration sources and integration targets.

Set your context to the integration of interest and select the Pages-view for a one-stop shop with all the properties of all the dependencies of that integration.

I plan to replace system "X". Which integrations will need to be updated? Which applications might be affected down the line?

By selecting a view that displays several degrees of dependencies, such as the Block Diagram view, you can see more than just direct dependencies, both when looking “up” or “down” the data flow path.

Set your context to the application of interest and select the Block Diagram view.

Be careful to filter away reference- and component types that are not relevant. You can do this in the Quick Filters in the bottom left corner. 

Also, be sure to adjust the degrees of relationship-settings so that indirect dependencies also are shown.

What does a given data path look like end-to-end (visual representation)?

In this model we suggest modelling data paths as components in their own right, much in the same way as you might model a process as a single component. This high level abstraction is beneficial in more than one way.

Firstly, it serves to build a catalog of all your data paths. If relevant, you can also expand the model to include fields on the data path-component, for instance to attribute life cycle status to data paths.

Secondly and most importantly, because all the integrations that constitute a data path are connected to its component, you can easily use these references to exclude everything not relevant to that data path. You'll do this by creating an advanced filter in the Manage Perspectives-window.

Such a filter as the one illustrated above will exclude all integrations that don't belong to that specific data flow, while including applications and databases, irrespective of whether these also feature in other data flows:

Data Entry

All the information in this model can be collected and updated via a single Ardoq survey, including the data captured in the fields on the references between the integration component and the application- or database components.

Pro-tips:

  • Naming convention: Make it easy for your contributors and avoid duplication by defining an intuitive naming convention for your integrations, something like
    "<source>-<target>".
  • Use the Result Filtering-feature in the Survey Builder to allow respondents to filter the integration catalog by data flow.

Logical and physical integrations:

In this model, all integrations are abstracted into individual components, including integrations which one might have visualized as an arrow, e.g. integrations that connect two APIs directly to each other.

As long as the model is instantiated strictly via a survey such as the one above (one that creates and updates integration components), this will happen automatically. 

However, if adding data in the core app, it is important that a reference of type Data Flow always has a component of type Integration either as source or as target. Or else the above-mentioned filter that allows the visualization of a single data path will not work as intended.

Governance

Build a dashboard to track your progression and the quality of your data.

Expanding the scope:

Additional component types:

  • Middleware
  • Integration Platform

Additional reference types:

  • Uses Data (Application -> Data Object & Integration -> Data Object)
  • Stores Data (Database -> Data Object)
  • Accesses (Application -> Database & Integration -> Database)
  • Contains integration (Middleware -> Integration)
  • Platform (Integration -> Integration Platform)

Additional fields:

  • On references of type “Uses Data” (Application -> Data Object & Integration -> Data Object) and “Accesses”(Application -> Database & Integration -> Database): CRUD (Create/Read/Update/Delete) or similar
  • On references of type “Uses Data” (Application -> Data Object & Integration -> Data Object): “Master”/”Owns”.
  • On components of type “Integration”: Criticality
Did this answer your question?