Content

“Most organizations don’t know how to measure their tech debt, much less how to actively manage it. As unintuitive as it may appear, measuring and managing tech debt can create a competitive advantage for those who do it right.” [1]

This Solution enables you to document and manage the technical debt across your IT estate. It assumes that your focus is on debt associated with your application portfolio, and as a minimum, you must have implemented the Application Lifecycle Management Solution, or have a portfolio of Application components against which to record technical debt. You may also record debt items against underlying technologies and services that support your applications, such as specific components of your applications, databases, servers or other platform services, if you have modeled your IT estate to this level of detail.

If you have documented your business capabilities with the Business Capability Modeling and Realization Solutions, you can map your technical debt against them to gain a better understanding of the business implications of existing technical debt.

A simple survey enables you to capture and categorize individual Technical Debt Items. This survey can be distributed to stakeholders, such as application owners, via Discover. The Solution includes visualizations to help you understand the costs and risks associated with your technical debt portfolio, explore the potential costs and benefits of remediation, and prioritize the alternatives based on available budget and resources.

If you’re using the Insights & Impacts part of the Strategy to Execution Solution, you can identify the Initiative or Initiatives that are underway to remediate debt, and use Ardoq’s automated process to help you follow up remediation activities, prompting updates when actions fall due for completion. If you’ve implemented the Application Risk Management Solution, you can also link identified technical debt to its associated risks, allowing you to update your risk register as your application teams uncover and record known technical debt.

For a primer on technical debt, and why it is important, see An Introduction to Technical Debt.

By identifying technical debt, describing and quantifying it, and linking it to your Ardoq knowledge graph, you are able to make key decisions concerning which items of technical debt should be prioritized, and make the case for the support, sponsorship and initiation of initiatives to address those that are selected for action.

Making debt visible, and understanding its impact on an organization is a key first step to addressing it. This Solution involves creating Debt Item components, each representing a distinct item of technical debt. Estimates are given for the cost of living with the debt, the cost of remediation, and the impact and likelihood of a serious incident associated with the debt. Debt Items are linked to one or more assets that are impacted (such as Applications or Technology Services), and associated with the Quality Characteristic that is affected. This allows technical debt to be analyzed from different perspectives, addressing the following questions:

This Solution encourages you to identify, record, and quantify technical debt. It gives you the information to prioritize which debt items need immediate action, which should be planned now, and which can be either delayed until later or ignored. Armed with this information, you can articulate the business impact of your technical debt and the consequences of leaving it unremediated. When you have the budget, the Solution will help you start the initiatives to address the debt you have prioritized and track their progress.

Visualizations are provided to communicate awareness of the size and impact of technical debt across your IT estate. These can be explored from different perspectives:

Connection to components that are used in other Solutions allows further technical debt perspectives to be explored:

The Technical Debt Management Solution facilitates the documentation, classification and quantification of technical debt items once they have been identified, so that they can be communicated, their implications can be understood, and efforts to remove them can be prioritized. In effect, the management of technical debt. At the heart of the Solution lies the Debt Item component type. It is used to describe different types of technical debt, and is associated with the assets that are impacted by it with the Impacts reference type. The component types that may be impacted by Debt Items are:

It is assumed that components of these types are already part of your Ardoq model, forming the model of your application and technology portfolio, but you can, of course, choose to apply the Solution to just a subset of these component types. For more details about creating and connecting these components to describe your portfolio, see the following Solution documentation:

It is possible that a single Debt Item impacts more than one asset, and it is permitted to show this with a single Debt Item component having multiple Impacts references to multiple Applications, Technology Services and/or Servers etc.

Various methods have been devised for describing and quantifying technical debt.

The SQALE Method [2] uses a quality model derived from ISO 25010:2011 [3] to classify technical debt, and proposes relating the cost of remediation to the cost of the debt’s business impact to form a cost-benefit analysis and guide prioritization. McKinsey also recommend quantifying technical debt in terms of cost [4].

Gartner also propose using a quality model, such as ISO 25010, to classify technical debt. But instead of using financial measures to compare the cost of living with the debt with the cost of remediation, they propose considering technical debt as a source of risk, inviting you to consider:

This differs from the SQALE approach, in that impact is seen through the lens of a “worst case scenario”, rather than the day-to-day cost of living with the debt. They recommend using a 1-5 numeric scale for this assessment, and relating this to “the likelihood that the unthinkable will happen” (which is also given a 1-5 value). With this approach, Technical Debt is a very close relative of Risk. Technical debt items can then be compared using the PAID model [5], which plots debt items against four quadrants: Plan, Address, Ignore, Delay. This can be shown on an Ardoq bubble chart, with the bubble size showing the cost of remediation.

Forbes Technology Council Member Dr Ken Knapton describes an assessment approach that leads to a score that is calculated for every technology in the enterprise [1]. It considers attributes that describe supportability, expected remaining life span, stability and span (the technology’s footprint across the organization). These are summed and the total is multiplied by a combined assessment of the technology’s business criticality and its alignment with the strategic technology roadmap. The result is a score, normalized to a scale between 0.0 and 1.0, with a higher score representing a higher level of concern that warrants attention. A disadvantage of this approach is that it views all debt associated with an individual technology as a single composite score, and so does not allow the comparative evaluation of different technical debt items for prioritization. Nor does it represent explicitly the possibility of the same technical debt issue applying to multiple technologies or applications.

This Solution supports assessment and visualization of Debt Items that reflect these different approaches to looking at costs and impacts. They complement each other, considering the cost of living with the debt as well as taking into account the potential cost or impact of an incident that might occur as a consequence of the debt. Both approaches involve the documentation of individual technical debt items so that they can be recognized and managed appropriately. The “Knapton approach” is somewhat different. Instead of recording and managing individual debt items, it involves the calculation of a score against a technology. The approach is therefore less specific, and for this reason has not been implemented here, though its consideration of business criticality and span are reflected in the Solution’s visualizations. However, if you have implemented the Technology Portfolio Management Solution and wish to implement the “Knapton approach”, this could be accomplished using a survey for each Technology Product to capture the attributes and a calculated field to turn them into the normalized score.

Support for these different perspectives on costs and impacts associated with technical debt places some information demands on the description of each Debt Item. Gartner’s PAID model requires input scores on a scale of 1.0 to 5.0 for the “worst case” consequential impact of the Debt Item, and for its likelihood. These are handled as two numeric fields that are populated in the survey that creates a new Debt Item.

Both the PAID model and the more financially specific cost-benefit approach use an estimate of the remediation cost (the bubble size in PAID, and the Y-Axis in the cost-benefit bubble chart. Asking directly for input of a financial amount can create the false impression that an accurate number is being sought (which will frequently produce a response of “I don’t know”). Instead, we have adopted a list field (Debt Remediation) with the following options:

This more clearly acknowledges that this is a “guesstimate”, and invites an appropriate response. A calculated numeric field, Debt Remediation Cost, automatically translates the chosen list item to a numeric estimate by applying a corresponding number of days effort multiplied by a daily rate. The effort and rate amounts can be easily changed by editing the calculated field’s Gremlin code. See Technical Debt Management: Metamodel for full details of the Gremlin code of this and all other calculated fields.

The cost-benefit approach also requires an estimate of the day-to-day business cost of living with the debt item. The same approach described above is taken to calculate this estimate, with Debt Current Impact presenting the following options:

Similar Gremlin code converts these options to financial amounts in the numeric calculated field Debt Impact Cost.

A measure of business criticality is calculated and stored as a number in the field Total Criticality. It adds up the business criticality scores of all the impacted Application components that are associated, either directly or indirectly, with the Debt Item. If there are two impacted Applications, each having a Criticality of 5, the Debt Item’s Total Criticality will have the value of 10. This is used in the cost-benefit bubble chart to form the size of each bubble.

Many authors on the topic of technical debt, including the authors of the SQALE approach and Gartner, advocate using ISO 25010 System and Software Quality Model to classify technical debt. While this is not the only classification model that has been proposed (for example, see [6] and [7]), the use of a quality model such as this to consider the functional and non-functional qualities of technology solutions is widely endorsed by enterprise and solution architecture practitioners. Its use goes beyond just classifying technical debt, as it can be used to classify requirements of new business systems and as a framework against which to evaluate the health of an existing solution architecture, or to compare competing candidate architectures (for example, as part of the architectural evaluation of competing RFP bids).

Three example quality models are provided with this Solution:

See Technical Debt Management: Solution Configuration Guide for details of how to configure the Solution to adopt your chosen Quality Model.

We follow the same approach to modeling quality models that was introduced in the Application Risk Metamodel for modeling control frameworks and corporate policies. See How to represent Policies, Principles, Standards and Frameworks in Ardoq for details. The standard itself is represented with an Information Artifact component which can contain a URL pointer to the source document. Each Quality Characteristic or Sub-characteristic is represented with a Requirement component. If more than one tier is to be modeled, then Quality Sub-characteristics will use the parent/child relationship to show that they are children of their parent Quality Characteristic. The Ardoq Quality Model, for example, is modeled like this:

Each Debt Item has a single Impacts reference to the Requirement component in the first level of the quality model that best describes that system quality characteristic that is most affected by that Debt Item. This allows Debt Items to be classified according to the quality model, facilitating detailed analysis of different impacts on the IT estate using the quality model to provide a framework and a common language. This is especially useful if the same quality model is used to classify quality requirements and capabilities of new or existing systems.

A Debt Type calculated text field in the Debt Item component is automatically populated with the name of its impacted Requirement (quality characteristic). This makes it easy to filter or color code with conditional formatting Debt Items by quality characteristics when visualizing and analyzing technical debt.

You may choose to extend the quality model to include characteristics that are specific to an organization. For example, a business in a highly regulated industry may wish to adapt ISO 25010 to include regulatory compliance. Alternatively, an organization may choose to adopt a quality model entirely of their devising. The same modeling approach that has been used here can be easily adopted and the Solution will adapt easily to accommodate a different quality model. See Technical Debt Management: Solution Configuration Guide for details.

Some organizations like to model their architecture guiding principles in their architecture repository and link them to assets or issues that do or do not conform to them. The approach described here to represent a quality model and link it to technical debt can be extended to model the situation where failure to comply with a principle is considered to be evidence of technical debt. The set of architecture guiding principles can be modeled using the same approach as the quality model: the set of principles will be modeled with an Information Artifact component (including a URL pointer to the source documentation), and each architecture guiding principle will be represented as a Requirement component.

Assets (such as Applications, Servers or Technology Services) that fail to comply with a principle can be linked to a Debt Item with an Impacts reference from the Debt Item to the Asset. That Debt Item will in turn have an Impacts reference to the Requirement component that represents the broken principle.

The GOV.UK Service Standard, provided as an example quality model, is a popular example set of service design principles.

If the Application Risk Management Solution has been implemented, you can choose to create Impacts references from Debt Items to corresponding Risk components.

This echoes the Gartner approach that recognizes the linkage between technical debt and risk. Linking the two in your Ardoq model allows you to record risks associated with technical debt in your risk register and take action to reduce risk scores as you remediate the connected debt items.

If you are using Ardoq to model change initiatives (see the Strategy to Execution Series for details), you can create an Initiative component for each initiative to remediate technical debt, with an Impacts reference from the Initiative to the Debt Item(s) that will be remedied.

This allows you to record budgets and track progress of these initiatives in the context of your strategic change portfolio. If you do this, a field in the Debt Item will automatically record the start and end dates of the Initiative, allowing you to show the actions you are taking to address technical debt with a Timeline visualization.

An automated process is included with this Solution that enables you to record and track your action decisions, interacting with Initiative and Risk components where appropriate. A high level view of the process is shown below.

Full details of how this process has been implemented, and how to configure and run it can be found in Process Playbook: Technical Debt Management.

A key purpose of documenting and describing technical debt is to be able to communicate its potential impact and use such communications to prioritize the allocation of funds and resources to its remediation.

Two complementary bubble charts can be used to examine the extent and impact of a set of Debt Items and prioritize action to remediate them.

The cost-benefit bubble chart, inspired by SQALE’s Debt Map [2], compares the cost of living with a Debt Item with the cost of remediating it. The business impact cost is an annual estimate, so if that number is greater than the estimated remediation cost, there is likely to be a return on investment within 12 months. The size of each bubble shows the combined business criticality of the applications impacted by the item, and the color indicates the type of debt item, classified according to the impacted quality characteristic.

This chart only uses the day-to-day impact cost, and not the “worst case scenario” impact that might result from an associated risk incident. To consider this aspect of your technical debt, use the Gartner PAID model:

Here we see the same Debt Items, but from a different perspective. Note how the four items in the “Plan” quadrant have quite serious “worst impact” scores, representing the potential cyber risk in the case of the two Debt Items of type “Security”, and the potential risk of failing to comply with regulatory requirements in the case of the “Interaction capability” risk. It can also be insightful to change just one element of the bubble chart. For example, just changing the bubble size from Debt Remediation Cost to Total Criticality reveals that some of the items in the Plan quadrant are highly business critical yet relatively low cost to remedy.

The combination of these bubble charts will help you to communicate the costs, business impacts and risks associated with technical debt, and determine which Debt Items demand the most immediate attention.

Technical debt can also be viewed from the perspective of the quality model. The Dependency Map below shows how much technical debt is linked to the different Quality Characteristics in the quality model. One can quickly see which quality characteristics carry the most technical debt and the greatest potential impact from it.

A more traditional Block Diagram view shows how selected Debt Items impact Applications, their owners, the organizational units that consume them and the business capabilities they realize.

Reports can be used to show the results of more complex graph searches that connect technical debt to other component types. The report below shows which Business Capabilities, Applications and their consuming Organizational Units are ultimately impacted by technical debt. The search finds the business capabilities and organizational units that are associated with applications that are impacted by debt, either directly or indirectly (because they are dependent upon a component, such as a Data Store, Technology Service or Server, that itself is impacted by a debt item.

And this report shows technical debt from the perspectives of Applications and their owners:

For some visualizations and dashboard widgets, it makes sense to show all Debt Items. Where a Debt Action has been recorded, there may be some Debt Items that should be excluded from many of the visualizations. These are the Debt Items that have already been remediated, or where a decision has been taken to ignore the debt. We use the term “Live Debt” to mean Debt Items whose Debt Action is neither Ignore nor Remediated. By default, the Perspective TD - Live Technical Debt Views filters out Debt Items with a Debt Action value of Ignore or Remediated. Some of the Reports and Dashboard widgets also reference “Live Debt” in their titles; these apply a similar filter so that the user is only seeing debt that remains active and is considered of some significance.

The Solution gives you considerable freedom to decide the level of granularity that each Debt Item represents, since a single Debt Item can point to just one asset, such as a single Application, or many assets potentially of different types. This begs the question “what is the right level of granularity for a Debt Item?”. Should there always be a one-to-one mapping between Debt Items and impacted assets, or are there good reasons to have Debt Items that cover multiple assets?

In making this decision, we recommend that you are guided by thinking about the task of remediation. If a proposed Debt Item might relate to multiple assets (e.g. Software version needs upgrading), consider whether its remediation would be a single initiative or a number of separate ones, and choose a level of granularity for Debt Items that will lead to each Debt Item having a single remediation initiative associated with it. Applying this to the example of Software version needs upgrading, one would be unlikely to choose a single Debt Item to represent all assets that are running old versions of software. But one might choose a single Debt Item to represent a particular software product, or a single vendor contract, even when that Debt Item might impact many Application components, each representing a deployed instance of the software. A single remediation initiative might manage the upgrade process across the estate.

Although the focus of the Solution is on the recording, classification, quantification and management of technical debt, you will want to develop systematic approaches to discovering technical debt so that you can add it to Ardoq, make it explicit, and decide how and when to deal with it.

We provide guidance here on two complementary approaches to discovering technical debt:

Architecture Reviews and Health Checks are a good way of surfacing technical debt. Many organizations conduct a review prior to a new solution going live, and such a review can be used to draw out decisions that were taken to incur debt during the solution’s initial development. A periodic Health Check review of an existing system can be used to identify debt that has been accrued over time, perhaps because requirements and contexts have changed, or because there were insufficient resources to keep the solution current. See How to use Architecture Reviews to identify technical debt for more details.

As we’ve seen, the term Technical Debt can be used to represent a wide variety of technical quality gaps, deliberate or inadvertent, prudent or reckless. Many of these can only be uncovered through a detailed architecture review or health check, involving those who best know about the solution architecture and how well it meets its required quality characteristics. But a well-populated knowledge graph is also likely to contain clues that might reveal the existence of technical debt.

Failure to keep a solution within the boundaries of vendor support can be considered a form of Technical Debt. This can apply to application software, supporting technology services, or underlying infrastructure. The decision not to upgrade or replace a technology that is heading out of support is a decision to incur technical debt, and should be recorded as such (though it often goes unrecorded). If you’ve implemented the Technology Portfolio Management Solution your technology catalog will contain date range fields representing the vendor support and extended support critical dates. The Solution contains the following reports which can help you to identify possible technical debt:

If you have mapped your Applications to your Business Capability Model, you can take a closer look at the applications that realize the same business capability. Were decisions taken in the past to deploy a new solution when the required functionality existed in an application that was already deployed? If so, you might consider that past decision to represent technical debt, and one that is worth revisiting. See Getting Started with Business Capability Modelling for more details of how to develop a business capability model and map your applications to it.

An application that offers poor business value, or is a poor fit with the organization’s current technical needs and future plans, is showing signs of a quality gap. The Application Rationalization Solution provides the ability to analyze business value, breaking it down into an assessment of its features and usability, its functional fit and its business criticality. Technical fit is composed of the application’s technical integrity, its maintainability and agility and its technical strategic fit. Collecting and analyzing these data gives you the opportunity to identify quality gaps that you might choose to record as Technical Debt which demands attention. See Getting Started with Application Rationalization for more details.

For more about the detailed implementation of the Solution and how to adapt it to fit your organization, see Technical Debt Management: Metamodel, Process Playbook: Technical Debt Management and Technical Debt Management: Solution Configuration Guide.

[1] K. Knapton, “Measuring And Managing Technical Debt,” Forbes. Accessed: Feb. 09, 2024. [Online]. Available: https://www.forbes.com/sites/forbestechcouncil/2022/08/10/measuring-and-managing-technical-debt/

[2] J.-L. Letouzey, “The SQALE Method for Managing Technical Debt”.

[3] International Organization for Standardization, “ISO/IEC 25010:2011 Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models,” International Organization for Standardization, 2011.

[4] McKinsey DIgital, “Demystifying digital dark matter: A new standard to measure and tame technical debt,” McKinsey Digital. Accessed: Feb. 09, 2024. [Online]. Available: https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/demystifying-digital-dark-matter-a-new-standard-to-tame-technical-debt

[5] S. Van Der Zijden, H. Dodd, A. Thomas, and Egiazarov, “How to Prioritize and Sell Technical Debt Remediation.” Gartner, Sep. 2023. [Online]. Available: Doc ID G00798329

[6] R. Verdecchia, P. Kruchten, P. Lago, and I. Malavolta, “Building and evaluating a theory of architectural technical debt in software-intensive systems,” J. Syst. Softw., vol. 176, p. 110925, Jun. 2021, doi: 10.1016/j.jss.2021.110925.

[7] A. Weaver, “What’s All the Fuss About Tech Debt?,” CTO Academy. Accessed: Feb. 14, 2024. [Online]. Available: https://cto.academy/how-to-best-manage-tech-debt/

[8] “BS ISO/IEC 25010:2023 - TC | 30 Nov 2023 | BSI Knowledge.” Accessed: May 20, 2024. [Online]. Available: https://knowledge.bsigroup.com/products/systems-and-software-engineering-systems-and-software-quality-requirements-and-evaluation-square-product-quality-model?version=tracked

[9] “Service Standard - Service Manual - GOV.UK.” Accessed: May 20, 2024. [Online]. Available: https://www.gov.uk/service-manual/service-standard

[10] “Open Government Licence.” Accessed: May 20, 2024. [Online]. Available: https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/

[11] R. Kazman, M. Klein, and P. Clements, “ATAM: Method for Architecture Evaluation,” Software Engineering Institute, Carnegie Mellon University, Technical Report CMU/SEI-2000-TR-004, Aug. 2000.

[12] S. Field, “SARM,” SARM: A site devoted to the Solution Architecture Review Method. Accessed: Feb. 14, 2024. [Online]. Available: https://sarm.org.uk/welcome/