Capturing Stakeholder Feedback

It is common to represent the value or fitness of a component in Ardoq in a field, typically as a score between 1 and 5. This is fine if all we want to record is the consensus view, but what if we want to use Ardoq’s survey capability to solicit different views across a range of stakeholders, and analyse the results using Ardoq’s visualization capabilities. Where a single score is needed, we might choose to automate it by aggregating the varied opinions of different stakeholders in some way.

For the purposes of this article, we’ll use an Application’s Business Value score as an example, and target the owners of each Organizational Unit that consumes that Application for their feedback on the Application’s Business Value (from their perspective). But this approach can be applied to a wider range of situations: for example, organizational units that realize a business capability may wish to express their views on the capability’s maturity. The design approach adopted here can easily be adapted to collect feedback from different kinds of stakeholders about different components.

Reaching out with a survey to different organizational units to capture a set of scores that can be compared, contrasted and aggregated, is an attractive proposition. This article explains how to do it. But you should also consider the alternative, which is to bring people together to discuss their experiences. A key job of the Enterprise Architect is to facilitate conversations and promote shared understanding. Reducing opinions to a set of scores may enable statistical analysis, but it shouldn’t replace conversations, and bringing together different stakeholders in a room can be a catalyst for new, shared ideas and perspectives. These would be missed if the EA team only engages with its stakeholders remotely and separately. So before you proceed with adding a little more complexity to your model, we recommend you consider alternative or complementary ways of understanding stakeholder views.

The field Business Value is part of Ardoq’s foundation metamodel, being further broken down into four constituent scores: Business Strategic Fit, Features and Usability, Functional Fit and Criticality. It is often used alongside Technical Fit to form Gartner’s TIME Model [1], which provides valuable input to application and product portfolio triage.

But what if an application has different stakeholders who each have a different view of that application’s business value? By holding only one set of values for an application’s business value (and its constituent elements), we are assuming that either there is consensus among, for example, the different organizational units that consume the same application, or that any differences are somehow resolved and left outside the scope of the Ardoq knowledge graph.

In this article, we explore how to capture the distinct views about business value that are held by the different organizational units that consume the same application. The differences may give insights into issues specific to some of the organizational units concerning how they make use of the given application.

If you have multiple stakeholders whose feedback about applications you wish to capture, Ardoq has this capability built in, and it is a simple job to extend the provided solution metamodel so that Business Value, and its constituent elements, is captured from the separate perspective of each stakeholder. This extension makes use of the following built-in features:

What is the value of doing this? Isn’t life easier just having a single Business Value score for each application and ignoring the differences between its stakeholders?

Consider the following situation: Two business units in an organization realize the same business capability, using the same application. If asked separately, one of these units might score the application’s Features and Usability ‘5’, while the other gives it a score of ‘1’. Should the enterprise architect be happy just to see an average score value of ‘3’? If they could see each score, it prompts a number of questions that are surely worthy of investigation (and that go right to the heart of the role of the EA). What is the reason for such a big difference?

Capturing feedback from stakeholders in Ardoq can highlight these questions in this situation. But we need to extend the metamodel, since we cannot store multiple scores of we only have a single Business Value set of fields for each application.

The most obvious group of stakeholders to approach for feedback are the organizational units that consume each application. Or to be more specific, the owners of these organizational units, since they are individuals with an email address, each of whom can be sent a survey.

If you’ve followed the guidance on stakeholder mapping in this article [2], you may have defined a wider range of stakeholders using this metamodel:

Both situations can take advantage of capturing stakeholder feedback. For the first example, we would seek feedback from each consuming organizational unit, asking its owner to complete the survey, and storing their response on the Consumes reference. For the second example, we would seek feedback from each stakeholder, sending the survey to the owner of each organizational unit that is a stakeholder in the application, and separately to each person who is individually a stakeholder in the application. In this case, the survey responses would be stored as fields on their respective Is Stakeholder In reference.

The remainder of this article will assume that we have the first of these two situations. But it is easy to see how the solution can be applied when the second situation is preferred.

In order to avoid interfering with other Solutions that make use of the Business Value field and its constituent components in the Application component, we will retain these fields with their names and types unchanged. But the four ingredients: Business Strategic Fit, Functional Fit, Features and Usability and Criticality can become Calculated List fields if we wish to derive these automatically from the collected feedback from multiple stakeholders. As the list options are just 1, 2, 3, 4 or 5, you will need to decide how best to combine them. You might choose max, min or a rounded average, depending on your view of how to draw together what might be very different scores. Alternatively, you might leave the metamodel as it is, with these fields as manually input, and without an automated calculation that attempts to reconcile what might be conflicting feedback.

We do need to add the four ingredient fields to the Consumes reference so that we can capture and store the different stakeholder views. You should use a different, but related, field name for these. Your extended metamodel will now look like this:

Note that we have chosen to leave out a calculated Business Value field on the Consumes reference, to keep the solution simple. We’re seeking here to capture the different input views of each organizational unit, but if a separate combined Business Value score is also required, it can be added as a Calculated Number field called Org Business Value on the Consumes reference using the same weighted average calculation used in the Application component type.

You may, of course, add more fields here, for example to capture the rationale behind the set of scores in a text paragraph field.

The new survey to capture the feedback will be on the Organizational Unit component in the Organization workspace. There will be one survey for each Organizational Unit that will capture their feedback for all the applications they consume. And the scores they provide with their feedback will be placed in fields on the Consumes references as described in the preceding section, with a set of feedback responses on each reference that points to the corresponding application.

Add a single reference question to the survey and choose the restriction Field edit only, giving the question the title Application Feedback. For this question, choose the Consumes reference to the Application component type in the Applications workspace. Add a Field Question for each of the fields you’ve added to the Consumes reference. This will present the respondent with a dropdown for each application their organizational unit consumes, each dropdown containing the required feedback questions.

Now that we have the survey, we may want to create a broadcast so that all organizational unit owners are sent a task periodically asking them to complete the feedback survey on behalf of their organizational unit. As we’ve seen, the one survey will cover all applications that are consumed by their organizational unit. This is a straightforward Survey Broadcast for your new Application Feedback Survey, that will have a selected audience using the predefined query Owns. If you have some Organizational Units that don’t consume any Applications, you could apply the following reference filter rule to avoid sending people a survey with an empty list of applications:

Once you’ve added a suitable message and set the schedule and interval, you can save and launch the broadcast.

Below is a block diagram showing the feedback of five organizational units, and the results of combining them into a single set of scores for the application using an averaging function.

In this article we’ve shown how you can extend the “out of the box” foundation metamodel so that you can solicit feedback from organizational units about the value they derive from the applications they consume. We’ve shown how you choose to retain these as independent opinions for analysis, or combine them in a suitable way to calculate a combined Business Value score for each application that is derived from the collected feedback of its consuming organizational units. The same approach to using Surveys, Broadcasts and fields on references can be used to solicit feedback from a wide range of stakeholders about any component they have a relationship with.

[1] ‘Use TIME to Engage the Business for Application and Product Portfolio Triage’, Gartner. Available: https://www.gartner.com/en/documents/3905663

[2] ‘How to Record and Map Stakeholders with Ardoq’. Available: https://help.ardoq.com/en/articles/224784-how-to-record-and-map-stakeholders-with-ardoq