<aside> 🎯 In this chapter, you will learn how to dig deep on the options while avoiding pitfalls and driving toward a recommendation.
</aside>
With options identified and short-listed, it is time to do the detailed research that will inform the recommendation and ultimately the decision. Even without a structured decision making process, technical architects and decision makers have always performed this step. However, with clarity on the needs and the scope of the decision, the evaluation can be focussed and complete.
The evaluation can largely be performed using your research instincts, but there are several common pitfalls to be aware of and avoided by leaning on the TDR format and process. In particular, be aware that the goal of the Evaluation section is not to make a decision but instead to provide the inputs and evidence to objectively make a recommendation that is well reasoned and objective.
We will discuss the pitfalls in the context of techniques that will help you avoid them.
You have seen it before - a decision is made claiming it is the best option even though it does not address key needs. Perhaps the best choice was made, but a lack of transparency on how a choice relates to the requirements erodes trust. For the integrity of the decision making process, it is important to use the needs and requirements identified earlier to drive the evaluation.
<aside> 🚨 Pitfall #1 - Focusing on 2-3 politically charged requirements instead of Needs & Requirements that were identified.
</aside>
In our experience, needs and requirements naturally separate into two categories: feature checklists, and those requiring analysis. Checklist items are those that can be answered as yes/no or with a single value, potentially with a short description. Other items require more complex analysis to decide the benefits or downside.
A good place to start is to determine which of your requirements can be covered by a feature checklist. Review the Needs & Requirements that were documented earlier in the TDR. Which of your needs and requirements can be easily captured in a table with a single value (ex. checkmark, yes/no, numeric value)?
Starting the evaluation with a feature checklist (aka feature matrix) helps with getting the evaluation started even when the decision being address is large and potentially daunting. With this first step, the research is structured and it may be feasible to share the workload between team members.
Create a table in your TDR to capture the data and fill in what you know already from past experience or earlier research. Below is an example of a feature checklist. Customize the format of the table to suit your needs.
| Option 1 | Option 2 | Option 3 | |
|---|---|---|---|
| Available for self-hosting | yes | yes | no |
| Cloud agnostic | yes | no | yes |
| Available for one-time cost | by special request | yes | no |
| Custom branding support | limited - logo only | no | yes |
| Familiarity | none | low - new for everyone but Jane (developer) | high - used previously by team |
Often these data points can be found in online material. However, make sure you are aware of the source of information as vendor or sponsored material typically frames features in beneficial ways.
If you can’t find conclusively whether an option satisifies a checklist feature, add questions marks or a note in the table as needed to keep the evaluation moving. You can either return to it later or it may be inconsequential to the recommendation and decision.
Here are a few additional tips for evaluating checklist requirements: