you're reading...
Service Management

Selecting ITSM tool is a game of chance!

Certain words carry negative connotation in business world – chance, game, chaos, random, uncertainty…. to name a few. These terms are perfectly normal in the scientific world and in fact they lay foundations of modern science.

So, when I came across a paper “Don’t Make ITSM Tool Selection a Game of chance” (Ferris 2013) I was curious. The paper acknowledges the wide range of tools and advocates criteria based selection. The main theme of the paper is to emphasise the need for careful planning before going to market. I am sure no one would disagree. Especially I like the emphasis on process and organisational change management.

In this blog let us probe further about the “chance” of selecting a perfect tool:

Can there be more than one tool in the market place that “best fit” our organisation?

If the answer is “yes” then there is a game of chance.

Can we come up with “thorough” selection criteria, to zero-in a single tool?

Yes, we can.

If you have answered “yes” for both the questions then there is a conflict. Unfortunately the organisations (especially Government, due to probity requirements), want to pretend that there is a single answer to their needs. These organisations want to narrow down their selection criteria to choose a single tool. They do succeed by reverse engineering the criteria from the tool they want to choose. This approach will tick all the boxes relating to probity and transparency.

Hybrid car selection

Let us take an analogy. I want to buy a car, to take me from point A to B with minimum two passengers, with low emission.

If I come with generic criteria, I may short list around 5 to 10 hybrid cars.

After this stage, my personal bias plays an important role in making my final selection. If I am a Toyota fan, I will not look anything other than Prius. If I am used to driving only Mercedes, Toyota will be rejected. Please note that both the selection will meet my objective “taking me from point A to B”.

This “selection” and “rejection” becomes complex in Government and corporate world. Since we want a perfect answer, we need to expand our selection criteria by manipulating the selection criteria to result in a single answer. I can easily do this by including the specifications of Prius and some criteria that can be satisfied only by Prius. Please note that in the selection criteria, I have not mentioned that I want a Prius, so I will satisfy the probity requirements.

The same decision making model can be extended to ITSM tools. It is possible to manipulate the selection criteria to arrive at a single answer. If we come up with generic selection criteria, you will come up with 5 to 10 tools. Selecting a single tool from that pool, is a “game of chance” because ALL the tools will meet your business objectives. There are other factors that play in your final selection.

In psychology, the decision making comprises of two systems. (Kahneman, 2012)

System1 operates without voluntary control. It is more like intuition

System2 allocates attention to details, including computations

In the first hybrid car selection decision making model, I combine both aspects to arrive at a decision. In the second model, I have forced “system2” to arrive at a decision which was already taken by System1.

Let us analyse ITSM tool selection. A typical ITSM RFT lists around 600 – 800 criteria (see my earlier blog: https://psymphony.wordpress.com/2013/09/18/fast-tracking-800-criteria-itsm-tool-selection/)

I have seen criterion like: “tool shall support Facebook type “like” icons in the knowledge article interface” categorised as a “core” criteria. “Core” means that the tools that do not satisfy these criteria will not be shortlisted. At that time of this RFT, I was told that there is only one in the market that will fit this criterion. ( I am sure we will have many more tools now). In other words, at that time it was obvious that the organisation is trying to choose a pre-determined tool.

You may argue that the above criteria may be important for some stakeholders. Well, during such arguments we need to go back to the business case, if there is one! The business case will state “we want to go from point A to B”. Having a “like” button is not going to impact your end customer satisfaction, improve productivity, reduce cost.

What is the “chance” of your chosen tool is the right fit for your organisation?

It is an interesting question. Ideally you want to answer 100%. But you know intuitively that it is not the right answer. The reasons are:

  • different stakeholders have difference expectations
  • the market place is continually changing. There may be another tool/or new feature added to the rejected tools after you have chosen the tool (and locked for 5 years)
  • the organisation needs change over time
  • some of the “core” features are not used at all
  • the “cool features” that made you to invest in the tool does not add business value

Now let us try to answer this question through the lens of conditional probability. Let us try to answer this question using Bayes theorem. (here is a non-mathematical introduction to Bayes theorem: http://www.youtube.com/watch?v=XR1zovKxilw)

In the Bayesian interpretation, probability measures a degree of belief. Bayes’ theorem then links the degree of belief in a proposition before and after accounting for evidence.

For example, you may believe that the ITSM tool you have chosen is a “best fit” for your organisation. Then you will start gathering evidence which may impact your belief either way.

I will give the example provided in Wikipedia (http://en.wikipedia.org/wiki/Bayes%27_theorem) which is intuitive to understand the concept.

Suppose someone told you they had a nice conversation with someone on the train. Not knowing anything else about this conversation, the probability that they were speaking to a woman is 50%. Now suppose they also told you that this person had long hair. It is now more likely they were speaking to a woman, since women are more likely to have long hair than men. Bayes’ theorem can be used to calculate the probability that the person is a woman.

To see how this is done, let W represent the event that the conversation was held with a woman, and L denote the event that the conversation was held with a long-haired person. It can be assumed that women constitute half the population for this example. So, not knowing anything else, the probability that W occurs is P(W) = 0.5.

Suppose it is also known that 75% of women have long hair, which we denote as P(L|W) = 0.75 (read: the probability of event L given event W is 0.75). Likewise, suppose it is known that 15% of men have long hair, or P(L|M) = 0.15, where M is the complementary event of W, i.e., the event that the conversation was held with a man (assuming that every human is either a man or a woman).

Our goal is to calculate the probability that the conversation was held with a woman, given the fact that the person had long hair, or, in our notation, P(W|L). Using the formula for Bayes’ theorem, we have:

where we have used the law of total probability. The numeric answer can be obtained by substituting the above values into this formula. This yields

i.e., the probability that the conversation was held with a woman, given that the person had long hair, is about 83%. More examples are provided below.

Moving back from having interesting conversation with woman in train, let us apply the theorem to choosing ITSM tools.

The probabilities we need to know are:

  • what is the probability of choosing a tool that could be best fit?
  • what are the evidences that the tool is a best fit?

The first question is not very difficult. You may consider the entire market space of 300 tools or narrow your search based on high level criteria. Let us assume you have short listed 5 tools that are comparable in terms of features, cost, support etc.

If this is the case, the probability of choosing a tool is 20%. ( 1 tool out of 5 )

The second question is tricky. You may have your own way of evidence gathering. My proposal is to benchmark around 30 companies that are comparable to your organisation. You will ask only two questions:

  • what tool you use?
  • are you satisfied and recommend to others? (Net Promoter Score).

For this exercise, let us assume 50% of the sampled organisations recommend tool-A.

Let us develop the equation for finding a tool-A that is bestFit.

You have chosen tool-A from the list of 5 tools. The probability of bestFit means, the comparable organisations use a ITSM tool and recommend it. (it is like “long hair” attribute in the Wikipedia example)

P ( tool | bestFit)

= P(bestFit/tool) * P (tool) / P(bestFit)

= P (bestFit/tool)* P(tool) / P (bestFit/tool) P (tool) + P (bestFit/otherTool) P (otherTool)

P (tool) = 0.2 ( we can choose 1 tool out of 5 choices)

P (bestFit/tool) is the proporation of companies that use your chosen tool (tool-A) and happy with it

P(bestFit/otherTool) is the proportion of companies that use other tools and happy with it.

P (tool) = 1/5 =0.2 (probability of choosing tool-A from a choice of 5 tools)

P (other tool) =4/5 =0.8 (probability of choosing other tools from a choice of 5 tools)

Let us consider 3 situations:

Case A: All the comparable businesses use tool-A and recommend it

Case B: 50% of the comparable business use tool-A and recommend it

Case C: None of the comparable businesses use tool-A

Case A:

P ( tool | bestFit) = P(bestFit/tool) * P (tool) / P(bestFit)

= P (bestFit/tool)* P(tool) / P (bestFit/tool) P (tool) + P (bestFit/otherTool) P (otherTool)

P (bestFit/tool) = 1 (100% of companies use tool-A)

P (tool) = 0.2

P (bestFit/otherTool) = 0 (nobody uses other tools)

P (otherTool) = 0.8 (probability of choosing other tools)

= 1 * (0.2) / 1* (0.2) +0 = 100%

The result is intuitive. If all the businesses you have surveyed are using tool-A and happy with it(the selected tool), then the probability that you are going to be happy is 100%.

Case B:

50% of the comparable business use tool-A and recommend it

P ( tool | bestFit) = P(bestFit/tool) * P (tool) / P(bestFit)

= P (bestFit/tool)* P(tool) / P (bestFit/tool) P (tool) + P (bestFit/otherTool) P (otherTool)

P (bestFit/tool) = 0.5 (50% of companies use tool-A)

P (tool) = 0.2

P (bestFit/otherTool) =0.5 (50% companies use other tools)

P (otherTool) = .8 (probability of choosing other tools)

= 0.5 * (0.2) / (0.5* (0.2) + 0.5 * 0.8) = 0.2

You may be surprised at the result. We have only 20% chance that tool-A is the bestFit for the organisation even if 50% of your comparable businesses recommend your chosen tool. The best-fit is low because there are many other tools in the market that are used by the comparable organisations.

If there more than 50% of the sample organisations recommend tool-A then, the best fit probability will increase.

Case C:

None of the comparable businesses use tool-A

We can see the probability of best fit is 0, since numerator becomes 0. This is again intuitive.


One limitation of my model is the reliance of market acceptance. If you have chosen a tool from a start-up company that has best-fit features, my equation will warn you that the probability of “best fit” is zero. In reality, it could be the best fit for your organisation. However, we are taking a calculated risk by choosing a tool nobody uses.

Closing thoughts

The tool selection process needs to change. My limited experience with the selection process reveal that it is slow, effort intensive, bureaucratic and hypocritical!

We need a nimble, low overhead, frequent tool selection process to get the optimum benefit from the dynamic market space. Like any business change, the new selection methodology should address the technology, process and people of aspects.

I am keen to hear “best practices” around the globe in ITSM tool selection process.


Ferris, K. 2013. IT Service Management Monopoly: Don’t Make ITSM Tool Selection a Game of Chance. [online] Available at: http://macanta.com.au/mcnta-wp/wp-content/uploads/2010/01/IT-Service-Management-Monopoly.pdf [Accessed: 29 Oct 2013]

Kahneman, D. 2012. Thinking, fast and slow. Australia: Penguin


About Murali Ramakrishnan

Murali is the Managing Director of the boutique consulting firm "Process-Symphony". Process-Symphony specializes in IT enabled business process orchestration. http://www.process-symphony.com.au http://www.kloudax.net.au


No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: