When evaluating solutions and making important purchase decisions, don’t skip applying critical thinking skills throughout the process. You should think about the perspective of information sources. Learn how accolades and industry awards being shown to you on websites and in marketing pitches are won. What methodology was used? Were the solutions tested? What criteria was examined? In the ever-growing list of “trophies for everyone” industry awards, it is becoming more difficult to decipher real awards from paid advertisements. In this article and in an upcoming webinar, I will provide a few tips to help you enhance your investigative prowess.
Introducing Critical Thinking
Before we examine the most commonly touted industry analyst accolades, let’s sharpen our critical thinking skills. What is critical thinking? Critical thinking is the ability to think clearly and rationally. It includes the aptitude to engage in reflective, independent thinking. Someone with critical thinking skills is able to do the following:
- Understand the logical connections between ideas
- Identify, construct and evaluate arguments
- Detect inconsistencies and common mistakes in reasoning
- Solve problems systematically
- Recognize the relevance and importance of ideas
- Reflect on the justification of one’s own beliefs and values
Critical thinking is a disciplined process of skillfully conceptualizing, applying, analyzing, synthesizing, and evaluating information. It considers clarity, accuracy, precision, consistency, relevance, sound evidence, good reasons, depth, breadth, and fairness.
People who think critically are keenly aware of the innately flawed nature of unquestioned human thinking. Emotion, prejudice, bias, complexity, and vested interest do distort personal reality. Critical thinkers recognize these challenges and understand everyone makes mistakes.
Critical thinking can be applied to many areas of our lives – not just analyzing the analysts. We do live in a world filled with fake news, propaganda and conflicting reports. The World Economic Forum predicts critical thinking will rise to become the second most important skill in the Fourth Industrial Revolution.
As you start applying critical reasoning to evaluation processes, continually develop a checklist of questions to assess evaluation methodology, reasoning and detect potential biases. Here is a partial list of questions that I have created from my own investigations.
- Who is the source? What is the motive?
- Is the source qualified? Can they impartially review vendors?
- Has the source received anything of value from the vendor?
- How is the firm they work for structured? How do they earn a profit?
- Were payments made to the firm by evaluated vendors? Why?
- Were all relevant vendors considered? Was a significant vendor omitted?
- What assessment criteria and process was used?
- Did evaluators conduct “hands-on” tests of solutions?
- What information, data and evidence was collected? How was it gathered?
- Does the award sample accurately reflect the larger population?
- Did the assessment address topic depth, breadth and complexity?
- What assumptions were made?
- How were vendors involved in the process?
- Do other sources conflict with the evaluation findings?
As you delve into these questions, you will uncover a wide variety of “open industry secrets” and known issues with awards that are often overlooked or misunderstood. While almost all sources, reports and awards do provide value, it is important to understand award context for appropriate use and weighting in a vendor evaluation process.
Never use an award as a shortcut to replace your own personalized vendor selection process.
To Learn More
Please join me in the the upcoming webinar to further examine this topic.
It is well for the heart to be naive and the mind not to be. -Anatole France