Is your judgment noisy?

Photo: Collected

What is noise? In their book with the same title, authors Daniel Kahneman, Olivier Sibony and Cass Sunstein describe 'noise' as the unwanted variability in judgment especially in the fields of law, medicine, business, recruitment and admissions where the errors of judgment have substantial consequences. 

To illustrate the concept of noise, try opening up the stopwatch application on your phone and practice timing 10 seconds with your eyes open a few times. 

Next, use the lap button to guess how long 10 seconds is with your eyes closed several times. Chances are that most of your guesses would be a few seconds more or less than the 10 seconds you were trying to get. 

This variation in time away from the real value is what the authors are referring to as 'noise' and it is a bigger problem than you think when it comes to our decision-making and predicting capabilities. 

In his bestselling book, 'Thinking, Fast and Slow', Daniel Kahneman described how bias affects human judgment, in 'Noise.' he and his co-authors posit that errors in human judgment have two components, bias and more importantly, noise. 

However, while biases in our judgment are easier to identify and widely addressed, the authors maintain that not only is noise ignored but also harder to detect. 

Why is having noise in judgment so problematic? Imagine you were being tried in court for robbing a bank, the authors show that the sentence you receive can not only vary wildly (between three years in jail to twenty years in jail, in one example) depending on which judge you get ('the first lottery') but also the weather, the mood of the judge and whether or not it is almost lunchtime.

In other words, not only can the judgment for the same case vary from person to person but also within the same individual depending on their personality, mood, and preferences which is not only unfair but often costly when it comes to large and small organisations. 

Throughout the book, the authors list case study after case study in fields ranging from medicine, job recruiting to forecasting to demonstrate how ubiquitous noise is. 

Notably, the authors discussed how the person speaking first in a meeting can inadvertently create noise in the decisions made by the whole group. 

In another instance, the authors showed how doctors were more inclined to prescribe cancer screenings earlier in the morning than in the afternoon which should not be the case.  

Why is noise so hard to detect? The authors suggest that without training most people tend to think causally (cause and effect or by forming a narrative) while noise is only detectable when thinking statistically. 

Unsurprisingly, the book delves into the psychology of judgment and uses statistics to explain its core concepts. 

Another interesting aspect of the book is that the authors attempt to make predictions. The book categorically outlines how unreliable, inconsistent and 'noisy' human judgment is in the area of forecasting and suggests that decision-makers rely on statistical models or even AI to enhance prediction capacities. 

However, the authors do acknowledge the limitations of AI (for example, AI trained on biased data can make equally biased forecasts) in certain situations and instead recommend organisations to improve their forecasting and decision-making capacities by aggregating the independent judgments (i.e., votes) of a group of decision-makers. 

This way, the group can avail the 'wisdom of the crowd' and make less noisy decisions. To illustrate, the group cannot share their opinions or findings before voting and should instead vote first, work out their disagreements and then finally vote once more to make an informed decision.

Similarly, the authors propose a smorgasbord of solutions to firms and decision-makers to not only detect noise through regular 'noise audits' but also reduce the noise substantially through 'decision hygiene techniques' including the aforementioned aggregated independent voting technique, specified checklists, uniquely structured meetings, decision observers to detect bias and many others. The appendix at the end of the book is a useful resource for carrying out such initiatives.

Furthermore, the book offers advice on how to make our personal judgments less noisy, for example, by being more open-minded and constantly adjusting our forecasts in light of new information. 

The authors also remind us to be suspicious of individuals who project unattainable levels of confidence regarding events that they could never predict as accurately. 

Finally, the authors also acknowledge that noise is not always undesirable as the variability in judgment is valuable in fields that value the diversity of opinions such as in art, movie or book reviews, debates or even the stock market. 

However, they warn us against taking noise in judgment for granted in areas where fairness and consistency are important and unfortunately, the noise in fields such as law, medicine and business are too high to ignore. 

For instance, it is quite common in the field of medicine for patients to seek a second opinion from another doctor so that mistakes or disagreements in this regard may not cost lives. Still, the authors urge decision-makers to carry out their own 'noise audits' to determine whether it is cost-effective for them to undertake noise-reducing strategies.

Overall, I found the book to be very insightful in understanding the limitations of my judgment and how to be less 'noisy'. This book could be a game-changer for decision-makers or firms seeking to develop or improve the reliability and consistency of their decision making and forecasting capabilities given that they are willing to adhere to the necessarily rigid principles of 'decision hygiene' delineated in the book.

However, the book is far from perfect, especially because of the fact that the 400 pages of the book could have easily been simplified and consolidated without the digressions into advanced psychology or often repetitive case studies. 

As a result, casual readers may find the book very dense and often inscrutable thanks to the often trivial terminologies the authors use to explain their concepts. 

If you want to apply the important concepts of the book without having to trudge through the entire 400 pages, skipping parts 3 and 4 of the book is a good idea.