Biased merit and diversity

In two earlier posts I discussed the roles of skill and luck in being successful and getting rewarded. Recently I read a Harvard Business Review* (HBR) article on the topic and wanted to get briefly back to this important topic.

As the fashion goes today, diversity is the hot topic on improving performance in organizations and promoting equality.  At the same time organizations face the problem of increasing diversity in the correct and effective way. This starts with the definition of diversity. The HBR article mentions that Millennials see diversity as “valuing ope participation by employees with different perspectives and personalities”, while older generations see diversity as “equitable representation and assimilation of people from different demographic groups”.

Even if we manage to agree on whether to promote diversity of thought and personalities, different demographic groups or still something else, choosing the right people is still a challenge. An obvious answer is to use merit based assessments. Yet, even then we face problems, since, it turns out, the very definition of merit depends on the person being evaluated.

The HBR article cites a study, where a consulting firm’s recruiting process was observed to see how merit is defined. It became clear that merit is not a fixed personality attribute or a specific behavioral model for a specific situation. When the consulting firm had interviewed job candidates and the interviewers came together to discuss their observations, they ended up trying to reach consensus on the maybe-maybe not candidates; the clear no-gos and top performers were not discussed. When discussing the still open cases, a person’s evaluation was highly influenced by his racial background: “For example, black and Hispanic men were often seen as lacking polish [, meaning communication skills,] and moved to the reject pile, even when they were strong in other areas, whereas white men who lacked polish were deemed coachable and keep in the running.”

The italics in the above passage are mine, since the word coachable really caught my attention here. The passage says that black an Hispanic men lacking polish were perceived to have incomplete skill sets, maybe without improvement potential, while white men with similar skill sets were seen amenable to coaching, something that might be even desirable to educate a new employee into the company culture.

In another passage from the article the difference in social behavior were discussed: “Nonwhite were rejected for being unassertive, but in white, modesty was seen as a virtue.” Here, like with the lack of polish, we see how some candidates missing a character are taken to be incomplete and also lack the potential for further development, while other candidates with lacking skill sets are seen as fertile ground for education and personal development.

Quite obviously merit based evaluation and remuneration is not a panacea, all the more if we have no clear, for all equal definition of merit in a given situation. Thus, we should be aware and acknowledge that our very definition of merit can be biased. Acknowledging this fact is one step closer to doing fair, merit based evaluations and promoting diversity based on true, equally assessed merit.

*Harvard Business Review July-August 2016: We Just Can’t Handle Diversity

Edit: Corrected typos on 26.3.2017

Nationalities and athletic performance for sale

Reading the newspapers this weekend, I spotted an article on a topic that has gone under my radar: buying athletes to gain success in international sports competitions. With buying of athletes I am not referring to illegal bribery, but to the practice of sponsoring a foreign athlete and having him consequently change his official nationality to that of the sponsoring country. This practice is nothing new, but has become more prevalent in recent years, as nations want more fame and fortunes. It is not just countries looking for the best talents but also athletes trying to find the best opportunitiesSome voices say that these practices of nationality and athlete shopping are worrying, if not outright morally wrong.

A knee-jerk response to the question on the fairness of these practices might be to condemn them as immoral since sports is about fair competition between nations and should be not driven by motives other than the joy of sport and competition.

When looking at such a counter-argument, as I present above, for trading athletic skills for money, it seems to be on shaky grounds. What is fair competition? Why should sports be done only for the joy of sport and competition? Yet another question is, if it is even relevant or desirable for nations or countries to compete against each other in sports. I will briefly discuss each of these questions and after that, I will draw the argument together with some analogies to the business world.

Sports and fair competition

If we say that sports, and especially sport competitions between nations, should be fair, I suppose we are requiring equal chances to win, equal chances to exploit potential or something similar. Without going too deep into a philosophical discussion on fairness, I just want to point out that even the concept of fairness is vague and not unlikely to vary between individuals. It is obvious that everyone prefers a “good” or “successful” life to a “bad” or “unsuccessful” one. Yet, we are not equally successful. But whether the observed inequalities are fair or unfair, is not always to decide. For those interested, I recommend reading John Rawls who has intensively studied fairness. In reading Rawls, it becomes clear that even equal chances to use our natural talents may not be fair. Thus, the requirement of fairness in sports would need a more rigorous definition to provide a good argument. In any case, I would claim that it is acceptable to pursue activities which you like, are good at and for which people are ready to pay.

Sports for the sake of sportsmanship

One argument against countries’ buying athletes into their national teams and athletes shopping for better prospects might be the concept of sportsmanship, the idea that sports is something pure and pursued just for its own sake. My immediate counter question is, why it should be so.

Many of us do sports, gymnastics, weightlifting, horseback riding, running, skiing and so on. Some of us train for health and fitness or to challenge our own limits. Some train for competitions where they can measure their skills against those of others. I’m inclined to think that many of these reasons belong or at least relate to the concept of sportsmanship, but they might just as well be goals on their own merit. We may play football on a Sunday afternoon with our friends just for fun, to spend time and rejuvenate from the hard workweek. But even then, I would argue, the purpose of the football game is to mingle with friends, recuperate and regain strength. These are stronger motives than just playing football for footballs sake.

The aspects of respect, ethics and fairness related to sportsmanship play a role in how we treat others during sports, but shouldn’t we always treat others with respect and according to high ethical standards? If we are to use these ideals to limit the number of ways how sports is to be pursued, why shouldn’t we also apply them in other areas of life? Why would it be any more acceptable for an office worker to move abroad after a better job and eventually change his nationality than for an athlete? I would argue that it is in both cases acceptable to maximize the benefits you can gain from using your skills, assuming that you are doing within the limits of the law.

Finally, the concept of fairness is still somewhat vague, so the concept of sportsmanship leaves us without a solid argument against our question on nationality shopping.

Utility and the competition of nations

The Olympic games are maybe the best example of a sports event where nations stand to find out who runs faster, jumps higher and is the stronger one. Winning sports competitions gives good publicity for countries, abroad and domestically. The politicians of a country with troubles in its internal affairs may receive a welcomed boost for their image, if their home country is successful in the Olympic games or other high profile sports events. Likewise, the individuals winning the respective competitions gain publicity, receive fame and fortune and can hope for even better opportunities in the future, as advertisers want to use them to increase sales, or the athletes can use the publicity to become known professionals in other areas of life.

Winning sports events can be seen as a consumable good like anything else, and people (and nations) have different preferences for goods. For example, ice hockey is quite liked in the USA and Canada so those countries have a high interest in getting the best players into their national teams, although their domestic supplies are already ample enough. Analogously, in Germany football enjoys high popularity so Germany could be argued to have a high interest in getting good football players into its national team, although here again the domestic supply is high enough. The main point is that if a nation receives benefit from trading something else for better performance in a sports activity, within legal boundaries, it should not trouble other countries. If some countries are willing to make that trade, two countries benefit. There might arise externalities that reduce the utility of other countries, but that can take place whenever people change their home country. Therefore, athletes changing country or nationality should not be seen as a special problem when compared to people in other professions.

As the people in today’s world are connected with each other, and people from different countries work together, the old meaning and purpose of nationality seems less important. Does it then matter, if an athlete has little or no connections to the country he is representing? Companies hire consultants to work for them, mainly to benefit from the consultant’s expertise. Companies also cooperate with celebrities for marketing purposes, paying for the use of a celebrity’s fame in hopes for increased sales. But these consultants or celebrities do not necessarily have any deeper connection to the company that is just one client among others. Likewise, an athlete could treat a country as a client, who buys his services in hopes of increased fame, visibility and financial benefits.

Pursuing talent in sports and business

We often hear about successful companies and their managers, but the home countries of these companies are less often at the center of the view. Countries may of course provide good policies, legislation and infrastructure for companies to flourish, but it is in the end the company that gets the most credit for its own success. Why shouldn’t this be the case with athletes? Furthermore, why shouldn’t countries try to get the best athletes into their teams and athletes go and grasp the best possibilities they are offered?

Countries try to get companies to invest in them, and individuals take jobs in those companies and countries that give them the best prospects. Since an athlete’s job is to compete in sports and, at least to some degree, entertain the audience, why shouldn’t he look for the best possible employer and the best possible country for pursuing his profession? On the other hand, why shouldn’t a country try to improve its success in the field of sports by offering good working conditions for athletes and recruiting talents from abroad? If we still accept that everyone wants a good life and that it is acceptable to maximize the benefits from your profession and if we also add the premise that a good life entails some degree of material well being, the conclusion is that athletes should be allowed to change their nationalities if they are so better off. Likewise, countries should be allowed to offer athletes citizenships. This all of course assumes that the actions take place within the boundaries of the law.

Who gets the spoils?

One piece of criticism against athletes changing their citizenship is the fact that their parents may not be following them, but stay at their, possibly poorer, home country.

But don’t all parties benefit, if a young talented athlete represents a foreign country, is paid well and can use his money to help his parents in his poor home country? This same practice takes place among people who are doing other jobs, coming from poorer to richer countries and sending money back home to help their families.

Arguably the initial media publicity from winning a sports event and the ensuing benefits in reputation and the monetary goods go mainly to the country an athlete is representing, but depending on his contract the athlete may be able to represent his home country later on. And when asked about his background, the athlete is able to promote his original home country, so it is not obvious that the arrangement would not be beneficial to all.

Even if money is not the main question, an athlete is likely to benefit if he is able to train and compete more often with the best and learn from them. Being the only super star in a losing team may not be equally satisfying as being among the middle-class in the world’s best team. In addition, a team may benefit more than the new team member’s individual skill set might suggest, if there are benefits arising from complementary skills between the team members. The audience may also enjoy the following sports events even more when they see more exiting matches or their favorite team becomes more successful. Of course, such changes come at the expense of other teams losing and some athletes not being able to play in the vest teams. Therefore, these externalities would have to be smaller than the added benefits, if we are to claim the change as beneficial as a whole. But in any case I would not claim that athletes changing nationalities and countries going for foreign athletes in hopes of more success in sports is wrong or necessarily counter-productive.

The idea of talented people moving abroad to improve their quality of life is discussed in the newspapers every now and then. Countries have to stay competitive, i.e. offer their citizens the possibilities for a good life, in order to avoid too many talented people moving abroad. If too many move abroad, a country’s capability to offer good opportunities to future generations gets undermined, as the current generation has less talent at its disposal. Another question is, how prevalent such changes are. If most people stay in their original home country, the flow of people between countries should not excessively undermine any country’s future. Quite the contrary, like with international trade, people moving abroad is likely to be beneficial because talents meet, get pooled and are able to combine their efforts more effectively and efficiently. As discussed before, some countries may also have different preferences for different goods and different capabilities for producing those goods. Therefore athletes, and people in general, moving between countries to maximize their potential may not be just acceptable, but also recommendable, for the benefit of individuals and a nation.

Skill or luck – on outcomes, rewards and fairness 2/2

Who gets the prize?

In my previous post I showed how skill and luck play a role in most activities and why it is important to distinguish between the two and their impact on outcomes. I also argued that evaluating outcomes and awarding people only on that basis is not always right. Yet, we often evaluate outcomes, since they are often easy to grasp and measure. However, as already mentioned, if an individual’s or organization’s impact on a specific outcome is negligible in comparison to the impact that luck has, evaluation solely based on outcome is not fair. The individual should rather be evaluated based on his adhering to the agreed process used to reach the outcome, as previously discussed. Also, among top performers luck, paradoxically, plays a relatively larger role in deciding outcomes. Therefore, we should in evaluating performance and providing merit pay attention to the three following points:

  1. Level of personal influence
  2. The process used to reach the outcome
  3. Level of competition

If we have no or very little personal influence on our success or failure, we should not be overly rewarded for good outcomes, nor punished or left completely unrewarded when the outcomes are less favorable. As mentioned before, luck dominated events might be shoved towards the skill dominated end of the spectrum, but this requires honing our skills and executing the event so as to maximize our impact. In this case, even if luck still dominates the outcome, we will have done our best to tilt the scales to our benefit; had we not followed the optimal process, the outcome would have been even worse. This implies that the correct process of working towards a goal and becoming proficient in that process should be merited, although measuring this might be more complicated than simply observing the outcome.

Finally, we should always observe the level of competition. If a person finishes last in the hundred meter dash finals in the Olympics, he is not a failure in sports and hardly beaten by the casual sprinter. Since luck has a larger role in defining the exact outcome at the top level, especially here we should give rewards for effort and process, not for the outcome. Of course outcomes do matter and they should be given merit. After all, if an outcome gets no merit, why would anyone go through the process required to reach that outcome? We just have to pay attention to the division of merit: as the pressure of reaching good outcomes and the desire for the prize get too high, the correct process may not be adhered to anymore, making the achieved outcomes also questionable.

Bonus or layoff, either way deserved?

As a practical example on rewarding individuals in a group of top performers I raise the rank and yank used by some companies to encourage their employees for better performance. The purposefulness of this policy is easily evaluated with the skill-luck model.

With rank and yank I refer to the practice where employees are annually ranked into multiple categories with fixed percentage quotas, and their future career development and remuneration is category dependent. For example, 10% are evaluated as top performers and they are given above average bonuses and promoted to more demanding position, 80% are average and receive the average bonus while keeping their current job and 10% are below average and have to be fired. Here we have at least three problems: role of skill and luck in an outcome, paradox of skill and the correctness of the evaluation. I will disregard the correctness of evaluation from further discussion, since it is a separate topic, but it is obvious that if the results of an evaluation are not correct or are inconsistent with other evaluations, any decision based on that becomes questionable.

As mentioned before, if the outcome of an individual’s work is largely dominated by luck, firing that individual due to poor outcomes hardly seems justified, assuming that the individual has tried to maximize the potential outcome to the best of his abilities. Also, if the individual belongs to a group of talented individuals, luck is bound to have a larger effect on the exact individual performance inside the group, again making the following gratification and firing of employees questionable.

Companies are reviewing and changing their evaluation and reward systems to help their employees learn and improve their skills constantly. For example, General Electric is using an mobile application that enables the supervisor to encourage and give positive feedback for  positive actions while asking to consider changing less desired ones. Here we see how placing the work process under focus might be the way how personal performance is evaluated and remunerated in the future.

Ain’t I lucky being so skillful

As a final thought a few words on the division of well-being on the global level since, here again, it is a question of skill and luck.

Nobody can choose the family and society to which they are born, nor can a person choose the time to which he is born. Yet, based on family ties and structure of the society, the easily accessible life and career paths are limited. A society offering good healthcare, public education for all and a stable government is much more likely to bring about successful individuals than a society plagued with high child mortality, low level of education and turbulent politics. Against this background, it makes me think what is the obligation of the richer countries to assist the poorer ones to reach higher standards of living? What is our obligation as individuals living in the richer countries to help those living in the poorer ones?

If my success is mostly dictated by luck, by having been born in the right country at a good time to the right parents, how much of the ensuing well-being belongs to me and how much should I share with others? I might claim that my success in studies and at work are a result of my own efforts, but they are also based on both nature and nurture, on my education (broadly understood) and genes, both of which are outside factors, or luck. This notion contradicts the definition of skill in the beginning of part one of this post, where I defined DNA and level of proficiency attained through education and practice to be individual attributes. As it seems, drawing a line between personal attributes and external factors is not easy. This makes the final distinction between skill and luck less clear, giving all the more reason for us to think about it and try to reach a fair solution.

Skill or luck – on outcomes, rewards and fairness 1/2

Coming towards the end of my Coursera course on model thinking, I was introduced to the concepts of random walks and skill-luck model. Especially the skill-luck model appealed to me as it can be used to explain success on a personal and organizational level and predict, if success is due to skill or luck and if success is lasting. The following is based on the Coursera course Model Thinking by Scott E. Page and the course readings, mainly on How to Entangle Skill and Luck from Michael Mauboussin at Legg Mason Capital management from July 15th 2010. I have split my text into two parts to make the individual posts shorter.

The skill-luck model

The skill-luck model states that an outcome is based, with varying degree, on both skill and luck. Here skill denotes the knowledge and ability of an individual or organization to steer the future events in to the desired direction. Luck, on the other hand, incorporates the noise, random effects, that affect the outcome and cannot be controlled by the individuals or the organization.

To make the definitions more concrete we could say that the one hundred meter dash is mostly skill based, as the winner is based on the speed of the individual, and the speed is mostly dependent on strength, running technique, reaction time and other things that are either inherited in the DNA or a result of hard training and therefore dependent on the individual.

An example of a pure luck event is lottery. One cannot practice or have since birth any advantage that could help in choosing a winning number (clairvoyance excluded). The winning number is solely based on chance, or luck, as people often say.

A general formulation for the skill-luck model, presented also at the Coursera course by Mr. Page, is

Outcome = A*Luck + (1-A)*Skill

where A is constant between zero and one. The formula shows that, depending on the event, luck or skill may dominate, or they may be of roughly equal importance.

Why use the skill-luck model?

The skill-luck model has relevance on both personal and organizational level. In the society, most people arguably want to be successful, and so do organizations, composed of those very people. Some end up being successful, some not. Sometimes success is lasting; sometimes it barely greets us before already taking part.

Success is often looked up to and merited, while failure may be frowned upon or even punishable. Yet, if success or failure is based more on luck than skill, we should be careful in giving praise and delivering punishment. If success and failure take place randomly, it would imply that the accompanied fortunes and fame are provided without, or due to little, personal merit.

Identifying whether something is luck or skill dominated serves at least five purposes.

  1. Ability to assess outcomes.
  2. Anticipating outcomes and regression to the mean.
  3. Giving guidance where we are most likely to be misled.
  4. Fair evaluation and appraisal, which also promote learning.
  5. More optimal resource distribution.

Assessing outcomes

With the skill-luck model we can assess, whether something is based on skill or luck. A good rule of thumb is, that you cannot lose on purpose in an activity that is dominated by luck, or at least losing on purpose is very difficult. An indicator of a skill dominated activity are lasting streaks. If a person consistently beats others and provides well above average performance consistently, providing that the observed sample is statistically representative, the event is likely to be skill dominated. Not all skilled people have streaks of success, but most streaks are done by skillful people, since the probability of streaks by chance is so low. A good example of a long-lasting streak is the already mentioned hundred meter dash, which has been dominated by Usain Bolt since 2008.

Another method for evaluating the role of skill and luck is to establish the base rate and compare it against actual observations. For example, when examining the role of skill in winning sport matches, the base rate would be the equivalent of a coin toss experiment, when the events consist of teams or individuals going one-on-one. If the group’s variance in the observed performance (for example percentage of games won) is much larger than the variance of the coin toss alternative (winning a game has a 50/50 chance), some contestants do consistently better than odds would have it, and so skill plays a relatively large role in the event. Yet, streaks and clusters can also be random. For example, if we throw a coin long enough, we would expect to get 20 heads in a row. The difference between luck and skill is also in the frequency of such streaks, which are achieved more often by skilled individuals, although they might be helped by some luck to keep the streak going on.

Here, in an MS Excel file,  is an example of a 6-game league between four teams (the method is described by Mauboussin in his paper). Each team plays 2 games against every other team. 1 in the table in the Excel file means that a team won the specific game, 0 indicates the losing team. As we can see from the calculations, skill would seem to play a large role in this specific game, where team 1 has been unbeatable. To ascertain such results in real life, large enough samples (also temporally) have to be ensured, which is not the case with this example, but it illustrates the point.

Anticipating outcomes and regression to the mean

After having identified the role of luck and skill in a specific event, we are more ready to anticipate outcomes of events. If an event is skill dominated, the most skillful are going to man the top positions. If the event is dominated by luck, anyone could win and predicting the top performers, not to mention the winner, becomes difficult. In a skill dominant event it is reasonable to expect that the most skillful will be the best performers, while the consistency of top performance is an indicator of a skill dominant event. This notion is not circular logic, rather it states that first we have to observe, if an event is skill dominant, as we have done in the previous paragraph. After enough observations have been done and the role of skill determined, we can predict, that in skill dominant events current success should be followed by future success, even if we have only few observations from individual performance.

The dominance of luck or skill has interesting consequences for anticipating outcomes. If an event is dominated by skill, it is quite certain that the few best will always be at the top, since the effect of random events, or luck, is so small. However, it is not certain that the absolutely best will win. Since the best can be very close to another in their skillfulness, the outcome is dominated by luck, by the small variations during the event and their effect on personal performance. Therefore, the top three or top five may consist of the same individuals from one event to another, yet the winner may never be the one with the most skill. This is called the paradox of skill. A further implication is that it is not always possible to find the best or most skillful, only the winner of a single event. If we cannot remove the effect of luck completely, small random effects make it difficult to distinguish the differences between individuals, when they are nearly equally skilled. The table below illustrates the paradox of skill. Player 1 is the most skilled, but due to small differences in skill, the least skilled Player 3 wins the game.


If an event is dominated by luck, it expresses the statistical property known as mean reversion: after an extreme event the next event is likely to be closer to the mean. This implies that extremely high or low values are often followed by mediocre results, which are in general more likely to take place. This also implies that if well above average performance does not regress to the mean or does so very slowly, an event is skill dominant.

To understand how regression to the mean can take place within groups while the variance of results in the whole population (consisting of those groups) stays the same, I encourage you to read pages 16 – 19 in Mr. Mauboussin’s paper.

Guidance to prevent us from being mislead

Not seldom are we inclined to draw conclusions from too small samples. For example, if an event is dominated by luck, we might nevertheless attribute the success of individuals to higher than average skill, although the outcome was dominated by luck. Therefore it is important to distinguish between skill and luck dominant events and also ascertain, what kind of tests are required to reach a solid conclusion.

Fair appraisal and evaluation and promoting learning

The roles that skill and luck play determine fair assessment and appraisal. If we are praised or punished for events that occur due to luck, the consequences are unlikely to be seen fair, eventually even morally wrong. An interesting point is that in some cases people are conditioned to being rewarded for giving bad feedback and going unrewarded for giving good feedback. This leads to restraining from praising a good performance, be it due to skills or luck, and calling errors in to attention, be they due to skill or luck. See Daniel Kahneman’s observation in Wikipedia for more information.

Mauboussin states in his paper, quite right in my opinion, that when luck is dominating an event, we should praise people for sticking to a prescribed process and becoming more proficient in the process and its execution. Only this way do we learn, by deliberately sticking to a process, repeating it and honing it. That is a time consuming process and not always pleasant or easy. However, this repetition improves our skills and reduces the role of luck in the considered event, making it more skill based. Therefore we should, as the cliché often goes, praise and give prize for hard work and effort, not necessarily for the good outcome.

Optimal resource allocation

If we know an event to be dominated by skill, we can improve the outcome by allocating more to the disposal of the most skillful ones, assuming the they still have excess capacity to use those resources. For example, two factories produce similar goods, but the other one has a more advanced technology that leads to lower production costs. However, this factory does not have enough capital to invest into more production capacity that is needed in any case to produce all the demanded goods. We might reduce the funding (bank loans for investments into new equipment) to the other factory and use it to increase the capacity in the more efficient one, thus reducing total average production costs while increasing capacity. If both factories would be efficient one day and completely inefficient the other, production thus being a luck dominated effect, such re-allocation of capital would not make sense.

Like in the case of randomly efficient factories, we should not take a bank loan and ask the last week’s lottery winner to play with the whole sum. As lottery is a pure game of luck, investing in a specific individual does not increase the chance of success.

Edit 10.9.2016: Corrected typos.