Law in the Internet Society

AI Decision Making in the Workplace

-- By IppeiKawase - 21 Nov 2020


Artificial intelligence (AI) has been used in many ways in a wide range of areas. One of such fields is the workplace. Many employers now have adopted AI to recruit employees, assess their performance, and dismiss them. What issues arise from these changes in the workplace? This essay will explore how AI is used in the workplace, what issues lie in such use, and the legal challenges to address these issues.

How AI is used in the workplace

According to Cambridge Dictionary, AI is defined as “the use of computer programs that have some of the qualities of the human mind, such as the ability to understand language, recognize pictures, and learn from experience.” This illustrates that it is obviously not human but a human-like computer program. How do we, humans, use these computer programs in workplaces? One of such situations is recruitment. For example, employers decide what characteristics of candidates should be evaluated and develop algorism to select candidates who have such preferable characteristics for the employer.

The dictionary definition is misleading. "AI" refers in current parlance to "machine learning," which is not actually learning but pattern matching, finding recurrent similarities in very large collections of unstructured data. Whatever one may think about the potential of "general artificial intelligence," about which I have been skeptical for fifty years, there's no relationship between machine learning applications and AI except hype by people with products to sell. Precision about technology is as important as precision about law or political history in understanding the issues we discuss.

Another example is that algorism itself decides such characteristics based on data which contains information of employees who performed well in the past. These algorisms enable employers to find and select suitable candidates more efficiently.

So long as "suitable" can be defined as "like other people we know already."

Especially in the second example, it is often said that it leads to fair recruitment because it is a computer, not human, that decides the recruitment standard.

"It is often said" is not a substitute for evidence. Who, credibly, says this? What does the extensive literature about implicit discrimination in automated hiring say about this point? What have you read that would convince your reader that there is the slightest foundation for this claim?

These kinds of AI use can be seen in other situations; employers review and evaluate employees to decide their salary or promotion or dismiss them.

We need to distinguish between "decision support" and "making decisions." What is the evidence that HR decisions are automated outputs, rather than decisions made by people on the basis of data and analysis produced by software?

What issues arise from the use of AI in the workplace

In this part, I will explore issues of AI use in the workplace in general and look at the actual example in which such an issue appears in the form of a lawsuit.


The main issue is that the use of AI may lead to discriminatory or unfair treatment of employees, including potential employees. As for the first example of algorism above, because employers decide the characteristics of a desirable person before developing algorism, there is a possibility that such employers decide, intentionally or unintentionally, in an arbitrary way. As for the second example, although there is no human intervention in the process of deciding characteristics, if data on which algorism relied contains biased or limited information, the outcome of the decision is likely to be biased. In this case, people other than the employer cannot understand how the algorism works to conclude unless the employer makes it public and available to others. Also, the possibility is that even some employers who use such technology cannot grasp the mechanism of it, just as ordinary people have little knowledge of how smartphones and the Internet actually work despite heavily relying on them. This difficulty in understanding the mechanism of AI seems more significant than traditional technologies because AI itself has the ability to understand and learn.

What does this last sentence mean? If it is intended as a direct observation about the behavior of software, it's too blunt: machine learning applications "learn" some things but cannot learn others. Consider, for example, an ML application that learns how to beat the best human chess players, but can never learn that chess is a game, or that it is playing. In your example, why don't we talk about the problem of spreadsheets in the workplace? Is that because spreadsheets do not "learn," so we attribute decisions made by people using spreadsheets to people? What about hammers or keyhole saws?

Example - Uber Case

In fact, recently, we have seen the actual lawsuit resulting from such use of AI in the workplace. On 26th October 2020, a UK-based union, App Drivers & Couriers Union (ADCU), which represents four former Uber drivers, filed legal action in the Netherlands against Uber over the use of an algorithm to dismiss the drivers. They argue that the drivers are wrongly accused of ‘fraudulent activity’ as detected by Uber’s AI systems and that the dismissals violate General Data Protection Regulation (GDPR), specifically a provision to protect people from unfair automated decision making.‍

What does the filing of a lawsuit prove?

Financial houses use software to scrutinize the actions of their own traders in order to detect illicit or prohibited trading behavior. When they act against a trader whose behavior was initially discovered by a program, at what point does the intervention of the people who use the tool change the content of the employee's claim from "the program fired me unfairly" to "the bank fired me unfairly," and why does that matter?

What legal challenges are to address issues

So what kind of legal challenges do we face now to address these issues? This part will examine how GDPR regulations address these issues with the example of Uber case above and point out some challenges which GDPR faces.


Article 22 GDPR provides, “The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her.” With regard to the term “based solely,” the European Data Protection Board (EDPB) explains that if human reviews and takes account of other factors in making the final decision, that decision would not be ‘based solely’ on automated processing. The EDPB further explains that you cannot avoid the Article 22 provisions by “fabricating human involvement.” In the Uber case, the plaintiff claims that Uber’s decisions are based exclusively on automated processing of personal data of the drivers and there is no significant human intervention, by showing the fact that the messages sent by Uber to the drivers contain largely standardized and very general texts. In contrast, Uber argues that the activities of the drivers have been assessed by their employees.

A factual issue in which the principles are agreed, correct?


As this case illustrates, I think that at least GDPR provides workers with legal tools to fight against certain types of AI systems adopted by employers. However, although the EDPB notes that “fabricating human involvement” cannot be an excuse to escape from Article 22, the standard for judgment of whether an employer fabricates human involvement is not clear. Considering that there can be human involvement with AI systems to a certain extent, it seems not so difficult to explain such involvement afterward. In addition, the argument regarding the messages sent by Uber seems not so persuasive because it is possible for employers to make their employees involved in the decision-making of dismissal and send messages whose content is the same as those automatically generated by AI.


At this moment, AI has been heavily incorporated into our society, including workplaces, but AI itself has not become independent and there is a certain kind of human involvement to a varying degree. While GDPR addresses legal issues arising from the use of AI in workplaces, we need to further clarify what is a decision made solely by AI, through examining individual cases in the future.

I think the best route to improvement is to consider the fundamental technical and social issues net of the concept "AI," which is not helping you. Under current technological conditions, collecting and analyzing all or substantially all employee behavior on the job is feasible, allowing employers to make decisions on bases previously unobtainable in the discipline of the "industrial army" of 20th century capitalism. Unless the precise details of the employer's analytical process and the software involved in it are immediately relevant, the issues are the same whether the employer uses "AI," "ML," or spreadsheet formulas to decide whom to hire, promote, discipline and fire. Concentrating on the employment law issues themselves, rather than the employer's particular choices in decision support software seems more promising.

You are entitled to restrict access to your paper if you want to. But we all derive immense benefit from reading one another's work, and I hope you won't feel the need unless the subject matter is personal and its disclosure would be harmful or undesirable. To restrict access to your paper simply delete the "#" character on the next two lines:

Note: TWiki has strict formatting rules for preference declarations. Make sure you preserve the three spaces, asterisk, and extra space at the beginning of these lines. If you wish to give access to any other users simply add them to the comma separated ALLOWTOPICVIEW list.


Webs Webs

r2 - 29 Dec 2020 - 15:08:55 - EbenMoglen
This site is powered by the TWiki collaboration platform.
All material on this collaboration platform is the property of the contributing authors.
All material marked as authored by Eben Moglen is available under the license terms CC-BY-SA version 4.
Syndicate this site RSSATOM