Posted in Essays on November 12, 2020 23 min read
Presumably, since the emergence of humans, the notion of the infallible has existed. Many a time, it took the form of superhuman beings. Until today, a variety of religions have come and passed, but one relatively recent phenomenon – according to cosmological timescales – stands out the enlightenment. It was then, that societies de-deified nature and aimed to grasp it through reason and control it through science, reconceptualizing all animate beings as inanimate subparts, that could be controlled and changed at will (Guyer, 2015, p. 155). Out of this reconceptualization of the world emerged technology. In due time, its capabilities developed exponentially and demonstrated their capacity to create ever-more utility. Today, they have arrived at a point where, for instance, machine learning has the capacity to create humanly inapprehensible models that have extremely high predictive possibilities. Simultaneously, brain imaging technology has emerged that can predict actions before they arrive in an individual’s own consciousness. In due time, the notion– currently going from strength to strength – developed, that technology and its underlying rule of operation, logic, could be infallible. The notion that God is dead has not reached the ears of all men, today, many believe that:
[t]o the extent that ethics is a cognitive pursuit, a [technological] superintelligence could do it better than human thinkers […] that questions about ethics, in so far as they have correct answers that can be arrived at by reasoning and weighing up of evidence, could be more accurately answered by a superintelligence than by humans’
(Bostrom, 2003, p. 3)
In other words, the idea that Artificial Intelligence (A.I.) could end the search for justice and morality, and hence govern humans better than themselves. Whereas we are still far from such extreme measures as declaring an A.I. as sovereign, the steppingstones towards it are being laid. Slowly, technologies such as the FAST system are making their way into the arsenal of law enforcement offices. Their central tenet is to avert damage, by detecting crimes before they happen. In this particular case, the system enables officers to detect potential terrorists by means of their vital signs, body language, and other physiological patterns (Furnas, 2012). Yet, the consequences of such developments for our systems of law, justice, and morality often remain overlooked as engineers around the world press ahead to develop technological solutions to our most fundamental disputes of social organization.
Meanwhile, one need only to look into the past, where different authors have explored imaginary futures corresponding to these developments. Ranging from Orwell’s dystopia of 1984, which reflected on the effects of Big Brother surveillance to Philip K. Dicks short story “The Minority Report”, and its film adaptation “Minority Report” by Steven Spielberg, which deal with the complexities of thwarting crime before it happens. Specifically, in the latter, a crime-unit with the ability to predict crime has been created whose task it is to detain criminals before they themselves may even know that they will commit a crime. These works of science fiction allow us to consider the effects of such interventions before they reach maturity in the real world. Hence, they can allow us to change or stop such developments preemptively (Wood, 2018, pp. 4).
That being the case a specific entry into the collection of potential justice futures deserves our attention, “Psycho-Pass”. Written by Gen Urobuchi and Naoyoshi Shiotani, this series depicts an ambivalently dys/utopian Japan ruled by an advanced technological entity capable of predicting crimes. As a work of hard science fiction, it tries to stay faithful to the laws of nature and avoid committing scientific errors (Wood, 2018, p. 3). Psycho-Pass is an anime, meaning it is hand drawn and computer animated form of visual media deriving from Japan, characterized by certain visual aesthetics such as enlarged eyes. This near-future post-humanist cyberpunk crime fiction work of art, presents its plot in noir style in the vein of cultural staple “Ghost in the Shell”(1995) and draws on western movies such as Blade Runner (1982), Gattaca (1997), and Brazil (1985) (Nakamura, 2017, p. 127). The aim of this essay is to utilize the world illustrated in Psycho-Pass to infer about the effects of techniques of pre-emptive justice such as the FAST system on society. For this purpose, the world of Psycho-Pass will be explained and interpreted. Thereafter, different moments of the show and the implications they carry will be highlighted in isolation.
Psycho-Pass is set in a Japan of the 22nd century, where “one’s mental state and personality can be measured into numbers. Every emotion, desire, social deviation, and mental inclination is put into record […] This value that determines the standards of the human mind, even an individual’s soul itself, […] is known to the public as ‘psycho-pass’” (Sai & Gotou, 2017, pp. 6). This amalgamation of information occurs through the deployment of a mass surveillance and welfare system called the “Sibyl System”. With the help of psychometric scanners, installed in all populated areas of Japan, the system continually assesses citizen’s mental states and consequently assigns each a “Psycho-Pass”, a hue as a visual representation of their mental condition. The more mentally ill or stressed a person is, the increasingly opaque their Psycho-Pass becomes. Additionally, through its capability of assessing citizens biological make-up, the system also determines their work prospects, assigning everyone a job that fits their talents and leads them towards happiness.
However, the show’s focus centers on another responsibility of the Sibyl System, that of maintaining the social order. In this regard, the system continuously evaluates immediate and future risks to the social order and tries to eliminate them in advance. In this endeavor, again, it gauges citizens emotions and thought processes to determine whether they are inclined to commit a crime. Subsequently, it assigns them a “criminal coefficient”, another component of their algorithmic identity, which quantifies their propensity to commit illicit or harmful behavior. In case this measure exceeds a certain point, it calls upon the members of the Criminal Investigation Department, tasking them to apprehend the individual before it has the opportunity to commit a crime, or, if their crime coefficient is high enough, executing them on the spot (Wood, 2018, p. 7).
The majority of the main characters of the series forms part of the Criminal Investigation Bureau and assume either of two roles: enforcers or inspectors. Enforcers form the powerhouse of the department. These all have high Criminal Coefficients and have in the past been removed from society, only to be brought back for the sake of finding and eliminating criminals like themselves. They are assigned this job so that normal citizens do not have to do such gruesome work that could worsen their mental health. The enforcers face serious consequences for transgressing orders and must not act without the approval of their superiors, the Inspectors. These, in turn, are taken from society’s elite and have more stable personalities resulting in their low Crime Coefficients. Both make use of “The Dominator”, a futuristic gun that directly communicates with the Sibyl System to assess an individuals’ Crime Coefficient, and from it arm the gun in one of three modes: unarmed, paralyze, lethal eliminator, according to the level of the coefficient (Nakamura, 2017, p. 126). Insofar, the gun becomes a direct expression of the Systems will and partially unburdens the CID agents from the moral choice of shooting. The Sibyl System acts as the judge, jury, and executioner.
In the west where individuality is a primary virtue, literature and film would immediately seek to characterize such a system as oppressive, as it constrains individualism. However, probably due to its Japanese setting where conformity is celebrated, the series never paints society under the Sibyl System to be unacceptable. Rather, it celebrates and forces us to consider its merits. In fact, many people are shown to have as much autonomy as they wish for; they are able to have children, quote literature, make music, and even pursue subversive discourse. Even when the main character Akane Tsunemori, questions the Sibyl System itself, no one seeks to punish her for it (Sakurai). It is exactly this ambiguous treatment, that invites a nuanced discussion of the show.
In Psycho-Pass the pre-emptive justice administered by the Sibyl System allows for society to be almost perfectly secure, as all threats to the social order are eliminated in advance. However, the question is whether its surveillance, security, and repatriation measures make for a free or oppressive society remains and can be considered from two paradigms:
According to Booth’s (1991) conception of security, pre-emptive justice, to be equated with perfect security, emancipates us. This he argues on the basis of emancipation being ‘the freeing of people […] from those physical and human constraints, which stop them from carrying out what they would freely choose to do’ (Booth, 1991, p. 319). In this sense, insecurity would signify the existence of certain threats towards the self, which keep one from engaging in specific behaviors. Security, on the other hand, is the absence of such threats that disinhibit our desires. In this view, the society of Psycho-Pass is approaching the apex of freedom, for crimes are prevented, security is absolute, and people are fully emancipated.
However, the approach of “governmentality” towards this perfect security gives a contrasting picture. “Governmentality” is a Foucauldian concept and refers to how populations are managed. To elaborate, it refers to how the relationships of power between liberty and domination are negotiated. Or in other words, it stands for “the conduct of conduct” (Burchell, 1991, p. 2), which is an activity that shapes the conduct of some persons by allowing them to act within a more or less open sphere of possibilities. Governmentality, then, is engaged in defining that sphere of possibilities (Foucault, 1983, p. 220-221). However, not only in the realm of institutions but even on the level of individuals and their relations with things. The most illustrative to me seems Foucault’s own explanation:
What does it mean to govern a ship? It means clearly to take charge of the sailors, but also of the boat and its cargo; to take care of a ship means also to reckon with winds, rocks, and storms; and it consists in that activity of establishing a relation between the sailors […] and the ship […] and the cargo […] and all those eventualities like winds, rocks, storms and so on.
(Foucault, 1991, p. 93-94)
This is to say, that the relationship of people and things towards eventualities are constitutive of the security of a population. Therefore, increased security means increased regulation of people and things, hence less freedom. This is to say that increased security comes at the price of constraining how people and things circulate.
In this light, the Sibyl Systems removal of potential threats to the social order, be that in the form of treatment or elimination, is a double-edged sword of security and oppression. To further illustrate, the Sibyl system in this imaginary Japan with its predictive capabilities prevents certain eventualities, that it judges to be a threat to the social order. Thereby, it impedes citizens’ experience of that range of eventualities, whether that is being the victim or the perpetrator of a crime. In the same vein, it gives people the security to have and be good at their jobs, by dictating them to pursue specific jobs. In essence, the social order is maintained and improved by limiting what is acceptable, and the Sibyl System’s Psycho-Pass and Criminal Coefficient, are the primary means of doing so.
Despite the Sibyl’s systems efforts of producing a healthy and secure society, the anime shows people who remain excluded. The most prominent contradiction to the social order are the enforcers themselves. As the anime progresses, the viewer learns that some enforcers do, in reality, not possess the will to engage in criminal actions. In this category falls the lesbian enforcer Yayoi Kunizuka. Her increased criminal coefficient, one learns, was never due to criminal inclinations, but rather a result of her unusual lifestyle as an artist in the underground music scene. This is a fascinating revelation, for the fact that it signifies, that the manner in which the Sibyl System determines criminal risk may rather depend on the intelligibility of a person and its corresponding predictive ability. That is to say, that conforming average people, more readily intelligible, who do not represent a difference may enjoy a lower criminal coefficient. In essence, it suggests that her independent art is an unwelcome opposition to the uniform mass culture that the Sibyl-society has created.
This development points to a major criticism of preventative justice and the Sibyl-society, that is best illustrated by relying on Mouffe’s (2013) criticism of closure theories. That is the notion, that theories or actors that proclaim truths, merely exclude and hide opposing forces from the visible terrain, thus creating an antagonism wherein the hegemonic actor denies the identity and seeks the destruction of the other (Mouffe, 2013 p. 3-7). In terms of the anime, we can say that exactly such a complication arises from the fact that neither happiness nor justice are essential categories. To elaborate, an unjust action at one time may be seen as a just action in another time. Even though the Sibyl System works under a single paradigm with essentialist categories, we are faced with the fact that the average citizen is completely content with their life. Consequently, almost all citizens believe that the Sibyl System is infallible, and always knows the best course of action. Yet, it needs to be recognized that the infallibility of the Sibyl System, is in effect, only a product of its exclusion and absorption of those that do not fit into its paradigm. For being different leads to a clouded psycho-pass and a high criminal coefficient, and the threshold at which individuals are determined to be criminal is a relation of power, necessary for the continued operation of the system as it can only achieve happiness for the selected majority. More importantly, the Sibyl System justifies itself to the “civilized” majority of good citizens, exactly by closing out “barbaric” criminals. On the question, whether through extermination or incorporation of these outsiders the system can ever be all-encompassing and provide for everyone, we must consider the notion, that differentiation from any hegemonic social order is continual and contingent. In fact, a battle with the outside must never be stopped, as it is the justification and is constitutive of the stability of the internal social order.
In the opening episode of the series, the new inspector Akane Tsunemori engages with her first case: A man has previously been identified to have an elevated Criminal Coefficient. When the Sibyl System asks him via a hologram to undergo ’emergency therapy’, he does not believe that he would be released again. Knowing that his arrest is inevitable and wanting to enjoy himself before his end, he abducts a woman, rapes, and tortures her. When the Inspectors arrive, his criminal coefficient has risen to the level of lethal elimination and they proceed to shoot him with the dominator, after which his body explodes, splattering blood around the room and on the woman he was raping.
In this instance, and many others in the series, character’s did not have criminal intentions. Rather, it was them being labeled as criminals by the Sibyl system that hastened their descent into criminality. The notion of such a self-fulfilling prophecy is not alien to criminal theory, however, in this case, a machine did the labeling. Whereas, we could imagine a human being able to consider another person in what Buber (1958, p. 3-34) called, an I-Thou relationship. Meaning, a relationship wherein one considers another being in terms of an indivisible unity that stands in relation to the self and is therefore interpreted contextually. A machine, on the contrary, lacking all forms of empathy can only conceive of an individual in an I – It relationship, seizing and measuring their characteristics and how they factor into a dependent variable or a specified form of utility. Therein, it measures, reifies, and reduces dynamic phenomena to create a data object that represents the threat the individual poses. However, “the criminal” in an I – Thou relationship, is different from a criminal as an I – It data object. The former may only engage in criminal acts under specific circumstances or may not be a criminal according to legal or political standards. Yet, the data object cannot make distinctions, it is either positive or negative. Considering, the example of someone who steals something back, that is originally their property but had been stolen previously, we can clarify that distinction. According to the data-type, he is a criminal, whereas according to legal standards, he may not be (Cheney-Lippold, 2017, p. 51). At the example, of Psycho-Pass we may say, that had this man’s algorithmic identity not superseded his authentic identity, he may not have been labeled and descended into criminality.
In one of the most powerful moments of the series, the inspector Akane Tsunemori, stands face to face with the villain of the show, Shogo Makishima. Shogo is holding a knife to the throat of Tsunemori’s friend, threatening to slash it. As Tsunemori tries to shoot him with the dominator, it will not let her, because his crime coefficient remains abnormally low. We later learn that this is because he is “criminally asymptotic”, a rare human that cannot be grasped accurately by the system. Makishima slides the inspector an ordinary armed rifle saying, “If there is someone who can judge me, it’d only be those who can become a killer based on their own will.” Following his statement with: “Now you feel the importance of life in your index finger, don’t you? That’s the weight of a decision under free will.” Despite making his intentions clear Inspector Tsunemori cannot bring herself to kill Shogo Makishima for she cannot believe that the Sibyl System could be wrong. Consequently, he slashes the woman’s throat and leaves.
In an uncommon twist, it is the villain who affirms the humanist values that the viewers are likely to hold. What Makishima laments here, is that pre-emptive Justice judges’ people by their propensities and characteristics rather than their authentic inclinations. This absolutely deterministic outlook makes humans into mere livestock, that is to be discarded when it is corrupted. Rather, he favors a Kantian notion, that gives individuals the ability to be moral, precisely by giving them the chance to act against their inclinations (Guyer, 2014, p. 218-220). However, in a society of pre-emptive justice, individuals never get the opportunity to oppose their inclinations and become moral beings. In his view, this disregard for the right and its emancipatory horizons makes the world meaningless. The good alone cannot make life worth living.
Throughout, the series, the Inspector Akane Tsunemori, had continually affirmed the goodness of the Sibyl System and believed it to be superior to other forms of the law, for while it may give false positives from time to time, it remains an objective arbiter. However, at the end of the first season, that conception is shattered, as both Makishima and Tsunemori enter the lair of the Sibyl System and find it to be a wetware hivemind of people whose brains have been surgically removed and connected in a special container. Where, Makishima, notes that it was its supposed impartiality was the basis of its trust:
What a farce. They promised a society managed by fair and impartial machine intelligence. A law free of petty human ego. People only accepted Sibyl because that’s what they thought they were getting. But it was a lie – Sibyl’s arbitrary and unfair. It’s nothing but a committee of pickled brains in mason jars.
(Urobochi, 2013)
The Japanese only accepted the Sibyl System with its pre-emptive justice under the assumption that it would be impartial and therefore good. If they’d known that this was not the case, they would have disavowed its technological tyranny. This episode serves as a cautionary tale, that technology can never be impartial and technological solutionism is incapable of solving morality. Rather, the contemporary notion of technology as an impartial tool, veils the fact that “technology is society made durable” (Latour, 1990, p. 103). That is, technology is always constructed according to human intentions and tables of values. That is to say, it will perpetually favour one thing over another according to the programming we have given it. Processes and algorithms only further and favour the goals we have furnished them with. To conclude, every law has to cast itself as impartial, for us to consider it legitimate. Yet, no law can achieve that standard, and technology cannot be the solution either.
In Psycho-Pass, Akane Tsunemori is confronted with one last decision: to tell the truth about the Sibyl System and threaten the social order or to remain quiet under a biased system. In a Hobbesian moment, she decides on the latter, as she believes the Sibyl System to be generally good, whilst vowing to find a better solution.
The verdict on technological pre-emptive justice is as follows. Indeed, one could say pre-emptive justice would make the average person freer by removing threats to its safety. Yet, there is no entirely average person, and therefore, everyone’s freedom is limited as this form of governmentality defines and narrows the sphere of the admissible. In fact, happiness and justice can only be created for a civilized, to a certain extent homogenous inside sphere that shares the same conceptions of these goods. Those who live independently, according to their own values will invariably be punished for their lifestyle, as the technology fails to accommodate them in its framework. The judgement of technology too must be considered separate from human judgement, since the guiltiness of a data point, does not necessarily align with its corresponding individual. Indeed, accepting technological pre-emptive justice entails acknowledging the meaninglessness of the world and discarding our claims of human dignity, virtue, and vice. Yet, a preference for this must be left to personal deliberation. Ultimately, the valuable insight remains, that despite the ethos of the century to naturalize technology as neutral, it is to the contrary always imbued with the partialities of its creators.
References
Booth, K. (1991). Security and emancipation. Review of International studies, 17(4), 313-326.
Bostrom, N. (2003). Ethical issues in advanced artificial intelligence. Science Fiction and Philosophy: From Time Travel to Superintelligence, 277-284.
Buber, M. (1958). I and Thou, trans. Ronald Gregor Smith. Edinburgh, T & T
Burchell, G., Gordon, C., & Miller, P. (1991). The Foucault effect: Studies in governmentality.
Cheney-Lippold, J. (2017). We are data: Algorithms and the making of our digital selves. New York, NY: New York University Press.
Foucault, M. (1983). Beyond structuralism and hermeneutics: The subject and power, after word. I Hubert L. Dreyfus & Paul Rabinow (Red.), Michel Foucault: Beyond Structuralism and Hermeneutics: The Subject and Power (pp. 208-226).
Foucault, M. (1991). The Foucault effect: Studies in governmentality. University of Chicago Press.
Furnas, A. (2012, April 17). Homeland Security’s ‘Pre-Crime’ Screening Will Never Work. Retrieved from https://www.theatlantic.com/technology/archive/2012/04/homeland-securitys-pre-crime-screening-will-never-work/255971/
Guyer P. (2014). Laws of freedom: the foundations of Kant’s moral philosophy. In Kant (pp. 201–241). London: Routledge.
Latour, B. (1990). Technology is Society Made Durable. The Sociological Review, 38(1), 103-131. doi:10.1111/j.1467-954x.1990.tb03350.x
Mouffe, C. (1996). Democracy, Power and the “Political”. In S. Benhabib (Ed.), Democracy and Difference. Contesting the Boundaries of the Political (pp. 245-256). Princeton: Princeton University Press.
Mouffe, C. (2013). Agonistics. Thinking the World Politically. London & New York: Verso.
Nakamura, M. (2017). Emancipation in postmodernity: political thought in Japanese science fiction animation (Doctoral dissertation)
Sai, N. & Gotou, M. (2017). Psycho-Pass: Inspector Shinya Kogami: Volume 1. Milwaukie: Dark Horse Manga.
Sakurai, J. (n.d.). Psycho-Pass’s Ambiguous Dystopia (Part One). Retrieved from https://www.byline.com/column/12/article/139
Sakurai, J. (n.d.). Psycho-Pass’s Ambiguous Dystopia (Part Two). Retrieved from https://www.byline.com/column/12/article/167
Urobuchi, G. (Writer), & Shiotani, N. (Director). (2012, November 2). Nobody knows your mask [Television series episode]. In J. Wada, K. Tobori & W. Okamura (Producers), Psycho Pass. Tokyo: Fuji TV.
Urobuchi, G. (Writer), & Shiotani, N. (Director). (2012, December 21). The saint’s supper [Television series episode]. In J. Wada, K. Tobori & W. Okamura (Producers), Psycho Pass. Tokyo: Fuji TV.
Urobuchi, G. (Writer), & Shiotani, N. (Director). (2013, January 25). Sweet poison [Television series episode]. In J. Wada, K. Tobori & W. Okamura (Producers), Psycho Pass. Tokyo: Fuji TV.
Urobuchi, G. (Writer), & Shiotani, N. (Director). (2013, February 1). The town where sulfur falls [Television series episode]. In J. Wada, K. Tobori & W. Okamura (Producers), Psycho Pass. Tokyo: Fuji TV.
Urobuchi, G. (Writer), & Shiotani, N. (Director). (2013, March 8). Where justice is found [Television series episode]. In J. Wada, K. Tobori & W. Okamura (Producers), Psycho Pass. Tokyo: Fuji TV.
Urobuchi, G. (Writer), & Shiotani, N. (Director). (2013, February 15). Iron heart [Television series episode]. In J. Wada, K. Tobori & W. Okamura (Producers), Psycho Pass. Tokyo: Fuji TV.
Wood, M. A. (2018). Algorithmic tyranny: Psycho-Pass, science fiction and the criminological imagination. Crime, Media, Culture
Previous Next
Recent Comments