|
>
> |
| META TOPICPARENT | name="FirstPaper" |
|---|
It is strongly recommended that you include your outline in the body of your essay by using the outline as section titles. The headings below are there to remind you how section and subsection titles are formatted.
She Fell in Love with ChatGPT? . Her Students Were Already There.
-- By OrnaMadigan - 25 Mar 2025
Intro
-She Fell in Love with ChatGPT. Like Actual Love. With Sex. Despite sounding like clickbait, this is the very real title of a recent episode of The Daily, featuring Ayrin, a woman in a genuine sexual relationship with ChatGPT? . At first, the story feels easy to dismiss, until 20 minutes in, when shockingly we learn that Ayrin is not an outlier. A current elementary teacher reports that in her experience “3[%] to 5[ %] of the class,” have AI partners. While many responded with shock, for myself, a former teacher with 2 years of under-funded, over-crowded baggage it felt inevitable. Three years on, the whir of Chromebooks and ping of Google Classroom alerts, still haunt me. Even then, it was palpable that we were rushing to digitize the classroom with no idea the repercussions. Three years later, my worries have only worsened.
Still, even now, I feel unsure how to confront the problem, individual cases feel too narrow, sweeping movements too ambitious. And so, sitting down to write, I focused on what I do know: I know that the insidious surveillance practices adopted under the guise of Covid need to be dismantled, and fast. I also know that parents are a party with unique leverage who have remained oddly silent. While sweeping First Amendment challenges may seem idealistic, I see them as a powerful future tool worth considering, even if only to provide ammo for advocates.
School Surveillance
While I have previously outlined the growth of School Surveillance in previous posts for this class, I provide a brief summary below.
What Is It?
The Covid-19 pandemic forced schools to shift to fully remote learning overnight. Into this abyss emerged digital learning platforms, promising solutions. Platforms like Google Classroom, and Nearpod enabled teachers to assign virtual worksheets, run virtual class activities, and overall maintain a sense of normalcy. Schools scrambled to keep up, distributing Chromebooks, Wi-Fi hotspots, and student accounts. However, as schools transitioned into a post-Covid world, the perceived efficiencies, and financial investments, led many to turn these bandaids into pillars of in-person learning. But with these efficiencies came new risks–cyberbullying, child pornography, unfettered access—prompting schools to expand contracts with surveillance platforms to ensure round-the-clock monitoring of any data accessed via school wifi, devices, and even school-connected accounts.
Should We Be Worried?
Yes. As school-surveillance systems gain ubiquity, its normalization follows suit, fostering a disturbing acceptance of privacy invasions, and leading to widespread apathy towards the handover of personal digital data. Moreover, as demonstrated by recent scandals even sophisticated tech platforms are struggling to safeguard user data. Yet, we are allowing these platforms to collect and store vast amounts of students’ sensitive data, despite a documented rise in school-targeted cyberattacks. As well, these platforms are often using under-trained, low-paid human reviewers, that are given access to student’s “lengthy chat logs…nude photographs and, in some cases… students’ names,” daily. Surveillance shapes not only the adults students become, but puts them at grave risk of exploitation now.
But Doesn't It Protect Students?
Even more striking is how the magnitude of these risks contrasts with the limited evidence that these platforms actually improve student safety. While Gaggle claims it has “prevented 722 students from committing suicide,” this number is impossible to corroborate. In reality, only a few schools have actually cited Gaggle in halting threats. According to one school district, more than 83% of the items flagged were minor violations, such as a file named “Odyssey Essay” with the word “bastard” in it. Among those flagged as major violations, were at least a dozen flagged for the use of the word “gay.”
The First Amendment Approach - Gaggle A Case Study
While I’ve previously focused on teacher and student resistance to surveillance, this paper centers on parents—partly inspired by the buzz around Netflix’s Adolescence. Parents not only have the motivation but also the power to drive change. Yet many, who personally scan their own children’s texts, have bought into the safety narrative. Alerting and empowering them will require a multi-pronged approach—op-eds, TV, podcasts—but a First Amendment challenge could play a vital role not only in bringing the issue to the forefront, but also showing a mode of redress. Moreover, the parallels with First Amendment precedents may help make these issues more accessible to a generation of parents who struggle to keep pace tech’s rapid advances
What is Gaggle?
Although the extent of surveillance differs by platform, for purposes of First Amendment discussion, I will use Gaggle. Gaggle, is a subscription based 24/7 student-activity monitoring platform that integrates with student’s Microsoft 365 and Google G Suite accounts. Once a school district contracts with Gaggle–$162,000 annually–Gaggle utilizes “keywords, algorithms, and machine learning to identify content that indicates students planning self-harm, bullying, abuse, or school violence.” Gaggle scans students' emails, chat messages, photos, and any other content uploaded to their Google Drive or Microsoft accounts at any time. Additionally, although Gaggle claims not to access students' social media accounts, such as Instagram or Tik Tok, if the student has signed up for their social media account using a school issued email, it is then privy to any email alerts or snippets sent by the platforms. When Gaggle identifies “problematic” material, it sends the data to one of the 125 human reviews, who then determine if it should be elevated to school officials or in emergencies, generally after-school hours to law enforcement.
The First Amendment and Student Speech
Student free speech in schools is a complex issue—students have constitutional rights, but schools also require authority to maintain order and safety. Fortunately, the Supreme Court is no stranger to defining these bounds. In 1969 the court decided the seminal case Tinker v. Des Moines, holding that students do not “shed their constitutional rights to freedom of speech or expression at the schoolhouse gate.” But it wasn’t until 2021 in Mahanoy Area School District v. B.L. that the Court addressed student speech in the digital age—offering a framework for surveillance platforms. In Mahanoy, the Court ruled in favor of a student who was suspended from her high school cheer team for the contents of a Snapchat posted on her personal account after school hours. There, the Court identified three factors that schools must consider when determining whether they can regulate off-campus speech: (1) whether the speech falls within the zone of parental responsibility, (2) whether the regulation would leave students with no opportunity to engage in the speech at all, and (3) the school’s interest in protecting unpopular expression. Applying these 3 factors to Gaggle it quickly becomes evident that it cannot pass muster.
In Loco Parentis
While it’s true that the internet offers broader and harder-to-regulate avenues for exploration and communication than, say, a trip to the library, this reality shouldn’t obscure a key point: when students are browsing the web or chatting with friends online during their free time after school, it’s their actual guardians, not school officials, who are tasked with protecting, guiding, and disciplining them. While schools may argue that students’ outside activities impact their behavior during school hours, that does not give them the right to reach into students’ homes or claim superior authority over parents in guiding and monitoring their children. While courts may recognize schools in loco parentis position during the school day, they have drawn the line at the final bell.
24-Hour Regulation
Because Gaggle’s surveillance is constant, it creates a chilling effect on students’ freedom of speech. While this burden falls heaviest on low-income students—who rely most on school-issued devices to access the internet—the impact extends to all students. The ever present fear that any message or action taken online could be privy to the prying eyes of school officials instills a fear in all students, and it’s not necessarily just for risky activities. Schools’ experiences with Gaggle reveal an alarmingly high rate of false alerts, sending a message to students that anything they say or do could be misinterpreted and reported—fostering a constant state of anxiety and self-censorship. A reasonable counter-argument might suggest that students can simply opt out of using their devices and interact with peers in person instead. But in today’s deeply digital world, that’s not a real choice. Opting out online means being cut off entirely. This tension is only heightened by the fact that schools themselves require students to participate in online spaces, further blurring the line between optional and mandatory.
Unpopular Speech
Although recent events on our own campus might cast doubt on the claim from Mahanoy that “America’s public schools are the nurseries of democracy,” it remains the guiding precedent set by our courts. The whole point of school is to challenge students to explore and learn. Stemming the free flow of ideas greatly stumps intellectual curiosity and pigeon holes students from being free thinkers. Moreover, over the past decade, there has been a drastic change in the way students participate in political and social dialogue. For instance, as noted during the recent election cycle, 70% of young people stated they received information about the 2020 election from social media. Long gone are the days of neighborhood rallies, and here to stay are dedicated instagram pages, chat-rooms, and reddit threads. Yet, Gaggle routinely flags political posts, LGBTQIA+ language, and other expressions, even when there’s no threat or disruption. It’s evident that Gaggle poses a barrier to open debate and hinders the development of a well-rounded, informed citizenry capable of engaging with differing viewpoints and working collaboratively toward the common good.
Conclusion
Listeners of The Daily may wonder how we got here, but the answer is hiding in plain sight. While the factors driving students toward AI relationships are complex, our school systems offer unsettling clues. Many parents feel unprepared to understand AI, but that shouldn't stop them from advocating for their children’s rights. As Mahanoy suggests, you don’t need to follow every tech trend to know when a line’s been crossed—and even a brief look at Gaggle’s harms makes that clear. The solution therefore is twofold: raise awareness and arm parents with tools—like First Amendment arguments—to fight back.
You are entitled to restrict access to your paper if you want to. But we all derive immense benefit from reading one another's work, and I hope you won't feel the need unless the subject matter is personal and its disclosure would be harmful or undesirable.
To restrict access to your paper simply delete the "#" character on the next two lines:
Note: TWiki has strict formatting rules for preference declarations. Make sure you preserve the three spaces, asterisk, and extra space at the beginning of these lines. If you wish to give access to any other users simply add them to the comma separated ALLOWTOPICVIEW list. |
|