Social media monitoring tools were once big business for developers marketing to local police departments. In the wake of revelations that those tools were being used to surveil activists and protestors, however, and a change in the social media platforms’ policies to prohibit use of their data for surveillance by law enforcement, K-12 schools became a new target market for this industry. Under pressure to combat school shootings, a number of school districts have purchased social media monitoring products that purport to detect and prevent violence at schools, as well as cyberbullying and threats of self-harm. Original research by the Brennan Center has shown that purchases of monitoring software increased significantly between 2011 and 2018, with the numbers likely to keep rising.
There is little empirical proof that these tools are effective. Social media is often highly contextual and open to misinterpretation. Automated tools have documented failings that are exacerbated when it comes to non-English languages and even American slang. The companies selling these tools have provided only anecdotes of ostensible threats, some of which have proven to be simple misunderstandings while others likely would have been flagged by a concerned parent or peer without the need for broad surveillance. And when perceived threats on social media lead to punitive measures, students of color are disproportionately likely to be punished. At the same time, online surveillance curtails children’s ability to express themselves freely.
In light of these shortcomings, we recommend that school districts not undertake social media monitoring. When they choose to do so, officials must disclose the existence and scope of these tools to parents and other stakeholders, continuously assess their effectiveness, and put measures in place to ensure that they are not deployed in a way that disadvantages Muslim youth, youth of color, and other marginalized students.
