Published: July 22, 2023
To read the incident report with a timeline of the events, please click <a href="/about/reports/2023-07-22-timeline">here</a>.
<h2>Letter to the Community</h2>
by Jay Graber
Last week, users on Bluesky reported an account that used a racial slur as its handle. Within an hour, Bluesky’s moderation team took down the account, as we consider racial slurs to be unacceptable hate speech. We’re sorry that this handle slipped through — we should have had automated filters on user handles. Our Black community deserved better. If we could go back in time, we would have added a filter before bringing on any users. The next best thing we can do is to fix the issue and share a plan for moving forward.
After the account was taken down Wednesday night, the team began working on an automated slur filter. With the help of open-source contributors who had proactively submitted some basic word lists, we implemented a simple filter that night on a noncomprehensive list to prevent slurs in handles. When we merged two incomplete word lists together, it was done through a confusing PR that appeared to remove some ableist slurs, which caused controversy. Open source code sometimes means that incomplete work is assumed to be final by onlookers, and this was such a case. All of the terms that were removed in that PR were added to the final backend filtering system that we ended up with. The next day, we deployed a more comprehensive technical fix that would flag fuzzy matched handles and allow human moderators to take context into account when making decisions. A few days later, we expanded that system to include profile display names, custom feed names, and list names.
People have asked why we didn’t already have a technical solution in place that would prevent this problem. We had a list of handles that were reserved for technical reasons, pulled from a <a href="https://radwebhosting.com/client_area/knowledgebase/196/List-of-Banned-Subdomain-Prefixes.html">list of commonly banned subdomain prefixes</a>, but didn’t set up automated moderation to catch slurs. This was a mistake. Our team initially relied on human systems not out of malice, but because the automated filtering of slurs is a <a href="https://en.wikipedia.org/wiki/Scunthorpe_problem">notoriously tricky problem</a>. The right technical solution was more complex than throwing in a list of banned slurs along with the subdomain prefixes. Slurs need to be caught when they appear within handles, not just when the word is the handle itself. But lots of innocent words or names contain substrings that an automated system would filter out as a slur. The use of slurs is also dependent on context — there are users on Bluesky right now whose handles include reclaimed slurs for groups to which they belong. This reclamation of language is important, but the harm of seeing certain words in a hateful context means that some handles should be forbidden outright. We failed to take into account how important it is to implement some automated systems up-front, even imperfect ones, when leaning on the high-touch human approach to handle moderation.
We’re always trying to balance human-led Trust &#x26; Safety with automated labeling and filtering. In general, we prefer to have humans involved in moderation decisions. The more comprehensive solution for slur filtering we’ve arrived at in the past week is a two-step system for automated filtering and flagging of user handles. This solution combines an automated filter for some handles with a system that automatically flags others for a human moderator to review when they’re created. Keeping a human in the loop will help us take into account context that a purely automated system would miss. We’ll continue to improve this system over time.
On the community &#x26; communications side, we’ve been accelerating some changes we were already working on making. We’ve been consulting with Black community builders to advise us on how to create a space that is safe for marginalized communities, and to help us detect and prevent these sorts of issues in the future. We’ve been growing the Trust &#x26; Safety team. And we’ve been working with communications consultants, since we realize communicating with the community is also an area where we failed here.
Many users expressed dismay at the team’s silence in the wake of this incident. We realize that the silence from our personal accounts could come off as if we didn’t care or were okay with the usage of slurs in handles. This is absolutely not the case, and we apologize for this failure in communication. We’re a very small team that juggles a lot, and though getting an intense volume of feedback and some harassment from users through the app is part of running a social site, it makes doing the rest of our jobs difficult. We’ve started leaning on the support of communications and T&#x26;S professionals, who helped us post updates about last week’s incident from the main Bluesky account at @bsky.app. However, this change in tone was a noted contrast to the way team members casually interacted with the community from individual accounts in the past.
We’ve now created a new account, <a href="https://bsky.app/profile/did:plc:eon2iu7v3x2ukgxkqaf7e5np">@safety.bsky.app</a>, to communicate about Trust and Safety with the community. We plan to be more transparent and communicative from the Safety account, especially in high-profile cases that require either code or policy changes — starting with an in-depth incident report about what happened last week.
Since we reached 100k users, we’ve been <a href="/about/blog/6-02-2023-beta-update">encouraging people</a> to submit moderation reports through the in-app flows, and to file support requests through the in-app form. We’ve been moving this direction because it’s become increasingly infeasible for individual team members to address moderation and safety issues directly with the community from their personal accounts. Every moderation report that you submit in-app is read and handled by a human moderator, generally within 24 hours. We’re developing more structured feedback channels as we grow, but in the meantime, if you have something that you’d like to share with the team, you can do so <a href="https://blueskyweb.zendesk.com/hc/en-us/requests/new">here</a>.
Bluesky is a small team trying to do something big. We’ve made mistakes in the past, and despite our best efforts, we will probably make mistakes in the future. But it’s always our goal to correct those wrongs, and as we work to improve over time, we hope you’ll see that our actions support our intentions. We ask that you give us grace as we figure it out, and trust that we’re working as hard as we can to build a better, safer, more resilient social web.
To read the incident report with a timeline of the events, please click <a href="/about/reports/2023-07-22-timeline">here</a>.

2023-07-22 Letter to the Community

Published: July 22, 2023
To read the letter to the community, please click <a href="/about/reports/2023-07-22-letter">here</a>.
<h2>July 12 Incident Report: Timeline of Events</h2>
The timeline below isn’t comprehensive of every post or report that was involved in the week’s events, but we listed all the events that we think are core to understanding what happened. Please let us know if you feel we’ve omitted important context.
To protect users’ privacy, we don’t explicitly name the handles of Bluesky users that were involved in this situation, but to differentiate them from other involved people, we use letters. To refer to the same person multiple times, we use the same letter.
All timestamps are in Pacific Time, and we provide the timestamp in seconds if available.
7/12/2023 5:27 PM
Bluesky user A posts a screenshot of an account that had a racial slur @***.bsky.social as its handle. This account was created on 6/26/2023. At the time of the screenshot, the account had 4 followers, 3 following, and 1 post.
7/12/2023 5:28:51 PM
Bluesky user B reports @***.bsky.social. This is the first report on the profile of the evening, and is followed by 448 more reports.
7/12/2023 5:37:26 PM
Triage content moderator X escalates the report.
7/12/2023 6:25:16 PM
Escalation content moderator Y takes down the account @***.bsky.social.
7/12/202, 6:42:35 PM
Escalation content moderator Y takes down another account using a slur as its handle @**.bsky.social.
7/12/2023 7:44 PM
A Bluesky community member opens a GitHub <a href="https://github.com/bluesky-social/atproto/pull/1314">PR</a> with a code change: “add slurs to reserved words”
7/12/2023 9:06 PM
A Bluesky team member opens a <a href="https://github.com/bluesky-social/atproto/pull/1318">PR</a> with additional changes to the one that the open-source community contributor made.
7/12/2023 9:38 PM
A Bluesky team member merges the PR. This deploys a quick basic fix that adds a list of banned words for handles.
7/12/2023 10:30 PM
A Bluesky team member merges a <a href="https://github.com/bluesky-social/atproto/pull/1319">PR</a> titled “tweaks and additions to slur list.” The file diff shows that some slurs were removed and others were added. The PR receives many comments asking for explanation. The slur lists were in the process of being moved to the backend, and a team member later posts this follow-up explaining the changes.
<img src="/about/images/reports/2023-07-12/pr-comment.png" alt="comment on a PR" />
7/12/2023 11:37 PM
The Bluesky team posts:
<img src="/about/images/reports/2023-07-12/bsky-app-post-1.png" alt="post from @bsky.app" />
7/13/2023 6:19 AM
An account that was created on 7/1/2023 changes its handle to a racial slur @*********.bsky.social
7/13/2023 6:23:19 AM
Bluesky user C reports the account @*********.bsky.social. This was the first report of this account after it changed its handle, and was followed by approximately 700 more reports.
7/13/2023 6:30 AM
Bluesky user D posts a screenshot of the account @*********.bsky.social.
7/13/2023 6:31:42 AM
Triage content moderator Z escalates the report.
7/13/2023 8:51:29 AM
Escalation content moderator Y takes down @*********.bsky.social.
7/13/2023 8:29 PM
A Bluesky team member merges another <a href="https://github.com/bluesky-social/atproto/pull/1336">PR</a> with a more comprehensive technical fix: “Handle validation improvements.” This creates a two-step system for automated filtering and flagging of user handles. This solution combines an automated filter for some handles with a system that automatically flags others for a human moderator to review when they’re created. Keeping a human in the loop will help us take into account context that a purely automated system would miss.
7/14/2023 11:53 AM
The Bluesky team posts an announcement about the updated community guidelines and Terms of Service that went into effect on that day.
<img src="/about/images/reports/2023-07-12/bsky-app-post-2.png" alt="post from @bsky.app" />
7/14/2023 9:38 PM
The Bluesky team posts a longer thread below:
<img src="/about/images/reports/2023-07-12/bsky-app-post-3.png" alt="post from @bsky.app" />
7/17/23 2:39 PM
A Bluesky team member merges another <a href="https://github.com/bluesky-social/atproto/pull/1351">PR</a> that extends the fix above to profile display names, list names, custom feed names, and any rkeys.
To read the letter to the community, please click <a href="/about/reports/2023-07-22-letter">here</a>.

Reports

Bluesky

Links

Connect