In today’s gaming landscape, communication plays a crucial role, and when players are given the chance to chat, their language isn’t always family-friendly. Roblox, a well-known online game creation platform, has long utilized simple censorship methods to filter chat content. Traditionally, this meant replacing inappropriate words with a series of pound signs. However, Roblox is now integrating artificial intelligence to intelligently rephrase offensive messages instead of merely censoring them.
“Utilizing AI filters to block inappropriate text has always been pivotal to our safety measures and fostering a respectful environment,” stated Rajiv Bhatia, Roblox’s VP of User and Discovery Product, in a blog post. “Today, we’re introducing an additional feature to enhance the civility of in-game text chat: we’re employing AI to automatically rephrase messages, starting with profanity.”
Now, when a player types an inappropriate term, the AI will automatically revise the message before it appears in the chat. For instance, “oh shit, are you OK?” would become “are you OK?” Similarly, “hurry tf up” will be transformed into “hurry up.” While the latter may not be explicitly stated, Roblox is well aware of its implied meaning.
Everyone in the chat will notice that the AI has rephrased the original message; however, only the person who sent it will receive a warning regarding their use of improper language.
This update comes just two months after Roblox introduced facial recognition, a measure designed to categorize users by age to minimize interactions between children and adults. This, however, has not prevented legal challenges. Nebraska recently filed a lawsuit against Roblox, citing alleged failures in ensuring child safety, following similar actions from Texas, Florida, Louisiana, Kentucky, and various localities.
Users who send inappropriate language will be informed that their message was rewritten by AI and reminded to adhere to community standards.
Evolution of Language
With the creation of new slang constantly evolving, much of it can be deemed unsuitable or offensive. Roblox reveals that the AI is trained using a set of specialized models derived from a broader framework, incorporating in-game chat samples to keep pace with evolving language. When new slang emerges and the AI is uncertain whether it is inappropriate, it escalates the evaluation to a more advanced AI model, which deduces its relevance based on context.
Players who repeatedly try to go around the new safeguards or persist in using banned terms will face penalties as per the game’s Community Standards policies.
While this new system may still experience occasional errors and will gradually adapt to new slang, Roblox remains optimistic about its effectiveness. “Our trials have demonstrated that this comprehensive approach has significantly enhanced our filtering capabilities,” Bhatia explained. “The filters are now more adept at recognizing leet-speak, where letters are substituted with numbers or symbols, as well as more nuanced attempts to evade our restrictions.”
