Xbox's new voice reporting system is designed for "a world that keeps changing"

Between the veil of anonymity and lack of accountability, in-game voice can be a minefield, especially when we interact with strangers. Even with the best will in the world, moderating these spaces is a near-impossible task, but Xbox looks as though it has come a step closer to achieving just that. 

Revealed on June 12, Xbox has launched a new safety feature for Xbox Series X|S and Xbox One which will enable players to report voice activity that breaches community standards. The feature allows players to capture 60-second video clips of in-game voice incidents and is rolling out to Alpha and Alpha Skip-Ahead Xbox Insiders this week. 

To get an idea of the thinking behind the new feature, as well as how it works we spoke with Xbox corporate vice president Dave McCarthy. 

“There’s basically a buffer running in the background”, said McCarthy, explaining how the system will allow players to record voice activity that’s already taken place. Say I’ve encountered something, it will capture the last 60 seconds back from the button press.” Afterward, players have “24 hours to decide” if they want to file a report, after which time the encrypted recording will be automatically deleted.  

McCarthy was keen to address privacy concerns and to stress that none of this data is stored on a permanent basis by Microsoft either. “We’re looking at a storage of 90 days on our side if an enforcement decision rules that there’s no violation. If there is a violation we keep that clip [for] up to 366 days and then it’s auto-deleted after that.” 

The corporate vice president also went into detail surrounding the security of local data. “There is encryption in place which prevents any editing or ability to remove it from the console itself - not dissimilar to a lot of techniques we use on game clips.” All of this is intended to prevent tampering and to ensure that reports can’t be falsified. 

The social network

While the technology is no doubt impressive, it is poised to massively increase the workload for Microsoft’s moderation team. Xbox’s solution for this problem, however, is not without its risks. 

McCarthy explained how, using “AI advances”, Xbox intends to “optimize the flow of content to [its] moderators by separating workloads and filtering out noise in the system. With a lot of confidence and automated scanning, [our system] can tell [when there’s] something we actually don’t need our moderators to look at.” 

AI is a powerful tool but can be a double-edged sword, especially if the AI in question reflects the biases of those who created it - a problem that can lead to sexism, queerphobia, and racial profiling in the very worst cases.

However, to his credit, McCarthy seemed very much aware of this potential problem. “Language and culture experts are key for us to train our [AI] models and update them on a regular basis.” He conceded that this is a “never-ending job” but also claimed that the process for optimization “does sharpen” when moderators make decisions, allowing for a “feedback loop” which, in turn, can be used to better teach the AI. 

“We’re always auditing our automated systems… you have to stay up on top of things, in terms of understanding local trends, local slang, and local developments.”

Walking the talk?

Xbox’s voice reporting technology is very much a work in progress, but McCarthy sees it as a chance for players to come together. 

“Any ally that witnesses [inappropriate] stuff in a game can decide to upload the clip and report abuse they saw against another player. We’re really keen to see how players support one another when they come across behavior that’s in violation of our community standards.” 

The feature’s emphasis on “ease of use and not interrupting gameplay” is also likely to encourage this sort of behavior, since, to hear McCarthy say it, the feature is sufficiently streamlined that the cost to your play experience is virtually nil. The corporate vice president made it clear that the feature doesn’t “interrupt the flow”.  

On paper, it’s a noble principle; allowing players to look out for one another and for folks to call out poor behavior. Microsoft mostly has a positive record when it comes to inclusivity, with the notable exception of its willingness to welcome the controversial Bobby Kotick into the fold, who has often been accused of turning a blind eye toward sexism at Activision Blizzard. However, it’s also clear that Microsoft has identified that at least the appearance of inclusivity is vital in modern video game communities.

When it comes to the millions who use Xbox for online play, these changes seem poised to bring about marked improvements to the user experience. McCarthy’s commitment to building a community “that is welcoming to everyone” may be ambitious, but the addition of the new voice reporting system seems to be a significant step in that direction.

Xbox has also committed to supporting indie development for another 10 years, a move that will hopefully make our list of the best indie games a whole lot bigger.