What is AI Message Moderation?
AI Message Moderation allows you to enable up to four different moderation tools to ensure your Rooms message content is auto-moderated in the manner that best suits your community and context.
The four different moderation tools can be configured and updated by speaking to your Client Success Manager.
Each tool has various moderation policies that your district can tailor to its preferred method(s) of monitoring message content
AI Message Moderation is available as a part of the Group Connect product package, reach out to your Client Success Manager for more information.
Reported Content Dashboard
Administrators will benefit from a consolidated dashboard* of both manual and automated reports of inappropriate message content.
This dashboard allows them to take action on reports where needed, but with auto-moderation, it can function more as an audit log of all messages automatically resolved by AI Message Moderation.
*This Moderation dashboard is already available for clients using manual message moderation. AI Message Moderation, and reports created from AI moderation, feed into the existing Moderation dashboard.
Investigating a Report
From the Report View, easily switch between reports, marking a report resolved, and choose to block a message if the report is currently unresolved
After clicking into a report in the Reported Content Dashboard, you will be shown a screen that contains 4 main tab on the left side of the screen:
Overview
Overview
Contains the same details as the dashboard screen for the individual report, including report date, reporter, sender, assigned to, severity, reason, and reporter comments.
Members
Members
Shows the members in rooms included in the report, their rooms role, and what campuses they are a part of.
Notes
Notes
Creates a note thread for those reviewing the report to store comments/notes about the content.
Activity Log
Activity Log
A log of the changes made to the report as steps of action are taken.
Keyword Blocklists
Custom lists of keywords and regex (regular expression) patterns to block unwanted content if these keywords are detected in a message.
Supports blocking keywords, static phrases, email addresses, and web domains.
Organizations can have keyword blocklist policies that only apply to their community, and can have up to 4 keyword blocklists.
AI Semantic Filters
Leverages AI to detect harmful content by finding messages semantically similar to known harmful phrases.
Expands upon static phrases included in keyword blocklists by accounting for the different ways a user might word a harmful phrase.
AI Semantic Filters cannot be customized, but organizations using Rooms can opt into leveraging our auto-moderation provider's default AI semantic filters policies.
AI Text Harm Detection Engine
Leverages advanced ML model that detects 40+ types of harmful content across 30+ languages.
Expands upon keyword blocklist and AI semantic filter functionality.
Organizations can have AI text harm detection engine policies that only apply to their community; districts can also use custom severity levels for messages reported by this auto-moderation tool.
AI Image Moderation
Analyzes images for inappropriate content. Districts can have AI image recognition policies that only apply to their community.