Intern Drew Bent and designer Kitt Hirasaki have created automatic tools to improve the quality of discussion on Khan Academy videos and CS explorations.
Our engineers have put an emphasis this summer on improving moderator tools. We have taken on a few new volunteer ‘Guardians’ who help us moderate discussions. They are helped out by our users who do a great job flagging low-quality posts.
However, even with all this, the number of posts the Guardians have to sift through can be daunting. Drew and Kitt came up with a naive, but surprisingly effective, automatic way to detect low-quality questions and answers *before* they are posted. We call it our Robot Army.
By standing on the shoulders of giants, we came up with several heuristics that can detect and prevent offensive and low-quality posts. We look for everything from abnormal uppercase-to-lowercase ratios to content with little substance. We also try to prevent users from asking for things like up-votes.
Notifying the user of a low-quality post is a delicate manner. Rather than condemning the action, we take more of a glass half-full approach and prompt the user to clarify her question or answer before posting it. We’ll even go as far as to given them contextual hints on how to improve it, whether it be ‘Use standard capitalization’ or ‘Try to not repeat the ! character so many times.’
At that point, the user may or may not choose to improve her post. If she goes with the latter, then her post will initially be hidden from the site.
While the heuristics used in our low-quality detection are fairly conservative, there are undoubtedly going to be false positives. In this case, a Guardian will able to see the post in our ‘Pending Approval’ moderator queue and approve it, making the post appear to everybody.
We hope that these changes will help improve the quality of discussion. As for you, our users: keep on asking great question, writing helpful answers, and flagging offensive posts!