“If you like me, why would you talk like this to me?”
Ask Microsoft’s Chinese chatbot Xiaoice about Tibetan spiritual leader the Dalai Lama and that’s how she responds. She also skirts around highly sensitive topics like the 1989 crackdown on Tiananmen Square protests, Chinese President Xi Jinping or even President-elect Donald Trump.
Xiaoice has become hugely popular since she was introduced in 2014. More than 40 million Chinese smartphone users chat with her on the social media platforms WeChat or Weibo every day, according to Microsoft. A quarter of them have declared their love for her.
Companies use chatbots to connect with consumers and promote their brands. Chinese often use Xiaoice simply to have something resembling a real conversation with a close friend.
Xiaoice is an experiment in artificial intelligence. She mines Chinese websites to gather material for conversations. She can glean vital context from voice messages and images. If you send her a photo of a pop star, for example, she might respond by saying how much she likes the singer’s music.
Refusing to talk about sensitive topics suggests Xiaoice has been programmed to avoid prohibited words when asked about them. Microsoft declined to comment.
China pours incredible resources into patrolling the internet, tracking down prohibited content and unwanted posts with an army of more than two million censors.
To get around them, Chinese bloggers and dissidents get creative. Instead of writing Tiananmen, online users instead reference the date of the massacre — June 4, 1989 — often simply writing 64 or 8964.
Xi Jinping is called Xi Baozi, or Steamed Bun Xi, a nickname he earned after a publicity stunt where he bought steamed pork buns at a small store in Beijing and carried his own food tray.
But censors eventually figure out new phrases and nicknames and restrict them online.
China Digital Times recently reported the growing attention to Xiaoice’s evasive answers, highlighting tweets from Chinese dissidents like Su Yutong, who lives in Germany. Su tried several banned topics with Xiaoice, posting screengrabs of conversations like the one below on Twitter:
CNNMoney also had a go. We asked Xiaoice about China’s president:
And then we name dropped President-elect Donald Trump, and asked her about Tiananmen Square:
Tech giants such as Facebook, Twitter and Google are blocked in China, in part because the government fears open discussions about the very topics Xiaoice refuses to acknowledge.
Amnesty International has documented a marked uptick in censorship in China over the past year, including an increase in controls on social media.
Microsoft still operates in China, and has been criticized in the past for complying with the country’s strict policing of the Internet.
If Microsoft has programmed Xiaoice to sidestep sensitive topics, it indicates a worrisome trend, said Patrick Poon, an Amnesty researcher. Earlier this week, Facebook was reported to have built a censorship tool to get into China.
“They are legitimizing what the Chinese government is doing, which is convincing foreign tech companies to suppress freedom of speech,” Poon said.
— Tim Schwarz and Serena Dong contributed this story.
