MIT predict what topics will trend next on Twitter

Trends Map showing what's trending now around the worldThose smart people at the Massachusetts Institute of Technology think they can predict what will trend on Twitter, and there findings could have significant consequences both for Twitter and those working in digital advertising.

A professor and his student think they can estimate with 95% accuracy what Trending Topics you will be seeing on Twitter during any given day.

According to MIT:

“Associate Professor Devavrat Shah and his student, Stanislav Nikolov, will present a new algorithm that can, with 95 percent accuracy, predict which topics will trend an average of an hour and a half before Twitter’s algorithm puts them on the list — and sometimes as much as four or five hours before”.

The programme needs to be “trained” as it goes through data in a sample set and establishes meaningful patterns. However, this algorithm  does not assume anything about the shape of patterns.

Shah explains that this is because “I don’t know that things that trend have a step function. There are a thousand things that could happen.” Due to this, they “just let the data decide.”

To create the algorithm Shah and Nikolov conducted an experiment with training set  of data. This data set contained 400 Twitter topics, half of which trended, the other half which didn’t.  Then in real time, the applied their algorithm to live tweets,  and were able to predict trending topics with 95% accuracy and a 4 percent false-positive rate. They also believe that as the training set increases in size, the accuracy of their predictions will improve still further.

This is significant for Twitter, as they could obviously charge an increased sum for advertising content relating to posts likely trend. Premium products like this are going to be critical to the company’s business model as they progress.

Commenting on the findings, Ashish Goel, an associate professor of management science at Stanford University and a member of Twitter’s technical advisory board said:

“People go to social-media sites to find out what’s happening now. So in that sense, speeding up the process is something that is very useful.”

It’s also very useful to advertisers and digital marketers. If they could predict what was going to trend, they could alter their copy to make sure it fitted into popular topics, and was seen by as many people as possible.

  • Abdallah Al-Hakim

    interesting – how much sooner can this algorithm make these predictions compared to the current system? is it a matter of minutes or hours? Sounds pretty neat!