New technology to help predict popularity of hashtag compounds

shreyoshi's picture
If Twitter is one of the most effective tools in the hands of marketers today, the hashtag is a honing device, it is what adds that extra edge to the tool. And researchers at IIT Kharagpur and IIT Guwahati have just come up with a model that could optimize the use of this tool through early prediction of potential hashtag compounds.
The immense popularity of the hashtag has led to it being widely adopted in various other social media, including Facebook, Google Plus and the like. It has evolved into a language of its own, and in this diachronic process, perhaps the most common trait noticed is the compounding or coalescing of two or more individual hashtags to form a new hashtag.
So, for example, #Wikipedia and #Blackout come together to form #WikipediaBlackout; #Bihar and #Verdict and #2015 are joined to form #BiharVerdict2015. Clearly, because these lexemes are largely the creation of the individual user, the same event or news may be referred to by a variety of hashtags.
Therefore, if Flipkart is to offer a Christmas sale next month, the company could promote it on social media using the hashtag #ChristmasDayBigBillionSale or it could go with two separate hashtags - #Christmas2015 and #BigBillionSale. But does it have a way to predict, standing now and here, which would catch on in a bigger way?  
In the US, should Republican Donald Trump’s campaign tweet with #TrumpforPresident or would #DonaldTrump and #USElections2016 be likely to have a higher frequency of usage and hence garner a higher share of mind space in the months leading to the Presidential contest?
This is precisely what the software application developed by researchers Suman Kalyan Maity and Animesh Mukherjee from the Department of Computer Science & Engineering at IIT Kharagpur and Ritvik Saraf from the Department of Mathematics and Computing at IIT Guwahati purports to predict.
Research While some hashtag compounds gain a higher frequency of usage over time (even higher than the individual constituents), many are soon lost into oblivion. Noting this trend, the team proposes a prediction model that can predict with 77.07% accuracy whether a particular hashtag compound will become popular in the near future, that is two months after compounding. This is done using a machine learning application where the most important steps are feature selection and feature engineering. 
At longer time frames, that is six and 10 months after compounding, the prediction shows 77.52% and 79.13% accuracy respectively. The researchers had also asked human subjects to guess whether a hashtag compound will really catch the imagination on social media and found the overall accuracy of human prediction to be 48.7%. “Notably, while humans can discriminate the relatively easier cases, the automatic framework is successful in classifying the relatively harder cases,” notes the paper.
Thus, this has strong implications for recommending trending hashtags, since the popularity of alternative compounded tags can be predicted even before actual compounding has taken place.
Commercialization - While the development of the application was primarily driven by fundamental research, the software could become a potent tool in the hands of marketeers. “The application can be used by any organization, commercial or otherwise, for targeted marketing or promotion of campaigns on Twitter. It can even be plugged directly into the trending hashtag application that exists on Twitter,” says co-researcher Animesh Mukherjee, pointing to the scope for commercialization of the research. 
The team is currently in discussion with different companies to gauge the scope of application, and the interest in it. “Even outside the huge existing commercial realm that can make use of this research, we would be equally happy to make it available for students or budding entrepreneurs who may get some start-up ideas based on this,” Mukherjee adds. 
Though formal adoption of the application in industry circles is yet to occur, the work has been noted in eminent quarters. US-based digital marketing expert A J Kohn writes of the research, "Intriguing research and somewhat sad that we're fairly predictable. Potential marketing application here for someone who is really trying to get ahead of the curve."