Over the past few years, a growing number of Wall Street and cryptocurrency analysts have started to rely on a relatively underutilized source of information to try and beat their respective markets with: social media.

For obvious reasons, social media is seen as a potential treasure trove of market sentiment information, and the macro analysis of forum messages, twitter comments and other social data to gain an edge has quickly been embraced by veteran quants. However, the ability of this data to actually and accurately predict price trends and develop new market alpha is still hotly contested by many crypto traders.

Hopefully, our latest study might finally put an end to this debate.

Trading Crypto with Social Media Data

At Santiment, we gather a massive amount of information from social media to try and determine its impact on the crypto market. As you read this, our system is collecting all incoming messages from over 1000 crypto-specific social media channels, including hundreds of Telegram groups, crypto subreddits, vetted twitter accounts, professional trader chats not indexed by Google and more.

We’ve already developed various market indicators using this dataset, like our ‘Social Volume’ metric which shows the amount of coin-specific mentions on crypto social media over time:

Though the initial results looked promising, in the early days of building our social dataset we too weren’t really sure whether this tool will end up producing any actionable insights about the crypto market, or mostly prove to be trivial intelligence or useless noise.

We quickly found out it was going to be the former.

Timing Price Tops by ‘Peak Hype’

The first real clues about the usefulness of social media data came when we created Emerging Trends, a handy list of the top 10 words with the highest growth ih crypto-related social mentions, which automatically updates every hour.

The initial goal of this tool was to build an easy way to discover fresh and quickly developing topics in the crypto community before they hit the ‘mainstream’, and many of our customers still primarily use it in this way.

However, we quickly discovered that this tool also had a nifty side effect. Whenever a particular coin’s name would appear in the top 3 positions on the list, it usually meant two things:

1. 1. The coin is experiencing a strong price rally. This, of course, made sense - even in 2020, outside of Bitcoin and Ethereum most cryptocurrencies only ever attract serious attention from the general crypto community during a pump.
   
   
2. 2. A price correction was imminent. Think of this as ‘peak hype’ - the bigger the spike in a coin’s social mentions, the more likely everyone interested in the project has ‘bought in’ already, leaving mostly sell pressure.
   

This second point was a big deal. If we could prove this to be true beyond anecdotal evidence, it would turn our Emerging Trends tool from a nifty ‘market overview’ tool to an entirely new leading price indicator for cryptocurrencies.

So after more than a year of data gathering - and some nicely executed trades based on this information by the Santiment team and community members - we recently put this theory to a proper test.

The Social Backtest

To stress-test the theory that a high placement on our Emerging Trends list indicated a looming correction, we used a so-called ‘event study’ which analyzes the coin price a set amount of days before and after a certain signal fires, and then averages this price information for all observed signals.

Our signal ‘trigger’ was simple - any time that a coin’s name appears in the top 3 words on our Emerging Trends list.

(One important caveat - the top 5 cryptocurrencies were excluded from this study. Unlike with mid and low-cap coins, the spikes in the mentions of Bitcoin or Ethereum aren’t always purely pump related. They also happen around major project announcements and often near local bottoms as well. The impact of social data on top cryptocurrencies will be the subject of a future study.)

So far, there have been 198 instances that satisfied our selected criteria - a coin’s name appearing in the top 3 positions on our Emerging Trends. So we analyzed the average coin price for these 198 signals - from the 2 weeks before to the 2 weeks after these coins claimed a top 3 position on our Emerging Trends list:

As you can see, there is a sharp increase in the price of these coins before the signal triggers. This further gives legs to our theory that price pumps are in fact the biggest reason why all of these coins suddenly get talked about a whole lot more than usual.

The 0 point on the X axis marks the average price of the analyzed coins on the day that the signal triggers. The most important part of the backtest is what happens afterwards:

Within the next 12 days after a coin claims a top 3 position on our list of Emerging Trends, its price drops by an average of 8.2 percent!

Again - this is on a sample size of almost 200 observed signals.

Now, an experienced analyst would also ask about the behavior of the market itself. Afterall, the market might have been going down anyway, which would’ve contributed to a decline in the prices of these coins.

For this reason, an event study also looks at the market itself and tries to nullify its impact by removing a calculated beta.

What does that mean? The below graph shows the results of our backtest alongside the average price behavior of Bitcoin, used here as a proxy for the crypto market:

With this information, we can now go back and calculate a ‘beta’, or a measure of how much each of our analyzed assets typically reacted to the market. Then, we can use this to remove the market’s impact on their price performance.

With the beta calculated, here are the results of our backtest after we removed the impact of the market from the analysis (in orange):

Even though the downtrend has reduced slightly after the signal triggers, the general behavior remained very much the same. With the overall market impact removed, the analyzed coins lost an average of 6% in the 12 days after appearing on our list of Emerging Trends.

Automated vs Human Signals

The initial results of our event study demonstrate the massive potential of social media information as a leading sell indicator in cryptocurrencies. That said, placing aggregated data in proper market context is key to profitable trading in the long-term. And this is something that no amount of automation can achieve.

This is also why we at Santiment don’t only provide hard data about the crypto market, but also focus on producing daily insights and analysis to contextualize this market behavior in a way that only a human analyst can. You can read our latest market reports and analysis over on insight.santiment.net:

Of course, many of our insights in the past have been inspired by different coins showing up on our list of Emerging Trends. ‘Coin go moon’ is often an interesting starting point to try and explain the fundamental factors that have enabled the rally, and to determine where the coin might go from here.

In a small portion of these insights - or whenever we identify clear trends in both the coin’s social and its on-chain data - we also give explicit SELL calls.

We recently started doing this more often, but for the purpose of this analysis, we looked at 18 insights where we explicitly called a ‘top’ based on the fact that a coin’s name appeared on our list of Emerging Trends.

Here is the price performance of the coins featured in those 18 insights, 2 weeks before and 2 weeks after we wrote about them:

Minus some ebbs and flows, the general pattern is again the same - the coins pump prior to our article(s), and begin to decline shortly afterward. This time, however, the actual size of an average downtrend is much stronger compared to automated signals: in the 12 days after we published our ‘top’ calls, the price of these coins decreased by an average of 18.1%!

The same pattern holds when we calculate the ‘beta’ to remove the impact of the market on the price of these coins:

While the sample size is considerably smaller, this further validates our belief that - in order to truly be effective - market data needs to be interpreted by experienced analysts. Information is power but only in the right hands, which is why the Santiment team and vetted community members publish unique analysis about the crypto market each and every day. Again, you can find all of our market reports and daily insights on insights.santiment.net

Conclusion

As laid out above, the results of our event study clearly showcase the potential of social media data as a leading sell indicator in cryptocurrencies. The second backtest also suggests that an enrichment of data through human interpretation typically yields the best results.

When used to predict downtrends, the appearance of a coin in the top 3 words on our Emerging Trends list suggests an average downtrend of 8.2% for automated signals and 18.1% for human-created signals.

In the future, we will attempt to extend this study to include the largest cryptocurrencies as well, and analyze the predictive power of other social metrics, like our ‘crowd sentiment’ indicators that we have recently introduced to Sanbase Pro.

In the meantime, you can check out Emerging Trends live on Sanbase and try its predictive power for yourself. The tool is still 100% free - for now *wink wink*