Click here to Login

Introduction to sentiment analysis applied to the stock market

Updated on 2012-07-11

What the majority of people think of this stock or this currency?

By trying to answer this question, many traders and investors are gauging the general sentiment regarding a particular asset. Later, they may take a trading decision or create a trading system based on these sentiments. This is what is called sentiment analysis.

Many traders take decisions in the financial market solely based on what other people think and what they recommend. They may go in the same direction as the crowd and enter long gold (for example) if the general sentiment is bullish or they may use the contrarian investment style and go against the crowd by entering a short position on the gold.

In today's market, using computers and automated systems to perform sentiment analysis is a necessity. You must be able to analyze the general sentiment of thousands of stocks and backtest/optimize thousands of strategies.

Data required to perform sentiment analysis

Sentiment data is available in numeric or text format. The CAPS rating from the Motley fool for example are numeric values used to measure trader's sentiment regarding a particular stock. News or tweets are another type of sentiment data available in text format.

As you might have guessed, using text in sentiment analysis is a much more difficult task than using quantifiable data. In that case, the sentiment analysis consists of automatically identify the emotion and felling behind a given text using basic or advanced text-mining techniques.

You can do this part of analysis yourself or you can subscribe to a trading service that does the job for you. Of course, the issue with the second solution is that the system that converts text data into numeric values is a black box and there is generally no way to know how the conversion occurs exactly. Thestocksonar is an example of such service.

In the next paragraphs, I will show you few services you can use to get historical sentiment data. I will also show you how to use each service to plot sentiment data on a chart or create trading systems based on one or several of these sentiment data.

How to use tweets to analyze market sentiments

Sentiment analysis can be applied to any kind of asset (Stocks, currency pairs, ETFs...).
StockTwits and Forex Tweets are two items used to download tweets to a QuantShare custom database. The first item gets tweets for the U.S. stock market and the second ones gets tweets for popular currency pairs.

When using the stock downloader, the data is saved under a custom database whose name is "stocktwits". Make sure you download data every day to build an historical database. To display tweets for your favorite stocks, select "Tools -> Database Data" or "Data -> Edit Databases".

To get the number of tweets per day for any stock type the following formula:
a = GetDataCount("stocktwits", "title");

How to deal with StockTwits data

Simple way to analyze the sentiment of every tweet

This technique can be applied to market news too. It consists of measuring the number of positive words and subtracting it by the number of negative works in each tweet or trading news.

First, you must download the following trading indicator: News Sentiment Indicator.

Once done, you can use the new trading indicator (news_sentiment) to measure sentiment. The indicator gets three parameters: Database name, field name, list of positive words and a list of negative words.

Example of an intraday trading system:
sentiment1 = news_sentiment('stocktwits', 'title', 'buy,good,better,bull,increase,positive,jump', 'sell,bad,lower,bear,decrease,negative,drop,fall');
buy = sentiment1 > 0;

QuantShare sharing server contains also another very interesting function. This one tells you whether a text contains a specific word. You can get this function here: News parser function.

As I said before, the same technique can be applied to stock news. You can find more information about how to use and get news data here: How to create buy and sell trading rules based on News Data.

Advanced sentiment analysis

Optimization: If you want to know which word or combination of words (in a news or tweet) is followed by an increase in stock prices then you should use the "GetField" function (Word Occurrence in News Titles - Quantitative Trading Analysis).

This function gets a word from a list by specifying its index.

a1 = GetField("buy;sell;up;down;increase;decrease;good;bad", 0); // Returns "buy" (index = 0)

Trading system example:
Optimize("i1", 0, 7, 1);
Optimize("i2", 0, 7, 1);
a1 = GetField("buy;sell;up;down;increase;decrease;good;bad", i1);
a2 = GetField("buy;sell;up;down;increase;decrease;good;bad", i2);
buy = ContainsWords("stocktwits", "title", a1) and ContainsWords("stocktwits", "title", a2);

When optimizing the above trading system, a total of 64 (8*8) trading systems are generated. Each one enters a long position if a tweet contains the words specified in "a1" and "a2" variables.

APIs: Tweets data contains also an "account" field that gets the author of each tweet. Analyzing the authority of this author can help us discard some authors or weight any result by author's reputation. Services such as Klout or PeerIndex can be used for this purpose.

The CAPS Stock Rating

CAPS rating is a popular measure released by the Motley Fool. More information about this sentiment measure is available here: Daily CAPS Ratings. This link allows you to get a downloader that retrieves CAPS rating for every U.S. Stock. Historical data are obtained using the following item: Historical Data of CAPS Ratings.

Here is how to use CAPS rating to create a trading rule:

caps = GetData("caps", "rate", Zero) > 4;

The "caps" variable will get a signal on bars where the stock had a CAPS Rating above 4 (the highest score).

Sentiment analysis is also about combining sentiment data from several sources. Here, you can for example create a system that uses both tweets and CAPS rating data.

buy = GetData("caps", "rate", Zero) > 4 and GetDataCount("stocktwits", "title") > 10;

The above buy rule enters a new position if a stock is highly rated and has a least 10 related tweets.

Other pools and surveys data you can use in sentiment analysis:
Media sentiment alerts
AAII Sentiment Survey - Historical Data
Stock Picks from Wikinancial
Stock market prediction using traderbots

Market sentiment

Market sentiment analysis determines whether the market is bullish or bearish on the current or future fundamental outlook.

One way to determine the general market sentiment is by calculating the sentiment of each stock and then aggregating the data using the composite tool (Tools -> Composites).

How to create a composite index/indicator
Create a stock index or a trading indicator using the composite tools
4 original breadth indicators you should consider in your market timing strategy

You can also get ready-to-use indices; for instance, the Tweet Sentiment Analysis downloader gets a measure of the sentiment of the overall market from

Other data for market sentiment analysis:
Permabear Sentiment Index
Roubini Sentiment Indicator
Rosenberg Sentiment Index
TSP Sentiment Survey - Percentage of Bullish and Bearish Traders

no comments (Log in)

QuantShare Blog
Search Posts

Recent Posts

Create Graphs using the Grid Tool
Posted 1227 days ago

Profile Graphs
Posted 1332 days ago

Previous Posts

More Posts


Create an account
Affiliate Program
Contact Us
Trading Forum
How-to Lessons
About Us
Terms of Use

Copyright 2024
Social Media
Follow us on Facebook
Twitter Follow us on Twitter
Follow us on Google+
RSS Trading Items

Trading financial instruments, including foreign exchange on margin, carries a high level of risk and is not suitable for all investors. The high degree of leverage can work against you as well as for you. Before deciding to invest in financial instruments or foreign exchange you should carefully consider your investment objectives, level of experience, and risk appetite. The possibility exists that you could sustain a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all the risks associated with trading and seek advice from an independent financial advisor if you have any doubts.