Click here to Login









RSS feeds transformation

Updated on 2009-07-25





QuantShare is capable of downloading RSS feeds and converting them into CSV files for easier parsing. This task is accomplished by the downloader and is fully automated. It loads the specified RSS feeds, converts them into CSV content, then applies the CSV parsing and inserting rules to add the data into your local databases.

The RSS feed comes generally in a format like the following one: (This is an example of an element (item) of the yahoo news feed for the Google ticker symbol)

<item>
<title>This Week Dumbest Stock Moves (at Motley Fool)</title>
<link>https://www.quantshare.com</link>
<description>The description</description>
<pubDate>Fri, 24 Jul 2009 19:37:46 Etc/GMT</pubDate>
</item>

After reading and parsing the RSS feed content of the Google stock, the downloader will convert every 'item' tag into a row. The 'item' node children will be separated by a string or a character. (You decide what string or character to use as a separator).

The result will be: (assuming we used a semi-colon ';' as a separator)

This Week Dumbest Stock Moves (at Motley Fool) ; https://www.quantshare.com ; The description ; Fri, 24 Jul 2009 19:37:46 Etc/GMT

The next step will be to define the CSV parser settings, adding columns and assigning a database field to each one of them. For example, in the above line, the forth column must be associated with the 'date' field of the database of your choice. The transformed content is then parsed and referenced database fields are filled.

Common problems and solutions:

How to deal with nodes content that contain more than one line?
Multiple lines nodes’ content will be transformed into a single line text; the 'new line' character will be transformed into the following text '||'. You can write a Pre-Script formula to transform this '||' into a 'new line' character, leave it as it is, or perform any other transformation.
Example: The Google finance RSS feed, that downloads news for stocks, has multiline nodes’ content.

What if a node’s content contains the separator I choose?
The separator you choose is removed from every node’s content. So whatever separator you pick, your content will always be correctly parsed.

What if the description or title contains html tags?
All html tags are removed from nodes' content, only visible text is kept.











no comments (Log in)

QuantShare Blog
QuantShare
Search Posts




QuantShare
Recent Posts

Create Graphs using the Grid Tool
Posted 1407 days ago

Profile Graphs
Posted 1512 days ago

QuantShare
Previous Posts

RSS feeds transformation
Posted 5556 days ago

Stocks: Market Capitalization
Posted 5558 days ago

Ranking System Engine
Posted 5560 days ago

How to deal with StockTwits data
Posted 5570 days ago

Trend following systems
Posted 5573 days ago

Working with the formula editor
Posted 5575 days ago

Optimization of trading rules
Posted 5586 days ago

Volatility and trading systems
Posted 5588 days ago


More Posts

Back







QuantShare
Product
QuantShare
Features
Create an account
Affiliate Program
Support
Contact Us
Trading Forum
How-to Lessons
Manual
Company
About Us
Privacy
Terms of Use

Copyright © 2024 QuantShare.com
Social Media
Follow us on Facebook
Twitter Follow us on Twitter
Google+
Follow us on Google+
RSS Trading Items



Trading financial instruments, including foreign exchange on margin, carries a high level of risk and is not suitable for all investors. The high degree of leverage can work against you as well as for you. Before deciding to invest in financial instruments or foreign exchange you should carefully consider your investment objectives, level of experience, and risk appetite. The possibility exists that you could sustain a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all the risks associated with trading and seek advice from an independent financial advisor if you have any doubts.