Click here to Login

QS Trading Software: Database Optimization

Updated on 2010-10-12

Unlike many software programs that store each symbol data into a different file. QuantShare stores all historical quotes in a single database file. This file contains the database layout, each stock quotes and it may also contains additional information.

Intraday, Tick and custom databases are stored in separate databases (Intraday and tick databases will be available in few weeks). Each time you create a new custom database (Historical or Intraday), a new file, with the same name as the database name, is created in your account directory. Each time you add or remove data from this database, the trading software will access this file and perform the appropriate actions. Intraday custom database are associated with files that have "db1" as extension, while Historical custom databases have "db0" as file extension.

Using a single file instead of thousands (There are more than 8000 stocks in the U.S. Stock Market), greatly improve quotes reading process; screener and simulation will run faster. It also makes easy to copy and backup databases.

Database Fragmentation

But even with this, adding and removing data/quotes several thousands of time will make the database file more and more fragmented. Fragmentation occurs when the system tries to add new data to a file but cannot found free space close enough physically to that file on the drive. The result is that the file will be composed of pieces of data located at different areas on the drive and consequently the application will take much longer to read the database file. This cannot be noticed when displaying charts, however you will notice an increase in the time taken by the simulator or screener to process the data.

Database Optimization Tool

Fortunately there is a solution, and this solution is a built-in tool whose role is to recreate database files and to reduce the fragmentation of each one of them. The database optimizer tool is scheduled to be run by the trading software once a month. The interval can of course be changed by selecting Accounts, Application Settings then Databases tab. You can force an optimization for the next re-start by clicking on the "Schedule databases optimization for the next start". The tool is always executed at application startup.
We have seen improvement of more than 200% in certain cases in the backtesting process after executing the optimizer tool.

Note that the optimization process takes 2 to 15 minutes depending on the number of database files and their sizes and that the optimizer cannot be done efficiently if there are not enough space on the disk (at least 10 times the size of your database files - >4Gb).

no comments (Log in)

QuantShare Blog
Search Posts

Recent Posts

Create Graphs using the Grid Tool
Posted 1298 days ago

Profile Graphs
Posted 1403 days ago

Previous Posts

Stock Market Prediction
Posted 5017 days ago

How to search for a trading item
Posted 5038 days ago

More Posts


Create an account
Affiliate Program
Contact Us
Trading Forum
How-to Lessons
About Us
Terms of Use

Copyright 2024
Social Media
Follow us on Facebook
Twitter Follow us on Twitter
Follow us on Google+
RSS Trading Items

Trading financial instruments, including foreign exchange on margin, carries a high level of risk and is not suitable for all investors. The high degree of leverage can work against you as well as for you. Before deciding to invest in financial instruments or foreign exchange you should carefully consider your investment objectives, level of experience, and risk appetite. The possibility exists that you could sustain a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all the risks associated with trading and seek advice from an independent financial advisor if you have any doubts.