Hi. I'm trying to optimize an intraday trading system and the performance is very slow, so I'd like some suggestions on how to speed up the performance. The multi-core optimization option is turned on. A few specific questions:
1. My source data (in the QS database) is 1 minute bars, but the system I'm testing is setup to test 10 minute bars. Would the system test / optimization be faster if I imported 'prebuilt' 10 minute bars into the QS DB? Is it a big performance 'hit' for QS to have to build 10 minute bars from 1 minute bar data?
2. A single test of the system with no optimization takes between 30 - 45 seconds. But when I optimize a single variable (the RSI entry level), the first optimization iteration takes under a minute, but each subsequent optimization iteration takes a huge amount of time - I'm 25 minutes into an optimization right now, and only 3 iterations have been run (and the first one took under a minute)! Any ideas on what might cause this behavior? My system specs are below; fairly robust. It looks like the hard disk is the bottleneck (100% read usage for minutes at a time), but I don't know why the HD would be working so hard for so long when I have available RAM.
For context, I'm running Windows 8 on a 6-core i7 with 12 GB of RAM, 1 TB HD at 7200 RPM. I'd welcome performance suggestions that are germane to the specific issues I mentioned above.
One update - while running the optimization mentioned above, I've validated that I have about 40% of memory available (about 5 GB), and QuantShare is showing numerous hard faults (page swaps) in the Windows 8 resource monitor. No other applications are showing hard faults, so they aren't contributing to the issue. The disk monitor also shows QS hitting pagefile.sys.
1/ Is it a big performance 'hit' for QS to have to build 10 minute bars from 1 minute bar data?
No
2/ I don't know why the HD would be working so hard for so long when I have available RAM.
You can specify whether to read data from the hard drive or store them into memory (Accounts -> Application Settings -> Memory Management -> Load quotes...).
Please let me know if you trading system uses a lot of money management code?
Did you try with a very basic trading system? What about the performance?
Yes, I'm using the 64-bit version.
Yes, my configuration is setup to load quotes into memory.
Yes, the system I was trying when I had the performance issue used a bunch of money management code.
I ran an optimization on the same time period and symbols but using a simple system with no money management code. This worked well -- it took 8 minutes to run through 400 iterations. Very reasonable. So I guess that means the issue is with one of the money management scripts or one of the custom functions used in the first system I tried to optimize. I'll try to pinpoint which one...
For anyone who experiences a similar issue, in my case the problem was a custom built MM script with a bunch of metrics / calculations. When removed, the optimization worked fine.
Trading financial instruments, including foreign exchange on margin, carries a high level of risk and is not suitable for all investors. The high degree of leverage can work against you as well as for you. Before deciding to invest in financial instruments or foreign exchange you should carefully consider your investment objectives, level of experience, and risk appetite. The possibility exists that you could sustain a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all the risks associated with trading and seek advice from an independent financial advisor if you have any doubts.
1
