Export to CSV the whole database including symbols - Page 2

Export to CSV the whole database including symbols

Back to threads

Previous - 0 1 2 - Next

QuantShare
2013-04-29 18:27:13

Don't forget to choose the "Intraday" database

vernon
2013-05-01 06:55:42

I know how to do that. But, I want to use a script to export "all" the data in the database for the one stock. Using the select feature of the database only lets you export one day at a time for min data. I want to export" all" the days with one command forintraday data.

Thank you.

QuantShare
2013-05-01 10:09:17

Using the script editor, to export only one symbol, replace the following line:

Symbol[] symbols = Symbols.GetSymbols();

By

Symbol[] symbols = new Symbol[1] { Symbols.GetSymbol("GOOG") }; // Replace "GOOG" by any ticker symbol

Or by

Symbol[] symbols = new Symbol[1] { Symbols.GetSymbol(Charts.GetSelectedChart().SymbolName) };

To export data for the selected chart symbol

Benn
2013-07-20 19:59:30

I'm using your example from above, but getting a runtime null-pointer exception error "Object reference not set to instance of an object". From a little testing, seems like the GetDatabasesData call is failing and the DatabasesData object is null. I do have the custom database earnings_cal downloaded and I can view it correctly. Any ideas?

Symbol[] symbols = Symbols.GetSymbols();
System.Text.StringBuilder text = new System.Text.StringBuilder("");
for(int i=0;i < symbols.Length;i++)
{
Symbol sym = symbols[i];
DatabasesData data = Databases.GetDatabasesData(sym.Name, "Custom", "earnings_cal");
for(int j=0;j < data.Value.Length;j++)
{
double year = data.Date[j].Year;
if(year >= 2000 && year <= 2004) // Specify Period of data exporting
{
text.AppendLine(sym.Name + ";" + data.Date[j].ToString("dd/MM/yyyy") + ";" + data.Value[j].ToString());
}
}
}

System.IO.File.WriteAllText(@"c:\quotes.csv", text.ToString());

Read more: http://www.quantshare.com/title-95-export-to-csv-the-whole-database-including-symbols&kbaseid=forum-95-1793-389#ixzz2ZbNOSlRf
Follow us: @quantshare on Twitter

Earnings database (by Patrick Fonce, uploaded several months ago)

No notes

QuantShare
2013-07-20 22:36:16

Hi Benn,

"GetDatabasesData" function gets the following parameters: Symbol Name, Database Name and Field Name

So the correct syntax should be:
DatabasesData data = Databases.GetDatabasesData(sym.Name, "earnings_cal", "[A FIELD NAME IN THE EARNING CAL DATABASE]");

Seeker
2013-10-30 16:51:08

Its mind-blowing how a huge database is exported at such lightning speed! May i request one small modification to export only the data for the last trading day of the symbol. If the database has symbols whose last update was years back, it will take those final dates and those which are current will be exported with today's data.

I know that the last date's data for multiple symbols can be created and exported through the screener. But the fields in my custom database has text data (type String).

Also, the script discussed shows downloading one field from the custom database. How can we add more fields?

QuantShare
2013-10-31 19:12:06

To get data for a specific date, you can replace these lines:

double year = data.Date[j].Year;
if(year >= 2000 && year <= 2004)

By:

double year = data.Date[j].Year;
double month = data.Date[j].Month;
double day = data.Date[j].Day;
if(day == 30 && month == 10 && year == 2013)

To get data for multiple fields, just add a new instruction (Databases.GetDatabasesData) to get field's data then append this data in the "text.AppendLine(" line.

Seeker
2013-10-31 21:01:57

Thanks. What I needed was the data on the last date of the symbol which can be different for various symbols.

QuantShare
2013-11-08 19:26:32

Remove the date check code I referred in the previous email then replace:

for(int j=0;j < data.Value.Length;j++)

By

for(int j=data.Value.Length - 1;j < data.Value.Length;j++)

BBB
2014-05-16 23:19:43

I was running into memory exceptions when trying to export a very large database. A slight adjustment to the original script fixes the problem:

....
text.AppendLine(sym.Name + ", " + data.Date[j].ToString("dd/MM/yyyy") + ", " + data.Value[j].ToString());
}
System.IO.File.AppendAllText(@"c:\allQuotes.csv", text.ToString());
text.Length = 0;
text.Capacity = 0;
}

By changing the file Write to an Append, moving "System.IO.File.AppendAllText(@"c:\allQuotes.csv", text.ToString());" inside the final parenthesis, and then clearing the text.Length and text.Capacity values - the file will be written to after each symbol and memory usage will stay low. Hope this helps someone.

QuantShare
2014-05-17 18:25:49

Thanks a lot for this fix.

Seeker
2017-01-23 21:59:34

I am using the script to export data for last date for all the symbols. I have added one more field for previous close as quotes.Close[j-1]. It gives the error 'index was outside the bounds of the array''; may be because there are symbols which may have data only for one day. How to bypass this?

QuantShare
2017-01-24 02:59:05

In the loop, you need to start with j = 1

Example:
for(int j=1;j < data.Value.Length;j++)

Barbecue
2017-03-25 09:34:29

Hi,
How should I modify the script to export just a set of symbols associated with a specific index ?

Thanks !

QuantShare
2017-03-27 04:09:21

You need to update the first line and select a specific symbols filter. You can create one using (Symbol -> Symbols View)

Then use:
Symbol[] symbols = Symbols.GetSymbols("filter name", "");

Barbecue
2017-03-27 18:02:19

That worked !

Thank you !

jason
2017-12-23 13:21:16

So to export a custom DB, the following defines the custom DB field to export...

DatabasesData data = Databases.GetDatabasesData(sym.Name, "earnings_cal", "[A FIELD NAME IN THE EARNING CAL DATABASE]");

What if the custom DB has 4 columns you wish to export, is that possible to export several/all columns OR can this only be done for a single custom DB column?

QuantShare
2017-12-24 04:10:16

You would need to make four calls to "Databases.GetDatabasesData" each one with a different field name.

jason
2017-12-24 07:53:35

Can you help with that syntax? Would they be new/seperate objects as in....

DatabasesData data1 = Databases.GetDatabasesData(sym.Name, "divcalendar", "dividend");
DatabasesData data2 = Databases.GetDatabasesData(sym.Name, "recorddate", "dividend");

QuantShare
2017-12-25 04:23:11

Yes but you need to switch "divcalendar/recorddate" with "dividend". The syntax is Symbol, Database, Field.

After that you can change the "text.AppendLine" to something like:
text.AppendLine(sym.Name + ";" + data.Date[j].ToString("dd/MM/yyyy") + ";" + data1.Value[j].ToString() + ";" + data2.Value[j].ToString());

Get more messages

No more messages

Previous - 0 1 2 - Next

Back


Trading Items Ticker Symbols List of the National Stock Exchange, NSE, India Export Indicators Data - Custom Time Frame Leading Index for the United States Tweet Sentiment Index for the Forex and Stock Market Update the Stock Exchange Associated With U.S. Stocks - Market Fi...	How-to Lessons How to quickly select stocks based on the last value of a databas... How to export trading data to a CSV file How to import trading data from CSV files How to quickly download the most recent EOD data for your stocks How to display the number of notes per stock
Related Forum Threads Export Custom Database data - all symbols - including text fields Create and Export each database for each market How to import to database from a csv file? Export database by industry list How to export historical data for a list of symbols	Blog Posts How to Create a List of Ticker Symbols Different Ticker Symbols for Each Data Source Sentiment Analysis: How to measure the sentiment score of your st... Ranking stocks based on their correlation with the S&P 500 Index How to measure market strength with the composite tool