Click here to Login





                                                   Updating info on existing stocks?

  1

0
Josh McCormick
2020-10-20 12:07:06


I'm wanting to update some of the information associated with the symbols I use in QuantShare (without removing and re-adding the symbols). I've pulled a list of all my QuantShare symbols into a text file and a CSV field. I have a source for Fullname, Sector, Industry, and a few other fields that I want to update. What's going to be my best (automated) method to import the updates into QuantShare?


Kyra Packens
2020-10-23 15:30:50

  1

Best Answer
Hey there. I have a script to do this regularly when my data source updates their ticker metadata. Not the prettiest code, but I'm sure you can adapt it to your needs

Pasted here: https://pastebin.com/pJC3Egmz

You need to attach the CsvHelper https://joshclose.github.io/CsvHelper/ dll to your script to use it directly, or just replace that with whatever reading code you use.

Hope it helps.
-- KP



Josh McCormick
2020-10-28 16:00:40

  0

Interesting, thank you!


Josh McCormick
2020-10-30 23:59:27

  0

Kyra --

Uh... Kyra? Your reply wasn't just interesting. What you provided was ABSOLUTE GOLD!

I noticed that your script was looking for a .csv file name "SHARADAR_TICKERS", so I Googled for it and found it on Quantl, and started playing around with the data. NICE!

https://www.quandl.com/databases/SF1/data

Hmmmmm. It seems that I've already got a number of active (and verified) symbols that aren't in his database? But wait... he's got quite a number that aren't in mine (even after filtering out Canadian stocks, OTC, and de-listed symbols)! None of my ETFs are in there, but he clearly indicates that it's a database of public companies, not of stocks. (And now I see he's got an "SFP" database for funds.) He's got a custom scale for market cap and revenue for most companies? Extremely useful. **AND** there's a way to extract just the recently added companies? (I see Root, McAfee, and more?) This is absolutely beautiful.

Up until this point, I've taken my own list of symbols (companies, ETFs, mutual fund, indexes) and used a bit of shell script to extract some company information from Yahoo. The script locally stores Yahoo's JSON file for each symbol. After that (not shown), the real data is extracted from different objection, depending on the symbol type (Stock, ETF/Mutual Fund, Index, or Future). There's some absolutely fantastic (and unique) company information in the Yahoo JSON data (along with a ton of unnecessary page rendering info). Extracting the JSON from is done via a simple UNIX/Linux shell command. I choose the symbol via the $i variable, and it writes the JSON to a file named after that symbol. Here's the command:

curl "https://finance.yahoo.com/quote/${i}" | grep root.App.main | cut -d= -f2- | sed 's/.$//' > yahoo/${i}

Wow. Now I've got a lot to think about here. Yahoo provides a ton of info, but I've got to work through my own list of symbols. The SF1 database is limited to companies, but it is pretty authoritative and very useful for catching new listings. I think I want to merge the two approaches. This is AWESOME.

PS: Now that I think of it, if you didn't intend to disclose this much, let me know and I'll scrub it from my reply, okay?

Thanks again so much for your help Kyra!
Josh McCormick



Kyra Packens
2020-11-01 14:27:21

  0

Nice, glad you found that helpful!

Don't worry, the snippet I posted was plenty redacted, but I appreciate you mentioning it :D I certainly don't mind disclosing that I use the Sharadar data source for some of my historical data. In fact I recommend them (and I'm not affiliated). I spent a long time researching a historical provider with good coverage at a reasonable price after I decided free data was mostly more trouble than it was worth a few years back. I believe that their US Equity bundle (https://www.quandl.com/databases/SFA/data) is a particularly good value.

You mentioned that you are using some linux command line script for some of your work. Are you running Quantshare under emulation in linux, or doing data prep on a separate box? I'm curious since I haven't tried running Quantshare in linux and was wondering if you had a good experience with it if you're doing that.

Thanks,
-- KP




Josh McCormick
2020-11-02 17:08:41

  0

Kyra,

As it turns out, I'm not emulating Windows on Linux. A funny thing... these days, Microsoft is hoping to get people to do just the opposite and use Windows Subsystem for Linux in order to emulate Linux on Windows! But I'm not doing that either. For what I'm using it for (data prep, custom processing, notifications, alarms), a Raspberry Pi 4B (4GB model) is overkill. I *do* understand and appreciate your choice of a commercial data provider. (Many times, free data is only "free" if your time is worth nothing.)

Josh



Kyra Packens
2020-11-02 20:47:25

  1

Last I knew about unix scripting in windows would have been Cygwin, but I haven't done any serious shell scripting in quite some time. I wasn't aware of WSL, thank you for that.

> (Many times, free data is only "free" if your time is worth nothing.)

It's worse than that. When you're developing strategies, in my experience, it's common to run into situations where they actually zero in on problems with the data, especially if you are using optimization or data mining techniques. If you have poor quality data there is a very good chance that any strategies you develop against it overstate their results dramatically because those results are based upon exploiting problems with the data. When you go to the real market with these, you're going to lose money.

Sadly there's no guarantee that if you pay for some data that it's going to be free of these problems! Data quality is something that I'm always having to think hard about.


-- KP



No more messages
0




Reply:

No html code. URLs turn into links automatically.

Type in the trading objects you want to include: - Add Objects
To add a trading object in your message, type in the object name, select it and then click on "Add Objects"










QuantShare

Trading Items
Updating Margin Factor During Drawdown
Disable or Enable your Stop Rules
Custom Weight for Individual Securities
Hull Moving Average Strategy - Multi Time Frame Trading System
Basic in-Sample/out-of-sample trading technique

How-to Lessons
How to add a trading indicator to a chart
How to backup your databases (EOD, Intraday, Tick and Custom) and...
How to automatically draw Fibonacci Retracement for each new stoc...
How to create histogram charts
How to download historical quotes of indices

Related Forum Threads
Update existing Historical Earnings Surprise trading object?
Accessing Symbol Info in Screener, Strategy script and in MM tool...
STATIC WATCH LIST-add a symbol to existing static watch list from...
Screen for non trending stocks?
Updating BULL.L ends at June 10th

Blog Posts
Backtesting All Candlestick Patterns in 10 Minutes
How to get buy and sell orders for a portfolio based on a trading...
Update Chart Layout based on Active Ticker Symbol
How to Send Commands to QuantShare from Excel or Other Applicatio...
How to make sure your Historical Price Data is Split Adjusted









QuantShare
Product
QuantShare
Features
Create an account
Affiliate Program
Support
Contact Us
Trading Forum
How-to Lessons
Manual
Company
About Us
Privacy
Terms of Use

Copyright 2020 QuantShare.com
Social Media
Follow us on Facebook
Twitter Follow us on Twitter
Google+
Follow us on Google+
RSS Trading Items



Trading financial instruments, including foreign exchange on margin, carries a high level of risk and is not suitable for all investors. The high degree of leverage can work against you as well as for you. Before deciding to invest in financial instruments or foreign exchange you should carefully consider your investment objectives, level of experience, and risk appetite. The possibility exists that you could sustain a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all the risks associated with trading and seek advice from an independent financial advisor if you have any doubts.