Click here to Login





                                                   Making it easier to download JSON price data that is login protected?

  0

0
Josh McCormick
2020-08-18 22:25:45


As a side project, I'm downloading and analyzing some of the price data from my 401K provider. The data is in JSON format, and the fund manager requires the user to be logged into their account before their URL will provide any fund data (including price). I've got it all working, but my workflow is absolutely awful and time-consuming:

1. Log into the 401K provider's website
2. Download the 14 individual JSON files (one for each 401K fund)
3. Transfer the files to a Linux box
4. Convert the JSON price data to TXT (using 'jq')
5. Transfer the files back to the PC
6. Import the 401K price data into QuantShare with the Data --> Ascii Import tool.

I've been successful! But it is a lot of manual work. Are there any existing features or possible enhancements you'd be considering that would make this easier? Might there be some way that QuantShare's downloaders can allow the user to log into a page before retrieving data? Is it possible to directly import from (moderately complex) JSON files with the Data -> ASCII Import tool? (I didn't see any JSON options.)

Thanks,
Josh McCormick



QuantShare
2020-08-20 11:28:47

  1

Best Answer
Yes, it is possible to import using JSON. You need to import it as CSV data then use the "Pre script" tool to ignore currently imported line and use the "ReadJsonData" to read json.

Content.Rows[x].IsIgnoreLine = true;
JsonData json = Content.Net.ReadJsonData(text);

For the automated log in, this could be possible depending on how complex it is.



Josh McCormick
2020-08-20 17:50:01

  0

I'll give the prescript a try, thanks. I'll send a private email with more of the login details.


No more messages
0




Reply:

No html code. URLs turn into links automatically.

Type in the trading objects you want to include: - Add Objects
To add a trading object in your message, type in the object name, select it and then click on "Add Objects"










QuantShare

Trading Items
Consumer Price Index - Historical data
Price to Book Ratio Historical Data - Market to Book
ISE index options historical data
Market Cap Historical Data for the US Stock Market
Market cap, ROE, ROA and Forward PE for U.S. Stocks - Fundamental...

How-to Lessons
How to download historical EOD data for penny stocks
How to download Portfolio123 data into QuantShare
How to quickly download the most recent EOD data for your stocks
How to download earnings calendar data for various stocks
How to download and use U.S. stocks earnings data

Related Forum Threads
Downloader - Login and Download a file
how do you check if the price data is up to date?
ETF data download limited to 2011
Download fundamental data for U.S. stocks
How to download data for selected symbols ?

Blog Posts
Day Trading: A trading system that combines intraday and EOD data
6 places to download historical intraday Forex quotes data for fr...
Download Trading Data using the URL-Script
Download Trading Data using the Pre-Script
Download Trading Data using the Post-Script









QuantShare
Product
QuantShare
Features
Create an account
Affiliate Program
Support
Contact Us
Trading Forum
How-to Lessons
Manual
Company
About Us
Privacy
Terms of Use

Copyright 2020 QuantShare.com
Social Media
Follow us on Facebook
Twitter Follow us on Twitter
Google+
Follow us on Google+
RSS Trading Items



Trading financial instruments, including foreign exchange on margin, carries a high level of risk and is not suitable for all investors. The high degree of leverage can work against you as well as for you. Before deciding to invest in financial instruments or foreign exchange you should carefully consider your investment objectives, level of experience, and risk appetite. The possibility exists that you could sustain a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all the risks associated with trading and seek advice from an independent financial advisor if you have any doubts.