You have guest access to browse, login, or register.

Reply to this topic
Script to download stock quotes and history
Feb 12, 2007 12:39 pm

[panamacity]
panamacity

Total posts: 5
Quicken Mac 2003
Mac OS 10.3 and below
Like most people, I'm more than annoyed with the Intuit "sunset" policy that prevents Quicken users from downloading data after a few years of use.

I recently installed TurboTax, and was appalled to find that it had a bug that causes it to crash. Several other people have experienced the same crash. Last year, two family members lost all their data during TurboTax crashes.

So, I'm not big on buying more software from Intuit, but I do want to keep getting stock quotes, at least until GNUcash (which has massive dependency issues on OSX) or jgnash or Grisbi get far enough along.

I found several scripts that do most of the job, but decided I'd just make my own. Feel free to use it if you'd like. No warranty, use at your own risk. It works for me.

It's written to use zsh and tested on OSX 10.3.9. It will probably work on most Unix platforms with zsh-like shell environments - it doesn't use any particularly unique zsh components.

It's meant to be used from the command line. You could pretty easily trigger it from a cron job, to get the quotes on a regular basis. The name of each qif file is unique (it has the date and time in the name).

Before you use it, you should make a file called ~/stocknames that contains the symbols for the stocks you want to download. Save this script and run it from the command line, for example:

$ ./getstockquotes

You can give it one or two dates, if you want to retrieve historical data. If you don't give any dates, you get the current data.

Copy the text below into a blank file. Save the file as getstockquotes, then set the execute bit by typing chmod u+x getstockquotes

Here it is:

#!/bin/zsh

#

# getstockquotes < startdate enddate >

# Gets stock quotes from yahoo finance and saves them in a qif file.

#

# version 1.0 2007-02-11

#

# The stock symbols you want to download should

# be put in a file called ~/stocknames

#

USAGE='getstockquotes < startdate < enddate > >

date format=yyyymmdd

The resulting file, quotes_xxx.qif or table_xxx.qif

can be imported into programs that understand qif files'

# If you specified a date or two, get historical data.

now=`date +%Y%m%d%H%M%S`

if < $# -gt 2 >; then

echo $USAGE

echo Too many parameters.

exit

elif < ! -r ~/stocknames >; then

echo $USAGE

echo ~/stocknames is not readable

exit

elif < $# -gt 0 >; then

if < "$1" = "-h" >; then

echo $USAGE

exit

fi

# Massage the given dates into the form yahoo wants them

start=$1

# If you only give one date, the end date is assumed to be the current date.

if < $# -eq 1 >; then

end=`date +%Y%m%d`

else

end=$2

fi

startyear=`echo $start|cut -c1-4`

sm=`echo $start|cut -c5-6`

# Yahoo months are numbered from zero, not one

let startmonth=$sm-1

startday=`echo $start|cut -c7-8`

endyear=`echo $end|cut -c1-4`

sm=`echo $end|cut -c5-6`

let endmonth=$sm-1

endday=`echo $end|cut -c7-8`

# Quicken wants DOS line endings (hex 0D0A)

# so we put a ^M (0D, \r) just before the Unix line ending (0A).

echo "!Type:Prices\r" > table_$now.qif

# Yahoo won't give multiple quotes for historical data

for f in `cat ~/stocknames`; do

echo curl "http://ichart.finance.yahoo.com/table.csv?&s=$f&a=$startmonth&b=$startday&c=$startyear&d=$endmonth&e=$endday&f=$endyear&g=d&ignore=.csv" -s -o /tmp/tmp_$$.csv

curl "http://ichart.finance.yahoo.com/table.csv?&s=$f&a=$startmonth&b=$startday&c=$startyear&d=$endmonth&e=$endday&f=$endyear&g=d&ignore=.csv" -s -o /tmp/tmp_$$.csv

# The first line of the file from curl has a header (hence the tail +2).

# Yahoo puts out dates as yyyy-mm-dd, but Quicken doesn't take them that way.

# The yahoo historical format is in a different order, with extra data, too.

for g in `tail +2 /tmp/tmp_$$.csv`; do

# closing price

cl=`echo $g|cut -d',' -f7`

# year

dy=`echo $g|cut -c1-4`

# month

dm=`echo $g|cut -c6-7`

# day

dd=`echo $g|cut -c9-10`

# the format for Quicken

dt=$dm/$dd/$dy

# high

hi=`echo $g|cut -d',' -f3`

# low

lo=`echo $g|cut -d',' -f4`

# volume

vl=`echo $g|cut -d',' -f6`

echo \"$f\",$cl,\"$dt\",$hi,$lo,$vl"\r" >> table_$now.qif

done

done

echo "^\r" >> table_$now.qif

rm -f /tmp/tmp_$$.csv

echo "Result is in table_$now.qif"

else

# No dates specified, just get current quotes for all symbols

tick=''

for f in `cat ~/stocknames`; do

tick=$tick\&s=$f

done

echo "!Type:Prices\r" > quotes_$now.qif

# Notes on getting the data from yahoo:

# The flags that apply to getting quotes (not historical data):

# s-security, l-last, d-date, t-time,

# c-change, o-open, h-high, g-low, p-present?, v-volume

#

curl "http://download.finance.yahoo.com/d/quotes.csv?$tick&f=sl1d1hgv&e=.csv" -s -o quotes.csv

# The output from yahoo already has a ^M.

cat quotes.csv >> quotes_$now.qif

echo "^\r" >> quotes_$now.qif

rm -f quotes.csv

echo "Result is in quotes_$now.qif"

fi
Replies:
 

[Foobar, Uncle]

Total posts: 928
Number of years using Quicken: 6 to 10 years
Quicken Mac 2007
Mac OS X 10.4
This is a new post #1
of 8
Re: Script to download stock quotes and history
Feb 14, 2007 10:22 am 
Reply to this message  
re: losing all data in TurboTax.

Hmmm, never heard of making backups? doing a "save as" every so often?

Foob

--
- - -

Uncle Foobar & the Quicken 2008's

Proposal (Pimp My Quicken thread):

http://www.quickenforums.com/thread.jspa?threadID=600010099&tstart=0&mod=1169385651523

- - -

Uncle Foobar & the Quicken 2008's

Proposal (Pimp My Quicken thread):

Search: Pimp My Quicken
0 users found this answer helpful. Did you find this answer helpful? Yes No

[panamacity]

Total posts: 5
Quicken Mac 2003
Mac OS 10.3 and below
This is a new post #2
of 8
Re: Script to download stock quotes and history
Feb 18, 2007 01:25 pm 
Reply to this message  
re: "Uncle Foobar"'s naive comment

I won't waste my time pulling out a half-dozen links from people who have lost all their data after doing a TurboTax update this year. At least one of them had a backup copy on another (apparently connected) disk, and lost that copy, too.

You might want to look in to a problem more closely before writing demeaning comments about others.

Personally, I print all my interim results to a .pdf. TT is less likely to find and delete a .pdf.

But your comment also has nothing to do with this thread. Hopefully, others will provide feedback on the topic.

Replies to this message
  • Fuzzy_1 (Sep 8, 2007 3:33 am)



  • 0 users found this answer helpful. Did you find this answer helpful? Yes No

    [jldodge]

    Total posts: 3
    Windows XP
    This is a new post #3
    of 8
    Script to download stock quotes and history
    May 15, 2007 08:33 am 
    Reply to this message  
    This post is about: Quicken Home & Business 2007
    Replying to: panamacity (Feb 12, 2007 12:39 pm)
    Script to download stock quotes and history: Like most people, I'm more than annoyed with the Intuit "sunset" policy that...

    I apologize but I am a novice with scripts. Can someone give me the instructions on how I can load & execute this script? I have "lost" some historical pricing beyond 5 years and need to get the info reloaded in an automated fashion.

    Would appreciate any advice/suggestions or alternative approaches.

    Thanks in advance ...

    0 users found this answer helpful. Did you find this answer helpful? Yes No

    [Fuzzy_1]

    Total posts: 9
    Number of years using Quicken: 10+ years
    Quicken Mac 2007
    Mac OS X 10.4
    This is a new post #4
    of 8
    Re: Script to download stock quotes and history
    Sep 08, 2007 03:33 am 
    Reply to this message  
    This post is about: Quicken Mac 2007
    Replying to: panamacity (Feb 18, 2007 1:25 pm)
    Re: Script to download stock quotes and history: re: "Uncle Foobar"'s naive commentI won't waste my time pulling out...

    I tried this script and got the error:

    sbin/getstockquotes:33: parse error near `;'

    I tried the stocknames with each stock separated by a semicolon and with each stock on a separate line.
    I used stock sympbols from Yahoo eg: BSN.TO and REI-UN.TO
    Has anybody got this script working ?
    Cheers

    Replies to this message
  • msienkiewicz (Sep 8, 2007 5:28 pm)



  • 0 users found this answer helpful. Did you find this answer helpful? Yes No

    [msienkiewicz]

    Total posts: 143
    Number of years using Quicken: 6 to 10 years
    Quicken Mac 2006
    Mac OS X 10.3 and below
    This is a new post #5
    of 8
    Re: Script to download stock quotes and history
    Sep 08, 2007 05:28 pm 
    Reply to this message  
    This post is about: Quicken Mac 2006
    Replying to: Fuzzy_1 (Sep 8, 2007 3:33 am)
    Re: Script to download stock quotes and history: I tried this script and got the error: sbin/getstockquotes:33: parse error near...

    Hi Fuzzy_1

    I think that some of the characters got switched out or something when the script was posted to the forum.

    In particular, where the posted version has angle braces "<" it should have square braces "[".

    I will try uploading the copy of the script that I have working for me.

    In my 'stocknames' file I have one stock or mutual fund symbol per line.

    Replies to this message
  • Fuzzy_1 (Sep 9, 2007 5:23 pm)



  • Attachments:

    getstockquotes (3 KB) (88 Downloads)


    0 users found this answer helpful. Did you find this answer helpful? Yes No

    [Fuzzy_1]

    Total posts: 9
    Number of years using Quicken: 10+ years
    Quicken Mac 2007
    Mac OS X 10.4
    This is a new post #6
    of 8
    Re: Script to download stock quotes and history
    Sep 09, 2007 05:23 pm 
    Reply to this message  
    This post is about: Quicken Mac 2006
    Replying to: msienkiewicz (Sep 8, 2007 5:28 pm)
    Re: Script to download stock quotes and history: Hi Fuzzy_1 I think that some of the characters got...

    Thanks msienkiewicz,
    I downloaded the file and it works great.
    Now if we could get mutual funds working.
    I can get quotes of canadian mutal funds from an excel spreadsheet using a Web Query which gets the data from MSN Money. the query is <http://moneycentral.msn.com/investor/external/excel/quotes.asp?SYMBOL=CA:PHN150;CA:CHO100> to get two quotes. Can you creat a QIF file from an excel spreadsheet? or can this query be used in the aformentioned script?

    Replies to this message
  • msienkiewicz (Sep 9, 2007 9:37 pm)



  • 0 users found this answer helpful. Did you find this answer helpful? Yes No

    [msienkiewicz]

    Total posts: 143
    Number of years using Quicken: 6 to 10 years
    Quicken Mac 2006
    Mac OS X 10.3 and below
    This is a new post #7
    of 8
    Re: Script to download stock quotes and history
    Sep 09, 2007 09:37 pm 
    Reply to this message  
    This post is about: Quicken Mac 2006
    Replying to: Fuzzy_1 (Sep 9, 2007 5:23 pm)
    Re: Script to download stock quotes and history: Thanks msienkiewicz, I downloaded the file and it works great. Now if...

    The format for a stock quote QIF file is simple, you should be able to create it from an Excel spreadsheet. I use a Google spreadsheet to track company stock funds in my retirement account and export a CSV file that I use to import the prices into Quicken.

    You would modify the sheet so that the 1st line says
    "!Type:Prices" and the following lines contain
    Stock Symbol, Price, Date (and optionally High, Low,Volume) Then the last line of the sheet should have '^' in the first column. If you export to CSV (comma delimited values) the file should have the right format to import to Quicken. After exporting you may need to open the file in a text editor and make sure that the last column is '^' followed by a carriage return.

    Since you said the 'getstockquote' script worked fine, you should have an example of the QIF format. Also see in the Quicken manual, chapter 15, under "Importing a price file into Quicken".

    0 users found this answer helpful. Did you find this answer helpful? Yes No

    [swatman]
    Florida
    Total posts: 1
    This is a new post #8
    of 8
    Script to download stock quotes and history
    Mar 17, 2008 06:49 pm 
    Reply to this message  
    This post is about: Quicken Mac 2008
    Does anyone have a tweak for this script that will ignore errors due to bad ticker symbols in the file "stocknames"?

    I track LOTS of securities (bonds and the like) in Quicken and I have to use dummy symbols to keep the manually-entered security price history straight. Additionally, stocks are regularly delisted and the symbols disappear.

    These ultimately end up in "stocknames" and it causes big time errors in the output file. I've attached a file that results, for instance, from the non-existent symbol, ABYNY.

    Any help would be greatly appreciated.
    swman

    Attachments:

    table_20080317214723.qif (32 KB) (33 Downloads)


    0 users found this answer helpful. Did you find this answer helpful? Yes No
     
     
     
     
    XML RSS feed
    QuickFeeds
    Post to Del.icio.us
    Digg!
     

    In order to post a message, you must login.