Retrieving Information...

I'm wondering about how I would go about getting information from other people's databases. I know that this is possible, but I do not know how... I have an Information Technology project that I have to do for school in which the teacher wants me to make a hockey pool. He told me that I could just go by information of how the players did last season, but in my opinion that somewhat defeats the purpose of the hockey pool if all the information is known already...

Therefore, I would like to learn about how I could get information from other websites. What I had in mind was trying to get information from the ESPN website or something. I found that they have decently updated statistics about the players and goaltenders, and I'm pretty sure that they have enough information for me to work with on my hockey pool.

However, I have no idea about where to start... In fact, I don't even know what programming language I would have to use to do such a task... I already know a fair bit about PHP (not a LOT though, mainly just working on integrating the HTML into PHP and working with forms, membership systems, etc.) and I also know how to work with MySQL.

Would anyone be able to possibly provide with some links to tutorials in which I could learn how to do this, or maybe even teach me? I would greatly prefer tutorials over just a script because I would like to actually learn, step by step, how to do it rather than take someone else's code. If you want to just contact me directly and help me out over an IM program, my MSN is clickk@gmail.com. Thanks for any help in advance!

 

As the ex-System Architect for one of the biggest sports web sites on earth, I doubt you'll be able to actually get into anybody's database directly. At least, not if they truly know what they're doing.

What you CAN do, however, is simply scrape the appropriate web pages and pull the information directly from those. It's a fairly simple matter to save someone else's web page to your own computer. Then, if you're really ambitious, you could write a simple parser to extract information from it, format it into something usable, then load it in to your own system.

 

Well what I'm looking more into doing is have something that updates automatically on a regular basis. My IT teacher and one of his former student worked on a stock market "website/program" that allowed us to "invest" in stocks, with real-time updates of the real stocks and how well they were doing in the stock market. Apparently, it was the student who did all the work in grabbing the information, and my teacher has no idea how he did it so he's not able to help me.

However, all the information was retrieved from yahoo, so I'm sure there must be a way to do it... Would there be some way I can do it and have it automatically update itself on a regular basis? Or am I just misunderstanding your method about saving the page on my own computer and writing a parser to extract information...? Thanks for the help so far.

 

that student probably used one of these services
http://www.nyse.com/productservices/infoprod/datainformation.html#Realtime Datafeeds
for NYSE.

In any case, I can't find anything on espn's site that offers the same things. You'll probably end up screen scraping as mentioned above.

 

Ahhh... alright, well thanks for the help. I'll try to find a way to do this

 

Well, ESPN does have that kind of service, it's called Sportsticker and is a real-time feed for all sports activities. My old employer had it and we used it for live updates on our web site. Sportsticker is not free by any means and requires some dedicated hardware and connection ability. It sends XML data basically as a serial link.

Scraping and parsing an existing page is fairly trivial - that is until those you're scraping change the page, which is fairly common in the web world. A smart parser looks for keywords and doesn't care about positioning, although if the provider uses lots of tables, it can get very confusing really fast.

 

shoot us the link. If the feed is xml then it would even more trivial to write an app in .NET to handle the data

 

There is no link to shoot you. Sportsticker is a paid subscription service from ESPN, and is more of a wild feed. Think of the old stock market ticker machines you see in the old movies - now, substitute sports play-by-play data for what's being printed out. Oh, and they also do broadcast summary-style statistical data overnight.

 

;x this is going to prove more difficult than I had imagined...

 

Heh, well, depends how you get the data I guess.

If it were me I'd go ahead and try to get a screen scrape about now. I wrote a PHP script to scrape what's currently plaing on radioparadise.com a while ago, however they changed their website design and broke it (something to evaluate about screen scrapes, eh?).

As an application model I would have a central server scrape the data and conglomerate it, using different modules (to reflect different site designs and so forth), into a single RSS feed (or generic XML feed anyway) then simply have your desktop app read that.

 

That looks like a very intelligent way of doing it but unfortunately I have no clue what you're talking about :x If you can maybe link me to some tutorials that can do something like that or maybe even teach me... that would be great!