# Retrieving data from websites?

**URL:** https://boards.straightdope.com/t/retrieving-data-from-websites/624763
**Category:** Factual Questions
**Created:** [June 10, 2012, 10:56pm UTC](https://boards.straightdope.com/t/retrieving-data-from-websites/624763 "2012-06-10T22:56:27Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![Dog80](https://avatars.discourse-cdn.com/v4/letter/d/6bbea6/32.png) [@Dog80](https://boards.straightdope.com/u/Dog80)
#### Post date: [June 10, 2012, 10:56pm UTC](https://boards.straightdope.com/t/retrieving-data-from-websites/624763/1 "2012-06-10T22:56:27Z")

</div>

There’s this website that posts the wholesale gasoline price of the day. For example this is for last Friday: [http://oil.gge.gov.gr/?p=1856](http://oil.gge.gov.gr/?p=1856)

What I am mainly interested is the 95 RON price, in the example above 1.280,028.

Is there any way to automatically extract these values to an Excel without having to manually visit the site and copy-paste the value?

---

<div class="post-metadata">

### Author: ![voltaire](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/voltaire/32/313_2.png) [@voltaire](https://boards.straightdope.com/u/voltaire)
#### Post date: [June 10, 2012, 11:08pm UTC](https://boards.straightdope.com/t/retrieving-data-from-websites/624763/2 "2012-06-10T23:08:23Z")

</div>

The term describing what you want to do is known as _scraping_. [Here’s an article](http://www.poynter.org/how-tos/digital-strategies/e-media-tidbits/102589/how-to-scrape-websites-for-data-without-programming-skills/) that refers to a FireFox addon that sounds like it does what you want, but I have no personal experience with using it.

---

<div class="post-metadata">

### Author: ![voltaire](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/voltaire/32/313_2.png) [@voltaire](https://boards.straightdope.com/u/voltaire)
#### Post date: [June 10, 2012, 11:19pm UTC](https://boards.straightdope.com/t/retrieving-data-from-websites/624763/3 "2012-06-10T23:19:40Z")

</div>

Here’s the direct [mozilla.org link](https://addons.mozilla.org/en-US/firefox/addon/outwit-hub/) to the aforementioned FireFox addon, including some user reviews.

---

<div class="post-metadata">

### Author: ![Duckster](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/duckster/32/1244_2.png) [@Duckster](https://boards.straightdope.com/u/Duckster)
#### Post date: [June 10, 2012, 11:22pm UTC](https://boards.straightdope.com/t/retrieving-data-from-websites/624763/4 "2012-06-10T23:22:38Z")

</div>

Try the [U.S. Energy Information Administration](http://205.254.135.7/) site instead.

Specifically, —\> [http://205.254.135.7/petroleum/data.cfm#prices](http://205.254.135.7/petroleum/data.cfm#prices)

---

<div class="post-metadata">

### Author: ![Dog80](https://avatars.discourse-cdn.com/v4/letter/d/6bbea6/32.png) [@Dog80](https://boards.straightdope.com/u/Dog80)
#### Post date: [June 11, 2012, 7:05am UTC](https://boards.straightdope.com/t/retrieving-data-from-websites/624763/5 "2012-06-11T07:05:10Z")

</div>

> [@voltaire](#):
>
> The term describing what you want to do is known as _scraping_. [Here’s an article](http://www.poynter.org/how-tos/digital-strategies/e-media-tidbits/102589/how-to-scrape-websites-for-data-without-programming-skills/) that refers to a FireFox addon that sounds like it does what you want, but I have no personal experience with using it.

Thank you, that was exactly what I was looking for 🙂

**Duckster** , thanks for the link but I am in Greece so the prices in US are irrelevant to me

---

<div class="post-metadata">

### Author: ![Blakeyrat](https://avatars.discourse-cdn.com/v4/letter/b/ecd19e/32.png) [@Blakeyrat](https://boards.straightdope.com/u/Blakeyrat)
#### Post date: [June 11, 2012, 11:45pm UTC](https://boards.straightdope.com/t/retrieving-data-from-websites/624763/6 "2012-06-11T23:45:45Z")

</div>

I’d just like to note that scraping is frowned upon in the web world. If the site doesn’t have a public-facing API in 2012, they don’t want you making use of their data with anything other than a webpage refresh. They might block you if this is the case.

---

<div class="post-metadata">

### Author: ![Omphaloskeptic](https://avatars.discourse-cdn.com/v4/letter/o/bcef8e/32.png) [@Omphaloskeptic](https://boards.straightdope.com/u/Omphaloskeptic)
#### Post date: [June 11, 2012, 11:56pm UTC](https://boards.straightdope.com/t/retrieving-data-from-websites/624763/7 "2012-06-11T23:56:53Z")

</div>

It’s maybe worth noting that Excel can do this directly, at least sometimes. In Excel 2003 this is at Data \> Import External Data \> New Web Query…; in Excel 2007 it’s Data \> From Web. Type in the link and click on the table containing the data you want.

---

<div class="post-metadata">

### Author: ![UncleRojelio](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/unclerojelio/32/3160_2.png) [@UncleRojelio](https://boards.straightdope.com/u/UncleRojelio)
#### Post date: [June 11, 2012, 11:59pm UTC](https://boards.straightdope.com/t/retrieving-data-from-websites/624763/8 "2012-06-11T23:59:30Z")

</div>

Oh come on, If you are going to scrape data, at least do it the old fashioned way and write a PHP script to do it for you.

---

<div class="post-metadata">

### Author: ![fubbleskag](https://avatars.discourse-cdn.com/v4/letter/f/c2a13f/32.png) [@fubbleskag](https://boards.straightdope.com/u/fubbleskag)
#### Post date: [June 12, 2012, 1:32am UTC](https://boards.straightdope.com/t/retrieving-data-from-websites/624763/9 "2012-06-12T01:32:40Z")

</div>

PHP described as “the old fashioned way” made my day
