# Stealing web content, revisited for a good reason **URL:** \ **Category:** Factual Questions\ **Created:** [June 24, 2010, 1:23pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346 "2010-06-24T13:23:04Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Bricker](https://avatars.discourse-cdn.com/v4/letter/b/977dab/32.png) [@Bricker](https://boards.straightdope.com/u/Bricker)\ **Post date:** [June 24, 2010, 1:23pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/1 "2010-06-24T13:23:04Z")
I seem to recall that **Stoid** got in a bit of trouble with a few folks here by asking a variant of the question I’m about to ask… so let me explain why I’m asking. I have [The Stoker](http://www.rocksbarbque.com/). It’s a small appliance that monitors the temperature of the meat and interior of my Big Green Egg smoker, and (when connected to my home network) even lets me check and modify temperatures on my beef brisket from inside the house, using a laptop that connects to the device’s built-in web server. By setting up a port forwarding rule on my home router, I can also monitor and change the smoker through my phone’s browser when we’re out shopping. Unfortunately, there’s no authentication option. That means that the only thing protecting me from some yahoo from messing with my meat is security by obscurity. I’d like to fix that. I’d like to have another web server in the house be the one that’s visible to the public, and I’d like it to present whatever content my little internal built-in web server is presenting. This way, I could enforce some sort of authentication. Bonus: would there be a way to present a read-only version of the information as well as a “real” version? The device’s page just uses forms to accept changes to the desired temperatures and alert ranges. I’d love to able able to post in MPSIMS, “Hey, I’m cooking a pork shoulder – check it out at [http://www.bricker-big-green-egg.com:9119](http://www.bricker-big-green-egg.com:9119),” without worrying that someone here would think it was funny to change my cook temp from 225 to 525. So that’s the background. Is there a way I can code a web page to display exactly what Website B is showing, when the client can’t directly connect to Website B? And if there is, is there one method that would support interacting with the forms on Website B and another method that would just show the info without letting the client interact with the form? I know networking pretty well, but very little about coding. ---
**Author:** ![Jackmannii](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/jackmannii/32/311_2.png) [@Jackmannii](https://boards.straightdope.com/u/Jackmannii)\ **Post date:** [June 24, 2010, 1:28pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/2 "2010-06-24T13:28:56Z")
A coding solution I do not have, however: > [@Bricker](#): > > …the only thing protecting me from some yahoo from messing with my meat is security by obscurity. I urge you to adopt this as your sig. ---
**Author:** ![Unintentionally\_Blank](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/unintentionally_blank/32/5418_2.png) [@Unintentionally\_Blank](https://boards.straightdope.com/u/Unintentionally_Blank)\ **Post date:** [June 24, 2010, 1:31pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/3 "2010-06-24T13:31:32Z")
What OS’s do you have available? What is the firewall protecting your network? There’s many ways of skinning this cat. Several firewall products provide VPN functionality, which would handle the remote access to read/write data. Look up the curl or wget commands to pull the content from web page, which you could then place on -another- web server for read only use. ---
**Author:** ![Bricker](https://avatars.discourse-cdn.com/v4/letter/b/977dab/32.png) [@Bricker](https://boards.straightdope.com/u/Bricker)\ **Post date:** [June 24, 2010, 1:41pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/4 "2010-06-24T13:41:18Z")
> [@Unintentionally\_Blank](#): > > What OS’s do you have available? What is the firewall protecting your network? There’s many ways of skinning this cat. Several firewall products provide VPN functionality, which would handle the remote access to read/write data. Look up the curl or wget commands to pull the content from web page, which you could then place on -another- web server for read only use. I have Windows and a couple of Linux flavors available to host the second site… so basically I have IIS or Apache easily available. The problem with VPN is that I’d need a VPN client for my phone’s browser, or more accurately for my phone, period. Hmmm… looking up curl brought me to a page that lists this technique under “Don’t Do:” ```auto ``` But why not? Their advice is, “Don’t do this, because someone could hack the [example.com](http://example.com) site, and then you’ll be screwed,” but that’s not a concern here. (I’m assuming that ‘[example.com](http://example.com)’ gets resolved and connected-to server-side, not client-side - yes?) This may solve my “read-only” problem quite nicely. ---
**Author:** ![njtt](https://avatars.discourse-cdn.com/v4/letter/n/ecd19e/32.png) [@njtt](https://boards.straightdope.com/u/njtt)\ **Post date:** [June 24, 2010, 1:51pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/5 "2010-06-24T13:51:51Z")
Sheesh! Isn’t being able to have your meat Tweet you when it is cooked geeky enough for you already! ---
**Author:** ![Unintentionally\_Blank](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/unintentionally_blank/32/5418_2.png) [@Unintentionally\_Blank](https://boards.straightdope.com/u/Unintentionally_Blank)\ **Post date:** [June 24, 2010, 1:56pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/6 "2010-06-24T13:56:33Z")
Some firewalls support ssl VPN, which would work for any device that can connect to https:// In my case, there’s an address bar on the webpage the firewall provides that will proxy an website you enter. So, you log into [https://bubbashouse.com](https://bubbashouse.com) and enter 10.0.0.20/brisket and it’ll show you what the internal website is offering. ---
**Author:** ![Bricker](https://avatars.discourse-cdn.com/v4/letter/b/977dab/32.png) [@Bricker](https://boards.straightdope.com/u/Bricker)\ **Post date:** [June 24, 2010, 2:06pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/7 "2010-06-24T14:06:02Z")
> [@Unintentionally\_Blank](#): > > Some firewalls support ssl VPN, which would work for any device that can connect to https:// > > In my case, there’s an address bar on the webpage the firewall provides that will proxy an website you enter. So, you log into [https://bubbashouse.com](https://bubbashouse.com) and enter 10.0.0.20/brisket and it’ll show you what the internal website is offering. You have to expose your firewall management page to the public interface, then? Hmmm… that sounds workable, as long as it supports an ssl connection. What firewall(s) do this? I like the idea that I can solve this problem without ANY coding! ---
**Author:** ![Bricker](https://avatars.discourse-cdn.com/v4/letter/b/977dab/32.png) [@Bricker](https://boards.straightdope.com/u/Bricker)\ **Post date:** [June 24, 2010, 2:07pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/8 "2010-06-24T14:07:03Z")
> [@njtt](#): > > Sheesh! Isn’t being able to have your meat Tweet you when it is cooked geeky enough for you already! The very existence of this thread should answer that question in the negative. ---
**Author:** ![Unintentionally\_Blank](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/unintentionally_blank/32/5418_2.png) [@Unintentionally\_Blank](https://boards.straightdope.com/u/Unintentionally_Blank)\ **Post date:** [June 24, 2010, 2:18pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/9 "2010-06-24T14:18:02Z")
> [@Bricker](#): > > You have to expose your firewall management page to the public interface, then? > > Hmmm… that sounds workable, as long as it supports an ssl connection. > > What firewall(s) do this? I like the idea that I can solve this problem without ANY coding! NO! It’s not the management page! Tha would be Baaaaad! > **[Virtual private network](https://en.wikipedia.org/wiki/Virtual_private_network?wasRedirected=true)** > > A virtual private network (VPN) is an overlay network that uses network virtualization to extend a private network across a public network, such as the Internet, via the use of encryption and tunneling protocols. In a VPN, a tunneling protocol is used to transfer network messages from one network host to another. > Host-to-network VPNs are commonly used by organisations to allow off-site users secure access to an office network over the internet. Site-to-site VPNs connect two networks, such as ... ---
**Author:** ![Unintentionally\_Blank](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/unintentionally_blank/32/5418_2.png) [@Unintentionally\_Blank](https://boards.straightdope.com/u/Unintentionally_Blank)\ **Post date:** [June 24, 2010, 2:19pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/10 "2010-06-24T14:19:52Z")
Crap, that wasn’t very helpful. Cisco, smoothwall, astaro, openvpn, there’s a BUNCH of providers that give you VPN access using https. ---
**Author:** ![Bricker](https://avatars.discourse-cdn.com/v4/letter/b/977dab/32.png) [@Bricker](https://boards.straightdope.com/u/Bricker)\ **Post date:** [June 24, 2010, 2:34pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/11 "2010-06-24T14:34:07Z")
> [@Unintentionally\_Blank](#): > > In my case, there’s an address bar on the webpage the firewall provides that will proxy an website you enter. > [@Unintentionally\_Blank](#): > > NO! It’s not the management page! Tha would be Baaaaad! > [Virtual private network - Wikipedia](http://en.m.wikipedia.org/wiki/Virtual_private_network?wasRedirected=true) OK, so I’m obviously not understanding “an address bar on the webpage the firewall provides.” ---
**Author:** ![Kyrie\_Eleison](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/kyrie_eleison/32/7682_2.png) [@Kyrie\_Eleison](https://boards.straightdope.com/u/Kyrie_Eleison)\ **Post date:** [June 24, 2010, 2:48pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/12 "2010-06-24T14:48:00Z")
If you don’t want to go the VPN route, you could use an authenticating web proxy such as [squid](http://www.squid-cache.org/). I don’t know how you might disable forms though without either coding or maybe writing some impressive packet filtering rules. ---
**Author:** ![Bricker](https://avatars.discourse-cdn.com/v4/letter/b/977dab/32.png) [@Bricker](https://boards.straightdope.com/u/Bricker)\ **Post date:** [June 24, 2010, 3:00pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/13 "2010-06-24T15:00:06Z")
> [@Kyrie\_Eleison](#): > > If you don’t want to go the VPN route, you could use an authenticating web proxy such as [squid](http://www.squid-cache.org/). I don’t know how you might disable forms though without either coding or maybe writing some impressive packet filtering rules. Does squid let me authenticate independently to it? I thought squid would let me use it as a “helper” to pass NTLM-type authentication requests through it if the source site used NTLM, but can I tell squid, “Only accept proxy connections from someone who authenticates via username and password?” ---
**Author:** ![Kyrie\_Eleison](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/kyrie_eleison/32/7682_2.png) [@Kyrie\_Eleison](https://boards.straightdope.com/u/Kyrie_Eleison)\ **Post date:** [June 24, 2010, 3:10pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/14 "2010-06-24T15:10:44Z")
> [@Bricker](#): > > Can I tell squid, “Only accept proxy connections from someone who authenticates via username and password?” [Yup](http://wiki.squid-cache.org/Features/Authentication). ---
**Author:** ![Sailboat](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/sailboat/32/461_2.png) [@Sailboat](https://boards.straightdope.com/u/Sailboat)\ **Post date:** [June 24, 2010, 3:28pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/15 "2010-06-24T15:28:46Z")
> [@njtt](#): > > Sheesh! Isn’t being able to have your meat Tweet you when it is cooked geeky enough for you already! Having your meat tweet when it’s ready to eat is l33t. ---
**Author:** ![Duckster](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/duckster/32/1244_2.png) [@Duckster](https://boards.straightdope.com/u/Duckster)\ **Post date:** [June 24, 2010, 3:30pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/16 "2010-06-24T15:30:38Z")
You can avoid all the web scripting and use a quality router. I [just upgraded to this one](http://www.netgear.com/Products/WirelessRouter/WirelessRoutersforHighPerformance/WNDR3700.aspx), and so far in my configuring it, it’s appears possible for ordinary folks to be granted access to a web page to view your work while you have secure access to change settings. ---
**Author:** ![Musicat](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/musicat/32/20189_2.png) [@Musicat](https://boards.straightdope.com/u/Musicat)\ **Post date:** [June 24, 2010, 3:38pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/17 "2010-06-24T15:38:47Z")
> [@Sailboat](#): > > Having your meat tweet when it’s ready to eat is l33t. I thought it was a treat to beat your meat in the Mississippi mud? Sweet! ---
**Author:** ![Bricker](https://avatars.discourse-cdn.com/v4/letter/b/977dab/32.png) [@Bricker](https://boards.straightdope.com/u/Bricker)\ **Post date:** [June 24, 2010, 3:44pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/18 "2010-06-24T15:44:29Z")
> [@Kyrie\_Eleison](#): > > [Yup](http://wiki.squid-cache.org/Features/Authentication). OK, so this is a workable solution. I can use the PHP code above for the read-only, and use squid to allow inbound proxy connections for the full read/write access. You guys rock. Thanks. Now I’ll take a look at the hardware-only solution and see if it’s easier. 🙂 ---
**Author:** ![Unintentionally\_Blank](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/unintentionally_blank/32/5418_2.png) [@Unintentionally\_Blank](https://boards.straightdope.com/u/Unintentionally_Blank)\ **Post date:** [June 24, 2010, 3:47pm UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/19 "2010-06-24T15:47:24Z")
I haven’t watched the whole episode, but this is supposed to describe ssl VPN: [![](https://img.youtube.com/vi/GH38WNcfegY/hqdefault.jpg "Hak5 - Build a Free SSL VPN on Linux or Windows") ](https://www.youtube.com/watch?v=GH38WNcfegY) ---
**Author:** ![Stoid](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/stoid/32/272_2.png) [@Stoid](https://boards.straightdope.com/u/Stoid)\ **Post date:** [June 25, 2010, 4:06am UTC](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346/20 "2010-06-25T04:06:00Z")
> [@Bricker](#): > > By setting up a port forwarding rule on my home router, I can also monitor and change the smoker through my phone’s browser when we’re out shopping. > > Unfortunately, there’s no authentication option. That means that the only thing protecting me from some yahoo from messing with my meat is security by obscurity. > > . I am a computer geek going back almost 30 years. And the very idea of this is still completely bizarre to my little born in 1958 brain. [Next page](https://boards.straightdope.com/t/stealing-web-content-revisited-for-a-good-reason/544346.md?page=2)