Stealing web content, revisited for a good reason

I seem to recall that Stoid got in a bit of trouble with a few folks here by asking a variant of the question I’m about to ask… so let me explain why I’m asking.

I have The Stoker. It’s a small appliance that monitors the temperature of the meat and interior of my Big Green Egg smoker, and (when connected to my home network) even lets me check and modify temperatures on my beef brisket from inside the house, using a laptop that connects to the device’s built-in web server.

By setting up a port forwarding rule on my home router, I can also monitor and change the smoker through my phone’s browser when we’re out shopping.

Unfortunately, there’s no authentication option. That means that the only thing protecting me from some yahoo from messing with my meat is security by obscurity.

I’d like to fix that. I’d like to have another web server in the house be the one that’s visible to the public, and I’d like it to present whatever content my little internal built-in web server is presenting. This way, I could enforce some sort of authentication.

Bonus: would there be a way to present a read-only version of the information as well as a “real” version? The device’s page just uses forms to accept changes to the desired temperatures and alert ranges. I’d love to able able to post in MPSIMS, “Hey, I’m cooking a pork shoulder – check it out at http://www.bricker-big-green-egg.com:9119,” without worrying that someone here would think it was funny to change my cook temp from 225 to 525.

So that’s the background. Is there a way I can code a web page to display exactly what Website B is showing, when the client can’t directly connect to Website B? And if there is, is there one method that would support interacting with the forms on Website B and another method that would just show the info without letting the client interact with the form?

I know networking pretty well, but very little about coding.

A coding solution I do not have, however:

I urge you to adopt this as your sig.

What OS’s do you have available? What is the firewall protecting your network? There’s many ways of skinning this cat. Several firewall products provide VPN functionality, which would handle the remote access to read/write data. Look up the curl or wget commands to pull the content from web page, which you could then place on -another- web server for read only use.

I have Windows and a couple of Linux flavors available to host the second site… so basically I have IIS or Apache easily available.

The problem with VPN is that I’d need a VPN client for my phone’s browser, or more accurately for my phone, period.

Hmmm… looking up curl brought me to a page that lists this technique under “Don’t Do:”


<?php print read_file('http://example.com'); ?>

But why not? Their advice is, “Don’t do this, because someone could hack the example.com site, and then you’ll be screwed,” but that’s not a concern here. (I’m assuming that ‘example.com’ gets resolved and connected-to server-side, not client-side - yes?)

This may solve my “read-only” problem quite nicely.

Sheesh! Isn’t being able to have your meat Tweet you when it is cooked geeky enough for you already!

Some firewalls support ssl VPN, which would work for any device that can connect to https://

In my case, there’s an address bar on the webpage the firewall provides that will proxy an website you enter. So, you log into https://bubbashouse.com and enter 10.0.0.20/brisket and it’ll show you what the internal website is offering.

You have to expose your firewall management page to the public interface, then?

Hmmm… that sounds workable, as long as it supports an ssl connection.

What firewall(s) do this? I like the idea that I can solve this problem without ANY coding!

The very existence of this thread should answer that question in the negative.

NO! It’s not the management page! Tha would be Baaaaad!

Crap, that wasn’t very helpful. Cisco, smoothwall, astaro, openvpn, there’s a BUNCH of providers that give you VPN access using https.

OK, so I’m obviously not understanding “an address bar on the webpage the firewall provides.”

If you don’t want to go the VPN route, you could use an authenticating web proxy such as squid. I don’t know how you might disable forms though without either coding or maybe writing some impressive packet filtering rules.

Does squid let me authenticate independently to it? I thought squid would let me use it as a “helper” to pass NTLM-type authentication requests through it if the source site used NTLM, but can I tell squid, “Only accept proxy connections from someone who authenticates via username and password?”

Yup.

Having your meat tweet when it’s ready to eat is l33t.

You can avoid all the web scripting and use a quality router. I just upgraded to this one, and so far in my configuring it, it’s appears possible for ordinary folks to be granted access to a web page to view your work while you have secure access to change settings.

I thought it was a treat to beat your meat in the Mississippi mud? Sweet!

OK, so this is a workable solution. I can use the PHP code above for the read-only, and use squid to allow inbound proxy connections for the full read/write access.

You guys rock. Thanks.

Now I’ll take a look at the hardware-only solution and see if it’s easier. :slight_smile:

I haven’t watched the whole episode, but this is supposed to describe ssl VPN:

I am a computer geek going back almost 30 years.

And the very idea of this is still completely bizarre to my little born in 1958 brain.