Content delivery networks, advertising optimisers. Why are they effing up my browsing?

Over the last month or so, several websites have become basically unusable for me, whether I’m trying them with my Windows7+Firefox system or my Ubuntu+Chromium system. The damn thing just doesn’t load, no matter how long I wait. Adblock does not help. Chromium at least has the decency to tell me who it is waiting for, and the result is invariably, not the website whose content I wanted, but somebody else who is ‘managing their content’. Christ, just that phrase makes me think I should perhaps post this in the Pit instead.

Fr’instance, I can’t get The Onion, because Chromium is waiting for cdn.optimizely.com. Actually in the latter case, if I wait long enough, then when I finally get fed up and go to another site, as soon as I’ve clicked the bookmark, my browser decides it can finally show me everything it did manage to get from The Onion… which happens to be everything I wanted to see in the first place. But for other cases I never get to see anything.

In fact, the same thing is ALMOST happening with the SDMB. I am forever waiting in this case for “optimized-by.rubiconproject.com”. But, crucially, the site still displays and I can still view and make posts while my browser is waiting for whatever the hell it is waiting for.

Anyway, somebody please tell me what setting to change, to make these optimisers go and optimise themselves?

Seems like those optimizers are actually advertising and surveying/tracking companies. Are you sure your adblock software is actually blocking them?

There are several different adblockers. Make sure you’re using Adblock Plus with EasyList, which should block rubiconproject.com by default.

If you’re still having issues, just manually add those domains to your adblocker’s blacklist.

For other pages: In both Chrome and Firefox, you can push CTRL-SHIFT-I to bring up a timeline that clocks the loading times of every element on the page. You can see whatever’s slowing down the loading and blacklist it.

Call me old-fashioned, but I’m still blocking a whole lot of unwanted sites by
re-directing them to 127.0.0.1 in my hosts file.

Example line:


127.0.0.1     ad.doubleclick.net


You might be mixing up two things here. A CDN is a bunch of servers used to serve up content quicker than a general purpose web server can - often they’re spread around the world and designed so your browser will fetch from the nearest one. I gather you know this much already. There’s nothing inherently wrong with them.

But The Onion isn’t using Optimizely or Rubicon to deliver the page content. Optimizely and Rubicon are using their own CDNs to deliver javascript that’s used for testing and tracking. I don’t know much about Rubicon, but Optimizely is a test tool - The Onion is probably using it to test small changes to their page layout and measure which ones work better.

Are you using something like a VPN server, or Google’s Public DNS or OpenDNS? They can sometimes mess with CDNs.

Me too.
I use the hosts file provided by http://www.mvps.org/winhelp2002/, with additional domains added as they annoy me.

I found that file a year or so ago – But I decided not to use it when I saw how many zillions of entries it has in it! (If I’m not mistaken, the entire file is loaded into memory and kept there, I think.) So I just built my own, adding lines as I encountered ones that annoy me most frequently. (Since I have a little in-house network here, I have a bunch of other entries for those too.)

ETA: It so happens that this machine I’m using also has Apache web server on it, so all my 127.0.0.1 aliases actually do bring up a web page – my own home page, which is just a tiny stub with a few words to tell me it’s working.

You may want to add NoScript to your list of add-ons. (Actually I would suggest that you would be insane not to.) NoScript will block execution of the scripts, and for some sites that have been written to require that these data gathering scripts actually run, it inserts faked results, to trick everything into running.

NoScript is a bit of a pain in the first place, as you need to add sites manually to the allowed list. But after a while things settle down, and it becomes a matter of reflex adding new trusted sites as you visit them, and also instructional as you see just how ridiculous the Web has become.

As a matter of personal security, NoScript is mandatory IMHO.

Thanks for all the replies. I’ve made sure to reinstall adblock plus with easylist but that hasn’t helped. What still perplexes me is the question of WHY all these elements aren’t loading, not so much whether I want them to or not. If they did always load instantly I probably wouldn’t care that they were there. Which brings me to…

No, none of those. BUT, do you think it could be a DNS problem? My ISP said they were going through some upgrades a month or so ago, and now that I think of it, that’s about when these sites stopped working for me. Not to mention that my internet connection has also been broken completely for long periods since then. Apart from switching ISPs, do you think I should moan to my current one about whether its DNSes are working okay?

If they’re NOT, what should I do? Perhaps I could actually start using Google’s public DNS!

Considering you are having problems on 2 different machines with different browsers and OSes, I would highly consider that the problem is somewhere in your pipe, not the browsers.

Definitely check out different DNS servers. Also if you have a router try connecting without it (plug your ethernet cable right into a machine) and see if that helps too.

I use ghostery in addition to adblock. Ghostery’s emphasis is more on beacons, tracking cookies and the like, than on stuff that’s actually visible to the user. Plus, it lets you see all the extra cr*p that pages load, choose which ones you want to block. I’ve used it a few times to disable optimizely.com scripts when it was slowing down page loads.

Some technical details about your situation: your browser actually already has all the content it needs to display the page, it’s just waiting to download and/or execute a script that’s hosted on an optimizely.com server. If you tell it to ignore that script, your page should load just fine.

Agree with the others saying this problem is most likely due to a bad/slow DNS server.

As another note, correctly coded websites don’t block on third-party JS. IMO you’re in the right to email a complaint to those sites, and tell them to load third-party JS only after their first-party content has loaded. (Although that is admittedly tricky with A/B testing JS.)

… so, I finally found out what it seems to have been. My MTU was set too high. It was 1500, and should have been 1400.

Why the hell should I have had to learn what an MTU was and how to change it? Why was it set too high? Why did this crippling problem only start a couple of weeks ago? Effed if I know. And now, effed if I care.

Are you sure about that?

Yeah, how did you make that determination? Are you using PPPoE, or possibly some other encrypted or anonymizing connection?

Well I’m only sure about it in the sense that, I made the change and the problem immediately went away and hasn’t come back. Whether it was the most appropriate solution, I’ve no idea. And I found it from here: Site Unavailable

(Zen are my ISP)

That thread title described my situation perfectly. Notice how people also went straight to ‘it’s a DNS problem’. “wuggy” in post #5 brought up the MTU thing.

My router says PPPoA.

Apparently, that does not have the same MTU restriction as PPPoE. From Wikipedia (bolding mine):

Still, your modem, firewall, or router could be set to use a smaller than default MTU size. How it would have changed if it was working properly previously is a mystery, but then so is the effectiveness your found solution. Under normal circumstances, there is no reason to change the MTU size.

It’s like an episode of House, but for computers.