CISCO guru's...need help with routing problem

Factual Questions
1

Unfortunately I forgot to get a dump of the config and I’m writing this in my hotel room from memory.

Today was a BAD day. Customers firewall went tits up and there was no firewall to spare…and the customer didn’t have any documentation on their network at all. Essentially what they have is an external router with a T-1 and a frac-T in two serial ports going into a fast ethernet port which hooks into (the now defunct) firewall, going into some kind of filtering appliance which then hooks into their layer 3 switch that is serving up their VLANs on their local network. There is no routing on this network…it simply uses default routes with each device pointing to the device above it and the external router set up to do load balancing with unequal weighted default routes pointing at the T-1 as the primary and frac-T as the secondary.

Seemed like a simple problem, even considering the lack of documentation…problem is I couldn’t make it work no matter what I did. I tried for about 10 hours today before I gave up before my head exploded and headed back to the hotel to think about it. I’m drawing a blank…I have no idea why.

What was happening was that I could ping from the interior network all the way through the external router and even hit the ISP’s gateways…but I couldn’t go beyond that. Yet, the external router could ping and resolve to the ISP’s DNS server with no problem, so I know that both links were up. I tried shutting down first one then the other link. I eliminated the filter box. I reconfigured and eventually eliminated the 3560 layer 3 switch. I verified that the internal DNS was pointing outside (they are using Windows 2003 with some kind of weird conditional forwarding thingy I had never seen before…but I eliminated this as a possible problem by simply trying static addressing pointing directly to the ISP’s DNS servers).

I know this isn’t a lot to go on, and I doubt I’ll even get a response to this…but if anyone has any thoughts they would be gratefully received. Even off the wall stuff. This is only a default routing system…this stuff should be cake. Yet I can’t make it work!

-XT

2

Yow. Sorry, I don’t have anything to offer except empathy.

But I thought it might be appreciated after a hard day.

3

I know these are very basic considerations, but I just throw them out on the off chance: Do the hosts beyond the external router such as the DNS servers know how to route back to the interior network? Maybe internal addresses not getting NAT-ed when they should?

4

Appreciated. :slight_smile:

Sometimes it’s the simple things that bite you on the ass. Essentially Usram was on the right track…the problem turned out to be NAT. The old firewall (which has now been replaced and thankfully a backup image of it’s config found and re-applied) had a truely bewildering array of NAT rules…and this was preventing traffic from getting out correctly as I wasn’t applying them to the external router. I’m still not sure why I could ping the ISP’s gateway router (that shouldn’t have worked), but putting the new firewall in and putting the old config back on it (with some modifications) has done the trick anyway. And now I’ve started documenting this weird configuration so this doesn’t happen again. After I’ve puzzled out how the hell any of this crap actually manages to work of course.

Anyway, my thanks for the thoughts.

-XT