LinuxSA Mailing list archives

  From: Mark Newton <newton@atdot.dotat.org>
  To  : David Newall <davidn@rebel.net.au>
  Date: Thu, 7 Dec 2000 10:52:12 +1030

Re: "Host based" routers and BGP

On Thu, Dec 07, 2000 at 02:15:24AM +1030, David Newall wrote:

 > I'm not arguing with your call to use a dedicated router for routing, but I
 > do wonder if Ciscos really are as reliable as you think they are.  As an ISP
 > you would get to see a lot of hardware problems from cheapo PCs, and maybe
 > even from not-so-cheapo PCs, but then you get to see an awful lot of PCs, in
 > fact far more PCs than routers I would guess.  That's likely to skew  your
 > perception.

Ok, let's be clear about this.  In all my years in this industry, I have
never seen a Cisco develop a hardware failure.  Not once.

I've seen one or two which could be classed as "infant mortality" or
"bathtub curve" failures, where they had problems very early in their life,
but (a) they have always been replaced within 24 hours, and (b) once 
installed, they have always, without fail, continued to work reliably.

I've run Ciscos on factory floors where the cooling fan housing fills
up with iron filings and dust;  I've run them in Rittal cabinets in 
ceiling cavities where the summer daily temperature can reach 50 degrees
celsius;  I've run them on dodgy mains power where they've suffered
90-minute-long brownouts while nobody was there to see what was going on...
and I've never seen a failure.

It's easy to hear someone say, "Products from vendor X are very reliable,"
and brush it off.  But don't underestimate the significance of a statement
like, "I've been using these products for ten years and have had a failure
rate exactly equal to zero percent."  I'm sure some people have seen
failures; Statistically, they *must* occur -- But I've never seen one.

 > Since you brought up power supplies: I've seen enough never-turned-off
 > cheapo PC's to say with some authority that about 4 years is a common age
 > for them to blow up with power supply problems.  (I don't know how long to
 > expect from a quality PC, whatever type that is!  IBM maybe?)  How long do
 > you expect your Cisco to last without failure?  (And do they ever have power
 > supply problems?)

I expect it to last forever.  When I install some ISP or telco
infrastructure in a location which is 400km away from anyone who is 
qualified to fix it, where it'll be servicing several thousand customers,
I expect the phrase "telco-style reliability" to mean that I can reasonably
expect that it simply won't break.  If the power is unreliable, the
airconditioning fails, and storm damage to the roof allows the cooling
fans to fill up with bees nests, I still expect that equipment to run.

Just to make sure, I'll be installing a UPS and redundant power supplies
where I can, as well.  I don't actually expect to *need* the redundancy, 
but I'll install it anyway just in case on equipment which supports it.

 > Actually what's more interesting to me -- because I expect that Cisco
 > hardware really is superb -- is what ongoing costs you have with them. 

It costs an absolute fortune, make no mistake.  The equipment itself is 
more expensive from day one, and the maintenance contracts cost about 
10% of the value of the hardware in any given year.  But it's well worth
the money, because that maintenance contract means you have very rapid
response in the unlikely event that you'll ever get a hardware failure,
and the software support is almost always second-to-none.

However, on some of my network links, the amount of money I lose from
fifteen minutes of downtime is sufficient to pay the maintenance contract
on the routers for 12 months (even if the customer impact is zero; consider,
for example, what happens if one of the links to one of my cheaper sources
of upstream bandwidth fails, and BGP shifts the load on to one of my more
expensive sources without actually impacting customer connectivity:  I might
end up pulling many hundreds of gigabytes of traffic over a link which costs
a couple of cents more per megabyte to run, and that's going to cost 
thousands of dollars.  I'd rather pay the maintenance contract, thanks :-)

 > I'm willing to believe that the
 > Cisco you buy today will still be perfectly serviceable in ten years time or
 > longer, but they certainly have software upgrades at least as often as your
 > quality open source OS.  It seems that Cisco release security patches almost
 > as often as Windows, so the effort for upgrading software might be worse
 > than for host based routers. 

No, the upgrade philosophy is different -- You only ever upgrade them if
you're suffering from a bug which the upgrade is likely to fix (let's face
it, you're not going to upgrade due to new features, because that router
wouldn't be in place if it didn't already have the features it needed to
service customers).  There are very few downsides associated with continuing
to run an old-but-stable version of IOS, so although a new version of IOS
comes out every few weeks, upgrades are relatively rare.

 > (Worse in the sense of more often, although
 > it's probably a no-brainer each time, which in that sense is better.)

Yup;  Type a single command to load an OS upgrade from a file server.

 > If you don't buy a support contract do you get these patches for free, or do
 > you have to shell out extra cash? 

You shell out extra cash.  You often find that the amount of cash you shell
out is more than the cost of the maintenance contract, so there's no good
reason to avoid having one.

The cost of a maintenance contract should be factored in as part of the cost
of running the network.  You really shouldn't run critical network
infrastructure without one.

   - mark

--------------------------------------------------------------------
I tried an internal modem,                    newton@atdot.dotat.org
     but it hurt when I walked.                          Mark Newton
----- Voice: +61-4-1620-2223 ------------- Fax: +61-8-82231777 -----

-- 
LinuxSA WWW: http://www.linuxsa.org.au/  IRC: #linuxsa on irc.linux.org.au
To unsubscribe from the LinuxSA list:
  mail linuxsa-request@linuxsa.org.au with "unsubscribe" as the subject