LinuxSA Mailing list archives

  From: Alan Kennington <akenning@topology.org>
  To  : LinuxSA <linuxsa@linuxsa.org.au>
  Date: Fri, 8 Jun 2001 23:15:43 +0930

Re: Small Window Sizes, repeater latency, and Ethernet over Fiber ...

On Wed, Jun 06, 2001 at 08:13:16PM +0900, Richard Sharpe wrote:
> 
> Assuming TCP, and given that some Windows clients (NT SP1 at least, and
> maybe more) use a window size of 8192 bytes or thereabouts, and there are
> often a lot of ethernet hops, they could be right. Eg,
> 
> 
> [Node]----[Hub]----[Router]----[Enet to fiber]---[router]---[hub]---[Node]
>                                 link frm telco
> 
> If this were 10MB/s Ethernet, we have 6 hops over Ethernet plus the fiber
> link from end-to-end, for a min of at least 8mS end to end latency and a
> round-trip-time of at least 12mS (ack's can be smaller).

Richard,

Maybe the most convenient way of viewing the dealy-bandwidth thing is this:

max bit-rate <= window / round-trip-time.

More generally, bit-rate = min(line rate, window/round-trip-time).

In this case, you get for the ridiculously low 8k window:
8192*8/0.012 = 5461333 bits/sec
for the 12 mS round trip.

This way you don't have to know the line rate in order to knwo the limit
on bit-rate caused by the window.

So at 32 kByte window,
32768*8/0.012 = 21,845,333 bits/sec.

So bit-rate = min ( line-rate, 21.8 Mbits/sec ).

> But, a window of 8192 bytes will take about 6.6mS to fill up, after which
> we have to wait for ACKs to come back before we can send any more. Thus we
> waste almost 50% of the available throughput, and can only achieve about
> 5Mb/s. If the latency is any worse, then the throughput for any TCP
> connection going over the link above would be worse as well. Simple
> solution is to increase the window size. Linux uses a window of 32k, and we
> could go up to 64k without using the window scale option. 100Mb/s operation
> makes it worse. (We can use the delay-bandwidth product to estimate the
> minimum window size we need.)

It's a trivial rearrangement that I'm suggesting, but I just find it makes
it clearer for me.
I.e. I make the bit-rate the dependent variable rather than the window size.

> 2. What are the latencies of hubs and switches (in store-and-forward mode)
> and routers. Experience suggest that Cisco 3600s add very little, and that
> switch latency should be lower than link latency.

I don't have much of a clue on this.
But.... suppose a router at least reassembles an entire packet
before sending it off on its way.
This is fairly reasonable, although it's only the packet header that
really requires to be reassembled.
But if the whole packet, e.g. 1500 Bytes is reassembled, then that's
1500 Bytes off your window size every time you get a hop.
For, say, 20 hops round trip, that's 30 kbytes. So your whole delay
budget may be used up entirely in packet reassembly time.
I'm pretty sure that linux boxes definitely _do_ incur this delay.
If Cisco router latency is much less than 1500 Bytes / line-rate,
then clearly they're just reassembling packet headers.
If they're just doing the header reassembly, and then on-sending
the packet immediately, then you should get, obviously about
20 bytes (or 40 bytes for TCP reassembly too) of latency per hop.
That shouldn't affect the window budget much.

Just a few musings....

Cheers,
Alan Kennington.

-- 
LinuxSA WWW: http://www.linuxsa.org.au/  IRC: #linuxsa on irc.linux.org.au
To unsubscribe from the LinuxSA list:
  mail linuxsa-request@linuxsa.org.au with "unsubscribe" as the subject