RE: EF% vs. BCN

[Sanjay Agrawal <sanjay@xxxxxxxxxxxxxxxxxxxx>]

Title: RE: EF% vs. BCN

My analysis computed the worst case.

(a) What I computed was the maximum jitter that can happen if two packets of a given flow traverse two different network paths (which is not a good network design anyway) to

convince you that network jitter cannot increase beyond number of voice flows.

You should not have looping in you network.

 

b) I didn't make any assumptions about the switch fabric.  I assumed class based strict priority scheduling for classes where EF is serviced first . Class based queueing is implemented today in input or output queued architectures.

 

Sanjay K. Agrawal

Luminous Networks

 

 

-----Original Message-----
From: Krishna Pattabhiraman [mailto:Krishna@xxxxxxxxxxxxxxx]
Sent: Thursday, September 28, 2000 3:09 PM
To: 'Sanjay Agrawal'; 'stds-802-rprsg@xxxxxxxx'
Cc: Arun Sastry; Vinay Bannai
Subject: RE: EF% vs. BCN

Sanjay,

 

BTW, in your analysis of guarantees, I wouldn't call it worst-case guarantees.

(a) At any queue, the flows in that queue may have traversed different number of hops - there is an inherent assumption in your analysis that all flows have traversed the same number of hops. Thus, some flows may already have "looped around" and may have more than 1 packet in the queue.

(b) You have assumed that the switches on the path are "shared-memory" switches - thus, there is no contention at the input of output port apart from 1 BE packet or other EF packets for the output port. Its not unreal to assume that there are only shared memory switches - but typically >40Gb/s switches are crossbar or some MIN structure. In these switches, there is a potential delay in arbitration/conflicts. It could be O(N) packet times (where packet time is the time to traverse the fabric link from input to output) where N is the switch dimension.

 

Having said the above, I would like to add that we are re-iterating the same thing - the n/w has to provisioned such that the performance obtained using EF is acceptable. There is no doubt that EF is a simple approach to provide better performance than the normal BE traffic. If the user is happy with probabilistic guarantees we can increase the amount of EF traffic in the network.

 

EF is a good mechanism to provide for example FR-type service like {CIR, BR}, where CIR/BR is measured over some averaging interval which is much much more than a packet time. In other words, where asymptotic guarantees are enough. My understanding is that most of the SLAs fall into this bracket. On the other hand, if we are expecting the phone companies to start sending voice traffic over an EF domain then extra extra care has to be taken. 

 

Krishna

 -----Original Message-----
From: Sanjay Agrawal [mailto:sanjay@xxxxxxxxxxxxxxxxxxxx]
Sent: Thursday, September 28, 2000 5:18 PM
To: 'Krishna Pattabhiraman'; Sanjay Agrawal; 'stds-802-rprsg@xxxxxxxx'
Cc: Arun Sastry; Vinay Bannai
Subject: RE: EF% vs. BCN

Again, please refer to provision right definition. Provision right makes sure that the worst case or (in acceptable probability case) you will meet the intended QoS constraints. If 1/3 provisioning with worst case gaurantee is too low, and you can't use the remaining link bw for other services, diffserv architecture is probably not a right solution for your needs. If you are willing to live with some probabilistic guarantees (something like 99.99%), you can provision higher fraction (Please see Van Jacobson, Kathleen Nichol's document). Diffserv architecture never promises that for 100% provisioning levels you will meet QoS constraints for voice. How low is low and how high is high is entirely dependent on network service cost constraints. Also, low voice provisioning doesn't mean low link utilization. The remaining bandwidth is used by other classes that don't need stringent timing guarantees.

 

For worst case guarantees:

-------------------------------------

For a gig link for 1000 voice flows you get 20ms/ (.544ms + .012ms)  = 35 hops

for gig link for 10000 voice flows you get 20ms/(5.4 +.012) = 3.69 = 3 hops

 

This is lot of voice call at a gig link cost, maybe utilization for voice only traffic be low but remaining link is available for other data services. 

 

The point is that diffserv architecture with right provisioning does provide QoS guarantees for voice.

 

-Sanjay K. Agrawal 

-----Original Message-----
From: Krishna Pattabhiraman [mailto:Krishna@xxxxxxxxxxxxxxx]
Sent: Tuesday, September 26, 2000 3:24 PM
To: 'Sanjay Agrawal'; 'stds-802-rprsg@xxxxxxxx'
Cc: Arun Sastry; Vinay Bannai
Subject: RE: EF% vs. BCN

Lets extend your analysis. Lets take the example number of 500 voip calls - this occupies about 14Mb/s - which is less than 1/3rd the DS3 capacity. This will cause 6 ms of J2 - thus the number of hops is 3 before we see back2back voip packets of a flow. 3 hops is common occurence.

 

So, what needs to be defined is "right provisioning" for EF - which if conservatively estimated may be too low for certain topologies - and if optimistically estimated, EF may become a "better-than-best-effort" service.

 

Krishna

-----Original Message-----
From: Sanjay Agrawal [mailto:sanjay@xxxxxxxxxxxxxxxxxxxx]
Sent: Monday, September 25, 2000 8:27 PM
To: 'Krishna Pattabhiraman'; 'stds-802-rprsg@xxxxxxxx'
Cc: Arun Sastry; Vinay Bannai
Subject: RE: EF% vs. BCN

Hi Krishna,

 

What you are talking about is voice streams folding on themselves to cause jitter. That von't happen if you design your network right or provision it right . VoIP

streams generate 1 68 byte packet every 20 milliseconds. Thus for voice streams to fold on itself you need least 20ms of jitter in the diffserv network. EF streams are shaped on the egress of each diffserv domain. Now the question what is the maximum size of diffserv network before you have to reshape the streams to 20ms of interpacket spacing.

 

 

 

Please refer to my attached document. Here are the numbers:

 

for 100 voip calls

 

For network consisting of DS3 links: 20ms / (.26ms +1.2ms) = 13 hops

For networks consisting of OC12 links: 188 hops

For network consisting of 100mbps links: 30 hops

for network consisting of 1Gbps links:  300 hops

 

 

-Sanjay K. Agrawal

 

 

 

 

 

-----Original Message-----
From: Krishna Pattabhiraman [mailto:Krishna@xxxxxxxxxxxxxxx]
Sent: Wednesday, September 20, 2000 12:43 PM
To: 'Sanjay Agrawal'
Subject: RE: EF% vs. BCN

Sanjay,

Here is an example - used usoft word because it was simpler for diagram. In this example, I'll show how delay accumulates over each hop. Even though the burstiness on a link does not exceed the # of voice flows, it is possible for delays to accumulate due to the "spatial" burstiness - over number of hops, it then translates to a burst in the flow itself even if it was shaped coming into the network.

 

Krishna

-----Original Message-----
From: Sanjay Agrawal [mailto:sanjay@xxxxxxxxxxxxxxxxxxxx]
Sent: Tuesday, September 19, 2000 6:07 PM
To: 'Krishna Pattabhiraman'
Subject: RE: EF% vs. BCN

Hi Krishna,

 

I am famaliar with Anna Charny's work. I have had few interactions with her regarding this. Van Jacobson, and Kathy have tried to convince her but without any luck. Problem with her assumption is that she is not considering provisioning for the critical link on the diffserv network. At each hop she is considers all the possible new flows. Total number of flows provisioned for the critical link are much smaller than number of flows than can accumulate at each hop. This equations is a result of this basic error. The maximum burst will never exceed the number of voice flows at any provisioning level unless there is provisioning error.

 

-Sanjay K. Agrawal

-----Original Message-----
From: Krishna Pattabhiraman [mailto:Krishna@xxxxxxxxxxxxxxx]
Sent: Tuesday, September 19, 2000 1:28 PM
To: 'Sanjay Agrawal'; Raj Sharma; Reflector RPRSG
Cc: 'Gunes Aybay'
Subject: RE: EF% vs. BCN

Hi Sanjay,

If you do not shape, and maintain a very small EF utilization in the network, you can get some delay bounds - however they are only reasonable for small utilizations and small number of hops in the network. But in general, for higher utilizations, all bets are off, and examples can be given when in fact burstiness/delays grow at least as fast as (1+util)^(hop count) times f(util) where f(util) is linear in util (EF utilization factor).

I dont know if you followed the discussion on the diffserv work. I would refer you to work by Anna Charny on this - u'll find references to it in the mailing list.

Following up on why EF should not be shaped (actually I had posed this question earlier!!) - If you shape aggregate EF traffic at each hop, the worst case behavior is even worse than if you do not shape it (intuitively this is because you can think of aggregate shaping as roughly running EF on links of small capacity which are fully utilized by EF, so by the above, you get worse behavior).

If you shape EF on an *edge-to-edge* aggregate basis (that is the aggregate of all EF flows between a particular ingress and egress pair), then you can indeed substantially improve the jitter and delay bounds. That requires identification of *edge-to-edge "flows" at each internal hop at the domain - so just looking at the DS codepoint no longer suffices. 

Regarding the ECN discussion - I did not propose ECN as a mechanism - I just chanced upon the posting and commented generally on it. Pardon me for nit-picking, RED (in its pure VJ/SallyFloyd form) is also not the best mechanism for transient or persistent congestion - there are variants that perform better. Again, best, better or worse depends on the metrics of interest for the service provide/eqmt maker.

I hope that helps.

Krishna

-----Original  Message-----
From: Sanjay Agrawal [mailto:sanjay@xxxxxxxxxxxxxxxxxxxx]
Sent: Tuesday, September 19, 2000 4:08 PM
To: 'Krishna Pattabhiraman'; Raj Sharma; Reflector RPRSG
Cc: 'Gunes Aybay'
Subject: RE: EF% vs. BCN

Hi Krishna,

 

I have few comments.

 

EF train will grow but it can never grow to be more than EF packets equal to number of active connections unless there is a provisioning problem. For provisioning please see Van Jacobson's and Kathleen Nichols's recent results in

http://www.ietf.org/internet-drafts/draft-ietf-diffserv-ba-vw-00.txt

 Voice streams produce packets periodically with a fixed time period. If provisioned right within the fixed time frame there will never be voice packets more than number of connections. Shaping won't help to reduce jitter it may help to reduce N degree buffering requirements at each node which is no big deal.

 

RED accomplishes both transient and persistent congestion. ECN helps remove persistent congestion through marking instead of dropping. Unfortunately, it is not widely utilized.

 

-------------------------------------

Sanjay K. Agrawal, Ph.D.

Luminous Networks

Cupertino, CA 95014

 

-----Original Message-----
From: Krishna Pattabhiraman [mailto:Krishna@xxxxxxxxxxxxxxx]
Sent: Wednesday, August 30, 2000 9:06 AM
To: 'Raj Sharma'; Reflector RPRSG
Cc: 'Gunes Aybay'
Subject: RE: EF% vs. BCN

Hi Raj,

I fail to understand why you think that EF traffic cannot be shaped? Even the best of scheduler/rate controllers add at least a packet to the incoming burst length. So, if you do not shape the traffic on the way out, these bursts get accumulated and hence at hop h, you will see O(h) burst sizes even for EF traffic (which might have entered the network in a tdm/cbr manner). In such cases, buffer sizing for EF traffic has to take into account such bursts, else we'll notice pkt drops even for EF traffic (which is not preferrable).

 

The above burstiness was just due to the rate limiter at each input port. Now in addition to that, we might have "spatial" burstiness  - this adds a "fan-in" dimension to the burst. So, the "EF train" could grow to be a big one and rate limiters at downstream nodes may end up dropping them.

 

Regarding congestion indication - there is persistent congestion and then there is transient congestion - the congestion that you are talking about looks like transient congestion and should ideally be dealt locally (pick your flavor of RED) - congestion of persistent nature needs to be propagated back to upstream nodes (using BECN etc).

 

Krishna

-----Original Message-----
From: Raj Sharma [mailto:raj@xxxxxxxxxxxxxxxxxxxx]
Sent: Friday, August 18, 2000 7:00 PM
To: Reflector RPRSG
Cc: 'Gunes Aybay'
Subject: RE: EF% vs. BCN

Hi Gunes,

Maybe I failed to clarrify.

EF can be bursty data or constant bit rate source.
EF could be rate limited. Once admitted at the ingress
it assumes a very unique personality. In fact, it cannot be
shapped. As a matter of fact, most EF are more like TDM
traffic.
It has a specific queue treatment: it must be
sentout immediately without preempting the current packet
or getting preferential treatment over another EF packet
in the queue.
This preferential treatment gives it a "speedup" over other packets
and hence all packets will get clumped near a choke point At this choke
point the rest of the traffic has to wait till the "EF train"
passes through.

So, this would be similar to assesing traffic congestion by the average
wait time for cars on a road upstream from a railroad crossing.
The train always gets priority over the cars.

Which point in time will you pick to signal congestion?

-----Original Message-----
From: Gunes Aybay [mailto:gunes@xxxxxxxxxxxxxxxxx]
Sent: Friday, August 18, 2000 11:33 AM
To: Reflector RPRSG
Subject: Re: EF% vs. BCN

I don't necessarily agree that EF packets will be arriving together when
EF traffic is given
the highest priority. In practice, EF will be used for types of traffic that
require bandwidth,
latency and jitter guarantee, which are typical of voice and multimedia
applications. EF can
also be used to emulate TDM services over IP. By definiton, I would expect a
traffic class
that requires commited bandwidth and controlled jitter and latency not to exceed
the committed
rate. In any realistic implementation, the ingress devices would actually have
to do policing and
shaping with minimal burst allowance to enforce this kind of behavior,
otherwise, I cannot
see how EF can be made to work in any kind of network topology. If individual
EF traffic flows
are shaped so that they do not exhibit bursty behavior,  it would be reasonable
to expect that
aggregatetion of EF  traffic would also not exhibit bursty behavior .

I beleive a typical end-to-end EF deployment over an IP network would look like
this:

- Assignment to EF class would be subject to network level provisioning. The
provisioning
    system would have to know the total and currently utilized EF capacity for
each congestion
    point in the network.
- EF traffic will flows will be relatively  uniform (i.e. not burst) over time.
- EF traffic flows will be subject to admission control at the original ingress
point of the network.
    A policing based admission control scheme is OK for traffic that is not
inherently bursty, but
    a shaping function may have to be used for bursty flows. (I don't see why
one would map
    bursty flows into EF class, though)
- EF traffic will be subject to aggregate admission control at the admission
point to the ring. (It is
    easier to implement an aggregate shaping function at this point)

In such an implementation, I would expect to see the state of the high priority
queue used for EF to
be near-empty in a sustained fashion, i.e., if the total bandwidth allocated for
EF is less than
the ring

It is also  interesting to look at the implications of spatial reuse and
failover conditions on the
network level provisioning  mechanism that is used to manage EF traffic  in a
ring topology.

Gunes Aybay
RiverStone Networks