|
Agree.
> While I
agree that it is probably still a little premature to be finalizing the metrics
...
That's why I mentioned that it would be considered
closed *for now*.
> This is also an iterative process.
Exactly!
I'm suggesting that we get started on the first
iteration now, since we'll learn a lot out of it, and it will help us move
forward quicker on next iterations. That's as opposed to waiting till everything
is in place and then start digging around.
Any comments are welcome.
Khaled Amer President,
AmerNet
Architecture Analysis and Performance Modeling Specialists Phone:
(949)552-1114
13711 Solitaire Way, Irvine, CA 92620 Fax:
(949)552-1116
e-mail: khaledamer@xxxxxxx
----- Original Message -----
Sent: Tuesday, August 15, 2000 2:14
PM
Subject: RE: RPR Perf. Metrics
Discussions
While I agree that it is probably still a little
premature to be finalizing the metrics (They certainly can't be finalized in
advance of the PAR), it is not premature to discuss them. The process
should proceed from the PAR criteria to specific questions supporting the PAR
criteria. These questions should then evolve into specific
metrics. The metrics are then grouped by like issues and scenarios are
developed that adequately test proposals against these criteria. Once
the scenarios are developed, models are built, etc.
This is also
an iterative process. In the development of the scenarios, we will come
across issues that weren't originally considered and we'll need to update the
metrics...
I think once the PAR is finalized, we will see additional
discussion on the metrics question. Of course, if we had some existing
models available for general consumption to provide a common understanding of
what people intended, we might also generate some significant discussion...
:-)
Taylor
At 01:35 PM 8/15/00 -0700, Raj Sharma wrote:
Khaled,
Dont lose hope yet !
My take on the silence is that it maybe
still premature to discuss this.
Although, it needs to be launched
now. Without having a good reference
model it maybe be difficult to scope
the perfomance. Not to be pedantic, but,
Taylor's tutorial suggested we
that we define a model before establishing
requirements or metrics for
performance.
I personally
feel we are biting more based on the broad nature of
performance
metrics we are going after. It maybe time to approach
this from a reference model
point of view and then establish performance
metrics.
Comments?
raj
-----Original
Message-----
From: Khaled Amer [mailto:khaledamer@xxxxxxx]
Sent: Wednesday,
August 16, 2000 12:25 AM
To: Reflector RPRSG
Subject:
Fw: RPR Perf. Metrics Discussions
- RPR'ers,
Based on the 'underwhelming' responses that I got back on this, we'll
consider the performance metrics discussion closed for now, and move on.
As I mentioned before, I want for us to work on identifying the
scenarios, and metrics that we want to use as a starting point. I know
that some of you already put extensive efforts into producing simulation
results, but we want to all get to run a consistent subset for various
proposals. I've already posted some preliminary thoughts and suggestions
(and got the same underwhelming response!).
Please share with us any thoughts that you have on this subset, and if
you have suggestions on more details. In the meantime, I'll work on
putting together more details too. I'd like to see us go to the Interim
meeting with some of these discussions already started, so that we just
work on ironing out any points of disagreements that we may have.
Hopefully, we can reach to some consensus on a starting subset that we all
agree to, and start running some coherent simulations after the Interim
and share the results in the November plenary. As you know, we're going to
have a lot of simulation work to do, so getting a head start on it can
help us avoid the situation where this becomes a gating factor in the
future causing delays in establishing the standard.
Thanks.
Khaled Amer
President,
AmerNet
Architecture Analysis and Performance Modeling Specialists
Phone:
(949)552-1114
13711 Solitaire Way, Irvine, CA 92620
Fax:
(949)552-1116
e-mail: khaledamer@xxxxxxx
----- Original Message -----
From: Khaled Amer
To: Reflector RPRSG
Sent: Saturday, August 05, 2000 8:48 PM
Subject: RPR Perf. Metrics Discussions
All,
Attached is the last version of the presentation that I gave in the La
Jolla plenary meeting in July. This includes the suggestions that were
raised in the discussions that we had during the meeting. Please have a
look at them and make sure that they cover everything that we discussed.
If not, please point out any points that I may have missed.
I'd like for us to reach an agreement on this before the interim
meeting that we're having in a few weeks in Santa Clara.
Please provide any input or concerns that you may have by Friday 8/11.
I'll plan on posting the 'final' version the following Monday for your
review. After that, we'll consider this closed (at least for now).
Another couple of points:
- I would like to request that we begin thinking about some simulation
scenarios that we can use as a starting point. We'll have to start with
some subset and then add on as we decide is appropriate. Along with this,
we should also identify in these scenarios a subset of the output results
or metrics for these simulation runs. We can start by addressing one or
more of the of the metrics that we have in the attached
presentation.
For example, I'll go ahead and make a strawman attempt at this. We can
use this strawman to open up the discussion for bashing it! How about
starting with the following subset of the goals that we identified for
this work such as:
- Ring performance:
- Metrics: ring utilization under heavy loads, global throughput and
goodput
- Fairness:
- Metrics: per class and per node throughput, end-to-end packet delay
for scenarios that demonstrate fairness (TBD)
- Congestion control:
- Metrics: per class and per node throughput in response to a
congestion condition occurring (TBD)
I know
that this is a rough strawman, and most of the items need to be defined in
more details. But we can use this as a starting point to trigger some
thoughts and discussions on the reflector. If you have any thoughts on this,
or just any thoughts on your mind (well ... related to this!), please speak
up. Let's start discussing this on the reflector in the coming couple of
weeks so that we can make good progress in the meeting, instead of starting
the discussion then.
- Several of you
indicated availability of traffic patterns that were taken off the Internet
or other networking environments that would be useful for us to use in the
simulations. In my mind, we need things like packet size distributions,
interarrival times, burstiness, application and protocol distributions, and
any other relevant characteristics of network traffics that we may want to
consider in the simulations. If you have information that would help with
this, please let us know on the reflector and we would appreciate you making
it available and discussing it during the next meeting.
Any other suggestions are most
welcome.
Khaled
Amer
President,
AmerNet
Architecture Analysis and Performance Modeling Specialists
Phone:
(949)552-1114
13711 Solitaire Way, Irvine, CA 92620
Fax:
(949)552-1116
e-mail: khaledamer@xxxxxxx
**********************************************************************
P. Taylor
Salman Phone:
202-364-4700x2297
Manager, Market
Research Mobile:
202-427-3319
OPNET Technologies,
Inc. E-mail:
tsalman@xxxxxxxxx
Washington, DC
20008 **********************************************************************
|