|
Khaled,
From the operation point of view, you will have to define what your
metrics points are.
This is what users like Excite, Sprint, or the Navy can use to help
them 1) engineer their network and
2) measure its performance. That is what I see the EXTERNAL
metrics are, regardless of the scenario.
For each scenario you may have to define what is acceptable. This includes
full mesh, partial meshed,
hubbed, over-subscribed network, under-subscribed network etc.....
You have captured the basic EXTERNAL metrics point from my point of
view: delay and goodput (end to end).
If you want to oversubscribe the network, stat gain factor is another
point for the metrics, but is can be derived from the goodput.
I agree with Raj and Taylor that we need to look at the scenarios
and other INTERNAL metrics may be come up.
Harry
Taylor Salman wrote:
While I agree that it is probably still a little
premature to be finalizing the metrics (They certainly can't be finalized
in advance of the PAR), it is not premature to discuss them. The
process should proceed from the PAR criteria to specific questions supporting
the PAR criteria. These questions should then evolve into specific
metrics. The metrics are then grouped by like issues and scenarios
are developed that adequately test proposals against these criteria.
Once the scenarios are developed, models are built, etc.
This is also an iterative process. In the development of the scenarios,
we will come across issues that weren't originally considered and we'll
need to update the metrics...
I think once the PAR is finalized, we will see additional discussion
on the metrics question. Of course, if we had some existing models
available for general consumption to provide a common understanding of
what people intended, we might also generate some significant discussion...
:-)
Taylor
At 01:35 PM 8/15/00 -0700, Raj Sharma wrote:
Khaled,
Dont lose hope
yet !
My take on the
silence is that it maybe still premature to discuss this.
Although, it needs to be launched now. Without having a good reference
model it maybe be difficult to scope the perfomance. Not to be pedantic,
but,
Taylor's tutorial suggested we that we define a model before establishing
requirements or metrics for performance.
I personally
feel we are biting more based on the broad nature of performance
metrics we are going after. It maybe time to approach this from a reference
model
point of view and then establish performance metrics.
Comments?
raj
-----Original
Message-----
From: Khaled Amer [mailto:khaledamer@xxxxxxx]
Sent: Wednesday, August 16,
2000 12:25 AM
To: Reflector RPRSG
Subject: Fw: RPR Perf. Metrics
Discussions
-
RPR'ers,
-
-
Based on the 'underwhelming' responses
that I got back on this, we'll consider the performance metrics discussion
closed for now, and move on.
-
-
As I mentioned before, I want for us to
work on identifying the scenarios, and metrics that we want to use as a
starting point. I know that some of you already put extensive efforts into
producing simulation results, but we want to all get to run a consistent
subset for various proposals. I've already posted some preliminary thoughts
and suggestions (and got the same underwhelming response!).
-
-
Please share with us any thoughts that
you have on this subset, and if you have suggestions on more details. In
the meantime, I'll work on putting together more details too. I'd like
to see us go to the Interim meeting with some of these discussions already
started, so that we just work on ironing out any points of disagreements
that we may have. Hopefully, we can reach to some consensus on a starting
subset that we all agree to, and start running some coherent simulations
after the Interim and share the results in the November plenary. As you
know, we're going to have a lot of simulation work to do, so getting a
head start on it can help us avoid the situation where this becomes a gating
factor in the future causing delays in establishing the standard.
-
-
Thanks.
-
-
Khaled Amer
-
President, AmerNet
-
Architecture Analysis and Performance Modeling Specialists
-
Phone: (949)552-1114
13711 Solitaire Way, Irvine, CA 92620
-
Fax: (949)552-1116
e-mail: khaledamer@xxxxxxx
-
-
-
----- Original Message -----
-
From: Khaled Amer
-
To: Reflector RPRSG
-
Sent: Saturday, August 05, 2000 8:48 PM
-
Subject: RPR Perf. Metrics Discussions
-
All,
-
-
Attached is the last version of the presentation
that I gave in the La Jolla plenary meeting in July. This includes the
suggestions that were raised in the discussions that we had during the
meeting. Please have a look at them and make sure that they cover everything
that we discussed. If not, please point out any points that I may have
missed.
-
-
I'd like for us to reach an agreement
on this before the interim meeting that we're having in a few weeks in
Santa Clara.
-
Please provide any input or concerns that you may have by Friday 8/11.
I'll plan on posting the 'final' version the following Monday for your
review. After that, we'll consider this closed (at least for now).
-
-
Another couple of points:
-
-
- I would like to request that we begin
thinking about some simulation scenarios that we can use as a starting
point. We'll have to start with some subset and then add on as we decide
is appropriate. Along with this, we should also identify in these scenarios
a subset of the output results or metrics for these simulation runs. We
can start by addressing one or more of the of the metrics that we have
in the attached presentation.
-
-
For example, I'll go ahead and make a
strawman attempt at this. We can use this strawman to open up the discussion
for bashing it! How about starting with the following subset of the goals
that we identified for this work such as:
-
Ring performance:
-
Metrics: ring utilization under heavy
loads, global throughput and goodput
-
Fairness:
-
Metrics: per class and per node throughput,
end-to-end packet delay for scenarios that demonstrate fairness (TBD)
-
Congestion control:
-
Metrics: per class and per node throughput
in response to a congestion condition occurring (TBD)
I know that this is a rough strawman, and most of the items need
to be defined in more details. But we can use this as a starting point
to trigger some thoughts and discussions on the reflector. If you have
any thoughts on this, or just any thoughts on your mind (well ... related
to this!), please speak up. Let's start discussing this on the reflector
in the coming couple of weeks so that we can make good progress in the
meeting, instead of starting the discussion then.
- Several of you indicated availability
of traffic patterns that were taken off the Internet or other networking
environments that would be useful for us to use in the simulations. In
my mind, we need things like packet size distributions, interarrival times,
burstiness, application and protocol distributions, and any other relevant
characteristics of network traffics that we may want to consider in the
simulations. If you have information that would help with this, please
let us know on the reflector and we would appreciate you making it available
and discussing it during the next meeting.
Any other suggestions are most welcome.
Khaled Amer
President, AmerNet
Architecture Analysis and Performance
Modeling Specialists
Phone: (949)552-1114
13711 Solitaire Way, Irvine, CA 92620
Fax: (949)552-1116
e-mail: khaledamer@xxxxxxx
**********************************************************************P.
Taylor SalmanPhone: 202-364-4700x2297Manager,
Market ResearchMobile: 202-427-3319OPNET Technologies,
Inc.E-mail: tsalman@xxxxxxxxxxxxx International Drive,
NWWeb: www.opnet.comWashington,
DC 20008**********************************************************************
--
Harry Peng
------------------------------------------------------------------
Dept: 1E11
Email: hpeng@xxxxxxxxxxxxxxxxxx
ESN: 39-52277
Phone 613-765-2277
Fax: 613-768-4904
Web: http://skywww/~hpeng/
-------------------------------------------------------------------
|