Thread Links Date Links
Thread Prev Thread Next Thread Index Date Prev Date Next Date Index

RE: RPR Perf. Metrics Discussions

While I agree that it is probably still a little premature to be finalizing the metrics (They certainly can't be finalized in advance of the PAR), it is not premature to discuss them.  The process should proceed from the PAR criteria to specific questions supporting the PAR criteria.  These questions should then evolve into specific metrics.  The metrics are then grouped by like issues and scenarios are developed that adequately test proposals against these criteria.  Once the scenarios are developed, models are built, etc. 

This is also an iterative process.  In the development of the scenarios, we will come across issues that weren't originally considered and we'll need to update the metrics...

I think once the PAR is finalized, we will see additional discussion on the metrics question.  Of course, if we had some existing models available for general consumption to provide a common understanding of what people intended, we might also generate some significant discussion... :-)

Taylor

At 01:35 PM 8/15/00 -0700, Raj Sharma wrote:
Khaled,
 
Dont lose hope yet !
 
My take on the silence is that it maybe still premature to discuss this.
Although, it needs to be launched now. Without having a good reference
model it maybe be difficult to scope the perfomance. Not to be pedantic, but,
Taylor's tutorial suggested we that we define a model before establishing
requirements or metrics for performance.
 
I personally feel we are biting more based on the broad nature of performance
metrics we are going after. It maybe time to approach this from a reference model
point of view and then establish performance metrics.
 
Comments?
 
raj
 
 -----Original Message-----
From: Khaled Amer [mailto:khaledamer@xxxxxxx]
Sent: Wednesday, August 16, 2000 12:25 AM
To: Reflector RPRSG
Subject: Fw: RPR Perf. Metrics Discussions
RPR'ers,
 
Based on the 'underwhelming' responses that I got back on this, we'll consider the performance metrics discussion closed for now, and move on.
 
As I mentioned before, I want for us to work on identifying the scenarios, and metrics that we want to use as a starting point. I know that some of you already put extensive efforts into producing simulation results, but we want to all get to run a consistent subset for various proposals. I've already posted some preliminary thoughts and suggestions (and got the same underwhelming response!).
 
Please share with us any thoughts that you have on this subset, and if you have suggestions on more details. In the meantime, I'll work on putting together more details too. I'd like to see us go to the Interim meeting with some of these discussions already started, so that we just work on ironing out any points of disagreements that we may have. Hopefully, we can reach to some consensus on a starting subset that we all agree to, and start running some coherent simulations after the Interim and share the results in the November plenary. As you know, we're going to have a lot of simulation work to do, so getting a head start on it can help us avoid the situation where this becomes a gating factor in the future causing delays in establishing the standard.
 
Thanks.
 
Khaled Amer
President, AmerNet               
Architecture Analysis and Performance Modeling Specialists
Phone: (949)552-1114             13711 Solitaire Way, Irvine, CA 92620
Fax:     (949)552-1116             e-mail: khaledamer@xxxxxxx
 

 
----- Original Message -----
From: Khaled Amer
To: Reflector RPRSG
Sent: Saturday, August 05, 2000 8:48 PM
Subject: RPR Perf. Metrics Discussions

All,
 
Attached is the last version of the presentation that I gave in the La Jolla plenary meeting in July. This includes the suggestions that were raised in the discussions that we had during the meeting. Please have a look at them and make sure that they cover everything that we discussed. If not, please point out any points that I may have missed.
 
I'd like for us to reach an agreement on this before the interim meeting that we're having in a few weeks in Santa Clara.
Please provide any input or concerns that you may have by Friday 8/11. I'll plan on posting the 'final' version the following Monday for your review. After that, we'll consider this closed (at least for now).
 
Another couple of points:
 
- I would like to request that we begin thinking about some simulation scenarios that we can use as a starting point. We'll have to start with some subset and then add on as we decide is appropriate. Along with this, we should also identify in these scenarios a subset of the output results or metrics for these simulation runs. We can start by addressing one or more of the of the metrics that we have in the attached presentation.
 
For example, I'll go ahead and make a strawman attempt at this. We can use this strawman to open up the discussion for bashing it! How about starting with the following subset of the goals that we identified for this work such as:

I know that this is a rough strawman, and most of the items need to be defined in more details. But we can use this as a starting point to trigger some thoughts and discussions on the reflector. If you have any thoughts on this, or just any thoughts on your mind (well ... related to this!), please speak up. Let's start discussing this on the reflector in the coming couple of weeks so that we can make good progress in the meeting, instead of starting the discussion then.
 
- Several of you indicated availability of traffic patterns that were taken off the Internet or other networking environments that would be useful for us to use in the simulations. In my mind, we need things like packet size distributions, interarrival times, burstiness, application and protocol distributions, and any other relevant characteristics of network traffics that we may want to consider in the simulations. If you have information that would help with this, please let us know on the reflector and we would appreciate you making it available and discussing it during the next meeting.
 
Any other suggestions are most welcome.
 
Khaled Amer
President, AmerNet               
Architecture Analysis and Performance Modeling Specialists
Phone: (949)552-1114             13711 Solitaire Way, Irvine, CA 92620
Fax:     (949)552-1116             e-mail: khaledamer@xxxxxxx
 

 

**********************************************************************
P. Taylor Salman                Phone:  202-364-4700x2297
Manager, Market Research        Mobile:  202-427-3319
OPNET Technologies, Inc.        E-mail:  tsalman@xxxxxxxxx
3400 International Drive, NW    Web:     www.opnet.com
Washington, DC 20008
**********************************************************************