Thread Links Date Links
Thread Prev Thread Next Thread Index Date Prev Date Next Date Index

Re: [802.3_DMLT] AW: [802.3_DMLT] P802.3br IET, P802.1Qbu and 802.1Qbb PFC (was RE: Announcement - D1.00 Task Force Review: IEEE P802.3br IET)



Cristian,

 

The 802.3br draft under review has an optional MAC Control entity above each of the two MACs. So currently the client could decide to send PFC on either MAC.

 

I think that some acknowledgement should be added to the 802.1 draft that there is an interaction if PFC and scheduled traffic are enabled together  should be added to the 802.1 draft. Possibly that should include adding something to the Annex N Buffer requirements for PFC on the impact of the choice of which path for PFC MAC Control PFC primitives on the PFC buffer requirements.

 

Perhaps also 802.3br should note that there is an interaction between PFC and scheduled traffic referencing the explanation in Qbu or Qbv.

 

Regards,

Pat

 

From: Christian Boiger [mailto:christian.boiger@xxxxxxxxxx]
Sent: Monday, December 15, 2014 3:48 PM
To: STDS-802-3-DMLT@xxxxxxxxxxxxxxxxx
Subject: [802.3_DMLT] AW: [802.3_DMLT] P802.3br IET, P802.1Qbu and 802.1Qbb PFC (was RE: Announcement - D1.00 Task Force Review: IEEE P802.3br IET)

 

(Peter, I only received your emails through the IET reflector. But they are in the 802.1 list archive. Perhaps this is a listserv feature?)

 

Hi Denis,

 

this is an interesting observation. As far as I understand the issue, the big question that needs to be answered on the .3 side is, do we need one MAC Control or two MAC Control. Until now my assumption was, that there are two MAC Control entities, one for the eMAC and one for the pMAC (as IET only inserts the MAC Merge below two MACs). In this case it is up to the higher layer to decide which MAC should be used for PFC frames. I agree with Pat that both options are useful, depending on the application. Preemption has several different use cases.

 

I do not remember any discussion about PFC in combination with Scheduled Traffic in the past. I assume this is still a missing part of .1Qbv. Without preemption Scheduled Traffic is not compatible with PFC (in my opinion). Theoretically it seems to be possible to calculate a schedule based upon the worst case interference due to PFC, but I don’t think that this is desirable. PFC would add a lot of delivery variation for the scheduled traffic frames, in order to create a schedule, one would need to compensate this at every hop (i.e. buffering for the worst case -> additional latency). Therefore it seems to be necessary to disallow PFC if Scheduled Traffic is used (like .3 pause is disallowed in combination with AVB). Additional delivery variation would also make the discussed time aware ingress gates less accurate.

 

Preemption would allow to use PFC in combination with Scheduled Traffic (using the pMAC for PFC frames). I assume this is a different way to look at it. However, the resulting performance would be worse compared to “normal” PFC operation (as you mentioned). The worst case latency for sending a PFC frame would be higher than the one currently shown in 802.1Q, but it is possible to calculate a worst case based on the schedule, i.e. the delay is bounded and known.

 

If preemption is used in combination with the Credit Based Shaper, PFC frames can be sent on the eMAC (AVB currently allows PFC frames, according to 802.1BA. But I am a little bit wondering if this is really true. PFC is currently not part of the worst case AVB latency calculation (this would make it worse), but this more an AVB issue than Preemption issue.) As AVB currently also guarantees a certain quality of service for the highest non AVB traffic class, it would be possible to calculate a worst case if CBS is used and PFC frames are sent by the pMAC, i.e. again the delay is bounded and known.

 

For other preemption use cases like Cyclic Queueing and Forwarding, the situation seems to be less clear to me. To use the eMAC for PFC frames seems to be possible, but it has also some potential disadvantages (e.g. reducing the available bandwidth for CQF frames).

 

Therefore I don’t think that 802.3br should define which MAC is used for PFC frames. It should be defined individually for each transmission selection mechanism (e.g. Scheduled Traffic, CBS, CQF, UBS, etc.), how it interoperates with PFC and which MAC should be used (the default in the absence of those mechanisms is perhaps eMAC?). It seems to be an additional open task for Qbu to make the necessary additions to the “Qbb” sections.

 

 

Regards

Christian

 

 

Von: Peter Jones (petejone) [mailto:petejone@xxxxxxxxx]
Gesendet: Montag, 15. Dezember 2014 22:42
An: STDS-802-3-DMLT@xxxxxxxxxxxxxxxxx
Betreff: Re: [802.3_DMLT] P802.3br IET, P802.1Qbu and 802.1Qbb PFC (was RE: Announcement - D1.00 Task Force Review: IEEE P802.3br IET)

 

(to: Pat Thaler <pthaler@xxxxxxxxxxxx>; STDS-802-3-DMLT@xxxxxxxxxxxxxxxxx; STDS-802-1-L@xxxxxxxxxxxxxxxxx)

 

Folks,

 

It seems like the IEEE 802 reflectors are removing “CCs”. I see the same email on both reflectors.

 

Trying again so both 802.1 and 802.3 interested people can see Pat’s response.

 

It seems to me that there probably isn’t a simple answer that’s best for everything. In the short term are we just going to document the impact of preemption on the PFC transmission latency?

 

Regards

Peter

 

_______________________________________________

Peter Jones             Cisco Systems

Principal Engineer      3600 Cisco Way

ENG Switching Software  San Jose, CA, 95134 USA

Tel: +1 408 525 6952    Fax: +1 408 527 4698

Email:                  petejone at cisco.com

Twitter:                @petergjones

_______________________________________________

 

From: Pat Thaler [mailto:pthaler@xxxxxxxxxxxx]
Sent: Monday, December 15, 2014 12:52 PM
To: Peter Jones (petejone); STDS-802-3-DMLT@xxxxxxxxxxxxxxxxx
Subject: RE: [802.3_DMLT] P802.3br IET, P802.1Qbu and 802.1Qbb PFC (was RE: Announcement - D1.00 Task Force Review: IEEE P802.3br IET)

 

(Peter, you email started +Tony and 802.1, but I don’t see their addresses in the to or cc.)

 

Yes Denis, I realize that in the general case having PFC in the non-Express path could mean an unbounded delay in sending the PFC MAC Control frame.

 

But if PFC MAC Control frames are sent in the express path, they could delay express traffic by more than 84 bytes because PFC can be enabled for multiple priorities so one could have

·         Priority x passes its high water threshold (or was previously paused and drops below its low water threshold) which triggers a PFC MAC Control primitive

·         Transmission of a PFC MAC Control frame to pause (or unpause) priority x

·         Priority y passes its high water threshold (or was paused and drops below low water threshold) triggering another PFC MAC Control primitive

·         Transmission of a PFC MAC Control frame to pause (or unpause) priority y

The delay of 139 octet times was too much for some TSN use cases. Even 84 octets which isn’t that much smaller is probably an issue for some.  The hold primitive was created to get the delay waiting for a preempted frame to complete to zero by preempting before scheduled traffic arrives.

 

Here we have a case where PFC can cause a delay of 186 octets.

 

An unbounded delay in sending the pause would be unacceptable for most uses of PFC. E.g. when PFC is being used to provide no drop operation for a protocol like FCoE which was designed to run over a flow controlled network with no congestion drop.

 

A delay in transmitting scheduled traffic is unacceptable in many 802.1Qbu scheduled traffic use cases.

 

Therefore, there is a conflict between these two capabilities in some cases. There may be some TSN links where one cannot use PFC to get no drop behavior on some priorities while meeting scheduled traffic latency objectives on other priorities. This is not necessarily a problem – PFC was generally designed to enable some protocols that are used in the data center. For example, one is unlikely to be running FCoE on an automotive network or on the manufacturing floor in an industrial machine.

 

There may be other networks where the scheduled express traffic is designed with frequent enough breaks to allow PFC  MAC Control frames on the premptable path  to be sent in the breaks between express traffic. Such breaks in scheduled express traffic would have to be greater than a max packet time plus inter packet time to allow a frame in progress to complete and the MAC Control frame to be sent.  For no drop in that case, one would need additional buffer headroom equal to the scheduled traffic schedule windows.

 

There may also be networks where express traffic can tolerate 84 octets or even 84 octet times * the number of PFC priorities of delay so that the PFC frames can go on the express path. On the higher speed networks where protocols like FCoE are generally used, 84 octets take a lot less time than on a 100 Mb/s or 1 Gb/s link.

 

Regards,

Pat

 

 

From: Peter Jones (petejone) [mailto:petejone@xxxxxxxxx]
Sent: Monday, December 15, 2014 12:15 PM
To: STDS-802-3-DMLT@xxxxxxxxxxxxxxxxx
Subject: [802.3_DMLT] P802.3br IET, P802.1Qbu and 802.1Qbb PFC (was RE: Announcement - D1.00 Task Force Review: IEEE P802.3br IET)

 

(+Tony & 802.1)

 

Hmmm, interesting.

 

As an aside, taking one step back, TSN in general is addressing some of the same “no network drop” cases that PFC states it addresses (from 36.1.1 PFC Overview, “PFC enables to not discard frames due to congestion for protocols that require this property.”).

 

That being said, I assume that prior to IET, PFC messages (as a M_CONTROL.request) are considered for transmission as “more important” than anything in the normal queueing/transmission selection process, and so the worst case delay in sending a message is an MTU work of transmission time.

 

I agree that we need to figure this out, and I’m wondering if this problem is only relevant to PFC, or are there other “control messages” that have expectation for only have an MTU’s work of delay time.

 

If we say that PFC uses the express MAC, then I think that makes the network scheduling very complex because you have to estimate how many PFCs need to be sent. I’m not a PFC expert, but what does it say about the max rate/min time between PFC messages? If we can push PFC though the express channel, it probably improves PFC performance at the cost of scheduled express traffic.

 

Regards

Peter

 

_______________________________________________

Peter Jones             Cisco Systems

Principal Engineer      3600 Cisco Way

ENG Switching Software  San Jose, CA, 95134 USA

Tel: +1 408 525 6952    Fax: +1 408 527 4698

Email:                  petejone at cisco.com

Twitter:                @petergjones

_______________________________________________

 

From: Beaudoin, Denis [mailto:dbeaudoin@xxxxxx]
Sent: Thursday, December 11, 2014 10:46 AM
To: STDS-802-3-DMLT@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3_DMLT] Announcement - D1.00 Task Force Review: IEEE P802.3br IET

 

The only concern for PFC frames on the non preempted path is that it now increases the receiver run out by the maximum preempted frame since a preempted frame can now delay the PFC frame, this delay allows the remote to send more regular traffic. It is even worse if a single max frame is preempted multiple times before the PFC frame is sent. My coworker pointed out that since a preempted frame can be preempted multiple times and each preemption could be a burst of preempted frames, than the run out was not fixed and could be huge! If the PFC frame was on the preempted channel, the preempted traffic would only be delayed by max of 84 byte times.

 

Regards

Denis Beaudoin  DMTS  Texas Instruments  dbeaudoin@xxxxxx W: 214-480-3287/77  M: 214-475-9193

 

From: Pat Thaler [mailto:pthaler@xxxxxxxxxxxx]
Sent: Thursday, December 11, 2014 12:33 PM
To: Beaudoin, Denis; STDS-802-3-DMLT@xxxxxxxxxxxxxxxxx
Subject: RE: Announcement - D1.00 Task Force Review: IEEE P802.3br IET

 

I recall any discussion in the task group of that and I’m not sure there is a uniform answer.

 

My own view, there will be a lot of cases where PFC and express traffic are not used at the same time. (I’d prefer to use express to describe the traffic on the path that can preempt – TSN is a broader term that includes uses of scheduled traffic or the credit based shaper without preemption.)

 

If they are both being used, in the most time sensitive use cases (e.g. we created the hold primitive because some found the possible 139 octet worst case delay before preempting too long) , one wouldn’t want sending a PFC to delay sending a scheduled frame.

 

On the other hand, in less extreme time sensitive use cases, it may be desirable to put PFC frames on the express path.

 

Something to discuss in January.

 

Regards,

Pat

 

From: Beaudoin, Denis [mailto:dbeaudoin@xxxxxx]
Sent: Thursday, December 11, 2014 6:25 AM
To: STDS-802-3-DMLT@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3_DMLT] Announcement - D1.00 Task Force Review: IEEE P802.3br IET

 

Typo…

 

Regards

Denis Beaudoin  DMTS  Texas Instruments  dbeaudoin@xxxxxx W: 214-480-3287/77  M: 214-475-9193

 

From: Beaudoin, Denis
Sent: Thursday, December 11, 2014 8:15 AM
To: STDS-802-3-DMLT@xxxxxxxxxxxxxxxxx
Subject: Re: [802.3_DMLT] Announcement - D1.00 Task Force Review: IEEE P802.3br IET

 

Ludwig, looking at the IET did we decide where 802.1Qbb pause frames are sent?

That is are pause frames considered TSN frames or non-TSN frames?

 

The reason I ask is that if pause frames are TSN frames, then Rx run out could be smaller than today, but assuming that TSN may not be supported by the link partner than what is required today is a minimum.

But if pause frames are non-TSN frames, then the pause run out is increased by the largest TSN frame plus IPGs times two plus some slop! If TSN frames can be 2000bytes, then this increases the Rx runnout requirements by 4000+ bytes.

 

Regards

Denis Beaudoin  DMTS  Texas Instruments  dbeaudoin@xxxxxx W: 214-480-3287/77  M: 214-475-9193

 

From: Winkel, Ludwig [mailto:ludwig.winkel@xxxxxxxxxxx]
Sent: Thursday, November 27, 2014 4:00 AM
To: STDS-802-3-DMLT@xxxxxxxxxxxxxxxxx
Subject: [802.3_DMLT] Announcement - D1.00 Task Force Review: IEEE P802.3br IET

 

Dear Colleagues:

 

This is the announcement of the review of the IEEE P802.3r Interspersing Express Traffic (IET) Task Force (TF) draft amendment D1.0.  Instructions for accessing the draft and submitting comments follow this announcement.

 

IEEE P802.3br is an amendment to IEEE Std 802.3-2012 which is available through the “Get 802” program at the URL:

http://standards.ieee.org/getieee802/802.3.html

 

The Project Authorization Request for this amendment is posted at:

http://www.ieee802.org/3/br/P802d3br_PAR.pdf

 

The Five Criteria Responses for this amendment are posted at:

http://www.ieee802.org/3/br/8023-DMLT-SG-1311-Winkel-5C-v2.3.pdf

 

The Objectives for this amendment are posted at:

http://www.ieee802.org/3/br/8023-DMLT-SG-1309-Winkel-Objectives-v2.3.pdf

 

Other related documents are posted at:

http://www.ieee802.org/3/br/index.html

 

___________________________________________

 

REVIEW SCOPE:

The scope of the review is the entire IEEE P802.3br D1.0 Draft.

 

REVIEW OPEN:

Thursday, 27th, November 2014

REVIEW CLOSE:

Monday, 1st, January 2015 11:59pm AoE

 

-------------------------------------------------------------------------

HOW TO ACCESS THE DRAFT

The draft is posted for Task Force review purposes only, and neither the draft nor access information should be copied or redistributed to others in violation of document copyrights.

 

Draft 1.0 should be used for commenting and may be downloaded from:

URL: http://grouper.ieee.org/groups/802/3/br/private/01_TF_Drafts/P8023br-D1.00p0.pdf

Username and password: distributed at meetings

 

The document is posted in Adobe "pdf" document format and can be downloaded and printed if desired. If you do not have Adobe Acrobat Reader, it is available for free downloading from Adobe at:

URL: http://www.adobe.com/prodindex/acrobat/readstep.html

 

_______________________________________

HOW TO SUBMIT COMMENTS

 

Acceptable comment types:

 

E = Editorial

ER = Editorial Required (an Editorial comment that the commenter feels very strongly about) T = Technical TR = Technical Required (a Technical comment that the commenter feels very strongly about)

 

For further information regarding comments, please see http://www.ieee802.org/3/bj/public/may12/diab_01_0512.pdf

 

1. Prepare your comments in one of the following two ways:

 

a. You are STRONGLY REQUESTED to use the comment tool in the TF private area at the URL:

 

URL: http://ieee802.org/3/WG_tools/filemaker/index.html

Please download and read the Comment Tool Tutorial:

http://www.ieee802.org/3/bj/public/may12/diab_01_0512.pdf

 

To use the tool, extract all components into a folder and do not move any component from that folder as this will cause the executable to not work correctly. The tool is:

 

Comment_Tool_Generic_Solution.exe

 

The comment tool runs under Microsoft Windows. It has also been run in VM environments on other platforms.

 

b. To submit comments in text (ASCII) form, please use the form below.  This will make it possible for the editor to properly record and track all submissions. Make as many copies of the template as necessary and submit as an ASCII text file or as part of your e-mail.

 

PLEASE NOTE: The ASCII method involves a manual transcription, hence you are recommended to check your comments to ensure accurate transcription when the comment file is made available. Please use the comment tool above if possible.

 

------CUT AND PASTE TEXT BELOW THIS LINE ONLY PLEASE--------

 

CommentID: (Leave Blank)

CommenterName:

CommenterEmail:

CommenterPhone:

CommenterCellPhone:

CommenterCompany:

Clause:

Subclause:

Page:

Line:

CommentType: (E, ER, T, or TR)

Comment:

CommentEnd:

SuggestedRemedy:

RemedyEnd:

 

--------------------end comment template------------------

 

2. Send your comments to: pthaler@xxxxxxxxxxxx, CC Ludwig.Winkel@xxxxxxxxxxx , mike@xxxxxxxxxxxxxxx and paste into the subject line:

 

IEEE P802.3br IET Task Force D1.0 comments

 

_____________________________________________________________

 

Thank you for your participation in this review and careful review of the draft.

 

Pat Thaler, Broadcom

Editor, IEEE P802.3br Task Force

 

 

Ludwig Winkel, Siemens

Chair, IEEE P802.3br interspersing Express Traffic Task Force http://www.ieee802.org/3/br/index.html

 

 

 

Mit freundlichen Gruessen / With best regards,
Ludwig Winkel


  Vice-chair of IEEE P2413

   Chair of the IEEE P802.3br Task Force

   Vice-chair of IEC SG8

   Convenor of IEC SC65B/WG16

   Convenor of IEC SC65C/WG17

   Convenor of IEC SC65C/MT9

   Convenor of CENELEC TC65X/WG1

 

Mail: Siemens AG , PD TI FC
        Oestliche Rheinbrueckenstr. 50
        76187 Karlsruhe, Germany
Tel.: +49 721 595-6098
Mobile: +49 172 6132658
mailto:ludwig.winkel@xxxxxxxxxxx

Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Gerhard Cromme; Managing Board: Joe Kaeser, Chairman, President and Chief Executive Officer; Roland Busch, Lisa Davis, Klaus Helmrich, Hermann Requardt, Siegfried Russwurm, Ralf P. Thomas; Registered offices: Berlin and Munich, Germany; Commercial registries: Berlin Charlottenburg, HRB 12300, Munich, HRB 6684; WEEE-Reg.-No. DE 23691322