Thread Links Date Links
Thread Prev Thread Next Thread Index Date Prev Date Next Date Index

Sponsor Ballot Recirculation

Dear All,

 

The 802.21 SB Recirculation-3 has been stopped as of last night Monday April 28, 11:59 EST.

Following are some of the reasons behind this.

 

[1] When we uploaded the resolutions from last recirculation, the resolutions for the comments were not properly documented in the Excel spreadsheet that was uploaded into the IEEE-SA database.

This was specifically true with regards to comments filed by Tony Jeffree in the previous round.

Tony also pointed out that in many of the resolutions we were pointing to a WG contribution for the resolution.

e.g. "Please see contribution 21-08-0056-01-0000-d9-0-primitive-and-message-consistency.doc" for resolution details.

However external (non 802 attendees) that are part of this ballot have insufficient information as to how to get to this document.

 

[2] EC members and others (Bob Grow, Geoff Thompson, Tony himself and others) have given us good feedback on the implications of this and how to deal with these issues.

Please see the attached email thread for details. The gist of what seems to have come out is:

- We need to document the resolution in Excel spreadsheet directly and give the commenter an idea of how the comment was resolved.

- If we do mention about a contribution, it’s best to include the URL so that others outside of 802 can get to these documents as well

- We can include all these contributions with our cover letter as well in the SB package and that way people who review things offline (RevCom members) can also get to these documents.

 

[3] Also in the last round of recirculation two of Tony’s comments (comments 142 and 145) have not been satisfactorily resolved as yet.

It has been pointed out that, until we complete the resolution to ALL the comments we cannot initiate another re-circulation.

 

As such the best thing at this point was to stop this re-circulation and for us to go back and rectify these issues before continuing with the SB.

Please let us know if there are any other questions/comments in this regard.

I have copied Paul, Geoff, Bob Grow and Tony to assist us with any further guidance in this matter.

There is an 802.21 teleconference scheduled for Wednesday morning. I would request Yoshi to allocate some time for this topic and would be happy to answer any questions as well.

 

Kind Regards

-Vivek

 

 

 

 


Title: RE: P802.21 Sponsor ballot recirculation

Vivek-

I want to throw my agreement with Bob's view that reviewers often wish to
do their work off line. I strongly agree with this. Further, Bob failed to
emphasize the extent to which you need to address your process to REVCOM as
well as the Balloting Group (especially external members).

It is REVCOM's job to audit your balloting process. As Bob said, the
personalities on REVCOM vary from year to year and, thus, so does the
emphasis of the REVCOM review. I'm not on REVCOM this year but my
particular area of focus was evidence that each comment was actually
considered by the balloting group (without consideration as to whether the
comment came from a balloter or observer) and whether an appropriate
decision was made and documented on each comment. Other things I looked for
were whether or not the resolution commitments were implemented in the draft.

If some of your frustration is with myBallot, then I certainly share that
as do a number of members of REVCOM and other Working Groups.

With particular consideration of Tony's point, if outside documents are
appropriate to support the comment resolution package then I would package
all into a supplementary zip file and provide a pointer to that file in the
cover letter.

Best regards,

Geoff Thompson

At 09:19 AM 4/26/2008 , Tony Jeffree wrote:
>Bob -
>
>The point you make about being able to review the ballot package offline
>is one I hadn't considered. In which case, if it is necessary to refer to
>external documents from within the comment database, presumably it would
>be acceptable for the referenced files to be included in the ballot
>package (along with change-marked drafts etc.)
>
>Regards,
>Tony
>
>At 17:04 26/04/2008, Grow, Bob wrote:
>>Vivek:
>>
>>As one might expect, as personnel change each RevCom has a slightly
>>different personality and varying number of individuals that are
>>sensitive to particular issues.  One consistency over the last few year
>>though has been in upholding the importance of the ballot group as the
>>body that owns the draft during Sponsor ballot, and the completeness and
>>clarity of ballot packages viewed by the ballot group.
>>
>>Last year, because RevCom saw frequent problems with handling of
>>comments, an ad hoc clarified the requirements.  The underlying
>>principle of the clarifications is that the recirculation package has to
>>provide to the ballot group (not the WG or just the commenter) a clear
>>response to the comment.
>>
>>If it is practical to include the details of the agreed change in the
>>response, do it.  Other files and the recirculated draft can be the
>>detailed response (e.g., changes to figures), but it is always good
>>practice for the response to indicate the substance of what did change,
>>especially when the remedy the commenter submitted is modified or
>>supplemented (Principle).  RevCom will examine a comment that is
>>rejected for a substantive response, something that the ballot group is
>>entitled to receive in the recirculation package.  A WG submittal can be
>>supplemental to the response, but I think only a WG submittal URL in the
>>response will cause you problems in RevCom review (or from the ballot
>>group as evidenced by Tony's complaint).
>>
>>For review, RevCom members get a package that includes the drafts, cover
>>letters, ballot summaries and myBallot comment database for each round
>>of balloting.  Many RevCom members will work offline to review those
>>submittal packages.  Therefore comment responses should be written so
>>that what is in myBallot is substantive enough for a ballot group member
>>or RevCom reviewer to understand the response without access to the web.
>>I do not think that referenced WG submittals would be considered to be
>>part of the ballot recirculation package (I personally don't consider
>>them part of the recirculation package).  Just like the RevCom reviewer,
>>I believe the balloter should be able to load the ballot package and do
>>their recirculation ballot work offline).
>>
>>A requirement of recirculation is to list the changes to the draft.  The
>>format for that list is not specified and RevCom is flexible in
>>accepting various ways of this being provided to the ballot group.  Some
>>Sponsors provide a detailed list of changes in the cover letter.  Others
>>use the comment responses with a change marked draft included in the
>>recirculation package (mentioning this in the cover letter is good).
>>Another alternative would be a cover letter summary of substantive
>>changes plus a change marked draft in the recirculation package for
>>details.  The consistency again though is that the list of changes is
>>part of the recirculation package.
>>
>>--Bob
>>
>>-----Original Message-----
>>From: Gupta, Vivek G
>>Sent: Saturday, April 26, 2008 4:30 AM
>>To: Tony Jeffree
>>Cc: Paul Nikolich; David Law; m.kipness@ieee.org; Grow, Bob; Geoff
>>Thompson
>>Subject: RE: P802.21 Sponsor ballot recirculation
>>
>>Tony,
>>
>>The failure I was referring to w.r.t the tool was the case where some of
>>our "internal notes" were exported as resolutions specifically w.r.t
>>your comments as per your original mail below.
>>
>>As regards to referring to external documents, the WG develops
>>contributions in response to comments and these are the documents you
>>find references to. Since these WG documents are publicly available we
>>can include a link to these documents to help SB commenters. Also we
>>will go ahead and document the remedy within these contributions in the
>>Excel spreadsheet though it gets very hard with detailed text changes,
>>figures, tables etc. We will go ahead and do that for all comments
>>submitted so far in the original SB and also in two rounds of
>>re-circulations conducted so far. We will include the updated
>>spreadsheets to be made available along with current draft for next
>>round.
>>
>>If there is anything else we can do please do let us know.
>>
>>Kind Regards
>>-Vivek
>>
>>
>>
>>-----Original Message-----
>>From: Tony Jeffree [mailto:tony@jeffree.co.uk]
>>Sent: Friday, April 25, 2008 4:07 PM
>>To: Gupta, Vivek G
>>Cc: Paul Nikolich; David Law; m.kipness@ieee.org; Grow, Bob; Geoff
>>Thompson
>>Subject: RE: P802.21 Sponsor ballot recirculation
>>
>>Vivek -
>>
>>Looking through the comment spreadsheet that was circulated with the
>>ballot (attached), I consider the following comments to have been
>>inadequately addressed (using comment numbers in column C):
>>
>>1, 11, 12, 13, 14, 18, 19, 20, 34, 37, 38, 39, 77, 82, 83, 84, 85,
>>99, 106, 107, 110, 111, 112, 113, 114, 134, 138, 142, 143, 145, 89,
>>90, 92, 171, 176, 181, 182, 183, 197, 217, 218, 244, 252, 255.
>>
>>In almost all of these comments, the problem is that you refer to an
>>inadequately identified external document to define the changes that
>>will be made. I can't see how that has anything to do with your
>>"Commentary" tool - it seems to me to be a straightforward failure to
>>document responses in a way that would be intelligible to the Sponsor
>>balloters. That is just an unacceptable way to conduct a Sponsor
>>ballot, period.
>>
>>Regards,
>>Tony
>>
>>
>>
>>At 20:52 25/04/2008, Gupta, Vivek G wrote:
>> >Tony,
>> >
>> >I agree with your assessment.
>> >
>> >We import the comments into an internal tool called Commentary, resolve
>> >the comments in Commentary and then import the resolutions from this
>> >tool into the Excel file which is then uploaded into myBallot. We seem
>> >to have made an error in importing some of the resolutions from the
>> >Commentary tool into the Excel file.
>> >
>> >I am not sure how to stop the ballot, but would request Mike Kipness to
>> >assist us in doing so.
>> >
>> >Kind Regards
>> >-Vivek
>> >
>> >-----Original Message-----
>> >From: Tony Jeffree [mailto:tony@jeffree.co.uk]
>> >Sent: Friday, April 25, 2008 9:41 AM
>> >To: Gupta, Vivek G
>> >Cc: Paul Nikolich; David Law
>> >Subject: P802.21 Sponsor ballot recirculation
>> >
>> >Vivek -
>> >
>> >I note that you have just started a further recirc of P802.21.
>> >
>> >I must protest at the state of the ballot resolutions that have been
>> >recorded on the My Ballot system for some of the comments I made on
>> >the previous ballot. For example:
>> >
>> >- Against my comment #145 the resolution detail reads:
>> >
>> >"Please refer to comment 2142 as well. Get an ethertype, MAC headers
>> >stay unchanged, and this works as is across bridges. VLANs and other
>> >elements of 802.1 architecture
>> >21-08-0086-00-0000-addressing-comment-2142"
>> >
>> >I have no idea what that means.
>> >
>> >- Against my comment #143 the resolution detail reads:
>> >
>> >"Upload Word versions of files for each section with Track changes on
>> >where these changes are made.Discuss this in a teleconference before
>> >updating the draft and going for a recirculation. based on PICS
>> >discussion, there is no need to have a one to one correspondence
>> >between "sha;;s"/"mays" and entries in the PICS table."
>> >
>> >Again, it is entirely unclear what this means - it seems to be a set
>> >of notes on ongoing work on resolving the comment, rather than a
>> >statement of what you are going to do to the draft.
>> >
>> >- Against my comment #142 the resolution detail reads:
>> >
>> >"Assigned to DJ21-08-0086-00-0000-addressing-comment-2142 Waiting
>> >update from DJ/VG"
>> >
>> >This looks like it is simply a statement of an action on DJ/VG, not a
>> >statement of how the comment will be resolved.
>> >
>> >It seems that this problem isn't confined to my comments. For
>> >example, see the resolution detail against Kimm Ronny's comment #252,
>> >which says:
>> >
>> >"See conribution
>> >21-08-0056-01-0000-d9-0-primitive-and-message-consistency.doc"
>> >
>> >This isn't an internal 802 ballot; voters that are not 802 attendees
>> >will have no idea what that document is, or where to find it. As a
>> >non-802.21 person, I also have no idea where to find it either.
>> >
>> >I haven't looked much further through the comment database, but given
>> >that the comment #252 problem was spotted from looking at a sample of
>> >two comments other than my own, I have absolutely no doubt that these
>> >are not isolated examples.
>> >
>> >Frankly, I am shocked that a draft is being issued for a
>> >recirculation ballot when it is patently impossible for the voters to
>> >determine how, or even if, all of the comments on the previous draft
>> >have been resolved by the working group.
>> >
>> >You must stop this ballot immediately and complete the process of
>> >properly documenting your ballot resolution on the previous recirc
>> >before attempting to re-start the recirculation.
>> >
>> >I am copying this to David Law (RevCom Chair) for his information also.
>> >
>> >Regards,
>> >Tony
>
>