Re: [802.3BA] [HSSG] Reach Ad Hoc
Chris and others,
As I noted in my previous e-mail, I send out a survey among a large
end user group (AMS-IX customer base). The responses I got (some 30)
were more or less similar to what you describe below. Based on these
responses I would support your conclusion below.
- Henk Steenman
On Jan 7, 2008, at 9:34 PM, Chris Cole wrote:
> Following the 12/06/07 Reach Ad Hoc conference call, I sent an email
> to
> the HSSG reflector (appended at the end of this email) requesting
> continued end user input on reach distribution for 10km and shorter
> applications.
>
> My email was forwarded to several end user email lists, for example
> NANOG (North American Network Operators Group). I subsequently
> received
> a number of emails from different data center operators commenting on
> their needs. This gave me the opportunity to follow up with some more
> detailed exchanges to better try to understand how optics are being
> used. While it would have been preferable for those emails exchanges
> to
> have occurred on the HSSG reflector, at least I can share the
> insights I
> gained.
>
> More then half the emails objected to reducing the 10km reach
> objective.
> There were three broad categories for why a 3km or 4km reach objective
> was seen an unacceptable. For a portion of the end users, 3km or 4km
> was
> not enough reach; they needed something a bit more then 4km for
> example
> 5km. For another portion, the extra 10km link budget was used to
> accommodate much higher number of connectors / patch panels then
> nominally in the standard, or just to deal with sloppy patch cord
> handling. The last portion of the end users had a significant fraction
> of links between 4km and 10km.
>
> The emails that preferred a 3km or 4km reach objective were in three
> broad categories of what was seen as the volume sweet spot, for which
> minimum cost was paramount. The three sets of reaches that were seen
> as
> capturing most of the volume were: 1) 500m to 1km, 2) <2km, 3) <4km.
>
> I received permission from several of the end users to share their
> emails with the HSSG reflector, and they are appended below. These
> emails are meant to be exemplary and there is no intent to imply that
> they accurate reflect the distribution of opinions for any of the end
> user groups.
>
> My conclusion from these exchanges is that there are many different
> ways
> in which optics are used. End users have adopted the existing 10km
> reach
> standard to a variety of applications, and we should be very careful
> before making changes that may break many applications that we are not
> even aware of.
>
> At this point, I support keeping the existing 10km reach objective,
> and
> do not support changing to a shorter reach.
>
> I would also encourage anyone on the HSSG reflector, who is also a
> member of any of the end user groups, like NANOG, to share their
> views,
> or summarize their take on the views of their group on the HSSG
> reflector. This will add to the credibility of this discussion.
>
> Chris
>
>
>> -----Original Message-----
>> From: Sam Silvester [mailto:sam.silvester@gmail.com]
>> Sent: Thursday, December 13, 2007 3:13 PM
>> To: Chris Cole
>> Subject: Re: IEEE 40GE & 100GE
>>
>> Hi Chris.
>>
>> Working for a reasonably small (geographically speaking - within a
>> single state / capital city) carrier, as long as (per your earlier
>> comments) 10km optics are happy talking to each other with no
>> attenuator required, I see very little value in a 4km reach version.
>>
>> If we're talking about a cost ratio of between 1.15 and 1.3, this
>> just
>> reinforces the fact in my opinion.
>>
>> Internally i.e. within our PoPs, we use 10km optics currently at 1GE
>> and 10GE (even for < 500m runs). We have some deployment of 40km but
>> in general due to geography, 10km does the job nicely.
>>
>> I'd suggest even if a 4km optic came along, I'd not bother as the
>> "end
>> user price difference" most likely wouldn't excite me enough to have
>> to carry another line of spares. I realise this is more or less in
>> line with other comments, but I figured you might like the input
>> anyway.
>>
>> Excluding "short runs" of 500m, I'd say the ratio of <4km links vs
>> <10km links is about 50/50.
>>
>> Note that I expect in an enterprise-style deployment the number may
>> well stack up with the 90% figure you mentioned in your email. I'm
>> still not sure a 1.15 - 1.3 ratio makes that worthwhile...
>>
>> Regards,
>>
>> Sam
>
>> -----Original Message-----
>> From: Justin M. Streiner [mailto:streiner@cluebyfour.org]
>> Sent: Wednesday, December 12, 2007 4:51 AM
>> To: nanog@merit.edu
>> Cc: Chris Cole
>> Subject: Re: IEEE 40GE & 100GE
>>
>> On Wed, 12 Dec 2007, Robert E. Seastrom wrote:
>>
>>> A practical question here: does anyone know offhand if 4km reach is
>>> adequate for interbuilding access (i.e., DC[124] to DC3) access at
>>> Equinix Ashburn, including worst-case interior wiring and cross
>>> connects? I'm thinking that's cutting it close. The enterprise
>>> people are substantially less likely to find themselves with a lot
>>> of
>
>>> interconnections in a GCE (Ginormous Campus Environment) than we
>>> are,
>
>>> and I suspect that skews the 90% number a bit. Folks who are more
>>> familiar with the layout of other facilities may wish to chime in
>>> here.
>>
>> I'm not in any of the Equinix facilities, but I do run a decent-sized
>> urban campus network and a 3km-4km distance limitation would be
>> cutting it really close for me in some cases. Some of the 10G links
>> on my backbone today do require multiple physical cross-connects,
>> which would eats into the link budget. Most of my backbone
>> connections
>> work find with 10G-LX4 ptics, but there are a few places where 10G-ER
>> is needed.
>>
>> I haven't read the draft spec yet to see what's being proposed for a
>> link budget at 3/4/10km, but that's just as important as the physical
>> distance.
>>
>> jms
>>
>> -----Original Message-----
>> From: Randy Whitney [mailto:randy.whitney@verizonbusiness.com]
>> Sent: Wednesday, December 12, 2007 8:46 AM
>> To: Chris Cole
>> Subject: Feedback on 100GE Proposed
>>
>> Chris,
>>
>> Per cross-posting to lists I am on I am providing you feedback on the
>> proposed change to the 100GE standard.
>>
>> For my deployment needs, 3km or 4km would *not* be sufficient in many
>> cases, so I would be strongly opposed to reducing the std down to 3km
>> or 4km.
>>
>> Best Regards,
>> --
>> Randy S Whitney +1 703 886 5642
> (0)
>> Verizon Business, Global Peering Manager +1 703 886 0512
> (F)
>> 22001 Loudoun County Pkwy, Ashburn, VA 20147 +1 202 997 8174
> (M)
>> randy.whitney@verizonbusiness.com uursw (AIM,
> YAHOO)
>>
>
>
>> -----Original Message-----
>> From: Robert E. Seastrom [mailto:rs@seastrom.com]
>> Sent: Wednesday, December 12, 2007 11:20 AM
>> To: Chris Cole
>> Cc: Justin M. Streiner; nanog@merit.edu
>> Subject: Re: IEEE 40GE & 100GE
>>
>>
>> "Chris Cole" <chris.cole@finisar.com> writes:
>>
>>> One of the points made by Ted Sealy from Sprint is that they take
>>> advantage of the extra link budget in 10GBASE-LR 10km link budget to
>>> account for extra connector loss, etc.
>>
>> Ted Seely and I are of the same mind on this. 2 dB sounds like
>> plenty
>
>> for connector loss right up until you have to deal with multiple
>> patch
>
>> bays in a structured system with amateurishly applied mechanical
>> splices. The difference between noting that the loss is a little
>> high
>
>> but the link still works so you roll with it, and having to spend
>> time
>
>> on the phone arguing with someone who thinks 24 dB link loss is A-OK,
>> will make the slight additional up front cost for the better grade
>> optics look very inexpensive indeed...
>>
>> ---rob
>
>> -----Original Message-----
>> From: Dale W. Carder [mailto:dwcarder@wisc.edu]
>> Sent: Wednesday, December 12, 2007 12:53 PM
>> To: Chris Cole
>> Subject: Fwd: IEEE 40GE & 100GE
>>
>>
>> Hi Chris,
>>
>> I appreciate the opportunity to provide some end-user perspective
>> from
>
>> our environment, the University of Wisconsin @ Madison and WiscNet.
>>
>> Most of what I saw in the links below was about link budget, but what
>> about ORL? We got bit pretty hard migrating from 1G to 10G through
>> old hand-polished SMF panels which had never been a problem before.
>> We ended up cutting & fusion splicing on new pigtails in numerous
>> locations.
>>
>> Data Center / super-computing environments:
>> - high performance connections are all within 300 meters,
>> with both FDDI grade MMF (largely abandoned) and also SMF-28.
>>
>> Campus environment:
>> - about (200) buildings all within 3km of our central
>> aggregation points on SMF-28.
>> - roughly 40% of point-to-point links have (2) patches
>> - roughly 60% of point-to-point links have (4) patches
>> (via an intermediate building's patch-panel)
>>
>> Metro Ethernet environments:
>> - about (10) links 8-10km, (4) 10-12km.
>> - We are running 10G-LX today on a few links beyond 10km.
>> - most have 6-8 patches
>>
>> Regional Long-Haul environment:
>> - (30) 80-100km links
>>
>> We are quite interested in what happens with HSSG, let me know if
>> there's anything else we can help with.
>>
>> Dale
>>
>> --
>> Dale W. Carder - Network Engineer
>> University of Wisconsin - Madison / WiscNet
>> http://net.doit.wisc.edu/~dwcarder
>>
>
>> -----Original Message-----
>> From: Brian Dickson [mailto:briand@ca.afilias.info]
>> Sent: Tuesday, December 11, 2007 6:17 PM
>> To: Chris Cole
>> Subject: Re: [HSSG] Reach Ad Hoc
>>
>> Hi, Chris,
>>
>> Bora Akyol forwarded your summary message on the 10 km vs 3/4 km
> issue,
>> to the NANOG mailing list, indicating that the IEEE needs operator
>> feedback on this issue.
>>
>> I'm pretty familiar with who needs to be deploying 40G and 100G, and
>> where and how they are likely to need these interfaces.
>>
>> There are primarily two places we expect to see *lots* of these
>> connections.
>>
>> One is - service provider backbones. These are the internal network
>> links that form major ISP/NSP network infrastructure. Those will fall
>> into a number of categories, ranging from DWDM, to CWDM, to
>> direct-over-fibre.
>>
>> The other is, the interconnections between ISPs. The number of
>> connections on these ingress/egress points is likely to be higher by
> an
>> order of magnitude - anywhere from 2x to 10x the links,
>> proportionally
>
>> speaking.
>>
>> It is worth noting that even when traffic levels may only be
> approaching
>>
>> 10Gbps, that 100G is *very* attractive, notwithstanding the price.
> Burst
>>
>> capacity is what IP network engineering is all about, not average
> usage.
>> And, even if an ISP is small compared to his neighbor, the links have
> to
>>
>> operate at one speed, which *needs* to be the highest-common-
>> multiple,
>
>> not the lowest-common-denominator. When in doubt, go high, is the
> rule.
>>
>> And, also worth noting is, that once the technology is available, the
>> switching capacity is anticipated to meet the line rate head-on.
>> Line-rate many-port switches will be the norm, and access to those is
>> best achieved at 100G.
>> Getting those cards affordable is a huge deal, and getting them
>> delivered in a timely fashion by the parts manufacturing community is
>> something we are *very* worried about. Even 10G cards are late to the
>> game.
>>
>> The places where ISPs interconnect, are a few dozen locations
> worldwide,
>> and those have the following common characteristics:
>>
>> A very large building, with cages, suites, and cabinets
>> A meet-me area where all interconnections generally run
>> Some big colocation providers who re-sell some of the building
>> real-estate, with their own meet-me areas
>> Hundreds to tens of thousands of companies colocating their equiment,
>> especially ISPs
>> Interconnections run from ISPs to common switching equipment (an
>> Internet eXchange, or IX)
>> Interconnections directly between ISPs bypassing the IX
>>
>> The former (IX connections) typically reach much further down the
>> list
>
>> of customers, many 10s to a few hundreds
>> The latter (private peering) typically use the same media type (SMF,
> or
>> sometimes MMF) regardless of speed - 100M, 1000M, 10G, 40G, 100G
>>
>> The buildings will typically be telco hotels (vertical, up to 30 or
>> so
>
>> floors), no more than 50m long and 30m wide, max fibre distance of
> 300m
>> typically, but as many as 8-10 patches per link, or as few as zero.
>> Some other buildings are more warehouse type - one floor only, but as
>> big as 300m x 500m or more, max fibre distance typically 1.5-2km,
>> with
>
>> probably fewer patches (2, 4, 6 are typical numbers).
>>
>> Based on this info, I would suggest 4km over 3km, but that 4km is
>> very
>
>> much of interest to the buying community, who (based on volume of
> links)
>> are definitely interested in keeping costs down. Many of those folks
>> were bitten in the @ss by OC-192 costs that were about 5x higher than
>> they should have been.
>>
>> Hope this information is helpful, and if you would like me to put you
> in
>> touch with some folks who operate the neutral data centers and run
>> the
>
>> fibres at them, I would be happy to oblige.
>>
>> Thanks for allowing me to provide some input,
>>
>> Brian Dickson
>> Afilias Canada
>>
>
> -----Original Message-----
> From: Chris Cole [mailto:chris.cole@FINISAR.COM]
> Sent: Tuesday, December 11, 2007 4:22 PM
> To: STDS-802-3-HSSG@LISTSERV.IEEE.ORG
> Subject: Re: [HSSG] Reach Ad Hoc
>
> During the November HSSG meeting, optics vendors made a presentation
> proposing changing the 10km reach objective to 3km or 4km. One of my
> motivations for working on the proposal was informal input from a
> number
> of 100GE end users that >90% of their data center and short
> interconnect
> needs would be met by a reach objective less then 4km (versus 10km.)
> With such a reach distribution, a 4km or less optimized reach
> objective
> would result in overall cost savings.
>
> As part of the HSSG effort to review this proposal, numerous requests,
> both informal as well as from the HSSG chair and Reach Ad Hoc chair,
> have been made for contributions to quantify the 10km and under reach
> distribution. While the optics vendors as suppliers can accurately
> represent the relative costs of optics alternatives, they can not
> represent end user requirements.
>
> To date, we have seen no end user presentation or data supporting
> changing the 10km reach objective to 4km or less. Unless such
> contributions are forthcoming, it is likely that there will be no
> motivation to make the change. This sentiment can be seen in the 12/7
> Reach Ad Hoc conference call minutes.
>
> I would encourage any HSSG participant that views their volume 100GE
> SMF
> needs as better met by a 4km or shorter reach objective to make a
> contribution containing reach distribution data in support of this
> position. Otherwise we will move forward with the existing approved
> objectives.
>
> Chris
>
> ________________________________________
>
> From: Andy Moorwood [mailto:amoorwood@EXTREMENETWORKS.COM]
> Sent: Tuesday, December 11, 2007 6:03 AM
> To: STDS-802-3-HSSG@LISTSERV.IEEE.ORG
> Subject: [HSSG] Reach Ad Hoc
>
> Colleagues, the meeting notes from our call last week are now posted
> on
> the IEEE website
> http://www.ieee802.org/3/hssg/public/reach/MeetingNotes_r1_1207.pdf
> Thank you for your contributions
> Andy