New York Day 4

Writing from La Guardia....

We started the day out on a down-note with Aaron feeling a bit under the weather. So, we started out one man short. Infinera called me last night about the OSC channel and we agreed to start troubleshooting around 10:30 AM. That meant we had to get the Abilene interconnect to the loaner T640 chassis up by then. Worked on the config last night and this morning.

In order to connect the new network with the old network, we're re-using a pair of Internet2 owned fiber from the 24th floor (new node) to the 14th floor (Abilene node). In order to do that, we have to move an existing Abilene connection to the new router to free up that inter-floor fiber. We chose the OC-192 interconnect between the MANLAN Nortel HDXc and the Abilene T640. That circuit has an OC-48 carved out of it for Surfnet that doesn't currently carry any traffic. Lowest impact. Lowest problem.

We decided to frame the inter-floor fiber as a 10GigE so we had enough OC-192 interfaces to the new Level3 backbone circuit. Tom played remote hands down at the Qwest POP to replace the PIC in the Abilene router. Couldn't get it to come up. Tried reseating and resetting the PIC. Nothing. Good light, just no link. Eventually, I moved the circuit to a different 10G interface and it greened right up. Bad XENPAK. Well, not bad, just not a Juniper XENPAK. Came from another vendor.......

Once we had that interface up, we tried getting the Surfnet circuit to come back up. No amount of config would make it happen. Eventually we saw the SONET circuit throwing RDI-P (Remote Defect Path) errors, which would indicate a problem on the Amsterdam side of the circuit. Talked with Caroline to see if she could follow up with them. Ball's out of our court. Moving along.

And move along we did. 20 minutes to get the router up to production speed and make sure I could route through it to the rest of the R&E world. DONE! Infinera calling!

We did a bit of work on the Infinera to verify that I configured everything correctly. Of course (*grin*) I did and all was well. We didn't get the IP connectivity via the OSC channel layer2 payload working, which means that I can't see the racklan at 111 N. Canal from 32 AofA. Since we could manage both sides, I decided to move on and work on the IP connectivity from Indiana on Monday. We configured an OC-192 up to a loop at the 32AofA side and I was happy. Done with that.

By then, Aaron started feeling better. Tom had been working on Ciena power and just finished. So we moved on to housecleaning items. Tom went over to 111 8th Ave. to plug a client fiber in for the new backbone circuit and run a copper cross-connect to the Level3 high speed IP drop that L3 was installing into our MFP. He got that interface up. We got everything documented and all is well.

We'll likely go back in the near future to clean up the racks a bit. We didn't get the Ciena control PC inverter going and we didn't get a chance to cut all the power cables and ethernet cables to length.

A VERY big shout out to Jim Shaffer for all his work. He did a great job staying on the contractors about the power install issues and making sure we had what we needed. I'm sorry to say we were a lug or two short, and he helped us out. Good support at the POP can make an install go so much more smoothly. I'd hate to think how this would have gone without him around.

Photos are updated. We're going to enjoy a nice quiet weekend before we get to deal with this again next week. :-)