When the Customer gets Annoying

[NETWizz]

I hate it when the customer gets annoying because they are nervous.

Usually they have had a bad experience in the past and now they are worried, so you agree to do your work on a Saturday...

Then the proceeding Tuesday, you get an email asking, "Just wanted to see if you could provide more detail about what you will be doing on Saturday."

My response: "Gutting the eleven year old Cisco equipment from the ABC half of the entire building on every floor every wiring closet; consolidating to fewer points of failure, optimizing the organization and cabling of patch panels, delivering GigabitEthernet to approximately 800 network ports /w Power over Etherent (PoE+) for the upcoming VoIP phone deployment and install of 30+ Wireless Access Points, connecting new switches between the wiring closets with the newer/better 50/125 micron OM4 multi-mode optical fiber with 10 Gigabit links between the switches."

 

[GTP]

So there!

 

[fencepost]

I hope you're bringing a team or that's going to be a really long Saturday.

 

[NETWizz]

Nope just me... Hoping to have it done by the afternoon.

 

[Markverhyden]

Wow!!! Obviously we do not know the dirty little details but I think you are being a bit over optimistic. I've done some large scale AP installs. 30 was a full day in itself and those were brand new drops. Replacements can take much longer.

 

[HCHTech]

Haha - reminds me of this!

 

[Porthos]

Nope just me... Hoping to have it done by the afternoon.

Sunday afternoon?

 

[NETWizz]

Nah, I am going to start at 7 AM in the morning and we are talking almost 800 ports spread over 6 closets. I am going to have the 16 switches /w 6 partially pre-configured, stacking cables ready, fiber patch cords, 1' patch cords etc.

I am estimating 15 minutes per switch to install with 2 hours miscellaneous (hopefully less).. I am going to go into ALL closest, tear out ALL the copper cable noting which ports were patched on a worksheet, then remove the OM1 "Orange" 62.5/125 fiber feed, and using a drill yank about the existing switches tossing everything into the hallway. I already adjusted the positioning of each patch panel such that it is a 48 port patch panel then 1 U empty rack space, then a 48 port patch panel, etc. I also pre-labeled and identified everything to be yanked.

At this point I put one of the six partially configured switches (they are all identical) in the top most position of each rack. Then I add the needed number of switches (the remaining 10 that are unconfigured) to each closet. Next, I slap all the stacking cables 2 -> 1... all the way down the switches with the last one back to the first one. Then plug them all into the UPS and let them boot. Connect my console cable to the first switch and execute a stack-enable on stack unit 1 within global config then a hitless-failover in global config... Then at privilege exec a stack secure-setup. It will give me two (2) topology options depending upon which direction I want to enumerate the switch members (i.e. clockwise or counterclockwise).. I pick the option for the second switch in the rack to be #2 (instead of the last switch) and walk away to the next closet...

Those other switches will reboot and join the stack... match the firmware... reboot and join again. They will then elect the second switch as standby and the stacking lights should show 1... 2.. 3 etc in order.

Then in each closet I plug into any console port and being the switches now have an identical config and are 95% configured AND fully updated, I set the IP address and hostname as well as pasting in the banner motd. (I will have a textfile ready with the six configs, and since the customer wants the asset tags in the banner, I will have the banners in the text file ready to paste in.) Then I type a "wr mem"


From there, I run back to the MDF with the Layer-3 switch and insert six 10GBase-SR modules and and run the fiber to the new LIU in the very top of the rack. I just keep them in order (red blue, red blue etc. or blue red, blue red, etc.) being consistent is key. Then plug the LC end into the transceivers (already have this configured).

Then in each closet I reverse (blue red, blue red etc. or red blue, red blue, etc.) the idea is to cross it over send to receive on the duplex fiber the first time, so I don't waste time flipping the ST pairs. anyway then in the top switch slide in a transceiver and plug in the LC end... each closet.

Back to the MDF verify I can see all the logical stacking units as neighbors.

Back to the closets, patch 800 ports with 800 cables I already opened.


I want to be done by noon, but realistically it might be 1 PM or 2 PM. At least the above is the plan, and I have tested each component. I have it to the point I mark on the boxes which asset number is in them and which position in which rack they go in even though 10 are not configured... I do it such that my text files are already lined up with the correct banner info.

 

[Mercenary Roadie]

You're dreaming even if you have a huge crew.

 

[NETWizz]

You're dreaming even if you have a huge crew.

I might be off by an hour or two, but I am usually pretty good and extremely fast. I do everything assembly line fassion and have every aspect of something worked out before I start.

I have already put more work into this than I will on Saturday. Tomorrow or the next day I am going to verify and configure everything and lay out my unique portion config overlay templates.

 

[glricht]

I have already put more work into this than I will on Saturday. Tomorrow or the next day I am going to verify and configure everything and lay out my unique portion config overlay templates.

Sounds like you've done a lot of pre-planning and it can definitely help. But I'm reminded of that old saying "the best efforts of mice and men ..." because just when things are going along well, something completely unforeseen pops up and the schedule goes down the tubes.

But wishing the best for you. Let us know how it went.

 

[Digital Micro]

On jobs like this in the past my best pre-planning was usually destroyed by a builder that "hide" surprises behind the walls and other places.

 

[fencepost]

Well, one thing I didn't think about - are all those APs simply going into the wiring closets instead of being mounted to the ceiling/high wall? Or, are they all replacing existing PoE units? If so and it's mostly going to be swapping them out I could see it going quickly, though I hope the old and new use the same brackets if that's relevant.

 

[NETWizz]

Oh, the APs are going in later. That is a completely separate portion of the project. I am taking care of only the structured network switching this go around. Then in another roll out a VoIP portion, then in a final portion the Wireless.

 

[Porthos]

Oh, the APs are going in later. That is a completely separate portion of the project. I am taking care of only the structured network switching this go around. Then in another roll out a VoIP portion, then in a final portion the Wireless.

Now the details emerge.
I was staring to think his name was Barry Allen. Lets see how many get that one.

 

[fencepost]

I was staring to think his name was Barry Allen.

Was he the chemical one or the magic one?

 

[NETWizz]

Now the details emerge.
I was staring to think his name was Barry Allen. Lets see how many get that one.

The hold for the APs is I am having the wiring drops installed. Otherwise, actually installing the first AP takes about an hour to configure the VLANs and generate the Subnets and trunk everything through... Then about half an hour to configure the first AP.

After that, it takes maybe a couple hours to add each port to the proper VLANs and Native VLAN to the WiFi management VLAN through the infrastructure.

Then it is a matter of barcode scanning each one and with a sharpie marking where they will be deployed on the box... Another hour.

Then about 1 hour to hang them all... and rename them.

Really it is pretty easy once the infrastructure is ready. Then for the VoIP phones, I am waiting on the telco to process a WAN circuit upgrade and with it change the QoS settings on the Telco side then agree on a a couple /24 subnets I need to route to them. Finally, I need to trunk the VLAN through the infrastructure (already done), specify the DHCP scopes and arguments, setup the default gateway, and setup QoS on my end. The configuration will take about an hour once the Telco does their part.

Then I have a phone guru who will work with the Telco to port all the numbers one at a time and individually get with each user to deploy their phone, show them how it works, etc. I suspect it will take her four days of hand-holding to get a full deployment.

 

[Porthos]

Was he the chemical one or the magic one?

Particle explosion one currently