OVHcloud Network Status

Current status
Legend
  • Operational
  • Degraded performance
  • Partial Outage
  • Major Outage
  • Under maintenance
FS#11601 — 46.105.106.0/24 - 46.105.107.0/24
Incident Report for Network & Infrastructure
Resolved
One of the 2 switches N5 rebooted continuously.
Kernel uptime is 0 day(s), 0 hour(s), 5 minute(s), 40 second(s)

Last reset at 351805 usecs after Wed Sep 17 05:48:53 2014

Reason: Reset triggered due to HA policy of Reset <<<<<<<<<<<
System version: 6.0(2)N2(4)
Service: eth_port_sec hap reset

plugin
Core Plugin, Ethernet Plugin


Update(s):

Date: 2014-09-17 07:38:33 UTC
The situation is back to normal.

The FEX are online and redundancy is OK.

Date: 2014-09-17 07:38:04 UTC
we will remount the FEX, it will happen gradually
FEX FEX FEX FEX Fex
Number Description State Model Serial
------------------------------------------------------------------------
100 fex100|51C01 Online N2K-C2232PP-10GE SSI14270LHS
101 fex101|51C01 Connected N2K-C2232PP-10GE SSI14190G2G
102 fex102|51C02 Online N2K-C2232PP-10GE SSI14210GJA
103 fex103|51C03 Connected N2K-C2232PP-10GE SSI14210AKD
104 fex104|51C03 Online N2K-C2232PP-10GE SSI14210ER4
105 FEX0105 Connected N2K-C2232PP-10GE SSI14260EF8
106 fex106|51C14 Online N2K-C2248TP-1GE SSI14270R4Z
107 fex107|51C13 Connected N2K-C2248TP-1GE SSI142810Y0
108 fex108|51C12 Online N2K-C2248TP-1GE SSI142803LX
109 FEX0109 Connected N2K-C2248TP-1GE SSI14270LKB
110 FEX0110 Connected N2K-C2248TP-1GE SSI144506L9
111 fex-ASA-IPMI Connected N2K-C2248TP-1GE SSI142602BX
114 FEX0114 Online N2K-C2232TM-E-10GE SSI180600PN
115 FEX0115 Online N2K-C2232TM-E-10GE SSI1809060P
116 FEX0116 Online N2K-C2232TM-E-10GE SSI1807008N
117 FEX0117 Connected N2K-C2232TM-E-10GE SSI181101P4
118 FEX0118 Connected N2K-C2248TP-E-1GE FOX1748GETH

Date: 2014-09-17 07:36:51 UTC
Performing supervisor state verification.
2014 Sep 17 08:13:44 %$ VDC-1 %$ kernel: [ 7.064544] I2C - Mezz absent
[####################] 100% -- SUCCESS

Supervisor non-disruptive upgrade successful.


Install has been successful.

Date: 2014-09-17 07:36:09 UTC
Module 117: Non-disruptive upgrading.
[####################] 100% -- SUCCESS

Module 118: Non-disruptive upgrading.
[####################] 100% -- SUCCESS

Install has been successful.

done for the 2nd Switch


we will update the 1st switch
install all kickstart bootflash:n5000-uk9-kickstart.6.0.2.N2.5.bin system bootflash:n5000-uk9.6.0.2.N2.5.bin

Verifying image bootflash:/n5000-uk9-kickstart.6.0.2.N2.5.bin for boot variable \"kickstart\".
[####################] 100% -- SUCCESS

Verifying image bootflash:/n5000-uk9.6.0.2.N2.5.bin for boot variable \"system\".
[####################] 100% -- SUCCESS

Verifying image type.
[########### ] 50%

Date: 2014-09-17 06:00:48 UTC
Module 114: Non-disruptive upgrading.
[####################] 100% -- SUCCESS

Module 115: Non-disruptive upgrading.
[# ] 0%

18 fex on this pair

Date: 2014-09-17 05:59:22 UTC
[####################] 100% -- SUCCESS

Module 100: Non-disruptive upgrading.
[####################] 100% -- SUCCESS

Module 101: Non-disruptive upgrading.
[####################] 100% -- SUCCESS

Module 102: Non-disruptive upgrading.
[####################] 100% -- SUCCESS

Comment by OVH - Wednesday, 17 September 2014, 07:49AM

Module 106: Non-disruptive upgrading.
[####################] 100% -- SUCCESS

Module 107: Non-disruptive upgrading.
[# ] 0%


Date: 2014-09-17 05:59:10 UTC
ontinuing with installation process, please wait.
The login will be disabled until the installation is completed.
le 2nd switch en cours d'update
il preload la nouvelle image sur les FEX

Performing supervisor state verification.
[####################] 100% -- SUCCESS

Supervisor non-disruptive upgrade successful.

Pre-loading modules.
[This step might take upto 20 minutes to complete - please wait.]
[*Warning -- Please do not abort installation/reload or powercycle fexes*]
[## ] 5%

[###### ] 25%
[####### ] 30%

Date: 2014-09-17 05:58:58 UTC
We cut all the fex on this switch to stabilize it.
The traffic is going through the 2nd n5 pair.

We will update the pair in a new version of NXOS

The reload of the first the switch will have no impact because it is isolated.
However, there will be some downtime during the upgrade of the 2nd switch, the time that fex is resumed with the new version.
Posted Sep 17, 2014 - 05:56 UTC