ITS Network & Systems Status

Jul 29, 2021

Unresolved incident: Wireless Connectivity.

Jul 28, 2021

Completed - The scheduled maintenance has been completed.
Jul 28, 18:00 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 28, 15:02 PDT

Update - We will be undergoing scheduled maintenance during this time.
Jul 28, 14:27 PDT

Scheduled - What: Maintenance on Student Activity Hub
When: 7/29/2021 from 3 PM to 6 PM

What: The Student Activity Hub (SAH) reports using Hana Production will be unavailable today (prod release date) from 3pm-6pm PT for scheduled enhancement migrations.

The expected duration for completing the implementation plan is approx. 30 minutes. The remaining time will be used for testing and verification of functionality.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Jul 28, 14:27 PDT

Jul 27, 2021

Maintenance: Active Directory minimum password length change

Completed - The scheduled maintenance has been completed.
Jul 27, 12:30 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 27, 12:00 PDT

Scheduled - ITS has scheduled the following maintenance:

A minimum Active Directory password length of 12 characters instituted. This is being implemented to improve security around Active Directory as password length is a key component of password complexity that fortifies against brute force password attempts.

What: Minimum Active Directory Password Length of 12 Characters
When: 6/27/2021 at Noon
Why: Password length is a key component of password complexity that will help fortify against brute force password attempts.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Jul 22, 15:34 PDT

Maintenance: Reload network switch scrm-1-1-1stflr-3850-stk

Completed - The scheduled maintenance has been completed.
Jul 27, 05:30 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 27, 05:00 PDT

Scheduled - ITS has scheduled the following maintenance:

START: Tuesday, July 27, 2021, at 5 AM
STOP: Tuesday, July 27, 2021, at 5:30 AM

SCOPE: Reload the switch to upgrade the IOS software code to resolve a software bug issue.

IMPACT: Loss of wired and wireless network connectivity to network switch scrm-1-1-1stflr-3850-stk for approx 30 minutes.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Jul 20, 11:28 PDT

Maintenance: Reload a-mcgill-0-1-9300-stk network switch

Completed - The scheduled maintenance has been completed.
Jul 27, 05:00 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 27, 04:30 PDT

Scheduled - ITS has scheduled the following maintenance:

START: Tuesday, July 27th at 4:30 AM
STOP: Tuesday, July 27th at 5:00AM

SCOPE: Reload switch to correct switch stack priority issue

IMPACT: Loss of wired and wireless network connectivity to devices connected to a-mcgill-0-1-9300-stk for a duration of 15 to 20 minutes.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Jul 19, 10:45 PDT

Maintenance: Reload wan-unex-grnwich-2-1-2960x-stk-1 network switch

Completed - The scheduled maintenance has been completed.
Jul 27, 04:30 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 27, 04:00 PDT

Scheduled - ITS has scheduled the following maintenance:

START:Tuesday, July 27th at 4:00 AM
STOP: Tuesday, July 27th at 4:30 AM

SCOPE: Reload switch to correct switch stack priority issue.

IMPACT: Loss of wired and wireless network connectivity to devices connected to wan-unex-grnwich-2-1-2960x-stk-1 for a duration of 15 to 20 minutes.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Jul 19, 10:16 PDT

Jul 26, 2021

Maintenance: Code Upgrade > m-nodem-rack-a2-3850-48

Completed - The scheduled maintenance has been completed.
Jul 26, 08:00 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 26, 07:01 PDT

Scheduled - ITS has scheduled the following maintenance:

WHEN : Monday, July 26th, 2021 7am-8am

WHAT : Maintenance - Code Upgrade > m-nodem-rack-a2-3850-48

The switch needs a code upgrade to fix bugs, improve performance and security.

IMPACT : No management access for devices for about an hour. No network interruptions. Should not be noticeable for customers.

IT Services will be conducting network maintenance for a code upgrade involving m-nodem-rack-a2-3850-48 on Monday, July 26, 2021 from 7:00 to 8:00 a.m. During this maintenance window, the switch will be receiving a code upgrade to fix bugs, as well as improve performance and security. There will be no management access for devices during this time, however, there won't be any network interruptions.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Jul 22, 15:57 PDT

Jul 25, 2021

No incidents reported.

Jul 24, 2021

No incidents reported.

Jul 23, 2021

ITS Datacom maintenance Friday July 23, 2021 9:00pm - 11:00pm software upgrade nodem-asr-1006 WAN router

Completed - The scheduled maintenance has been completed.
Jul 23, 23:02 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 23, 21:00 PDT

Scheduled - ITS Datacom has scheduled the following maintenance:

WHAT: Upgrade system software on nodem-asr-1006 WAN router (Central Utilities Building)
WHEN: Friday July 23, 2021 from 9pm - 11pm
SCOPE: Upgrade ROMMON, FPGA and IOS-XE software
IMPACT: Periodic loss of wired and wireless network connectivity for the following off-campus locations.

Rad-Onc-1200-Garden-View-891
4076-3rd-Ave-HNRC
UCOP-ANR-9335-Hazard-Way-Suite-201
BFS and Lib Annex 7835 Trade St
Pediatrics 7910 Frost St
NeuralComp 8110 La Jolla Shores Dr.
UNEX-Greenwich
CPC-(FPM)-ASE-Grenwch-dr
OSI 3636 NOBEL DR Ste 330
EH&S 3636 NOBEL DR Ste 340
Campus Counsel 3636 NOBEL DR Ste 490
6165 GREENWICH DR
F&PM-RB 16766 Bernardo Ctr Dr
Mt Soledad 7110 VIA CAPRI
GPH 3500 5th Avenue Suite 103
GPH 3500 5th Avenue Suite 302

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Jul 21, 14:28 PDT

Jul 22, 2021

Wireless Outage

Resolved - All problems with the listed services have been resolved.

*UCSD Protected
*UCSD Guest wireless networks

If you continue to experience problems or have any questions, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thanks again for your patience during the service outage.
Jul 22, 13:47 PDT

Update - We are continuing to investigate this problem. There is no additional information to share at this time. Updates and additional details will be shared here once they become available.

If you do notice any additional outages or impacts, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thank you for your continued patience.
Jul 22, 11:32 PDT

Update - We are continuing to investigate this problem. There is no additional information to share at this time. Updates and additional details will be shared here once they become available.

If you do notice any additional outages or impacts, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thank you for your continued patience.
Jul 22, 10:57 PDT

Identified - Our network team believes they have found the source of the problem impacting wireless and are working with our vendors to resolved it ASAP. Thanks again for your patience.
Jul 22, 10:21 PDT

Investigating - We are getting reports from various areas around campus of customers unable to access Protected and Guest wireless. Our network team is investigating and we will provide updates as we receive them. Thanks for your patience while we work on resolving this.

If you do notice any additional outages or impacts, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.
Jul 22, 10:02 PDT

Widespread Internet Outages

Resolved - All problems with the listed services have been resolved.

*SAP Concur production and test sites
*Oracle Financial Systems
*Qualtrics, including the Symptom Screening Tool
*LastPass

If you continue to experience problems or have any questions, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thanks again for your patience during the service outage.
Jul 22, 11:37 PDT

Monitoring - Most of the services are becoming available. We'll continue to monitor.
Jul 22, 09:53 PDT

Update - The services that we have identified below are part of a broader outage impacting many cloud based services and sites.
Jul 22, 09:41 PDT

Investigating - We are receiving reports of various services/tools that are currently unavailable due to a global widespread DNS outage:

*SAP Concur production and test sites
*Oracle Financial Systems
*Qualtrics, including the Symptom Screening Tool
*LastPass

We are also seeing reports of many web sites/services outside of campus that are currently unavailable. For information on how widespread this is, see:

https://www.cnbc.com/2021/07/22/several-major-websites-go-down-in-widespread-internet-outage.html

If you do notice any additional outages or impacts, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thank you for your patience.
Jul 22, 08:59 PDT

Reload v-lchtag-3-2-3850-stk network switch

Completed - The scheduled maintenance has been completed.
Jul 22, 04:50 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 22, 04:30 PDT

Scheduled - WHAT: Reload v-lchtag-3-2-3850-stk network switch
WHEN: Thursday, July 22nd at 4:30 AM
SCOPE: Reload switch to correct switch stacking order
IMPACT: Loss of wired and wireless network connectivity to devices connected to v-lchtag-3-2-3850-stk for a duration of 15 to 20 minutes.
Jul 12, 12:11 PDT

Network Maintenance--v-lchtag-1-2-3850-stk--Thursday, July 22nd at 4:00 AM

Completed - The scheduled maintenance has been completed.
Jul 22, 04:20 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 22, 04:00 PDT

Scheduled - ITS Datacom/Network Monitoring Team has scheduled the following network maintenance.

WHAT: Reload v-lchtag-1-2-3850-stk network switch
WHEN: Thursday, July 22nd at 4:00 AM
SCOPE: Reload switch to restore correct switch stacking order
IMPACT: Loss of wired and wireless network connectivity to devices connected to v-lchtag-1-2-3850-stk for a duration of 15 to 20 minutes.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Jul 12, 15:25 PDT

Jul 21, 2021

No incidents reported.

Jul 20, 2021

Network outage - Res Life Offices

Resolved - This incident has been resolved.
Jul 20, 17:28 PDT

Investigating - Our engineers are currently investigating possible issues with the following service:

*Network at several Res Life Offices across campus

These services may be slow or not available at this time. Updates and additional details will be shared here once they become available.

If you do notice any additional outages or impacts, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thank you for your patience.
Jul 19, 15:11 PDT

Kuali Research

Resolved - All problems with the listed service have been resolved.

*Kuali Research

If you continue to experience problems or have any questions, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thanks again for your patience during the service outage.
Jul 20, 14:32 PDT

Investigating - Kuali Research is currently experiencing connectivity issues, with users receiving "Page not Found" when attempting to access the application. Kuali Conflict of Interest and Kuali IRB continue to operate normally.

Kuali engineers are currently investigating this issue.
Jul 20, 14:02 PDT

Reload b-gsl-geisel-2-2-core-3850-stk network switch

Completed - The scheduled maintenance has been completed.
Jul 20, 04:50 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 20, 04:30 PDT

Scheduled - WHAT: Reload b-gsl-geisel-2-2-core-3850-stk network switch
WHEN: Tuesday, July 20th at 4:30 AM
SCOPE: Upgrade software code and reload switch to resolve software bug issue
IMPACT: Loss of wired and wireless network connectivity to devices connected to b-gsl-geisel-2-2-core-3850-stk for a duration of 15 to 20 minutes
Jul 12, 12:17 PDT

Jul 19, 2021

Kuali

Resolved - All problems with the listed service have been resolved:

Kuali

If you continue to experience problems or have any questions, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thanks again for your patience during the service outage.
Jul 19, 13:12 PDT

Identified - Kuali engineers are still actively working on the fix for the Dashboard and Omnisearch issues. Some of the "fixes" implemented overnight are taking longer than expected to run.

NOTE: Documents can still be accessed via lookups in Common Tasks or All Links.
Jul 16, 10:22 PDT

Investigating - Kuali users are currently receiving errors loading the Dashboard cards. At this time data is not loading into the Dashboard, causing the cards to be blank. Additionally, the Omnisearch isn't working. This issue only affects data loading into the Dashboard cards and the Omnisearch on the Dashboard. You should be able to use the Kuali application and access documents via Common Tasks or All Links.

Kuali engineers have diagnosed the problem and are actively working on a fix.
Jul 15, 13:30 PDT

Jul 18, 2021

No incidents reported.

Jul 17, 2021

Loss of connectivity for some UCSD applications and services

Resolved - At 8:42 AM until 9:25 AM, alerts were received for services we rely on that are hosted off campus, such as DUO and Incommon. IdentityNow is alerting for a loss of connectivity to it's virtual appliances that we host in VMWare. We also lost connectivity to AWS hosted VM's as well.

If you continue to experience problems or have any questions, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thanks again for your patience during the service outage.
Jul 17, 10:55 PDT

Jul 16, 2021

Campus Telephone System

Resolved - All problems with the listed servicehave been resolved.

*Campus Telephone System

If you continue to experience problems or have any questions, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thanks again for your patience during the service outage.
Jul 16, 08:05 PDT

Investigating - We are currently investigating an issue affecting incoming calls to the campus telephone system. Calls going out of campus are not affected at this time.

Please monitor this page for updates on the situation.
We apologize for the inconvenience.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Jul 15, 23:37 PDT

Jul 15, 2021

Maintenance: Financial Activity Hub (FinAH)

Completed - The scheduled maintenance has been completed.
Jul 15, 18:00 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 15, 15:00 PDT

Scheduled - ITS has scheduled the following maintenance:

What: Financial Activity Hub (FinAH) reports using Hana Production will be unavailable
When: July 15, 2021 from 3 PM to 6 PM

IT Services will be performing maintenance on Financial Activity Hub. During the maintenance Financial Activity Hub (FinAH) reports using Hana Production will be unavailable.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Jul 13, 09:53 PDT

Maintenance: Employee Activity Hub (EAH)

Completed - The scheduled maintenance has been completed.
Jul 15, 18:00 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 15, 15:00 PDT

Scheduled - ITS has scheduled the following maintenance:

What: Employee Activity Hub (EAH) reports using Hana Production will be unavailable
When: July 15th, 2021 from 3PM to 6PM

IT Services will be performing maintenance on the Employee Activity Hub. Employee Activity Hub (EAH) reports using Hana Production will be unavailable during the maintenance from 3pm-6pm PT.

The expected duration for completing the implementation plan is approx. 30 minutes with testing and verification being performed during the remaining maintenance window.
Jul 13, 09:52 PDT

Financial Activity Hub reports

Completed - The scheduled maintenance has been completed.
Jul 15, 18:00 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 15, 15:00 PDT

Scheduled - What: Financial Activity Hub reports will not be available for 3 hours
When: July 15, 2021 from 3 PM to 6 PM

ITS teams will be performing maintenance on The Financial Activity Hub (FinAH) reports. Hana Production reports will be unavailable July 15 from 3pm-6pm PT during the scheduled enhancement migrations.
Jul 12, 19:47 PDT

Network Maintenance: reload network switch n-mesa-nuevo-a-1-1-3850-stk on Thursday July 15,2021 at 5 AM

Completed - The scheduled maintenance has been completed.
Jul 15, 13:30 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 15, 13:01 PDT

Update - We will be undergoing scheduled maintenance during this time.
Jul 13, 13:08 PDT

Scheduled - ITS-Datacom/Network Monitoring has scheduled the following maintenance:

WHAT: Reload network switch n-mesa-nuevo-a-1-1-3850-stk
WHEN: Thursday July 15, 2021 at 5 AM
SCOPE: Reload the switch to upgrade the IOS software code to resolve a software bug issue
IMPACT: Loss of wired and wireless network connectivity to network switch n-mesa-nuevo-a-1-1-3850-stk for approx 30 minutes.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Jul 13, 13:01 PDT

Network Maintenance: reload network switch n-mesa-nuevo-digsign-P1-3560cx-8 on Thursday July 15, 2021 at 5:30 AM

Completed - The scheduled maintenance has been completed.
Jul 15, 06:00 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 15, 05:31 PDT

Scheduled - ITS-Datacom/Network Monitoring has scheduled the following maintenance:

WHAT: Reload network switch n-mesa-nuevo-digsign-P1-3560cx-8
WHEN: Thursday July 15, 2021 at 5:30 AM
SCOPE: Reload the switch after the successful reload/upgrade of network switch n-mesa-nuevo-a-1-1-3850-stk at 5 AM July 15
IMPACT: Loss of wired and wireless network connectivity to network switch n-mesa-nuevo-digsign-P1-3560cx-8 for approx 30 minutes.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Jul 13, 13:02 PDT

Network maintenance: Reload network switch ctri-4-2-3850-stk on Thursday July 15, 2021 at 4:30 AM

Completed - The scheduled maintenance has been completed.
Jul 15, 05:00 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 15, 04:30 PDT

Scheduled - ITS has scheduled the following maintenance:

WHAT: Reload network switch ctri-4-2-3850-stk
WHEN: Thursday July 15, 2021 at 4:30 AM
SCOPE: Reload the switch to upgrade the IOS software code to resolve a software bug issue.
IMPACT: Loss of wired and wireless network connectivity to network switch ctri-4-2-3850-stk for approx 30 minutes.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Jul 7, 13:33 PDT

Network maintenance: Reload network switch ctri-b2-2-3850-stk on Thursday July 15, 2021 at 4 AM

Completed - The scheduled maintenance has been completed.
Jul 15, 04:30 PDT

In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Jul 15, 04:01 PDT

Scheduled - ITS has scheduled the following maintenance:

WHAT: Reload network switch ctri-b2-2-3850-stk
WHEN: Thursday July 15, 2021 at 4 AM
SCOPE: Reload the switch to upgrade the IOS software code to resolve a software bug issue.
IMPACT: Loss of wired and wireless network connectivity to network switch ctri-b2-2-3850-stk for approx 30 minutes.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Jul 7, 13:32 PDT

Subscribe to Incident

About This Site