All Systems Operational

About This Site

Report a Problem: If you can't find a listing for a current issue, contact the ITS Service Desk - (858) 246-4357

Academic Services ? Operational
Business Services ? Operational
Network ? Operational
Communication and Collaboration ? Operational
Email ? Operational
Phones and Voicemail ? Operational
SSO ? Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Scheduled Maintenance
Maintenance: Copper Cable Cutover Apr 20, 08:00-17:00 PDT
ITS has scheduled the following maintenance:

What: Cut over of multiple fiber optic and copper cables on east campus
When: April 20th 8AM to 4PM

Due to work associated with the construction of the trolley guideway and re-alignment of Campus Point Drive, telephone voice services and data network services will be disrupted in some buildings on the east campus and some off-campus locations on as multiple UC San Diego fiber optic cables and copper cables will be cutover.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Posted on Apr 8, 16:35 PDT
IT Services will be upgrading the voicemail system server on April 21, 2021 from 5:00 to 7:00 a.m. During this time, saved voicemail messages (listened to and saved) will be inaccessible; however, new voicemail and access to the voicemail system during the upgrade will still be available.

What: upgrade to voicemail system server
When: 4/21 5:00am-7:00am

Saved voicemail messages (listened to and saved) will be inaccessible during upgrade; However New voicemail and access to voicemail system during the upgrade will be available at all times.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Posted on Apr 19, 13:31 PDT
ITS has scheduled the following maintenance:

START: Thursday, April 22nd at 4:00 AM
STOP: Thursday, April 22nd at 4:30 AM

IMPACT: Brief interruption of Loss of wired and wireless network connectivity to devices connected to wan-unex-grnwich-3-1-2960x-stk for a duration of 15 to 20 minutes.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Posted on Apr 19, 12:16 PDT
Past Incidents
Apr 19, 2021
VPN
Resolved - All problems with the listed service have been resolved.

*VPN

If you continue to experience problems or have any questions, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thanks again for your patience during the service outage.
Apr 19, 07:17 PDT
Update - We are continuing to work on a fix for this issue.
Apr 16, 07:19 PDT
Identified - We are continuing to investigate this problem. We now know the following:

VPN services were affected from 13:20 to 13:45. This was not a VPN issue but a problem with Internet routing between CENIC and their peering partner. Access to the VPN has been restored.

If you do notice any additional outages or impacts, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thank you for your continued patience.
Apr 15, 17:01 PDT
Investigating - Our engineers are currently investigating possible issues with the following service:

*VPN

These services may be slow or not available at this time. Updates and additional details will be shared here once they become available.

If you do notice any additional outages or impacts, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thank you for your patience.
Apr 15, 13:45 PDT
Apr 18, 2021

No incidents reported.

Apr 17, 2021
Completed - The scheduled maintenance has been completed.
Apr 17, 07:20 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 17, 07:00 PDT
Scheduled - ITS-Datacom/Network Monitoring has scheduled the following urgent maintenance:

WHAT: Configure/reload ecec-mesa-b-9500-dist to fix Cisco bug CSCvx58338 (can cause unexpected reload)
WHEN: Saturday April 17, 2021 at 7:00AM
SCOPE: Configure/reload ecec-mesa-b-9500-dist
IMPACT: Loss of wired and wireless network connectivity for the following network switches for approx. 20 minutes.
ecec-mesa-b-9500-dist
ecec-a-9300-48
ecec-b-9300-48
ecec-c-9300-48
ecec-d-9300-48
ecec-mesa-a-9300-48
ecec-mesa-b-9300-48
ecec-mesa-c-9300-48


Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Apr 15, 14:00 PDT
Completed - The scheduled maintenance has been completed.
Apr 17, 06:35 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 17, 06:15 PDT
Scheduled - ITS-Datacom/Network Monitoring has scheduled the following urgent maintenance:

WHAT: Configure/reload mandeville-b161-c9500-dist to fix Cisco bug CSCvx58338 (can cause unexpected reload)

WHEN: Friday April 17, 2021 at 6:15AM

SCOPE: Configure/reload mandeville-b161-c9500-dist

IMPACT: Loss of wired and wireless network connectivity for the following network switches for approx. 20 minutes.
mandeville-b161-c9500-dist
mandeville-b102-c9300-48
mandeville-b104-c9300-stk
mandeville-b115-c9300-48
mandeville-b147-c9300-48
mandevlle-b150-its-av-c9200-24
mandeville-b161-c9300-48
mandeville-b175-c9300-48
mandeville-b206-c9300-48
mandeville-b210-its-av-c9200-24
mandeville-111-c9300-48
mandeville-143-c9300-48
mandeville-219-c9300-stk

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Network
Apr 16, 07:33 PDT
Completed - The scheduled maintenance has been completed.
Apr 17, 06:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 16, 22:00 PDT
Scheduled - ITS has scheduled the following maintenance:

What: Cut over of multiple fiber optic and copper cables on east campus
When: April 13th, 14th, 15th, 16th: 10PM to 6AM for the fiber optic cables;

Due to work associated with the construction of the trolley guideway and re-alignment of Campus Point Drive, telephone voice services and data network services will be disrupted in some buildings on the east campus and some off-campus locations on as multiple UC San Diego fiber optic cables and copper cables will be cutover.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Apr 8, 16:33 PDT
Apr 16, 2021
Completed - The scheduled maintenance has been completed.
Apr 16, 06:50 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 16, 06:30 PDT
Scheduled - ITS-Datacom/Network Monitoring has scheduled the following urgent maintenance:

WHAT: Configure/reload mandeville-b161-c9500-dist to fix Cisco bug CSCvx58338 (can cause unexpected reload)

WHEN: Friday April 16, 2021 at 6:30AM

SCOPE: Configure/reload mandeville-b161-c9500-dist

IMPACT: Loss of wired and wireless network connectivity for the following network switches for approx. 20 minutes.
mandeville-b161-c9500-dist
mandeville-b102-c9300-48
mandeville-b104-c9300-stk
mandeville-b115-c9300-48
mandeville-b147-c9300-48
mandevlle-b150-its-av-c9200-24
mandeville-b161-c9300-48
mandeville-b175-c9300-48
mandeville-b206-c9300-48
mandeville-b210-its-av-c9200-24
mandeville-111-c9300-48
mandeville-143-c9300-48
mandeville-219-c9300-stk

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Apr 15, 12:53 PDT
Completed - The scheduled maintenance has been completed.
Apr 16, 06:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 15, 22:00 PDT
Scheduled - ITS has scheduled the following maintenance:

What: Cut over of multiple fiber optic and copper cables on east campus
When: April 13th, 14th, 15th, 16th: 10PM to 6AM for the fiber optic cables;

Due to work associated with the construction of the trolley guideway and re-alignment of Campus Point Drive, telephone voice services and data network services will be disrupted in some buildings on the east campus and some off-campus locations on as multiple UC San Diego fiber optic cables and copper cables will be cutover.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Apr 8, 16:32 PDT
Apr 15, 2021
Completed - The scheduled maintenance has been completed.
Apr 15, 18:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 15, 15:00 PDT
Scheduled - Employee Activity Hub and Financial Activity Hub. Scheduled: The Employee Activity Hub (EAH) and Financial Activity Hub (FinAH) reports using Hana Production will be unavailable April 15 from 3pm-6pm PT for scheduled enhancement migrations."

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Apr 15, 13:08 PDT
Resolved - All problems with the listed service have been resolved.

*Directory Services

If you continue to experience problems or have any questions, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thanks again for your patience during the service outage.
Apr 15, 07:27 PDT
Investigating - We are currently investigating possible issues with the following service:

*Directory Services

These services may be slow or not available at this time. Updates and additional details will be shared here once they become available.

If you do notice any additional outages or impacts, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thank you for your patience.
Apr 14, 06:49 PDT
Completed - The scheduled maintenance has been completed.
Apr 15, 06:30 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 15, 06:00 PDT
Scheduled - ITS-Datacom/Network Monitoring has scheduled the following maintenance:

WHAT: Finish replacement of ebu-1-2-2-6807 with ebu1-2-2-9500-dist and configure/reload ebu1-2-2-9500-dist to fix Cisco bug CSCvx58338

WHEN: Thursday April 15, 2021 at 6AM
SCOPE: Switch fiber jumper on and configure/reload ebu1-2-2-9500-dist

IMPACT: Loss of wired and wireless network connectivity for the following network switches for approx. 30 minutes.

ebu1-2-2-9500-dist
ebu1-0.4-9300-stk
ebu1-1.1-9300-48
ebu1-1.2-9300-stk
ebu1-1.3-9300-stk
ebu1-2.1-9300-stk
ebu1-2.2-9300-48
ebu1-2.3-9300-stk
ebu1-3.2-9300-stk
ebu1-3.3-9300-stk
ebu1-3.4-9300-stk
ebu1-4.1-9300-48
ebu1-4.3-9300-stk
ebu1-5.2-9300-stk
ebu1-6.2-9300-stk
ebu1-7.1-9300-stk


Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Apr 13, 13:21 PDT
Completed - The scheduled maintenance has been completed.
Apr 15, 06:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 14, 22:00 PDT
Scheduled - ITS has scheduled the following maintenance:

What: Cut over of multiple fiber optic and copper cables on east campus
When: April 13th, 14th, 15th, 16th: 10PM to 6AM for the fiber optic cables;

Due to work associated with the construction of the trolley guideway and re-alignment of Campus Point Drive, telephone voice services and data network services will be disrupted in some buildings on the east campus and some off-campus locations on as multiple UC San Diego fiber optic cables and copper cables will be cutover.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Apr 8, 16:30 PDT
Apr 14, 2021
Resolved - All problems with the listed service have been resolved.

*HANA

If you continue to experience problems or have any questions, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thanks again for your patience during the service outage.
Apr 14, 14:39 PDT
Identified - Our engineers are currently investigating possible issues with the following service:

*HANA

These services may be slow or not available at this time. Updates and additional details will be shared here once they become available.

If you do notice any additional outages or impacts, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thank you for your patience.
Apr 14, 08:58 PDT
Completed - The scheduled maintenance has been completed.
Apr 14, 10:30 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 14, 08:00 PDT
Scheduled - WHAT: Achieve System Maintenance
WHEN: Wednesday April 14th, 8:00am-10:30am
IMPACT: Students and Staff cannot view or run Degree Audit during the maintenance period

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Apr 1, 12:11 PDT
Completed - The scheduled maintenance has been completed.
Apr 14, 06:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 13, 22:00 PDT
Scheduled - ITS has scheduled the following maintenance:

What: Cut over of multiple fiber optic and copper cables on east campus
When: April 13th, 14th, 15th, 16th: 10PM to 6AM for the fiber optic cables;

Due to work associated with the construction of the trolley guideway and re-alignment of Campus Point Drive, telephone voice services and data network services will be disrupted in some buildings on the east campus and some off-campus locations on as multiple UC San Diego fiber optic cables and copper cables will be cutover.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Apr 8, 16:27 PDT
Completed - The scheduled maintenance has been completed.
Apr 14, 00:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 13, 21:00 PDT
Scheduled - WHEN : Monday, April 13th, 2021 9PM-11:59PM

WHAT : Change EIGRP router ID on both B-M core

After maintenance on M-Core, some routes are not being accepted, causing connection failures for a set of customers. They will troubleshoot tonight, change ID routers on M-B core that are currently auto populated from DNA-C Loopback 2045 and change that to Loopback 0. Change Metrics to route EIGRP routes to M-Core. Brief disconnections may be seen during the window.

IMPACT : Primary network path for inbound/outbound network connectivity will be taken down briefly due EIGRP computing routes again, during this window. Should not be noticeable for most customers.
Apr 13, 16:50 PDT
Apr 13, 2021
Completed - The scheduled maintenance has been completed.
Apr 13, 19:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 13, 17:01 PDT
Scheduled - What: Financial Aid application will be upgraded
When: April 13th from 5PM to 7PM

ITS development teams will be performing software upgrades to the Financial Aid application which will result the Financial Application being unavailable on 4/13th from 5pm - 7pm.
Apr 9, 12:16 PDT
Completed - The scheduled maintenance has been completed.
Apr 13, 00:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 12, 21:01 PDT
Scheduled - WHEN : Monday, April 12th, 2021 9PM-11:59PM

WHAT : Node-M Core Router supervisor failover and troubleshooting

After yesterday's maintenance on M-Core, some routes are not being accepted, causing connection failures for a set of customers. They will troubleshoot tonight, including failing over to the backup supervisor on M-Core. Brief disconnections may be seen during the window.

IMPACT : Primary network path for inbound/outbound network connectivity will be taken down briefly (due to Dual Supervisors) during this window. Should not be noticeable for most customers. Secondary network path through Node-B will remain active and passing all traffic including spanned VLANs to/from SDSC and TPCS during this window.
Apr 12, 15:26 PDT
Apr 12, 2021
Resolved - All problems with the listed service have been resolved.

*eTranscript


If you continue to experience problems or have any questions, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thanks again for your patience during the service outage.
Apr 12, 08:59 PDT
Investigating - Our engineers are currently investigating possible issues with the following service:

*eTranscript


These services may be slow or not available at this time. Updates and additional details will be shared here once they become available.

If you do notice any additional outages or impacts, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thank you for your patience.
Apr 12, 07:33 PDT
Completed - The scheduled maintenance has been completed.
Apr 12, 00:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 11, 21:00 PDT
Scheduled - ITS has scheduled the following maintenance:

WHEN : SUN, April 11th, 2021 9PM-11:59PM
WHAT :IT Services will be performing a Node-B and Node-M Core Router IOS upgrade to address bugs

IMPACT : The primary network path will then be taken down briefly, while the secondary network path through Node-B will remain active and passing all traffic including spanned VLANs to/from SDSC and TPCS during this window. During both windows, connectivity to Health Sciences will experience brief disconnects.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Apr 2, 15:27 PDT
Apr 11, 2021
Apr 10, 2021

No incidents reported.

Apr 9, 2021
Completed - The migration has completed successfully.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Apr 9, 06:11 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 9, 06:00 PDT
Scheduled - ITS has scheduled the following maintenance:

WHEN : Friday, April 9th, 2021 6am - 7am
WHAT : IT Services will be updating the admin site for the Status Page

The ITS Service Desk will be migrating the admin site from StatusPage.io to new Atlassian site.

IMPACT : The migration is expected to take 30 minutes or less. The Status Page will remain visible to the public during the migration, but there will be no access to modify the Status Page once the migration has started until it is finished.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Apr 8, 12:44 PDT
Completed - The scheduled maintenance has been completed.
Apr 9, 01:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 8, 21:00 PDT
Scheduled - What: The domain controller SKITTLES (132.239.0.140) is being replaced with a virtual machine.
When: The evening of April 8th, between 9:00 PM and 1:00 AM

SKITTLES will temporarily be offline during this process. When completed the hostname and IP address will remain the same.
Mar 31, 11:59 PDT
Completed - The scheduled maintenance has been completed.
Apr 9, 00:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 8, 21:00 PDT
Scheduled - ITS has scheduled the following maintenance:

What: Power cycle line cards of b-core router
When: Thursday 4/8/2021 from 9 PM to 11 PM

ITS network team will power cycle the line cards of b-core router. Errors are being generated by the card and the vendor recommends power cycling or reseating it then verify the errors ceased.

Network with alternate path won't be affected as we will power cycle the cards one at a time but might experience brief packet loss. Network without redundancy will have an outage for about 10-15 minutes while the card is rebooting. Any VLAN which spans vlan through the b-core will also be affected.

Connections with redundant or alternate path

its-nap-6509-20G-p
m-dist-6509-20G-po
b-dist-6509-20G-po
m-core-40G-po17
sec-arista7150-80G
VPN-b4110-via-b150
VPN-b4150-anyconne
VPN-b4150-anyconne
NAT-fortigate-agg1
border-mx0-200G-po
sdsc-nex7710-7-40G
b-ntwk-svcs9300-20
b-wrlss-6509-20G-p
agg1-vss-80G-po106
m-wrlss-6509-20G-p
agg2-vss-80G-po116
SIO-AGG-1-80 connected
wrlss-m-asr-1002hx
sio-colo-7702
border-mx1-100G
[resnet]b-rsnt-680
nodeb-f5-ldblncr-1
nodem-f5-ldblncr-1
n-core-6807-10G
scrm-6509
wan-m-asr1006
wrlss-b-asr1002hx-

Below are the buildings with single path and with vlan spanned and will experience an outage:

sdsc-b2-east-6807
hillcrest-switchroom
vlan 6 [muir]ad-test-forest
vlan 200[m]hds-srvr-int-inside
vlan 201 [m]hds-srvr-ext-inside
vlan 337 [b]pps-vlan
vlan 2856 [s]s-ogs-srvr-int-inside
vlan 2858 [s]s-ogs-srvr-ext-inside
vlan 3444 [its-colo]ucsd-ats-ogs-inside
vlan spanned through the core from ADCOM.



Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Apr 8, 16:41 PDT
Apr 8, 2021
Completed - The scheduled maintenance has been completed.
Apr 8, 00:00 PDT
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Apr 7, 21:02 PDT
Scheduled - ITS has scheduled the following maintenance:

WHEN : WED, April 7th, 2021 9PM-11:59PM
WHAT : IT Services will be performing a Node-B and Node-M Core Router IOS upgrade to address bugs

IT Services will be performing a Node-B and Node-M Core Router IOS upgrade to address bugs during two distinct maintenance windows

IMPACT : The secondary network path for inbound/outbound network connectivity will be taken down briefly (due to Dual Supervisors), while the primary path through Node-M will remain active. Certain customers who have VLANs spanned to/from campus locations to data centers at SDSC and TPCS will experience a brief interruption of service during this window. Some voice services would be affected as well.

Please contact ITS Service Desk at (858)246-HELP(4357) or servicedesk@ucsd.edu if you have any questions or concerns.
Apr 2, 15:26 PDT
Apr 7, 2021
Resolved - All problems with the listed service have been resolved.

*DUO All Deployments: Phone Call Delivery Issues to Chinese Numbers

If you continue to experience problems or have any questions, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thanks again for your patience during the service outage.
Apr 7, 07:44 PDT
Identified - dentified - We have identified the issue causing phone delivery issues to Chinese numbers on all of our deployments and are working to correct the issue as soon as possible.
Please, where possible, use alternative authentication methods, such as push or SMS.

Please check back here or subscribe to updates for any changes.
Apr 6, 01:38 EDT
Investigating - We have received reports from multiple customers about issues authenticating using phone callback to Chinese numbers with Duo. We are currently investigating and will post an update once we have gathered more information.

Please check back here or subscribe to updates for any changes.
https://status.duo.com/
Apr 6, 09:00 PDT
Apr 6, 2021
Resolved - All problems with the listed service have been resolved.

*Outlook

If you continue to experience problems or have any questions, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thanks again for your patience during the service outage.
Apr 6, 12:22 PDT
Investigating - Our engineers are currently investigating possible issues with the following service:

*Outlook

Outlook for Windows customers may receive a pop-up message when sending to some UCSD recipient that states "We won't be able to deliver this message." We are currently investigating the situation with MS engineers to determine the source of the problem. Customers can use OWA if they are encountering this problems. We will update within the next hour.

Updates and additional details will be shared here once they become available.

If you do notice any additional outages or impacts, please contact the ITS Service Desk at servicedesk@ucsd.edu or 858-246-4357.

Thank you for your patience.
Apr 2, 10:51 PDT
Apr 5, 2021

No incidents reported.