University of BRISTOL University of Bristol - ResNet Information Services banner
skip menus | Front Page | ResNet Website 

Archive for the 'Servers' Category

 

Database Issues - 14th Dec, 17:18 (resolved)Permanent Link to Database Issues - 14th Dec, 17:18 (resolved)

Wednesday, December 14th, 2011

The database server used by a large number of ResNet systems is currently experiencing problems.  The issues started at around 17:18pm on Weds 14th December.

Until the issues can be resolved, the following ResNet services are unavailable:

  • The My ResNet Control Pannel (http://my.resnet.bris.ac.uk) will be completely unavailable
  • The ResNet Activation System (http://go.resnet.bris.ac.uk) will be completely unavailable
  • All helpdesk troubleshooting tools will be unavailable
  • Various back-end scripts are failing, and may result in strange behaviour from the bandwidth monitoring systems (the ResNet Gadget/RSS feeds won't be updating) , the network status monitoring, DHCP updates will not happen etc.
Because ResNet don't run the database server, it's a little out of our hands at the moment.  The issue has been reported to the database team, and we will update this page as and when we have more information.
Update: 2011-12-15 09:35
The issue seems to be resolved (for now) and our systems seem to be working again.  However, reports from the database team suggest an underlying problem which they are continuing to investigate.

The ResNet servers are all running very slowly (resolved)Permanent Link to The ResNet servers are all running very slowly (resolved)

Monday, November 21st, 2011

About 25 of the servers which run ResNet are running *really* slowly at the moment (for various complicate and convoluted reasons) this means that customers may notice the following:

  • The ResNet DNS servers are slow and unresonsive, which effectively means that your connection may function slowly, or even not at all.
  • The My ResNet control panel (my.resnet.bris.ac.uk) is slow and unresponsive
  • The ResNet Activation System (go.resnet.bris.ac.uk) is slow and unresponsive
  • Most of  our backend troubleshooting tools are playing up as well, which makes troubleshooting individual problems difficult for the Service Desk
  • The Wireless ResNet authentication servers are caught up in this as well, so ResNet Wireless may also be unusable

Because of the widespread nature of the problem, we're expecting it to affect all ResNet customers (Wired and Wireless)

We're aware of the problem, and are doing everything we can to restore service as quickly as possible.  I've already started planning ways to stop the same issue hitting us again in the future.

Update: 21/11/2011 17:13
Good news and bad news.  The good news is that the load issues are beginning to resolve themselves and DNS seems to be back to normal.  The bad news is that somewhere in the last 2 hours we hit a (separate, but related) problem which means none of our web servers are currently running (so www.resnet/my.resnet/go.resnet etc are still dead)

We've got a lead on that problem though, and are working to get them back up again.

Update: 21/11/2011 17:50
And we're back in business!   Sorry for the inconvenience everyone.

System Maintenance, 2nd March 2011Permanent Link to System Maintenance, 2nd March 2011

Wednesday, February 23rd, 2011

Because of essential maintenance to one of the university database servers, the following services will be unavailable between 8:30am and 12:30am, Wednesday 2nd March.

The helpdesk connection troubleshooting tools will also be unavailable, so there will be a diminished support capacity for ResNet until about 1pm.

For more information about other services which will be affected by this maintenance work, see the IS website: http://www.bristol.ac.uk/is/news/2011/diversp17feb.html

Server failure (My ResNet, Registration, Slow connections)Permanent Link to Server failure (My ResNet, Registration, Slow connections)

Sunday, January 9th, 2011

It seems that one of our servers which provides My ResNet, the in-room registration system  and DNS for half of ResNet fell over on Friday evening.  Some people will have experienced slow connections as their connections failed over to the secondary DNS server.

As the office isn't staffed at weekends, I only happened to notice it by chance.  I've restarted the server and normal service was restored at around 14:45 on Sunday.

Sorry for the inconvenience caused.

Scheduled maintenance August 11th 2009Permanent Link to Scheduled maintenance August 11th 2009

Tuesday, August 4th, 2009

Information Services are carrying out some essential systems maintenance on the 11th August 2009, 10:00-10:30 which affects the following ResNet systems:

  • My ResNet
  • ResNet in-room Registration System
  • ResNet Helpdesk troubleshooting tools (including those used by the out of hours helpdesk)
  • ResNet Network Monitoring system (the traffic lights on the ResNet home page)

All fully registered ResNet connections will continue to function normally throughout this maintenance period, so most people won’t notice anything.

There are several other University systems which are also affected by this maintenance, for more details see the IS News website: http://www.bris.ac.uk/is/news/2009/koi28july.html

Server maintenance - Friday 17th AprilPermanent Link to Server maintenance - Friday 17th April

Thursday, April 16th, 2009

One of the ResNet servers is feeling a little under the weather, so we're going to have to take it offline first thing tomorrow
to run some diagnostics to confirm that the problem is what we think it is before we can log a hardware fault report.

The following services will be unavailable from 8:30am-10:30am, Friday 17th April 2009.

Currently registered connections will continue to function normally.

We will update this post when the maintenance is complete

Update: 09:42
Maintenance is now complete, and service has been restored.

Scheduled Maintenance - 16th April 2009Permanent Link to Scheduled Maintenance - 16th April 2009

Thursday, April 16th, 2009

Information Services are carrying out some essential systems maintenance on the 16th April 2009, 08:00-12:00 which affects the following ResNet systems:

  • My ResNet
  • ResNet in-room Registration System
  • ResNet Helpdesk troubleshooting tools (including those used by the out of hours helpdesk)
  • ResNet Network Monitoring system (the traffic lights on the ResNet home page)

All fully registered ResNet connections will continue to function normally throughout this maintenance period, so most people won’t notice anything.

There are several other University systems which are also affected by this maintenance, for more details see the IS News website: http://www.bris.ac.uk/is/news/2009/oraclepatch17march.html

My ResNet/Registration issues Sunday 15th Feb (resolved)Permanent Link to My ResNet/Registration issues Sunday 15th Feb (resolved)

Monday, February 16th, 2009

In the early hours of Sunday morning, one of our database servers encountered a problem and stopped accepting connections.  This meant that the My ResNet and Registration systems were unavailable, and in some cases were displaying unhelpful error messages.

The database team sprang into action on Monday morning and fixed the problem, and everything is back up and working now.  Sorry for any confusion/concern that this caused!

DNS server issues - resolvedPermanent Link to DNS server issues - resolved

Thursday, October 23rd, 2008

One of our ResNet DNS servers has developed a disk fault.

The majority of ResNet users will not be effected, as our other DNS server is ticking along nicely - however some users will notice that web usage feels a little unresponsive as their computer tries the dead server before failing over to the live one.

We swapped out the disk this morning at about 10am, in what should have been a transparent procedure with no expected downtime.  Things don't appear to have gone to plan, and the service is taking longer than expected to restore (as we've had to take the server offline while it re-synchronises its disks)

I'll update this post when we're back in business.

Update: 12:31
The disks are still syncing - but thanks to some fancy footwork, we've managed to move the DNS service over on to a backup machine, so we've got two DNS servers again.

Update: Friday 24th Oct - 11:09
Normal service has been resumed - thanks to a significant amount of mucking about with disks, a long and tedious synchronisation process, and an issue with the disk controller drivers.  I do have some more minor work to do to this server to tidy things up - but it's currently stable so I'll leave it alone for the weekend.

Single Sign On - not working (fixed)Permanent Link to Single Sign On - not working (fixed)

Tuesday, August 19th, 2008

It looks like the University single-signon system isn't working at the moment. I'm told that people are looking into it, but as far as I can tell it's a certificate problem.

This means that the following ResNet services are unavailable as there is no way to log in at present.

  • My ResNet
  • The in room registration system
  • All our administration tools

I'll update this post when the problem has been resolved.

Update: 2008-08-19 - 16:47
Problem is now resolved.  I don't have many details as yet, but access to all the above tools has been restored.