Servers

You are currently browsing the archive for the Servers category.

We’ve recently changed the DNS Servers used by ResNet, to some new servers running on some much better hardware.

Most peoples computers have moved seamlessly over to the new DNS servers, but there are less than 20 computers still trying to use the old servers. We’ve done all we can to encourage these computers to start using the new servers but it looks like they’re not configured in our recommended manner so will need manual intervention to move.

The old servers have now been switched off, and when we do so things will break for that small handful of machines. If your ResNet Wired connection stops working, here’s what you need to do.

  1. Reboot your computer. In some cases this will cause it to pick up the new settings automatically
  2. Check your network settings, and make sure you’re set to “obtain DNS server addresses automatically”

If restarting doesn’t work, and your connection is already set to “obtain DNS server addresses automatically” check to see what DNS Servers you’ve been given.

They should start with “137.222.8.x” – if they start with “172.16.67.x” your computer hasn’t picked up the new servers properly. If this is the case, give the IT Service Desk a call, or take your computer along to the Laptop Clinic and we’ll try and help you work out why.

Part of the infrastructure which runs the ResNet back end has developed a hardware fault, some of the services it’s running are currently in a degraded state. We need to move services off it asap.

The following services are affected:

– http://go.resnet.bris.ac.uk – will be unavailable for up to an hour while we move it
– http://my.resnet.bris.ac.uk – will be unavailable for up to an hour while we move it
– DNS (ResNet Wired) will be in a degraded state, and web browsing for 50% of currently connected ResNet customers will be slower than usual
– DHCP (ResNet Wired) will be working, in a non-resilient state
– The ResNet channel in portal.bris.ac.uk will be unavailable for up to an hour while we move it

Various other administrative tools (eg the troubleshooting system used by the Service Desk when troubleshooting ResNet connection problems) will also be affected.

Important: Wireless is not affected at all, so if you’re experiencing wired connection issues – try the wireless.

Updates will follow as we get them.

Update 24th March, 16:15
– http://go.resnet.bris.ac.uk – is now available again
– http://my.resnet.bris.ac.uk – is now available again
– The ResNet channel in portal.bris.ac.uk is now available again
– DNS/DHCP (ResNet Wired) – This is now back in service in a non-resilient manner (both the primary and secondary are on the same hypervisor) which will get us through tonight but isn’t ideal in the long term.

Update 24th March, 16:56
We’ve started to move people onto a new pair of DNS servers to improve resilience and reduce the load on the non-resilient hypervisor. Early indications are that the new servers are performing nicely and that people are moving across to them smoothly as their DHCP lease renews.

Update 25th March, 10:51
The temporary hardware we used to restore service yesterday is overcommitted and while it’s working, it’s at the limit of what it can handle. We will be moving services off it today onto a more permanent home, and we’ll try to minimise downtime while we do so.

Update 26th March, 09:28
Some services are taking substantially longer than expected to migrate. We kicked off the migration of 3 VMs last night, and they’ve still not fully completed. The following services are currently unavailable:
– http://go.resnet.bris.ac.uk – (still migrating)
– http://my.resnet.bris.ac.uk – (still migrating)
– http://www.resnet.bris.ac.uk redirects (the content is still available at http://www.bristol.ac.uk/it-services/advice/homeusers/resnet)

Update 26th March, 11:00
Everything is back up on new hardware.

Update 2014-01-24 10:50am As of about 10:45am the replacement server is up and handling DNS queries so we’re back in business.

Sorry for the disruption!


We had some maintenance scheduled for 9am-10am this morning, which would have resulted in slow DNS performance for a small number of users for a comparatively short amount of time.

Unfortunately we hit some issues, the work over-ran, and then the last stage of the work failed completely rendering a DNS server unbootable.

As a result, ResNet Wired is currently running on only one DNS server.

The server which is missing is the primary for 50% of ResNet, so half of our customers will be seeing slower than usual internet access as it’s taking longer than usual to resolve domain names.

We’re in the process of building a replacement server, and we hope to have normal service restored as soon as possible.

This issue does not affect the wireless network.

There will be a short break in service affecting all ResNet connections on Wednesday 27th June, between 9am and 9:30am while we make some changes to the network infrastructure.  Additionally the ResNet service should be considered at-risk until 10:30am.

The following services will be effected:

  • ResNet Wired – your computer will not be able to connect
  • ResNet Wireless – your computer will not be able to connect
  • My ResNet control panel will not be available
  • The ResNet Activation System will not be available

If you experience a problem with your ResNet connection during that period, please wait half an hour and then restart your computer before trying again.

We’re apologise for the short notice.  Updates on our progress will be added to this blog post when we have them.

Update: 2012-06-27 09:30
As far as we can tell, everything is back up again.  The interruption for the majority of users will only have been a couple of minutes, but if you’re having problems try restarting your computer.  If that doesn’t help, contact the IT Service Desk.

Because of essential maintenance to one of the university database servers, the following services will be unavailable between 21:00 and 21:30, Monday 13th February. This work is to fix the underlying cause of the problems we experienced before Christmas.

For the duration of the work, the following ResNet services will be unavailable:
The helpdesk connection troubleshooting tools will also be unavailable, but as it’s out of Service Desk opening hours that’s not going to make much of a difference!

For more information about other services (including a number of University Websites) which will be affected by this maintenance work, see the IS website: http://www.bris.ac.uk/it-services/news/2012/diversp1feb.html

The database server used by a large number of ResNet systems is currently experiencing problems.  The issues started at around 17:18pm on Weds 14th December.

Until the issues can be resolved, the following ResNet services are unavailable:

  • The My ResNet Control Pannel (http://my.resnet.bris.ac.uk) will be completely unavailable
  • The ResNet Activation System (http://go.resnet.bris.ac.uk) will be completely unavailable
  • All helpdesk troubleshooting tools will be unavailable
  • Various back-end scripts are failing, and may result in strange behaviour from the bandwidth monitoring systems (the ResNet Gadget/RSS feeds won’t be updating) , the network status monitoring, DHCP updates will not happen etc.
Because ResNet don’t run the database server, it’s a little out of our hands at the moment.  The issue has been reported to the database team, and we will update this page as and when we have more information.
Update: 2011-12-15 09:35
The issue seems to be resolved (for now) and our systems seem to be working again.  However, reports from the database team suggest an underlying problem which they are continuing to investigate.

About 25 of the servers which run ResNet are running *really* slowly at the moment (for various complicate and convoluted reasons) this means that customers may notice the following:

  • The ResNet DNS servers are slow and unresonsive, which effectively means that your connection may function slowly, or even not at all.
  • The My ResNet control panel (my.resnet.bris.ac.uk) is slow and unresponsive
  • The ResNet Activation System (go.resnet.bris.ac.uk) is slow and unresponsive
  • Most of  our backend troubleshooting tools are playing up as well, which makes troubleshooting individual problems difficult for the Service Desk
  • The Wireless ResNet authentication servers are caught up in this as well, so ResNet Wireless may also be unusable

Because of the widespread nature of the problem, we’re expecting it to affect all ResNet customers (Wired and Wireless)

We’re aware of the problem, and are doing everything we can to restore service as quickly as possible.  I’ve already started planning ways to stop the same issue hitting us again in the future.

Update: 21/11/2011 17:13
Good news and bad news.  The good news is that the load issues are beginning to resolve themselves and DNS seems to be back to normal.  The bad news is that somewhere in the last 2 hours we hit a (separate, but related) problem which means none of our web servers are currently running (so www.resnet/my.resnet/go.resnet etc are still dead)

We’ve got a lead on that problem though, and are working to get them back up again.

Update: 21/11/2011 17:50
And we’re back in business!   Sorry for the inconvenience everyone.

Because of essential maintenance to one of the university database servers, the following services will be unavailable between 8:30am and 12:30am, Wednesday 2nd March.

The helpdesk connection troubleshooting tools will also be unavailable, so there will be a diminished support capacity for ResNet until about 1pm.

For more information about other services which will be affected by this maintenance work, see the IS website: http://www.bristol.ac.uk/is/news/2011/diversp17feb.html

It seems that one of our servers which provides My ResNet, the in-room registration system  and DNS for half of ResNet fell over on Friday evening.  Some people will have experienced slow connections as their connections failed over to the secondary DNS server.

As the office isn’t staffed at weekends, I only happened to notice it by chance.  I’ve restarted the server and normal service was restored at around 14:45 on Sunday.

Sorry for the inconvenience caused.

Information Services are carrying out some essential systems maintenance on the 11th August 2009, 10:00-10:30 which affects the following ResNet systems:

  • My ResNet
  • ResNet in-room Registration System
  • ResNet Helpdesk troubleshooting tools (including those used by the out of hours helpdesk)
  • ResNet Network Monitoring system (the traffic lights on the ResNet home page)

All fully registered ResNet connections will continue to function normally throughout this maintenance period, so most people won’t notice anything.

There are several other University systems which are also affected by this maintenance, for more details see the IS News website: http://www.bris.ac.uk/is/news/2009/koi28july.html

« Older entries