Network problems

Posts in this category describe unexpected network outages, and what we’re doing to fix them.

We’ve been informed that some electrical testing will be carried out in January which will disrupt wired and wireless networking in the following locations:

Mon 4th Jan

  • The Hawthornes

Weds 6th Jan

  • 7 Priory Rd
  • Woodland Court
  • 5 Osborne Villas
  • 12 Osborne Villas
  • 16 Osborne Villas
  • 28 St Michaels Park
  • 32 St Michaels Park
  • The Hawthornes

We apologise for the lack of notice, and for the inevitable disruption that this will cause. Unfortunately we don’t yet have any details of how long the disruption will last.

At 12:00, Weds 17th April we lost all connectivity to the Stoke Bishop Halls.  We’re investigating, but currently the following services are not available:

  • ResNet Wired internet access
  • ResNet Wireless internet access
  • Eduroam
  • UOB telephones
  • Halls Admin connections

All residences at Stoke Bishop are offline, including:

  • Hiatt Baker
  • University Hall
  • Badock
  • Churchill
  • The Holmes
  • Durdham
  • Wills Hall
  • Grounds and Gardens Offices
  • Downs Workshop

We don’t have much information, but we’ve got a network engineer on his way to site to see what’s going on.  We’ll update this blog post as and when we get details.

Update Weds 17th, 12:44
It appears that all the fibre runs which connect Stoke Bishop halls back to the main campus have been cut through.  We’re currently trying to locate the position of the break, and will be liaising with our fibre contractor to get a fix put in place.  We don’t have any timescales for a fix at this stage, but will know more when we’ve been in touch with our contractors.  More updates to follow when we have them.

Update Weds 17th, 14:42
We’ve identified the location of the break (which is in the middle of the Hiatt Baker 2 building site) and our contractors are currently digging a bypass trench so they can splice in a repair to the fibre.   They will be laying the bypass duct much deeper than the existing duct, which was surprisingly close to the surface.  If all goes to plan, we’re hoping service can be restored by 7pm, but we’ll continue to update this post when more information becomes available.

We’ve just been sent this photo of the damage:
2013-04-17_small

 

Update Weds 17th, 17:58 (Mark, ResNet Manager)

Netwoking has been restored for all of Stoke Bishop.  Thank you to everyone who helped get this serious loss of service up and running again in a very short period of time – Fibre splicing is not easy!

And a welcome quote from Professor Robert Mayhew, Warden of Churchill Hall:

“Dear Mark,

Just to echo what you said on the Resnet blog, I am really impressed by the speed with which the Networking team managed to resolve this issue. The level of information for staff on the ground was also first class!”

See Update 11:16

This morning we lost the network link to Stoke Bishop at approximately 8:20. As a result, ResNet wired & wireless, eduroam and internal phones are not working.

The exact cause of the outage is not yet known, but our networks team is actively working to restore service.

We will post an update here when we have more information.

Update 09:46 by Mark, ResNet Manager – We are very confident this is a power outage in our comms room.  We are now waiting for Estates to visit and restore power to our equipment.

Update 11:16 by Mark, ResNet Manager – Power has been restored by the Estates Team and all networking has been restored to the Stoke Bishop area.

 

We lost connectivity to Northwell House at about 1pm today (Monday 4th Feb 2013)

We’ve managed to restore connectivity to the main building, but haven’t been able to get the Annex back up yet. So at the time of writing there is no ResNet Wireless, or ResNet Wired service in Northwell Annex.

We’re making arrangements to be on site as soon as we can tomorrow morning, and we’ll hopefully be able to bring it back to life then.

As a workaround, we suggest that any residents in the Annex use the common areas in the main building if you need internet access this evening.

Update: Connectivity to the Annex was restored shortly after 5pm Monday evening.

Well there’s been an awful lot of planning and preparation but we have finally launched ResNet Inclusive and perfectly on time too.  So from now, all UoB staff and students will be able to activate their ResNet connection as access is included in their accommodation rent; this includes any UoB person who’s just visiting.

Everyone will still need to activate their ResNet connection by visiting http://go.resnet.bristol.ac.uk via either the socket in their room or connecting to the ResNet-Setup wireless signal and follow the on-screen instructions.

The ResNet Inclusive connection is ResNet Standard but if they feel this is not enough for their needs then they can upgrade to ResNet Max for a small fee.  For details please see ResNet Packages.

What does this mean for ResNet customers?  Well, the registration process is much simpler because there’s no need to go through the on-line payment process which is great news for our international students that don’t always have Credit/Debit Cards.  This means more people won’t have to visit the Computer Centre to pay for ResNet.

I’d like to thank everyone that’s worked hard behind the scenes to make all this has happen.  The ResNet team have had to work closely with the Accommodation Office, Finance, the Secretaries Office and two external partners (Unite and A2Dominion) and without their help, understanding and cooperation this would never have happened.

Mark Elley
Residential & Mobile IT Manager

There will be a short break in service affecting all ResNet connections on Wednesday 27th June, between 9am and 9:30am while we make some changes to the network infrastructure.  Additionally the ResNet service should be considered at-risk until 10:30am.

The following services will be effected:

  • ResNet Wired – your computer will not be able to connect
  • ResNet Wireless – your computer will not be able to connect
  • My ResNet control panel will not be available
  • The ResNet Activation System will not be available

If you experience a problem with your ResNet connection during that period, please wait half an hour and then restart your computer before trying again.

We’re apologise for the short notice.  Updates on our progress will be added to this blog post when we have them.

Update: 2012-06-27 09:30
As far as we can tell, everything is back up again.  The interruption for the majority of users will only have been a couple of minutes, but if you’re having problems try restarting your computer.  If that doesn’t help, contact the IT Service Desk.

For updates, see the bottom of this post

We’ve had a large number of reports of customers experiencing problems with ResNet Wireless and Eduroam.  The symptoms being that it your computer asks for your username and password.  Some times it will accept them, but it often takes multiple attempts to get anywhere.

We’re actively investigating the issue, and we’ve been poring through mountains of log files trying to work out what’s going on.

Workarounds:
The ResNet Wired service is not affected by this problem at all, so if you plug your laptop into the wall you’ll have a nice fast stable connection that doesn’t repeatedly ask for your password.

For Eduroam, we don’t currently have a workaround other than “wait for 15 minutes then try again” as the problem appears to move to another location every so often and if you wait a bit then try again it may work better.

Update 2012-02-10 16:06:
We’ve managed to replicate the problem on a machine in the office, and we *seem* to have fixed the problem with that machine (it’s a bit early to tell, but I’ve connected/disconnected several times with no problems)

We’ve released a new version of the setup wizard with a couple of tweaks, which should help.  You can re-run the wizard by connecting to the “ResNet-Setup” wireless network and then going to http://go.resnet.bris.ac.uk wizard in your web browser.

Update 2012-02-13 10:50:
OK, so the new wizard only appears to have fixed things for my laptop in the office and not anyone elses.  This rules out client side certificate issues.  We’ve got a few more leads that we’re following up from deeper in the network infrastructure.  More news as we get it.

Update 2012-02-13 12:20:
The problem appears to be even more widespread than we thought, and is potentially affecting some eduroam clients as well.  We’re currently working closely with the core networking team to identify a course of action.

Update 2012-02-13: 14:00:
We’ve hit a point with this problem where the next step is to escalate it to Cisco (the vendor that makes our wireless networking kit)  So we’re in the process of raising a ticket with their support team.  Our best advice at this point is to use the ResNet Wired network instead where at all possible.

Update 2012-02-15:
Just a quick update to say we haven’t forgotten about this problem and we are still pursuing it.  We are currently waiting for Cisco to get back to us.

Update 2012-02-16:
I think it’s safe to say we’ve got enough detailed information to go on for now, so there’s no need to email the service desk.  We’ll update this blog post again when we have some news about a fix.

Update 2012-02-20:
We now think this is  a known bug in the firmware version our wireless controllers are running.  By all accounts it only surfaces if the controllers have been running for quite some time (as ours have) and a restart of the controllers should temporarily alleviate the problem.

Coincidentally there is a wireless maintenance slot scheduled for tomorrow morning, so we’re planning to restart things during that slot.  Upgrading the firmware is a larger job though and needs some planning.  There’s a meeting happening tomorrow which will determine when that upgrade can happen.  More news about that tomorrow.

Update 2012-02-21:
The upgrade to the new firmware has been scheduled for Tuesday 28th Feb and will be applied between 6am and 9am.  During this time all University wireless services will be unavailable.  See this news item for more details: http://www.bris.ac.uk/it-services/news/2012/wireless13feb.html

Update 2012-02-28:
We believe that the upgrades done this morning have fixed the problem.  If you continue to have problems with ResNet wireless, please contact the IT Service Desk and we’ll investigate.

Update: 13:20
And we’re back in business.  Service has been restored all ResNet accommodation.

History of this outage is shown below.

As of 12:30 today (19th Jan) just under a third of ResNet has lost internet connectivity.

Early investigations seem to imply that there is a power problem affecting Priory Road possibly extending as far as The Triangle.  This means we’ve lost power to the core networking equipment which feeds most of the centrally located accommodation – this has knocked out both Wired ResNet and Wireless ResNet services in the affected halls.

Hopefully, ResNet will just spring back to life when the power comes back on, but we will be keeping a close eye on it just incase.

Update: The following buildings are without service

  • Chantry Court
  • 33 Colston Street
  • Deans Court
  • The Hawthorns
  • Northwell House
  • Osborne Villas
  • 121 Redland Road
  • 29 St Michaels Park
  • 30 St Michaels Park
  • 31 St Michaels Park
  • 32 St Michaels Park
  • 33 St Michaels Park
  • Unite House
  • Winkworth House
  • 97 Woodland Road
  • Woodland Court
Update: 13:05
Power has been partially restored, and service should be available once again at Colston St, Chantry Court, Deans Court, Unite House, Winkworth House, 97 Woodland Road.
The following residences are still down
  • The Hawthorns
  • Northwell House
  • Osborne Villas
  • 121 Redland Road
  • 29 St Michaels Park
  • 30 St Michaels Park
  • 31 St Michaels Park
  • 32 St Michaels Park
  • 33 St Michaels Park
  • Woodland Court

We are experiencing a network issue that is affecting Deans Court North block.  All other Deans Court blocks are fully working.  We are looking into this problem and will update this post later on today with any further information.

We apologise for the inconvenience.

[Update 14:10 05/12/2011]

The connection to Deans Court North has been restored.

Just before 4pm today Hillside Woodside’s ResNet connection failed due to a power outage in the Burwalls comms room.  We are currently investigating this outage and hope to have this connection restored as soon as possible.  When we know the exact nature of the problem we’ll update this post.  Sorry for any inconvenience caused.

[Update 17:10 15/11/2011]

Our network team has arrived on site and have found that the fuse board that supplies our kit has tripped.  It’s now up to the electrical maintenance guys to reinstate power.  Once this is done ResNet connections should be back automatically.

[Update 20:21 15/11/2011]

It looks as though power has been restored, and Hillside Woodside has come back up.

« Older entries