From the archive – first published 19/05/2017
A consideration of the NHS ‘cyberattack’ concluding with a simple 4-point plan to avoid a similar event in your organisation
So, it turns out the NHS were using outdated software, even systems running on Windows XP, and, surprise, surprise, got ‘hacked’. Apparently they weren’t even the intended target for the malware, which was opportunistically looking for vulnerable computers. But, everyone who doesn’t take keeping their systems up-to-date is just waiting for this kind of event to happen to them. I have seen loads of organisations with terrible security habits. A major culprit being IT teams who batten down some hatches, often to meet ISO27001 type criteria, whilst leaving others wide open.
As we have seen, making it difficult to upgrade is one of the big problems. I bet, even now, you can think of software you rely on which is out of date, or has out of date dependencies. Often the costs of doing the required upgrade are prohibitive. Most organisations I see, like the NHS, have some systems that are so out of date they basically need complete re-writes.
There is no magic bullet to completely protect you from this. If everyone moved to Mac, or Linux, there would still be vulnerabilities and the need to keep patching. Exploits come out all the time for every conceivable system, the only thing preventing them making the headlines is their obscurity. However, there are things you can do to mitigate the risk.
As mentioned, organisations tend to bolt down computer access rights so only ‘experts’ can install software, the theory being the expert will not install malware and the system will be safe. Fine in theory, so long as the malware cannot install itself without human involvement, but there are other dangerous vectors, and Upgrade Lag is one of them. Some cases of Upgrade Lag are inevitable with sysadmins being, as they should be, reluctant to install an update until it has had exhaustive testing and they are happy with its reputation, but it’s a balance of risks here and time for testing is not the main cause of the serious incidents of lag.
Preventing staff in departments from having permissions on their own devices has negative consequences other than making it hard to upgrade, not least that it’s incredibly frustrating for staff trying to get things done. Software they take for granted on their home computers are years away at work. I would like to see a change to this, maybe give more staff more permissions, recognising the latent IT skills in staff even though they don’t have a job as an ‘IT Expert’. Amongst the usual array of technophobes, every department has a whole bunch of people who know a great deal about computers. Their abilities could be recognised and they could be given more permissions, not least so they can keep things up to date. One of the usual barriers to system administrators allowing staff permissions to install programs, is the fear that they will start installing all sorts of weird and wonderful software. The answer to that is to give sysadmins the software to monitor what programs are installed on machines across their network. There is a double win here.
It is worth pointing out the effect of organisations focussing their efforts on complying with targets for accreditations such as ISO27001 rather than keeping their eyes fixed on the actual reason for accreditation which is to make sure they are secure. Never let achieving the target become the only goal. This is a whole area of discussion in its own right.
Big organisations are better now at avoiding Upgrade Lag and have recognised the need to maintain some semblance of being current but they’re still woefully bad at it. Many have just about moved up to Windows 7 though I see applications requiring Internet Explorer 8 everywhere. Amazing. Others have procedures which could never keep pace with the speed of infection, especially from a targeted attack with intent to harm, unlike WannaCry which just got lucky.
The mindset underlying this is understandable. If a system has been working fine for years on Windows XP, why spend money to change it? If it ain’t broke don’t fix it. The trouble is the technology moves so fast that by the time the upgrade becomes an absolute necessity, like when Windows XP was no longer supported for security fixes, the cost of the upgrade is prohibitively expensive and possibly impossible. The expertise is hard to find, nobody is familiar with it, and the concepts have changed. A complete re-write is almost certainly the cheapest way forwards, but there are always other priorities and ‘security by obscurity’ seems the better option. Hoping for the best, a bit like not doing back-ups.
One of the keys to a successful upgrade process is to make it simple and one of the best ways of doing this is to put all your systems on the web. Then there is only one mission critical version of the application which can be backed-up and archived, or fired up on another machine within seconds. And this does not create a dangerous single point of failure, as you might suspect. All database systems rely upon a single point of truth and a web-based system ensures the single point of truth is easy to administer and very easy to backup. We run several servers in parallel, so, in the unlikely event that one server goes down you can simply type another URL into your browser and carry on where you left off. Far better than relying on someone’s legacy spreadsheet in an obscure shared directory somewhere, depending on ‘who knows what’ other software.
On a web-based system, if your personal device or computer is corrupted you can simply move to another one and carry on. The data is secure on a carefully administered computer which has security layers built-in from the ground up. Patches are applied appropriately and records are set to be delivered only by encrypted channels. Going web-based opens up more opportunities for Bring Your Own Device strategies where upgrade overhead and hardware management is substantially reduced. Organisations can then concentrate on devices like printers and the Internet connection, perhaps with virtual PCs for specific software. Virtually all my day-to-day applications are web-based. I use Word online, which is fine for basic writing, Excel online, Gmail, Google Calendar etc. There are even advantages to keeping my music player web-based and online. Using Chrome’s application mode, they all operate like installed applications with no upgrades to consider apart from Chrome itself
There are however some applications that you must install and you need to manage the upgrades in a timely fashion. You also need to deal with the Upgrade Lag which is certainly already in your organisation. People will always end up introducing Upgrade Lag and it’s important to make the time to prioritise its removal. We manage this by having Jubilee Sprints. Once a year we down tools from client work and concentrate on issues such as Upgrade Lag (also User Interface Lag and Technical Debt amongst other things). During the Jubilee Sprint we review all our systems, identifying the highest risks and the quickest wins, then we commit the time to getting them sorted. Afterwards we review how the lag was introduced and implement measures to avoid it in future. The aim is for all systems to be upgraded as soon as a suitable upgrade is available, and certainly nothing should ever rely on unsupported antiques, like Windows XP.
In conclusion, I would like to suggest a simple, four-point plan to dodge the kinds of threats that hit the NHS and others.
1. Identify the departments’ ‘ITers’
Allow your departments to identify IT experts. Give them qualifications, enhanced permissions on all the department’s computers and maybe a pay rise. The same principles as most organisations operate with First Aiders. They can make sure all computers and dependent applications are up-to-date where possible, or identify them for the IT department. They can also allow other staff to install and use everyday tools. The current restrictions on this are a huge frustration and a common complaint amongst keen and creative staff.
2. Device Independence
All software systems should be configured where if one device is broken another device can instantly be powered up and work can continue. Any indispensable devices should be identified and a strategy put in place to remove that dependency, probably by moving its functionality to a web-based system.
3. Jubilee Sprints
Schedule time, perhaps once a year, to identify the biggest Upgrade Lag risks and put in place a strategy to reduce them taking particular note of the quick wins (i.e. software that can be upgraded quickly).
4. Avoid Upgrade Lag in the first place
Develop a general strategy to avoid putting in systems where you must manage the upgrade process in the first place. Try looking to web-based solutions. Have a rigorous approach to avoid systems that demand upgrade expertise and have a procedure to install upgrades, however minor, as soon as possible. Avoid the big jumps through multiple major versions, say from ACME version 2.x to 5.x (what a nightmare), or Windows XP to Windows 10. That’s when things get really tricky, and expensive.I hope this is useful. Do let us know if you think I’ve missed out any big wins or got something wrong. Or even if it helps.