Email has become an essential way for football clubs to keep in contact with their fanbase; but defeating bad email in the form of spam presents a formidable challenge to their IT departments. Everton FC’s head of ICT Steve Fell explains to E&T how ‘The Toffees’ beat the problem.
Everton FC's head of ICT Steve Fell tells Keri Allan how the Merseysiders used Webroot’s SaaS Email Security Service to deal with a welter of spam and malware that threatened to overwhelm its defence.
Defeating bad email in the form of spam presents a formidable challenge to Everton FC’s IT department.
Engineering & Technology: How bad was the email spam problem for Everton FC, and when did it decide something had to be done to control it?
Fell: The reason behind the project was simply the amount of emails we were getting. Obviously Everton as a brand is well-known - an ‘easy target’, so to speak - so we had to do something about it. We had something in place, but to be honest, it was too time-consuming. On a daily basis, members of staff had to clear emails that were legitimate, and that was a burden for us. It was almost becoming a full-time job just to administrate the system we already had in place.
E&T: You say ‘time consuming’. How much time are we talking about here?
Fell: We were spending two-to-three hours a day clearing out emails: an hour in the morning, an hour at lunchtime, and an hour late in the afternoon. Of course, in-between those times we would have people ringing up saying that they haven’t received specific emails too.
E&T: So the old solution was hardware-based?
Fell: We had a single low-cost server held at one of our suppliers’ sites. [The solution] wasn’t spanned across multiple servers with load balance, etc. If we did want to spread it across a lot of servers we would have had to purchase them, and pay our supplier to host them, so the cost would have been phenomenal.
E&T: Are the benefits of the software-as-a-service (SaaS) solution now becoming obvious?
Fell: We now very rarely get a phone call from one of our clients or customers saying that they have been sent an email and it has not come through. We no longer have to spend time going through lists of what the system has wrongly perceived as spam and clear them out. Before implementing Webroot we were using our exchange server to route all our emails using DNS, but now Webroot does all our DNS queuing, and that has taken a big burden off it. Plus we have got a strange internal IP addressing which is public, not private; so sometimes it would get confused, and think an email going externally was going internally. And of course the other benefits include [the fact] that we’ve got no [new] hardware costs.
E&T: What about adding the additional level of security?
Fell: One of the features that we implemented is the searching for words in emails, both inbound and outbound.
E&T: What words?
Fell: Things like profanity, or even credit card details. Because of PCI-DSS [security standards] compliance, we are not allowed to take credit card details by email and so on inbound emails we can have Webroot classify them, and then put them aside; so we are not storing them internally, and [leaving] credit card details open to abuse.
E&T: So the Everton FC IT team suddenly has two or three hours a day freed-up. What are you able to focus on now that extra time is available?
Fell: We have been able to better monitor more important parts of the infrastructure. We’ve been able to focus on other projects, giving a better service, looking into technologies and into IT options; generally keeping an eye on other parts of our system security-wise, and making sure it is running smoothly.
E&T: Having time to reappraise the IT systems and how they run is usually likely to lead to proposals for upgrades, improvements, new projects. Then the IT department realises that there’s no budget available to embark upon these good ideas.
Fell: To be honest, we’ve looked to start loads of projects, but money’s the thing. I suspect there are probably a few people out there that think there’s a lot of money in football clubs; but I think most of the money goes on players, and [IT is] left- in most cases - to either do with what we’ve got, or improve upon it.
E&T: So where does this leave you?
Fell: We have done some projects. I started here four years ago, and since then we have migrated to active directory: from Exchange 5.5 to Exchange 2003. OK, it’s a bit old now; but to us there’s a massive difference.
E&T: How were you doing it previously?
Fell: We were doing tape back-ups, and we are now doing off-site backups. One of the recent projects was replicating our ticketing platform. Obviously, selling tickets is most important to us, and we were relying on one server to do that. If it went down then trade basically stopped. So we have just implemented a server based away from the ground, which replicates in real-time the ticketing server here. It’s a lot of worry taken away.
E&T: Any other projects to mention?
Fell: Another project was the build of the state of the art training ground, Finch Farm. We have used mostly Apple equipment there for video analysis, and put in wireless, which was then extended to Goodison Park. To a lot of people this is ‘old technology’, but for us it’s pretty new stuff. We upgraded the storage [at Finch Farm] - we’ve got about 18 Terabytes of storage now.
Also, we have been talking to Apple about doing podcasts of our players. They can plug in their iPod, log-in to iTunes, and download their own videos to analyse themselves away from the ground.
E&T: So do you take the secure route; leave new technologies until you can see how others have fared with them?
Fell: Absolutely; but it is difficult with the mix of departments. Take our Media and Communications department, for example: they want to jump ahead. You can see where they’re coming from - they are producing a lot of the media stuff on the Web for the fans, they have got to try these things out. I’m surprised no-one has contacted me yet to ask about trying out Windows 7. Generally, I’ll say leave it for 6-10 months...
E&T: So in that sort of situation, how do you get the balance right? Have you set guidelines - or do you look at each request individually?
Fell: The first thing I do is say ‘no’. I let people go and try to get a business case for it.
E&T: Does the IT workload ebb and flow with the football season?
Fell: I’ve heard people say ‘God, you must be quiet when the season finishes’, but it’s quite the opposite, as we are preparing for the next season. The end of the season is the start of the new season for us.
E&T: Presumably there are specific IT issues that come from working in a football club environment; what are they?
Fell: The main one is that, because Everton FC is a well-known brand, I’m very conscious of security. People trying to hack our network; or like the last project, reducing spam. Bear in mind at Finch Farm we’ve got an Academy, so we’ve got classrooms there with Apple hardware and PCs. That means that, from my point of view, security Web filtering is important - locking-down PCs, etc...
We don’t have IT people based at Finch Farm but we’ve had to up the amount of time we spend there. This is because the actual ‘football’ side of things has begun to embrace the technology, which is good. It’s the same old thing with IT.
E&T: How many people comprise the Everton FC IT team?
Fell: [The structure is:] I’m the head of ICT, and then I have someone who takes on first line support. He takes a lot of the calls and tries to solve the problems at point of call.
I then have two colleagues who are system analysts, who I class as second- and third-line support. These two guys aren’t often at their desks; they’re either out and about on the grounds, in the offices across the road or at the training ground.
E&T: What about your role as head of ICT? Do you get to be ‘hands-on’?
Fell: There are a lot of meetings, but I like being hands-on, so I try and do both. I sometimes get a bit of stick off the lads for poking my nose in.
Unless it’s an absolute emergency I’ll always hold back and the let the guys try and sort it out themselves… but there comes a time when you say ‘enough’s enough, I’m spending too much time on this, and I need advice off someone’; and they’ll come to me. We’ll go externally if need be, but generally we try to sort out everything ourselves...
E&T: The dilemma for a lot of senior IT professionals is that they find themselves drawn more into ‘pure’ management and strategic endeavour, and have less input into technological strategy.
Fell: There are a lot of meetings, preparation of projects and writing our policies, for me... I think that I have 15 policies to re-write at the moment: policies for Web use, for email use, for mobile devices, and so on...
We also administrate the club’s telephone system here, as well. We all enjoy it though and like being busy, so we try to get involved in everything. If anyone wants a plug putting on the kettle I suppose we’d get a call for that too! The departments here are diverse so by working with them you get to learn other things. It helps us to understand their requirements for IT.
E&T: IT leads the way - even if it is only just putting a plug on a kettle?
Fell: Gone are the days when if you worked in IT you were seen as a ‘guru’. I think IT has become just another service - which is quite sad actually, when you bear in mind how much businesses rely on the IT function.
E&T: But if you had to zero-in on aspects of being an IT professional, what would you say that you most enjoy about the job?
Fell: I suppose I’d have to say the challenges it brings - I just love all the new technologies that come out: the software, hardware, and the services.
I embrace it all...