My Way - IT at Assessment and Qualifications Alliance
Open Source came to the aid of schools and colleges exam board the Assessment and Qualifications Alliance - AQA - when it needed a secure extranet capable of delivering materials to 35,000 examiners and moderators at low cost.
Engineering & Technology: How many people are using the AQA extranet now, and what for?
Peter Morris: The extranet now has in excess of 22,000 active, registered users. Apart from a handful of internal users, these are all individual examiners and moderators engaged by AQA to assess the work produced by students seeking AQA qualifications.
E&T: Who are the examiners? And what do they use the extranet for?
Morris: They are almost exclusively active or retired teachers with substantial practical experience of teaching the subjects that they assess for us. They use the extranet primarily as a source and repository for the various kinds of material that they need to fulfil their roles, including question papers, marking schemes (documents that explain how to mark the papers, to encourage consistency), a plethora of guidance notes and training material, and several administrative forms.
E&T: What other applications does it provide?
Morris: More recently, we have introduced a messaging capability, and an online forms-based workflow application that supports the feedback process around examiner performance. We have a system of checks and balances that allow us to monitor and standardise the performance of individuals in those roles, again to promote consistency, and this workflow application enables that process to take place online.
E&T: Presumably secure emailing is a prerequisite feature?
Morris: We have added messaging and emailing capabilities, the facility to handle exam papers and mark schemes, and a certain amount of training material - though not yet including the streaming video formats that we intend to add eventually.
A user forum/bulletin board is one of several ideas that we have for future extension of the existing extranet, which will be reviewed and prioritised at the start of the next phase of enhancements.
E&T: How is the extranet supported?
Morris: The development team reached about a dozen at its peak, including business analysts, process analysts, systems architects, developers and testers, plus part-time involvement from the business users. The production support team numbers about 30 across all three principal AQA sites, but that team supports all production systems - not just the extranet.
E&T: How about your suppliers? Do they get involved with support?
Morris: We have support and maintenance agreements with the suppliers of the principal software components of the extranet technology stack: Red Hat for the JBoss Portal and others; Alfresco for the document and content management system (CMS); and EnterpriseDB for the underlying RDBMS. No external people are permanently assigned to support.
E&T: What was the size of the implementation team? And how did you gather the necessary skills?
Justin Codd: We engaged the services of a third party - Optaros - with experience in systems development utilising open-source-based technologies. Optaros consultants worked side-by-side with the in-house development teams, providing guidance and support during the development. As the project progressed the dependence on the Optaros staff reduced until in-house staff were conversant enough with the relevant technologies to take the lead in the ongoing development.
E&T: So what were you looking at in terms of support-team headcount?
Codd: In terms of the overall size of the support team responsible for the implementation the numbers were somewhat fluid. At its peak the team consisted of around 15 individuals with the numbers dropping back to around eight following the post-delivery departure of the Optaros team members.
E&T: One of the arguments in favour of adopting open-source platforms is cost savings. Did this apply in AQA's case? Or were cost savings more to do with the extranet deployment itself?
Morris: The first live deployment was immediately before Christmas 2008, when we implemented a pilot for just 200 examiners to support the January 2009 series of exams, so there were negligible financial benefits from that phase. We added a further 200 examiners and moderators in time for the March series, but it was not until April and May when we deployed to the current population of over 21,000 in time for the Summer 2009 series that the cost savings began to be significant.
E&T: Can you be more specific in terms of a figure?
Morris: Based on that series alone, we have saved fractionally less than £100,000 from the use of the extranet; primarily in the costs of printing and dispatching the materials that we used to send out by post and courier but are now available for download online.
E&T: Can one of you say something about the proprietary architecture that AQA migrated from?
Morris: We have not so much migrated away from our previous systems and architecture as added a suite of options around the open-source technologies. We still have large numbers of legacy systems and applications based on the original architecture, with no immediate plans to remove or reduce our commitment to it. However, the new capabilities will provide us with additional options for future development, particularly of outward-facing systems.
E&T: Can you reveal more about the identity of those legacy systems?
Morris: Because we have no wish to jeopardise our on-going relationships with the suppliers of our legacy systems, we are not prepared to identify them publicly for fear of implying any criticism of them.
E&T: Okay, that's understood. Let's turn to the question of convincing AQA management that open-source was the way to go. Was it hard to convince them that open-source platforms were suitable for mission critical applications?
Codd: I think it's fair to say that there was a certain degree of scepticism in some quarters when an open-source based solution was proposed. However, the in-depth due diligence and suitability assessments undertaken were typically more detailed and complete than those that had been undertaken for a number of previously implemented systems. It was the output of those assessments which helped to convince senior management that what was being suggested was both realistic and feasible.
E&T: What about the question of security? Did you have any particular concerns about the innate security and reliability of open-source software?
Morris: The principal concern was that the 'open-source movement' may not have reached an appropriate level of maturity and stability. As an end-user organisation, AQA relies upon the availability of support services from its network of suppliers: the concern was that the open-source market generally may not be sufficiently mature to offer the necessary levels of availability of these services.
E&T: Does AQA use open-source platforms in other areas of its IT infrastructure?
Codd: Yes, there are a number of open-source technologies actively being used within various different parts of the organisation, like Web content authoring and software development, for example. There are also a number of third-party appliance based systems deployed within the organisation (for example a popular brand of Web proxy platform) that 'under the covers' make heavy use of open-source technologies to deliver their services.
E&T: Had AQA used open-source prior to implementing the extranet service?
Codd: Open-source was deployed in the areas that would be regarded as more typically mainstream for such software - as server operating systems and Web servers - but not necessarily across other more general areas of [information] technology, such as databases and application servers.
E&T: Was there an inhibitor here, in terms of perception prejudicing reality?
Codd: One factor that helped to alter the perception of open-source within AQA was the realisation that a number of previously-deployed commercial software products had been found to make moderate-to-heavy use of open-source-based technologies. The realisation that other commercial organisations felt confident enough to sell products and services that - directly or indirectly - made use of open-source convinced us that maybe AQA should be examining whether such technologies could now offer more flexible and/or lower cost options for delivering systems and services.
E&T: What's the biggest ongoing challenge in the AQA IT infrastructure now?
Codd: 'Legacy' systems, a continually-growing software portfolio, data storage, and network/communications growth - all [these] generate their own sets of particular challenges to be overcome. In line with many other organisations perhaps one of the greater challenges typically faced is justifying and obtaining the required streams of funding for the ongoing maintenance and expansion of infrastructure, which the original implementation of is perhaps seen as a 'one-off' spend rather than part of a programme of continuous maintenance and evolution - which it frequently turns out to be.
E&T: But the investment is still there in one form or another?
Codd: I am not able to comment on specific details, [but] I can confirm that a substantial investment in core IT platform infrastructure is being made both now, and in the next few years.
E&T: Has contraction in the economy caused you budgetary restraints that have caused projects to be put on hold over the past 12 months?
Codd: While the economic situation has certainly caused AQA to reappraise a number of its IT-related goals and objectives it is typically 'business as usual' with an even tighter focus on obtaining the best possible value from current spending programmes. In addition, AQA has pressed ahead with the implementation of a number of key projects that will further enhance the capability of AQA to deliver both new and enhanced capabilities to the extensive portfolio of services already offered.
E&T: Finally, what about future plans? Does AQA intend to extend its use of open-source platforms?
Codd: Having now built and deployed an alternative infrastructure based on open-source technology we are keen to capitalise on it further wherever appropriate. There are a number of upcoming projects that could be good fits for the type of solutions infrastructure we have deployed for the extranet. We could expect a number of these new systems to make use of a number of the extranet related open-source technologies already deployed.