February 2009 - Volume 8 Issue 1
- Ambikesh Jayal & Martin Shepperd (Brunel University). An Improved Method for Label Matching in E-Assessment of Diagrams
- Ray Stoneham (University of Greenwich). Coursework Uploads and Zero Tolerance Deadlines
- Gillian Ragsdell (Loughborough University). Managing Knowledge about Knowledge Management: 'Practising What We Teach'
- S J Rees & Jing Lu (Southampton Solent University). Innovation and Emplyabililty in Knowledge Management Curriculum Design
- Paula Thomas (University of Glamorgan), Theo Tryfonas (University of Bristol) & Iain Sutherland (University of Glamorgan). An Analysis of the Curriculum Components of Computer Forensics Undergraduate Courses in the United Kingdom
- Lisa Payne (Coventry University). Using a Wiki to Support Sustainability Literacy.
- Andy Burn (Durham University). Thematic Analysis of Group Software Project Change Logs: An Expanded Study
- Mike Joy (University of Warwick). Undergraduate Computing Projects - an Investigation into the Student Experience
Editorial by Alan Poulter
University of Strathclyde
There is a fundamental irony in writing Editorials for Italics. On one hand, to write an Editorial involves reading and absorbing extremely useful papers on making teaching and learning both more effective and less onerous, while on the other hand, actual ‘teaching and learning’ (teaching preparation, marking etc) always seem to conspire against one undertaking the very reading activity on which Editorials depend.
The feeling of struggling against clashing deadlines inevitably drew my attention to the paper by Stoneham entitled “Coursework uploads and zero tolerance deadlines”. We all have the problem of how to enforce strict deadlines for submission of student work. However, things always conspire to reduce the effectiveness of uploading systems. Networks and servers can go down. The files submitted can easily bloat in size as students add more multimedia gee-gaws. Students may want to gamble that late submission penalties are too small to offset the higher marks that extra time might give. At the University of Greenwich Stoneham found that students who submitted work at the last possible moment gained lower grades on average. Concentrating on submitting work at day’s end also had a negative impact on their learning as students neglected other learning opportunities on days when coursework was due.
Not only do students have difficulties meeting deadlines. Stoneham tried to use FOI (Freedom of Information) requests to get statistics on the implementation of deadline policies in eleven major UK universities. Six responded that this data was not held centrally and that the cost of providing the data would far exceed the limits set for FOI requests while the other four did not respond at all! Informal enquiries revealed most had a zero-tolerance lateness policy in place.
Stoneham’s conclusions are thought provoking. A move to a 3am deadline helped spread the load of submissions, presumably as even the hard core last-minute submitters need their rest before 3 am. There was still a peak at midnight, but few students missed the 3 am deadline. Moving deadlines to weekends spread the load even more as students presumably completed early rather than risk social exclusion from their night life crowd. This also meant that no teaching sessions were lost, as they would have been with week night submissions.
Another paper on a classic teaching and learning issue, this time on Final Year Projects, is from Joy. “Undergraduate Computing Projects - an Investigation into the Student Experience” reports on the results of using a general questionnaire and a small number of semi-structured interviews to analyse the student experience of Final Year projects. Undergraduate Computing degree programmes normally contain a substantial individual project which a student undertakes in their ﬁnal year of study, as inclusion of a project is a prerequisite to programme accreditation by the British Computer Society (“BCS”). A project will typically take the form of a software engineering exercise, where the student will work on an item of software, going through the various stages of the software life cycle (specification, design, implementation, and, ﬁnally, testing), writing a technical report describing the software and the process, and outlining their achievements.
It is agreed that Project work is highly valued, since it supports students engaging in realistic activities which reinforce their understanding of the discipline, and draws on skills acquired in different modules throughout their degree. It brings together the supervisor and the student in regular one to one sessions, the only time on a course this happens. There is a broad consensus about the amount of work involved in an individual project (roughly 300 hours) and that its assessment weighting counts for a large majority of the Final Year assessment credits.
The study obtained data confirming, to a certain extent, these assumptions. The mean time spent was 369 hours, but some students claimed that they had spent around 1000 hours, the equivalent of 20 weeks full time working a 50 hour week! A significant number of students claimed 10% or more of their time on meetings with their supervisor. However, the data was not consistent: for example, of the 13 students who claimed to have spent 0% of their time in meetings, six also claimed to have spent at least 5 hours face-to-face with their supervisor. Of the five students who claimed to have spent 20% of their time in meetings, two spent 8 hours or less in face-to-face meetings. One interesting result Joy does not investigate is the large amount of online searching being performed, suggesting that it has now become a major source of information for students. In my experience, very few Computing students have been trained in this activity.
Two papers which look at the same topic from different perspectives are “Using a Wiki to support sustainability literacy” by Payne and “Thematic Analysis of Group Software Project Change Logs: An Expanded Study” by Burn. Payne employed a wiki to get groups of students working online together for an assignment on compiling resources linked to sustainability in computing. This is an up and coming area as there is evidence of IT employers wanting to recruit people with sustainability skills. In a Computer Weekly survey of ‘top IT professionals’ all respondents agreed that it was important for the IT industry to be environmentally friendly. A wiki was selected which provided private space for each group’s work; posts were identified and there were change tracking facilities. It was considered to be important that each individual’s work could be identified if necessary. It was felt to be particularly important that groups could not see each other’s work so that they created their submissions independently. The assessment used a scenario which gave an outline of a fictitious, small IT company which was seeking to adopt more sustainable practices. It was successful in getting students to collaborate and also learn about sustainability.
Burn carried out a “thematic analysis” on the change-logs of each of 26 student projects, creating a breakdown of activities over the life of each project; in total, over 4,200 log entries were categorised into one of six activity types. The analysis was used to explore the collaboration and behaviour of students in group work and to investigate the use of thematic analysis in assessing the collaboration aspect of group projects. This analysis was compared to public change logs available for open source projects to provide a real-world comparison with the student projects. Overall, the desirable activity types (‘administrative’, ‘corrective’ and ‘developmental’) were more prominent and evenly spread in open source projects than in the student projects, whereas the open source projects also had a significantly lower proportion of undesirable activity types (‘ambiguous’ and ‘misc’). Like Payne, Burns found that analysis of online activity, in this case project change-logs, were valuable when monitoring a group project. Perhaps more intensive use of wikis and other collaborative tools should be made in all courses to inculcate students in their effective use?
A related use of tagging concepts is “An improved method for label matching in e-assessment of diagrams” by Jayal and Shepperd. They propose a series of stages to tag content terms used in diagrams (e.g. in a UML exercise) to help match a student submission against a canonical answer and use the match as a marking tool for the former. This is a clever idea and needs to move up from syntactic concept matching to semantic matching. While I do not teach using UML, this approach could be re-engineered to work for taxonomy or thesaurus construction.
Two papers took novel perspectives on knowledge management (KM). In “Managing knowledge about knowledge management: ‘practising what we teach’“ by Ragsdell, she argues that KM can be used to underpin teaching activities. The delivery of her module on KM demonstrated principles offered by Nonaka et al’s (2000) Socialization-Externalization-Combination-Internalization (SECI) process of knowledge creation (KC). More generally Ragsdell suggests that, even if a module is not KM-based, following the basic principles that support KC will lead to an improved learning experience for students and lecturer alike. This module was evaluated only using University standard student feedback forms: why not use KM evaluation techniques as well?
If KM can be applied almost as a teaching and learning strategy in itself, in “Innovation and employability in KM curriculum design” Rees and Lu go further and show how an entire curriculum can be built around KM. Their new KM Honours Programme builds on existing strengths in activity-based learning and teaching while addressing explicitly the requirements of industry. The unique element is the “activity set”, derived from extensive analysis and discussion with the commercial sector, where technology or business-oriented activities are designed to focus on different aspects of knowledge management. These activities drive the course, not the content in specific modules. Each activity is reviewed by a current industry professional as to their perception of its suitability for employer needs. Two example activities are “Conduct an audit on a major networked information system” and “Present one set of information perfectly”. The latter could be fulfilled, for example, by demonstrating the relationship between the activity tasks and employer skills requirements as presented by TFPL.
Finally, “An analysis of the curriculum components of Computer Forensics undergraduate courses in the UK” by Thomas, Sutherland and Tryfonas reports on work building on a seminar (which I attended) where delegates were asked to contribute to an undergraduate curriculum design exercise, in this newly-forming area. Module titles, culled from web course curricula, were allocated by delegates to years of study in a ‘perfect’ computer forensics course. I can personally testify that while there was considerable agreement of what content should go where, there were serious issues with naming modules so as to unambiguously show their content, and this comes out in this excellent paper.
- Access, Delivery, Performance: the future of libraries without walls by Jillian Griffiths & Jenny Craven (editors)
- Advance Rails Recipes by Mike Clark
- Understanding IPv6 by Joseph Davies
The following titles may also be of interest:
- Online Assessment and Measurement by Scott Howell & Mary Hricko
- Online Assessment, Measurement and Evaluation: Emerging Practices by David D. Williams, Scott L. Howell, & Mary Hricko