We are, of course, bored with course boards: an online alternative to formal course meetings

 

 

Jon Dron

University of Brighton
Watts Building, Moulsecoomb
BN2 4GJ Brighton
00 44 1273 642480

Jon.Dron@brighton.ac.uk

 

Judith Masthoff

University of Brighton
Watts Building, Moulsecoomb
BN2 4GJ Brighton
00 44 1273 642490

Judith.Masthoff@brighton.ac.uk

 

 

 

ABSTRACT

 

This paper discusses the implementation of an online course board, an attempt to conduct formal course meetings using simple web-based threaded bulletin board technologies. This technique seeks to address some of the problems of conventional meetings where individuals can dominate proceedings, most attendees waste much of the time being bored and there is a heavy investment of time and money for little direct benefit. Although it provides solutions to some of the problems of traditional face to face meetings, the online discussion mechanism suffers from a number of weaknesses which negate many of the potential advantages. The paper describes some procedural solutions to these weaknesses that have been implemented, and technical solutions that are being incorporated into a purpose-built environment for running course boards.

 

Keywords

 

Course Management, Supporting e-learning, Collaborative Filtering, Adaptive Hypermedia

 

1.     Context

 

Educational systems are more than just collections of teaching events. A full perspective has to incorporate the complete system of which teaching is just a part (Moore & Kearsley, 1996). The management of learning is a process that includes not just the interactions between students, teachers and learning resources, but the larger context which provides the framework for those interactions, including the feedback processes which stimulate growth and change. This paper considers one aspect of this process, that of formal course boards.

 

Course boards are convened to discuss the progress of courses, examination results, learning and teaching strategies, academic standards and many other important issues. Unfortunately they are inherently boring for many participants as much of the time spent at a given meeting may be on topics of little relevance to them. Meetings may be dominated by individuals with their own axes to grind and agendas will usually cover far more areas than are of interest to any one participant. Such meetings are also expensive, incurring costs such as staff time, rooms, catering, travel and administration. Our study explores the use of asynchronous discussion forums for the conduct of electronic course boards for a foundation degree in eSystems Design & Technology. We identify a number of issues that have arisen in this process and discuss some of the ways we have addressed and are continuing to address them.

 

 

1.1     An overview of the course

 

The course is currently run by two Further Education (FE) colleges, coordinated and managed by the University of Brighton. The colleges and University are spread over an area of around 50 miles with relatively poor road and rail links between them. It was therefore decided early on to use web-based systems to develop, run and deliver the course.

 

Course modules are developed and run by teams from the colleges, with quality control, management and a managed learning environment (MLE) supplied by the University. Until recently, this employed a custom-built Lotus Domino system, although it has been running for the past semester on a Blackboard environment, a move that was forced by the loss of support for the original platform. Day to day management is delegated to site leaders at each college. Although all teams involved in module development have met face to face at least once, most coordination of development work is carried out electronically, using the same virtual discussion space that is used to teach. This environment is also used to train teaching staff in the use of online discussion forums to help learners, using a course based on Gilly Salmon’s five stage model for building an online learning community (Salmon, 1998)

 

The eSystems degree is run solely in a part-time mode. All students have occupations that allow them to practice the subjects taught and all are mature, with an average age somewhere in the mid thirties. The course adopts a blended learning approach. Most delivery is online through the MLE, with face-to-face meetings taking place at variable and negotiable intervals. Each college manages the learning of its own students, averaging around seven students per college per year group.

 

Delivery is mainly through asynchronous threaded discussion forums. In the past this was supplemented by web pages, a web-based chatroom and various web-based administration tools, most of which are now replicated using the Blackboard MLE. Each college maintains a separate discussion area and announcements, but links between the colleges are maintained through staff contact, a shared “café” area, a chatroom and some announcements and messages which extend across all colleges. The virtual environment is thus the glue that holds the course together and enables shared development of resources as well as the students’ learning space.

 

The key underlying ethos of the course is that learning should be dialogue-led, rather than through the provision of structured learning resources. This dialogue is mainly accomplished through the discussion forums. The underlying pedagogical framework is structured along lines suggested by Moore’s theory of transactional distance (Moore & Kearsley, 1996), which predicts that the greater the structure of an educational transaction, the lesser the dialogue and vice versa. With limited resources and little time for development, the mainly static structured web pages are mostly unsophisticated and the strength of the course lies in dialogue between students and staff. All participants in the system are therefore very familiar with the use of the MLE.

 

 

2.     The Course Board

 

The University of Brighton requires that a formal course board is held at least once per semester. Teaching staff must attend such boards, as well as invited student representatives. These are useful events, encouraging reflection and resulting in positive change. However, the benefits are balanced by the problems affecting such meetings considered previously. Because of the wide geographical distribution of the course board members, as well as the fact that all its students are in full time work and are experienced in online discussion, it was decided to employ the same mechanisms used in the delivery and development of the course to handle its administration. The chat room would have been an obvious choice of technologies for this. However, although it would have helped to deal with the problems of geographical separation, it would not have solved problems of temporal separation, nor the aforementioned problems of boredom and irrelevance of different parts of the meeting agenda. The first author of this paper (course leader of the degree) therefore proposed the use of online asynchronous course boards to help resolve these problems. There were two overriding requirements:

 

  1. any solution had to be simple and quick to implement and use
  2. University of Brighton regulations for formal course boards had to be adhered to.

 

The first requirement was met by using the same technologies that were used for delivery of the course, with some minor modifications. There was little time to develop more sophisticated solutions and the team were confident that users would be able to transfer their existing skills. The second requirement was more problematic. After prolonged negotiation and discussion with administrators and academics with a lot of experience of conducting course boards, the following solution was reached:

 

 

The original version of the course board discussion area (figure 1) provided simple navigation support with which the students were already familiar, allowing the viewing of messages by thread, by date, by category (the agenda items) or by poster, along with a search facility. In addition to this, we provided a threaded view of each agenda item. To help give an overview of the discussion, threads were initially shown collapsed, and participants could either expand a single thread or choose to see all threads expanded for the agenda item, which is the view shown in figure 1. In the current Blackboard-hosted solution, there are fewer options to customise the discussion forum, but much of the original’s functionality can be replicated to some extent and it includes some welcome improvements in some areas such as announcements of new messages and the ability to easily collate all messages.

 

Figure 1. Screenshot of the course board

 

 

2.1     Positive Outcomes

 

The asynchronous course board has been run five times so far. There have been a number of positive results:

 

 

 

2.2     Patterns of Use

 

Figure 2 shows page accesses for the course board throughout the eight days of one of its instances. It shows a typical pattern of use: a slow start, a dip after a fairly busy weekend and a significant rise on the last day. The actual page impressions shown are inflated due to the design of the Lotus Domino teamroom, but we are only interested in the relative usage per day. Many participants browsed pages at the end of the week, probably to ensure that they had not missed anything, although the results are skewed by the fact that the greater the number of messages, the more page impressions are needed to read them all.

 

The relatively high level of attendance over the weekend (November 9-10) is symptomatic of the part-time nature of the course. This was also the most common period for people to contribute postings (figure 3). Presumably this was due to having more time to reflect on the issues and carefully compose replies, though there are also probable effects due to reaching a critical mass of messages and the stimulation of other messages causing a small positive feedback loop.

 

Despite more apparent visits shown in figure 2, the number of contributions on the final day (figure 3) was small and included several participants registering their attendance. This suggests that the discussion was largely played out by this time, indicating that the decision to limit the meeting to a single week was reasonable.

 

Figure 2. Page impressions per day

Figure 3. Messages posted per day

 

 

2.3     Issues Arising

 

2.3.1     Non-Linear Parallel Discussions

 

It was very hard to summarise the discussion to provide the necessary formal minutes. A threaded discussion is by its nature non-linear and parallel. In a traditional meeting it is relatively simple to document the discussion because agenda items occur one at a time. It is common that several threads of asynchronous discussion might span the entire length of the course board. Worse still, although threads provide some measure of organisation they are notoriously weak at capturing the other links between messages (Turoff, Hiltz, Bieber, Fjermestad, & Rana, 1999). This leads to comments like the following appearing in messages (our italics):

 

“These points got lost somewhere in the discussion (they featured as responses from Jon in an earlier thread from Fitz I think) I'll repeat them with my responses:”

“Most of the issues have been commented on elsewhere, I think.”

“Perhaps our weekly module introduction pages, which are a little too detailed and prevent us getting straight to the work (see thread in Web site feedback), could be consoidated into one overview of the module page”

 

Messages thus form more of a web than a hierarchy, but this is not captured in the structure of a traditional threaded discussion.

 

To make matters worse, it is common for a single message to deal with multiple issues, some of which may relate to different threads. If a single issue from such a message is discussed, it is not always clear that this is so from the title of the message. This often leads to an undifferentiated mass of message titles which may point to postings discussing quite different topics. After the experience of the first course board clear guidance was given in the introduction to the course board that separate points should be raised in separate messages, and for changes of thread to be signalled in new titles for postings. This was only partially successful, not just due to carelessness on the part of the participants, but as a natural consequence of engaging in such discussions. In any complex discussion, there will be branches, changes of tack and changes brought about by misunderstandings (Hewitt, 2001). The problem has been exacerbated by the move to Blackboard, which truncates message titles so that it is very difficult to know the subject of the message without actually reading it.

 

 

2.3.2     Time Consuming to Catch Up

 

Catching up with the discussion was often time consuming for those who did not contribute constantly. The key benefit of being able to cherry pick specific areas of interest was partially lost due to the need to read sufficient threads to ensure that contributions were genuinely new. This requirement also acts as a disincentive to irregular contributors to participate which, in less active discussions, can result in a negative feedback loop leading eventually to an inadequate critical mass and hence failed online communities (Preece, 2000, p. 91).

 

 

2.3.3     Fun

 

One of the site leaders commented:

 

“Whoever suggested that an online course board was less time-consuming than a face-to-face one was wrong - even taking into account travel time! It is more fun, though!”
|

This reflects a general feeling the participants have found it much more engaging than traditional meetings. As participants are able to choose when and where they make their responses there is little of the flagging interest and boredom that afflicts traditional face to face meetings.

 

 

2.3.4     Archival Nature of the Discussion

 

A perfect record of the discussion to refer back to is clearly useful. However, this prevents contributors from using the option to not have a comment minuted which is available to most traditional face to face meetings. KKnowing that messages will persist forever changes the nature of discussion and inhibits some kinds of posting, leading to more carefully considered comments.

Some time was spent deciding whether or not authors of messages should be allowed to delete them or make retrospective changes. We have disallowed deletion as this would destroy coherence in threads, but have left it to the good sense of contributors to avoid making radical changes while allowing correction of minor errors and categorisation mistakes.

 

 

2.3.5     Investment of Time

 

For the course leader acting as moderator and for the site leaders (whose contribution has been especially large as all are module leaders too) the amount of time taken up by the course board is greater than time spent in a traditional meeting, even taking travel into account. However, for most participants it is a far less onerous task than attendance at a site far from home, at a time which may be inconvenient to spend a lot of time listening to discussions of little relevance to them.

 

 

2.3.6     Reaching Consensus

 

Online discussions commonly tend to drift around a point without reaching a conclusion (Hewitt, 2001). Although this can be beneficial in allowing multiple perspectives to be fully explored, meetings often need to reach decisions. For questions requiring simple yes/no responses, the method we have employed for capturing votes which requires the posting of an undifferentiated message makes it difficult to easily identify conclusions of discussions. Also, it is not encouraging for those who might vote but for whom the process is possibly not worth the effort. To compound the problem, it relies on the course leader to identify that a potential conclusion has been reached, often drawing together threads from several different places or which have split along the way. Although this is an important e-moderating skill (Salmon, 2000), it is far from easy to do.

 

 

2.3.7     Late arrivals

 

The parallelism of a threaded discussion means that the most interesting points are often reached towards the end of a course board. Figure 2 shows that this is recognised by many participants and the danger is reduced by a concentration of visits towards the end of the discussion with relatively few new postings. However, the benefits of asynchronous attendance are slightly negated by the need to be there at the end if all points are to be fully considered.

 

 

2.3.8     Discussion hogging

 

Like traditional meetings, online discussions may be dominated by a few enthusiasts. Although others may choose to ignore their contributions if they are of no interest, some points will gain prominence because they have champions and there will always be someone willing to further the discussion (usually at least the moderator).

 

 

2.3.9     Misclassification

 

Most postings have followed the structure imposed by threads and the meeting agenda, but some have been misclassified when posted leading to confusion, especially when they spawned further threads of conversation. Although some of this may be ascribed to poor interface design, the fact that all users were very familiar with the design of the discussion forum that we were using suggests that it would in principle be hard to avoid the occasional human error. This is not an issue that would usually affect a face-to-face meeting, where temporal and verbal cues mean that any departure from the current agenda item is likely to be intentional.

 

 

2.3.10     Loss of tacit dimensions

 

In a traditional face to face meeting there are many non-verbal cues and recognition of shared meanings which do not necessarily have equivalents in an online dialogue. Although all participants at these meetings have been expert users of the medium by virtue of being participants in the course, it is recognised that it is a different kind of interaction and a different kind of socialisation that occurs in this environment. It is not the purpose of this paper to explore this in detail (see (Preece, 2000, pp.150-154) for an excellent discussion), but the flip side of the benefits of focus discussed earlier is that some of the community bonding generated by a face to face meeting may be lost. This is exacerbated by the fact that although local groups know each other, many of the participants have never met face to face. As Seely Brown and Duguid (2000) observe, "digital technologies are adept at maintaining comunities already formed. They are less good at making them."

 

 

2.3.11     Hidden discussions

 

There is some evidence that participants at each site discussed the issues outside the course board. This raises some intriguing issues. On the one hand, such discussions may be seen to enrich the process and to ensure that messages posted are the subject of deeper reflection than might otherwise be the case. On the other hand, these discussions are lost to the whole community, therefore forestalling the possibility of serendipitous discoveries and what Seely Brown and Duguid describe (quoting Rabindranath Tagore) as “stolen knowledge” – the knowledge that the sender does not intentionally impart (Seely Brown & Duguid, 2000).  This draws attention to the fact that a course board is about more than the exchange of information. Instead, it is a process of knowledge construction, a formalised process bringing about group learning.

 

 

2.4     Conclusions to this section

 

Although our approach to constructing an online course board has clear benefits, the problems raised above show that there are also drawbacks. Some of these will be dealt with through organisational changes, but work has begun on a longer term technical solution. We are developing a web-based application based on the D3E (Digital Document Discourse Environment) system (Sumner, 2000) that will loosely integrate with the Blackboard environment to provide a more tailored solution to the needs of the course board. Some of the new system’s features are pragmatic, others more adventurous.

 

 

3.     Pragmatic solutions

 

The decision to use the D3E environment was taken mainly because of its document-centric discourse. All D3E discussions are centred around documents, usually split into subsections for finer grained commentary. The key document that we wish to focus on will be the agenda of the meeting, with a discussion attached to each agenda item, but it will also be used to provide an opportunity to discuss the minutes of previous meetings and tabled papers. In the current system this process is long-winded, involving the attachment of documents or transferring them to HTML format. Inevitably, such documents raise a number of comments and message threads, contributing to the confusion of single messages containing multiple points described earlier. It is hoped that a document-centric approach will help to keep discussions more tightly focussed on the matters at hand without as much of the thread drift that has been observed in the simpler system. As the document is permanently visible, it will be harder to ignore the topic at hand

Our extensions to the D3E environment include a dedicated voting mechanism, whereby votes on decisions will be made with a single click. This will serve a number of purposes, notably:

 

 

A separate action-point module away from the main discussion is being built to improve clarity and simplify the process of generating the official minutes by including an automated report generation option, which will also show the results of votes taken. Blackboard’s ability to collate messages has already been noted as a benefit of moving from the older Lotus Domino system. The new system will extend this by giving the minute-taker (and other participants) the ability to collate messages in a variety of sequences, including by date, thread, sender and subject. Although making minutes from these will still be a manual process, it is hoped that this will simplify the task.

 

The new system will give the ability to add texture to messages and their subject lines by changing colours, fonts, emphasis and so on. Although this is possible to some extent using Blackboard, the implementation is buggy, hard to use (requiring the use of HTML) and easily broken. By adding texture to the text it is hoped to slightly compensate for the absence of tacit and implicit cues that help communication processes in face to face course boards.

 

One of the strengths of the D3E system which caused it to be chosen is that message titles are taggable with an extendable selection of icons to make it clear what kind of message they contain. For instance, icons can be available to indicate that a message is informational, in disagreement/agreement with the previous message, important, a proposed action, a problem, a proposed solution, an improvement suggestion and so on. Such representations help to break the tyranny of message hierarchies by providing a further level of metadata adding another, more fluid layer of structure to the dialogue.

 

The system will allow users to receive email notifications when their own messages are replied to, as well as optionally letting them receive notifications of further responses in the ongoing thread. It is hoped that this will prove more engaging and encourage more timely participation.

 

The system will be more accessible to those with disabilities than the existing Blackboard system which, though not fatally flawed, has many features that would make it hard for users with various disabilities to use it. For example, the new system will provide full message titles in threads rather than Blackboard’s truncated versions, use fully compliant XHTML, avoid embedded formatting tags. Importantly, it will use a flexible and user-configurable stylesheet, which will also help with the adaptive solutions outlined below.

 

Although it is hoped that these improvements will simplify the formal processes of the course board, increase its usability and make it easier to contribute, there are limits to the problems it will solve. Up to this point the solutions we have proposed are evolutionary rather than revolutionary. In the next section we will explore more radical solutions to the problems. In so doing we highlight some aspects of the process of running any formal course meeting that are problematic whether they occur online or face to face.

 

 

4.     Adaptive solutions

 

If users were able to more easily identify relevant and/or interesting messages it would make participation even simpler, less time-consuming and more rewarding, magnifying the benefits of online asynchronous course boards. In the current system, most participants feel they have to scan all messages to see if anything relevant is said and the role of the moderator is central to making sense of it all. To summarise our earlier comments, it is time consuming to catch up, and increases the investment of time by course leaders and site leaders. Despite anticipated improvements resulting from the new system, the non-linear parallel discussions and misclassifications make it harder to find (and not miss) messages of importance. Late arrivals mean that messages of importance can be posted late in the week, and users have to revisit the course board regularly to keep an eye out for interesting new messages.  As a result, some discussants take too little time, ‘attending’ the course board, contributing their opinions on something they feel strongly about, and then failing to check regularly whether others might have said something they want to react to. Although this might be helped by the email notification process, there are alternative approaches that may add further value.

 

Our central strategy is to automatically emphasise relevant and important messages, thereby simplifying the process of following and contributing to the discussion. This is a potentially hazardous approach. One of the strengths of face to face meetings is that they provide an opportunity for knowledge sharing, community forming and serendipitous exchanges of experience that can enrich the teaching and learning of all concerned. Often, patterns can be discerned only when the experiences of many are brought together. However, this can only be brought about with sufficient levels of participation, so a system that makes this too difficult will fall at the first post. By making participation more rewarding we hope to increase levels of involvement and reduce the need to commit large amounts of time.

 

If we wish to emphasise important messages it is necessary to identify those that are important. In the next section we explore how importance is identified in existing course boards, before discussing how these importance factors will be used in the final system.

 

 

4.1     DECIDING IMPORTANCE

 

We have identified three significant features of a message which might make it more or less worthwhile for an individual to read: novelty, the message author and the message content. Related to both author and content are the opinions of others, which are thus a kind of meta-feature. This may not be a comprehensive list but is considered sufficient to provide a basis for building an effective adaptive system. The decision to read any message is in reality a unique case, its importance determined by a rich tapestry of context, the reader’s interests, prior knowledge, time available and so on. In seeking patterns we are not underestimating the significance of this fact, but our purpose is to seek sufficient understanding of the discourse to build a system to support the kinds of discussion found in a course board. To verify our assumptions, we have analysed the most recent course board (running under Blackboard) in some detail. The management information provided by Blackboard is limited and fallible when compared with the earlier Lotus Domino system, so our analysis has been restricted to messages and replies rather than patterns of reading.

 

4.1.1     Significance of novelty

 

The two main features of a message that reflect its novelty are:

 

  1. whether it has already been read
  2. the time elapsed since the first opportunity arose to read it.
  3. the age of the message

 

To illustrate this, it is informative to observe the effects of this on the behaviour of one participant who visited the course board seven times:

 

 

This illustrates that the fact that a message is new seems to be important and therefore worth building into a system, but it is not the only factor in predicting whether a user will send a reply. It is interesting to note that even at the time of her fourth visit, the participant was still answering messages that she had read on her first visit. This pattern seems common.

 

The time elapsed since the message was posted and the time elapsed since the first opportunity to read it (the two may differ) seem an intuitive measure of novelty: the older the message, the less novel it is[1]. For example, in the use of email, people often postpone reading certain messages because of a perceived lack of importance given to the sender and/or subject line. Inevitably, an unread message from two days ago becomes less important than one received in the last minute and as time goes by the chance of such messages being replied to diminishes considerably. Whether this is a good thing or not is a moot point. It is probable that this behaviour is to some extent shaped by the design of mail clients that by default draw prominence to recent messages by date order alone rather than thread. The effect of this is similar to the effects of search engine position on click-throughs, where people are far more likely to follow links on the first page of results than the second (Dron, 2002). This is given credence by observing the participant described earlier whose behaviour does not show a linear dependence on message age. Instead, her replies were either to new messages or to those posted at the start. This may be seen to result from two main factors:

 

1                     The content/sender of the early messages (many of which were started by the course leader and were thus intended to start discussion)

2                     The fact that the first view of the system that the user receives is automatically a threaded view which draws attention to the starting messages before those that are more recently posted.

 

It would therefore be dangerous in this instance to base the design of our new system simply on the observed behaviour of individuals in the current system. The reasons for changing the environment is our belief that systems affect behaviour (Senge, 1993) which in turn means that observations of the existing uses of the course board are only going to be meaningful when carefully interpreted. In designing a new system it is therefore more important to consider the potential gains that would be achieved by emphasising messages in proportion to their age. Currently, we can see no benefit from doing so and a mechanism to support it remains unimplemented. However, the importance of newly read messages seems to be clearly demonstrated so unread messages will be highlighted (the default behaviour of the D3E system) and we are adding a mechanism to indicate on login whether new messages have arrived, which will help users to decide whether to proceed further.

 

 

4.1.2     Significance of the author

 

Importance and quality of information may often be related to the perceived authority of the author. To test this assumption we made the following hypotheses:

 

 

We have analysed how this maps onto what happened in our most recent course board. Thirteen people participated in the course board: one course leader (CL), two site leaders (SL1 and SL2), four students (SR1..SR4), and six members of teaching staff (S1..S6). We have counted how many messages each sent, how often they were replied to, and by whom.[2] Shading indicates the participant’s site.

 

 

Table 1 - Significance of author

 

Total number of messages Sent

Total number of replies to messages sent

Respondents

CL

SL1

SL2

S1

S2

S3

S4

S5

S6

SR1

SR2

SR3

SR4

CL

39

46

5

9

3

 

 

 

1

4

2

5

5

11

1

SL1

13

15

5

 

 

 

1

1

 

2

 

1

1

2

2

SL2

5

2

1

 

 

 

 

 

 

 

 

1

 

 

 

S1

0

 

 

 

 

 

 

 

 

 

 

 

 

 

 

S2

2

2

2

 

 

 

 

 

 

 

 

 

 

 

 

S3

2

1

 

 

 

 

 

 

 

 

 

1

 

 

 

S4

1

2

2

 

 

 

 

 

 

 

 

 

 

 

 

S5

10

9

3

 

1

 

1

 

 

2

 

 

2

 

 

S6

3

1

 

 

 

 

 

 

 

 

 

 

 

 

1

SR1

11

15

5

2

 

 

 

 

 

1

 

 

 

7

 

SR2

8

5

4

 

 

 

 

 

 

 

 

 

 

1

 

SR3

21

12

3

2

1

 

 

1

 

1

1

3

 

 

 

SR4

4

2

2

 

 

 

 

 

 

 

 

 

 

 

 

 

 

We predicted that the course leader and site leaders would be regarded as more important by other staff, and therefore receive more replies. In the case of the course leader this is particularly true due to the nature of his postings, which were explicitly intended to elicit further contributions. Both the course leader and site leader SL1 did indeed get a lot of replies to their messages, but site leader SL2 did not. Closer observation suggests that this might be related to the content of the messages and their tone – it is a legitimate use of the discussion board that some types of message might inhibit further discussion, which seems to be the case here.

 

Our prediction that students would be interested in mail from other students is supported by the fact that most replies to student SR1 came from student SR3 who is in the same year at the other college. Similarly, most replies to SR3 came from SR1.

 

We predicted that people would be more interested in messages from others on the same site but this is not supported by the data. This seems to be mainly due to the fact that people at the same site were more able to talk to each other outside the discussion forum, an issue discussed previously.

 

Despite identifying a small correlation of the expected kind, we are not intending to implement a mechanism to enhance patterns embodying formal user roles in the new system. Partly this is due to the complexities of managing such a process, but also there is disagreement about the form it should take, in particular whether it would be better to enhance messages people would not otherwise be interested in or whether it would be better to amplify existing preferences. Although on the one hand we wish to encourage participation by making more relevant messages clear, were we to decide a priori which of those messages were relevant, we would be consciously shaping the discussion. If we think of the course board as a learning community, then the problems with this approach become clear as, in a sense, the designers would become the “sage on the stage” by proxy. While we would not wish to stunt the style of a moderator who may wish to take on that role, we would equally not wish to make it a structural part of the system itself. The situation may well be worse than that: Michael Moore’s theory of transactional distance posits an inverse relationship between structure and dialogue in any educational transaction (Moore & Kearsley, 1996). If we impose our own structure onto the dialogue then we may actually stunt that dialogue, which is the opposite of the intention of our system.

 

Rather than predetermining importance, we will give users the ability to rate messages and consequently use such ratings to identify people who provide the most interesting postings. This will thus add structure from the bottom up rather than the top down. To ensure that this does not result in ill feeling which would affect the dynamics of the meeting, it has been suggested that we should only use these ratings to affect the view of the person supplying the ratings. However, we have instead decided to make averaged ratings visible to all so that they might play the same role as (for instance) yawns in a face to face meeting. This remains a topic of debate and may be removed from the production environment. Trust is an important contributory factor in online discourse (Preece, 2000; Salmon, 2000) and such a mechanism may lead to feelings of insecurity.

 

We assume that users are interested in the subject of their own messages and therefore the responses to them. It is therefore useful to look at how close messages are in the threading to messages they posted themselves. For example, consider the threaded discussion depicted in Figure 4.

 

Figure 4 - Example of a discussion thread

 

It is probable that the author of message B will have a high interest in D, E, and F as they are responses to her own message. Her interest in G will be high, but may be less high than her interest in D, E, and F. Similarly, her interest in C (a reaction to the same message she reacted to) will probably be high, but less high than her interest in D, E, and F. Our assumption is that the further away from her message along the thread the less her interest will be. This is confirmed by an analysis of a typical discussion where, if an individual’s message has been replied to, they are significantly more likely to respond to that reply. For example, figure 5 represents a thread in the most recent courseboard. This fragment shows an ongoing conversation between A, C and D, with interjections by E and B.



In our new system the users may choose to receive email notifications of replies to their own messages, and optionally receive notifications of further replies in that thread. By proactively pushing these messages into the users email inbox it is hoped to stimulate further more timely dialogues, rather than relying on the users to choose to participate when they have the free time. In addition to this we will emphasise threads in which the user has already participated.

 

4.1.3     Significance of content

 

The importance of a message is likely to depend a lot on the subject of the message. We assume that staff will be interested in messages about their module, and that they will also be interested in their ‘topic area’. For instance, staff teaching a programming module, are likely to be interested in messages about programming even when related to other modules.

 

We have analysed on what topics people posted messages. In table 2, topic headings are given, as well as “Other” category for all messages that do not fit into those topics. FDxxx numbers indicate modules of the course. Shading has been used to highlight the modules that people teach on (if staff) or attend (if students). Note that topics are not necessarily the same as threads. As observed previously, the thread structure provides a poor representation of what is actually discussed. For instance, messages related to module FD133 have been found in three different threads: one directly related to FD133, one in Any Other Business, and one about the minutes of the previous meeting.

 

Table 2 - Significance of topic

 

SL1

SL2

S2

S3

S4

S5

S6

SR1

SR2

SR3

SR4

FD111

1

 

 

 

 

 

 

 

1

 

 

FD112

 

1

 

 

1

 

 

3

2

5

 

FD113

 

 

 

 

 

1

 

 

1

 

 

FD131

2

 

 

2

 

 

 

2

 

4

 

FD132

 

1

 

 

 

1

1

2

1

4

 

FD133

 

1

 

 

 

5

 

1

 

1

 

FD211

2

 

2

 

 

1

 

 

 

 

1

FD212

1

 

 

 

 

 

 

 

 

 

1

FD254

 

 

 

 

 

 

2

 

 

 

1

College Feedback

1

1

 

 

 

 

 

 

 

 

 

Website

3

 

 

 

 

 

 

2

1

5

1

Other

3

1

 

 

 

2

 

1

2

2

 

 

Everybody has posted messages on all topics they teach or attend (with the exception of S1 who did not post anything). Many members of staff (S2, S3, S4, S6) only posted messages related to the topics they taught.

 

Both second year students (SR1, SR3) have contributed to the discussion on first year module FD112. This module deals with programming, a topic they attend in FD132. Interestingly, first year student SR2 contributed to the discussion on second year programming module FD132. It seems therefore that students are not only interested in modules they attend, but may also be interested in other modules about the same topic.

 

From the staff, it is mainly the site leaders who post messages on modules they do not teach. SL2 seems to have in interest in programming (which he used to teach) in addition to his own modules. SL1 posted messages on FD131, which is taught at her college by a member of staff who could not attend this course board. Site leaders tend to act as stand-in for module leaders. SL1 also started the discussion about FD211, perhaps because she knew that the module leader at her site could not attend the course board till later in the week. This is also influenced by the fact that site leaders have a personal tuition role that means they are likely to be aware of problems that affect a wide range of modules.

 

It is clear that the topic of a message seems to be a good predictor of the interest a participant has in the message. Therefore, in our new system participants may filter messages based on keywords (such as module names, module topic areas). In the future, content-based and social filtering may be used to determine the likely appeal of the content of a message. This requires the system to know about how interesting the user has found other messages (or, in case of social filtering, the opinion of others). The explicit ratings that users give to messages will make this relatively simple to implement, but we are also looking at ways of capturing implicit ratings by observing whether the user replies to messages, or even reads them. A classic and still sophisticated example of this approach is the seminal GroupLens (Konstan, Miller, & Maltz, 1997).

 

A second way of using the content of messages is to consider their role in the context of the discussion. For instance, when a course leader proposes an action, and wants to take a vote on the acceptance of that action, the formal requirements of the meeting mean that this message has a high importance. Our new system allows the attachment of icons to represent the type of message being posted. Message openers (comparable to sentence openers in collaborative systems (Looi, 2001)) could also be used for this purpose.

 

These approaches should make it easier for the Course leader to summarize the discussion, or even partly automate this process using text summarization techniques.

 

 We are considering an extension of D3E’s ability to apply iconic representations of message type to allow the community itself to evolve message types, using a mechanism of the sort discussed in (Dron, Mitchell, Siviter, & Boyne, 1999) and (Dron, Mitchell, & Boyne, 2003). Thus, control over the process would not be centralised but reside in the evolving needs of the community itself. Whichever mechanism is used, this might require messages to be broken down into a sequence of smaller ones. For instance, at the moment (despite exhortations to avoid it) when a module leader reports on a module, she is likely to include problems as well as positive feedback, improvement suggestions, and perhaps proposed solutions. If the system allows a message to be considered as being composed of an arbitrary number of smaller sub-messages (any one of which may be replied to) then some of the problems we have observed relating to the disintegration of threads may be reduced. To some extent it is hoped that the document centricity of the D3E system will help this to occur naturally, but we are also building in a mechanism to make it possible to refer to messages outside the thread and subsections of any message. It is difficult to design an interface to this that is sufficiently simple and usable so it remains to be seen how effective this mechanism will be.

 

 

4.2     Representation

 

We have sketched a number of contributing factors to the importance of a message, and how they could be calculated. The question arises of how to represent them in the user interface. One alternative is to use the factors individually: different aspects of the user interface could be automatically adapted to give the user an indication of the importance of each message according to the dimensions discussed in the next section. Alternatively, they could be combined -calculating an overall predicted importance of each message given the individual measures. This overall importance could then be indicated to the user (discussed in the following section).

 

Individual measures could be combined in several ways. For instance, one approach could be to use a weighted addition:

 

Overall_Importance =  Σ wi * ContributingFactor i

 

Where wi is a weight reflecting the relative importance of ContributingFactori.

In a sense, this problem of getting an overall rating from a set of individual ratings is very similar to the group modeling problem discussed in (Masthoff, 2004). Empirical research showed that a multiplicative function (rather than an additive one as above) had better results, particularly if it was important to keep all individuals (in this case, all contributing factors) happy. Weights could then be used as exponents. Weights can be learned automatically if users rate messages (Masthoff, 2003).

 

 

4.2.1     Separate factors

 

Adaptive hypermedia techniques (Brusilovsky, 2001) and social navigation (Kurhila, Miettinen, Petri Nokelainen, & Tirri, 2002) can be used to adapt the interface to the relative importance of messages. The importance of the message based on individual factors, such as novelty, author, etc., can be expressed with the use of adaptive annotations. For instance:

 

 

All of the above navigation cues need to be customizable, a process which will be greatly facilitated by the rigorous use of stylesheets in the new system.

 

 

 

4.2.2     Aggregated factors

 

Instead of (or in addition to) showing all the importance factors individually, the overall importance of the message could be indicated in a number of ways:

 

 

In addition to supporting navigation by adapting the presentation of messages, more direct navigation guidance could be given as well. In particular, a “recommended messages” list could be constructed. One benefit of such an approach would be to help to reduce the “misclassification problem”. If somebody accidentally puts a message in the wrong location, then its importance due to content might go down slightly (as it may be further away from other interesting messages), but the keywords in the message, its novelty, and the importance of the author could push it up considerably. Unless, of course, the author is a boring person with a low author rating, in which case it might not be so bad to miss the message!

 

 

5.     Conclusions

 

The process of running an asynchronous course board has provided a number of benefits but equally has opened up a number of problems that a combination of process and software can help to solve. The basic augmentations of the D3E environment are progressing well and are expected to be used in next year’s course boards following trials over the summer.

 

Our adaptive solutions remain the subject of heated debate. It is unlikely that the University’s regulations would allow the system to prevent users from seeing certain messages and might frown on the automation of a process that might be more properly seen as a matter of individual rather than collective choice. Even where messages are not hidden, the social navigation features that are proposed are by their nature designed to shape the interactions of users. In so doing, the authors of the system are to a significant degree dictating the nature of the discussion and debate. If the suppositions that are being made are wrong or even at odds with the preferences and tendencies of the participants, then this might be positively harmful. Having said that, the traditional systems that have evolved for this purpose are equally constraining in their effects. Churchill told us that we shape our dwellings and afterwards our dwellings shape us (Churchill, 1943). We have lived within the constraints of traditional course boards for so long that it is not always obvious how they shape our perceptions and interactions within them. The strange formality of the processes that amplify social hierarchies  are often almost invisible to us: for instance, the bizarre practice observed by some of referring to the person leading the discussion as “chair,” the strict implicit rules preventing interruption of the person speaking, the difficulty of divergent discussion outside the agenda brief and the constraints of the meeting schedule, all contribute to shaping the range of potential interactions as effectively as any of even our most extreme suggestions.

 

As we depart further from the model of a familiar threaded discussion forum, one significant problem that the use of a more radical course board design raises is that the benefits of the current low threshold solution with which all are familiar and comfortable may be lost. Without great care there is a danger that we may adversely affect the ability of individuals to play an equal role in the community.   

 

In the process of running course boards through asynchronous discussion boards we have identified a number of issues that do not only affect these formal meetings but online discussions in general. Although we have focussed on ways to manage a very specific process there are many lessons that are applicable to the wider problems of online learning. Issues of trust, involvement, consensus and focus stand out as particularly pertinent to the needs of online learning. This is unsurprising, given that in reflecting on the process of running course boards it has become clear that the course board is primarily a means of enabling learning through a process of mediated discussion.

 

We started this analysis with the fairly simple belief that course boards were valuable due to the ability of participants to reflect on the course delivery and to effect change. In exploring some of the more exotic variations on how a course board might be constructed we have formed a richer picture of this process, including a much clearer notion of what matters to participants and how they are influenced in discussions, whether online or face to face. Whether our proposed solutions work or not, this outcome is very useful.

 

 

6.     References

Brusilovsky, P. (2001). Adaptive Hypermedia. User Modeling and User Adapted Interaction, 11(Ten Year Anniversary Issue), 87-110.

Churchill, W. (1943). HC Deb 28 October 1943 c403.

Dron, J. (2002). Achieving Self-Organisation in Network-Based Learning Environments. Unpublished PhD, University of Brighton, Brighton.

Dron, J., Mitchell, R., & Boyne, C. W. (2003). Evolving Learning in the Stuff Swamp. In N. Patel (Ed.), Adaptive Evolutionary Information Systems. Hershey, PA: Idea Group.

Dron, J., Mitchell, R., Siviter, P., & Boyne, C. (1999). CoFIND- an experiment in n-dimensional collaborative filtering. Paper presented at the WebNet 99, Honolulu, Hawaii.

Hewitt, J. (2001). Beyond threaded discourse. International Journal of Educational Telecommunications, 7(3), 207-221.

Konstan, J. A., Miller, B. N., & Maltz, D. (1997). GroupLens: Applying Collaborative Filtering to UseNet News. Communications of the ACM, 40(3), 77-87.

Kurhila, J., Miettinen, M., Petri Nokelainen, & Tirri, H. (2002). Use of Social Navigation Features in Collaborative E-Learning. Paper presented at the E-Learn 2002, Montreal, Canada.

Looi, C. K. (2001). Supporting Conversations and Learning in Online Chat. Paper presented at the International Conference on AI & Education, San Antonio, Texas.

Masthoff, J. (2003). Modeling the Multiple People that are Me. Paper presented at the User Modeling Conference 2003, Johnstown, Pittsburgh, USA.

Masthoff, J. (2004). Group modelling: Selecting a sequence of television items to suit a group of viewers. User Modeling and User Adapted Interaction, 14, 37-85.

Moore, M. G., & Kearsley, G. (1996). Distance Education: A Systems View. Belmont: Wadsworth.

Preece, J. (2000). Online Communities: Designing Usability, Supporting Sociability. Chichester: Wiley.

Salmon, G. (1998). Developing learning through effective online moderation. Active Learning, December 1998(9).

Salmon, G. (2000). E-moderating: The Key to Teaching and Learning Online. London: Kogan Page.

Seely Brown, J., & Duguid, P. (2000). The Social Life of Information. Boston, Massachusetts: Harvard Business School Press.

Senge, P. M. (1993). The Fifth Discipline- the art and practice of the learning organisation. Chatham: Century Business.

Sumner, T., Buckingham Shum, S., Wright, M., Bonnardel, N., & Chevalier, A.x. (2000, 23-26 May, 2000). Redesigning the Peer Review Process: A Developmental Theory-in-Action. Paper presented at the COOP'2000: Fourth International Conference on the Design of Cooperative Systems, Sophia Antipolis, France.

Turoff, M., Hiltz, S. R., Bieber, M., Fjermestad, J., & Rana, A. (1999). Collaborative discourse structures in computer mediated group communication. Journal of Computer Mediated Communication, 4(4).

 


[1] In contrast, it can also be argued that an unread message posted longer ago has more importance, as the author of that message has been awaiting a response for longer. Of course, this depends on the number of responses the author has already received.

[2] In this calculation we have ignored the thread “Actions” as this is intended for use by the course leader only, and the thread “Attendance” as all participants have to reply to this and their message will never be replied to.