Happy accidents or inspired designs?

Delving through the literature about online communities, I came across an earlier survey by Johnson (2001), treating online communities of practice as a naturally occurring phenomenon within designed online communities. Johnson studied successful communities of practice, to establish what organisational factors contribute to their success, as opposed to more traditional organisational decision making.

From my worldview, this contrasts the rigid doctrines of hierarchical institutions with the distributed, decentralised approaches of emergent networks. Interesting points concern:

1. The need for organisations to adopt these ways of working to survive in the face of rapid change
2. The concept of collaborative knowledge outweighing individual knowledge - some parallels with collective intelligence perhaps?
3. The tension between the need for the community to understand process, and the need for market & product development

All of these points have particular relevance for the organisation I work for, with rapid changes in business, learning, technology and government. Moving towards this way of working is particularly desirable for transparency and continued development, but is difficult to achieve in practice. My own interest is in understanding - from a qualitative perspective - what experiences encourage people to go beyond occasional participation and into sustained discourse (Hobson, 2013). I am also interested in understanding factors that lead to attrition, a point that Johnson notes as a common problem.

Johnson also addresses issues of the medium itself, our conceptions of community, and the effect of using face-to-face communication alongside online interactions. Effective online communities adhere to adult learning principles, with a breakdown of the normal instructor - learner differentiation. I note that this supports a move from hierarchical to networked organisation principles. Real and sustainable communities will form around real problems where motivation to seek out solutions is high.

Does such collaboration come about within a company naturally? Does it need any facilitation?  What role should learning and development play in all of this? Our role should be to support the development of collective and employee knowledge, rather than to impart it, leading to the moderator role. Here I can identify where my existing efforts have either met with success or struggled. Communities need goals to achieve, objective evaluation of that success [as a group], and also peer & self-evaluation.

There is a need to draw out the concepts of community that apply to work-based problems and those for formal education courses. A key difference is that for formal education, we are all expected to learn the same thing, and are assessed against this, whereas for real workplace learning, it is the variety of individual learning outcomes that allows the group to have greater intelligence than the sum of its parts. It is perhaps this necessary diversity of outcomes that makes online learning communities such a slippery but valuable prize.

Psychology naturally seems to underpin the development (or not) of a community, with perceived intentions having impact on whether learning happens as a result of comments from other members. 'Are they criticising me? How dare they!' or 'It's really helpful to get constructive thoughts on my work' - which of these a person thinks about comments from peers will likely depend on the underlying environment and individual mentalities. It's not simply a question of technology! My next stop for reading will be works by Palloff and Pratt, as mentioned by the author.

References

• Hobson, J. (2012). The problems with educating a quadriform society. Thinking and learning in the digital age [blog] 7 August. Available at: http://jimmyhob.blogspot.co.uk/2012/08/the-problems-with-educating-quadriform.html
• Hobson, J. (2013). Online communities - trying to walk the talk. Thinking and learning in the digital age [blog] 19 March. Available at:  http://jimmyhob.blogspot.com/2013/03/online-communities-trying-to-walk-talk.html
• Johnson, C.M. (2001). A Survey of Current Practice on Online Communities of Practice. Internet and Higher Education, Vol.4 pp.45-60.
• Malone, T. W. (2012). Collective Intelligence. Edge.org [online] 21 November. Available at: http://edge.org/conversation/collective-intelligence

Evaluating online communities

Thoughts on Ke & Hoadley (2009)

 

Important point to consider: do we expect online learning communities (OLCs) to appear spontaneously or through design? Study inherently seems to favour studies of those that are well-defined in terms of support & structure, so either well-designed or evolved.

Taxonomy of online learning community evaluations

Recognises that this is a divergent research area, attempts to categorise studies in terms of four key components:

1. Evaluation purpose
Notable distinction between proving and improving purposes – convincing organisations that the community has a value at all, versus looking for ways of systematically enhancing the interactions within it. In my case the interactions don't yet amount to a community, so the emphasis is on making interactions sustainable and of benefit to participants.

2. Evaluation approach
Approaches were sometimes summative, usually for proving, or formative for improving, sometimes with elements of both of these. There are also the participatory and responsive approaches – a choice of whether to include participant evaluation or not. Oliver (2000) and Patton (1997) are cited as primary references here. My approach will be based entirely on participant responses, with a view to formative evaluation of the interactions.

3. Measures for evaluation
Outcome vs process measures. The outcome view looks at the community as a static system, evaluating the raw technical set-up of the environment and the learning outcomes. Process evaluation takes an in-depth look at the factors that facilitate or impede learning within the system. My study will need to focus on the process, with a possibility for pairing this up with outcome evaluations from colleagues.

4. Evaluation techniques
The authors seem to use the term objective in place of quantitative; they also refer to qualitative and mixed-method approaches. An important distinction between the two main forms is made – objective approaches deliberately remove context from the data, focusing on what could be directly comparable between other studies. Qualitative studies allowed for more direct insights for the learning processes within a community. I will be focusing on qualitative approaches, although there is potential for identifying the best factors to use in future studies for building up an objective measure of communities in future.

Conclusions

This gives a very good critical analysis of the factors at play in evaluating OLCs, and can serve as a guide point relating to the higher level discussions of Newby (2010), Cresswell (2009) and Colquhoon (2006). The authors also point out a good number of shortcomings in current research. Partly these are due to the constraints of researchers performing their studies for their own purposes, rather than to sit conveniently into the wider body of research. They also point out that the offline interactions between participants play a large role in the actual learning process, and these are very difficult to find any record of. There are also no studies that show how a community has evolved over time.

Long term goals of the researchers are to establish a framework for understanding OLCs, possibly towards a central theory. They make no reference to the five stage model identified by Salmon (2003); they do refer to phases of community development (Palloff and Pratt, 1999). It will be useful to pursue this systematic approach for research to inform my own understanding, and note if any similarities or contradictions with Salmon’s model emerge, which has previously been central to my understanding of online learning interactions.

Reference:

• Colquhoon, D. (2006). Research Methods in Education Contexts. University of Hull.
• Creswell, J. W. (2009). Research Design: qualitative, quantitative and mixed methods approaches. (3rd edition) Sage.
• Ke, F. and Hoadley, C. (2009). Evaluating Online Learning Communities. Educational Technology Research and Development, 57(4), pp.487-510.
• Newby, P. (2010). Research Methods for Education. Pearson Education Limited.
• Salmon, G., 2003. E-Moderating: The Key to Teaching and Learning Online. 2nd ed. London: Routledge-Falmer.

Thoughts on Conrad

I've been going into more details on my literature review, and considering how other authors structure their writing, particularly from the perspective of what I find useful, and how it might influence my own writing. The first article is an interpretive study by Conrad (2002).

 

Conrad makes the point that while quantitative studies can give a useful overview of the area, understanding the experiences of users is a priority for development of communities. Although my context is somewhat different, her point about learners creating their own lines of defence sounds quite telling in the light of some initial comments observed in my study. Without understanding what barriers people put up, we can’t expect to engage meaningfully with them! Likewise, her point about our research agendas being shaped by our worldview matches with my research being guided by exploration of how network effects are re-shaping society, and the subsequent effects on education systems, both technological and organisational.

Writing a section about who, what, where, etc. is a useful step for grounding the paper, and setting the boundaries of the study, particularly the limitations of what might be achieved in the first place. She also breaks down the literature review itself to define different terms, building up how she wants the reader to understand her term of online community. She then examines each of the research questions in turn, looking at general patterns in responses, followed by particularly insightful comments by individuals. This approach will probably work very well for me, as I need to spot general patterns as well as bring out individual experiences.

Reference

Conrad, D. (2002). Deep in the Hearts of Learners: Insights into the Nature of Online Community. Journal of Distance Education, 17(1), 1-19

Reviewing the proposal

Considering points raised by Newby (2010, Ch. 2 & 14) to refine my research proposal has given me a mixture of outcomes. Some of these confirm that my basic methodology is correct, whilst some aspects might need tweaking.

Why this research?

My primary aim in this research is gaining a qualification – however I do need to relate the research to my work context, which has been a source of tension since in future I will need to contribute to projects that further divisional goals more directly.

Why am I interested in this area?

We are aiming to move towards a blended training model, with online and face-to-face components.  The use of forums to encourage interactions among participants has been running for some time, with uncertain results.

Why do I think that it is important?

The building of learning communities with increase responsibility for their own learning will become important to sustain learning across the organisation. Our current resourcing will not enable us to meet the continuously growing demand.

Will I be able to write enough to cover my task?

I have chosen this learning program because I know that there is the potential for enough research data to be gathered for a number of cohorts, and I can combine research on the participants with benefit for practitioners.

What experience do I have of this area?

I have made use of forums in my own study and introduced the idea to colleagues through an explorative example. This led to the introduction of forums in the context that I am researching, and I have been involved in an advisory and technical capacity since then.

Who else thinks this is worth exploring?

There is a slight difference of opinion amongst my colleagues as to the value of exploring this area further. Some of my colleagues however have expressed an interest in gaining insights into this area, and even those who don’t fully support it at least acknowledge their frustrations that the use of forums is problematic.

Creating the context for the proposal

Will it deliver on our personal goals?

Although it might meet the requirements of a modular assessment, it may not have a high enough ‘impact potential’ for further progression in some respects. The project should therefore be regarded as a stepping stone for future research efforts – my key learning points will be how successful the approach is in educating practitioners and giving further insights into a learning program. Once my methodology has been established I will then look for higher impact applications.

Does the topic have a political dimension?

Online education in general is an area of intense interest, along with the tools and media for achieving success. Political commentators and politicians may well have opinions on the area, but they are likely more interested in conventional learning matters at this time.

How much do we know about the area?

As stated previously, I have been involved with the use of online discussion for learning in a variety of contexts, both academic and work related. My initial literature review has uncovered some very useful information about how online communities can be investigated methodically, hence my approach to this project.

Identifying the research boundaries

Total amount of time available

In this case the time available is dictated by the time constraints of the assessment task, and my personal time for writing up my results.

What help can we expect from others?

The work does need to be primarily my own due to the assessment criteria, and the data collection should be fairly straightforward due to sending out questionnaires by e-mail. I do need to consider how much time it might take participants to answer my questionnaires, given that we will be approaching a time of higher workload.

Do we need specialist help?

No outside help will be necessary in this case since the data collection and interpretation is of my own devising, and I have sufficient access to journal articles at this time.

What if things don’t go according to plan?

I believe I have sufficient safety in terms of preserving data and the actual collection. My concern is with the possible response rate of participants within the time frame. For this reason I had already opted to research a program that has had multiple cohorts, and also the interactions have been digitally recorded due to the nature of the medium.

Assessing ethical issues

This has already been covered in the blog post on principles to some extent. As a summary, I will be carrying out my research within my workplace and sharing those results with colleagues. One potential issue could come from the sharing of participants’ responses to a wider audience, from a forum that was set up as a confidential area. For this reason I will need to ask colleagues about how to use their responses. I will treat responses anonymously by default, possibly with an option for their responses to not be quoted in reports. All postings on forums will remain confidential.

Preparing a case for a research programme

This research programme is for assessment purposes, but setting out my thinking will be of use in future, especially if I want to keep research as a positive influence on our evaluation strategy.

Getting an idea for research

From the sources of stimulus I have identified several different strands to my research idea & motivation:

1.   Literature

My review of the literature has revealed that there are a number of different theories cited by researchers, in some cases not including those that I have used as a theoretical basis for my own learning. Because of this I am wary of using quantitative methodology to tie results to a particular theory

2.   Policy

Ongoing interest in the field of online education interests me, particularly from the point of view of keeping practitioner education a key issue. Research that is carried out purely from a top-down institutional perspective is unlikely to produced sustainable benefits for the education system or society.

3.   Experience

Following on from previous points, my experience of online education is that it is most powerful as a constructivist environment, to support communities of inquiry and learning by doing.

Potential to finish research project

For this assessed research project I am confident that I can deliver the required results, and show that my methodology for design and data collection has been adequate. Questions about the impact of my research will need to be addressed for future efforts, since online learning projects will be focussed on areas of greatest business interest.

Arguing the case for the research

(a) Can the topic convince our judges?
The project should sufficiently convince my academic judges, although from a work perspective I will need to convince colleagues that the project has sufficient merit as a developmental tool for future work, and show that my approach to sharing results and insights can benefit the wider team.

(b) Is it feasible for me to undertake it?
I believe that this research project is well within my capabilities, and can be completed satisfactorily within the given time frame.

(i)  The field of enquiry
Research in this area is nothing particularly new, since there has been significant research into the field by academic researchers. Neither should this be considered as a theoretical study, although the context will always be different from others. The real interest here is how policies of using online education methods are succeeding against our objectives, and how the use of these approaches compares between academic origins and the actual implementation in the workplace.

(ii) The research issue / question
Related to the issues from previous questions, my key questions for research are based around the experiences of people participating in the use of new types of media, and ensuring that practitioner approaches can match the desire for business results.

Making sense of what we have

My interpretation of results may well draw on theory, although this will not be the primary lens. My main focus will be on judging whether activities have met objectives and how they can be modified for improvements, and also to identify what course of action should not be taken. My approach to truth here is realising how others perceive it, and using this to inform practice.

References

• Newby, Peter (2010).  Research Methods for Education Chapter 2 p.31-65

Blog post 5.2: Mechanism of my mind

Originally posted to eBridge, 2 March 2013

 

I've finished off my reading & reflections from last week at long last! My primary venture into reading about online community literature led me to the article mentioned in my last blog post (Ke & Hoadley, 2009). I note that the authors don't mention Salmon's (2003) model of online interaction, which has formed the basis for much of my understanding in this area, being the first model that I was introduced to on the course. It's interesting how the first mental representation we get will dominate our thinking on a subject!

As I started to outline in my previous post, the authors have done a great job of outlining how they reviewed the literature, and actually relates very well to the principles that have been introduced in our formal reading so far. Therefore I think I'll find it more helpful to pursue an approach influenced by their taxonomy, as it will hopefully allow me to connect my work with a wider body of research. However I will make a point of re-reading my work from the initial module and consider how this work affects my understanding, wither confirming it from a different angle, modifying it or destroying an incorrect assumption.

References:

• Ke, F. and Hoadley, C. (2009). Evaluating Online Learning Communities. Educational Technology Research and Development, 57(4), pp.487-510.
• Salmon, G. (2003). E-Moderating: The Key to Teaching and Learning Online. 2nd ed. London: Routledge-Falmer.

Blog post 5.1: Reviewing the literature (running behind)

Originally posted to eBridge, 24 February 2013

I'm running late this week, but thought it was better to put up a partial blog post than nothing at all. The primary piece of literature I've identified in my search is a comprehensive review of literature on online communities (Ke & Hoadley, 2009). My immediate thought is that this should save me a great deal of time, but there is a real danger of not being sufficiently cautious about accepting their methods and conclusions. The researchers do clearly set out their review, with criteria for how they chose studies that were suitable for inclusion. They also helpfully set out questions to ask of any study of online learning communities:

• What was the purpose of the study?
• What evaluation approaches were used in the study?
• What indicators or measures of online learning communities were observed in the study?
• How did the study collect and analyse data?

These will be very helpful for considering how to go about collecting my own research data. However I do need to beware that this review is made within the academic world, and likely does not include any kind of workplace learning. Participants in the online communities in question do so out of necessity - they must show that they have learned in order to progress in their studies or receive qualifications, whereas by contrast a workplace learning initiative cannot always count upon such participation from learners.

This actually seems like a good insight for how to move towards greater participation in online communities in the workplace. Having participation in online activities be part of the assessment could dramatically improve participation and uptake of learning, if used wisely, and with stakeholder support. In order to do this, I would have to show that online interactions can add value. Whilst it is unlikely a full collaborative community could be created in the first instance, progressively achieving enhancements to traditional training by provision of online communication tools could be a first step.

References:

• Ke, F. and Hoadley, C. (2009). Evaluating Online Learning Communities. Educational Technology Research and Development, 57(4), pp.487-510.