Re-examining my worldview

Image: ariss via freeimages

Research Philosophy

How do I identify with four of the main worldview areas, as discussed by Newby?

Scientism and Positivism

This school of thought demands predictable cause and effect - not easy to establish where people are concerned. Once people realise that others are able to predict their behaviour in any way, they tend to change their behaviour to avoid manipulation! It may be beneficial to look for patterns if you have a critical interest, such as a business issue, although this may actually overlap with more humanistic traditions (see below) because you will be seeking a model that is 'fit for purpose' rather than trying to establish an objective truth.

This is the case for the research carried out by Meadows (2004) and Tremain (2011, 2012), where the critical factors affecting examiner retention and job satisfaction needed to be identified. Establishing an all-encompassing theory or truth (positivism) may be too ambitious - and would be clouded by personal interests - identifying and monitoring the relevant factors was achieved through careful selection of questions and interpretation of data.

Future reading: Discussion of the work of Karl Popper (Newby, Ch.3)

Humanism, Phenomenology and Existentialism

Whist positivism demands an objective external truth independent of human influence, humanism treats 'truth' as a social construction, where one culture's truth may not be another's. This ties well with my understanding of social evolution (Hobson, 2012; Ronfeldt, 1996, 2012a,b) whereby societies develop distinct cultures of varying complexity through the addition of different forms of organisation. Conflicts arise between or even within societies due to disagreements about how society should be ordered (or not) by hierarchical institutions and free market policies. Ronfeldt (2013) acknowledges that even the term 'tribe' in his T-I-M-N framework is frequently contested by others, and progresses the model by engaging with and incorporating these disagreements into his model.

Phenomenology focuses on individual and collective experiences to form a rational basis for future action, by probing the differences between 'perceived' and 'experienced' world. Methods include description observation, reporting and reflection.

Existentialism centres around seeking to understand the world from a personal perspective, driven by conviction and desire. Methods could extend to asking participants for other forms of 'data' such as pictures, videos and stories of their lives - anything that conveys their viewpoint in richer detail than a questionnaire could. Existentialism has been applied to curriculum design, by designing a curriculum that centres around self-discovery.

Critical theory

This theory is concerned with political beliefs, particularly those that are left-of-centre and seeking to change society by making people aware of their circumstances in order to liberate them. Research carried out within a critical theory framework seeks to observe and expose individual facts which can then be combined to form an argument for change.

Aside: I am particularly interested in the references to Karl Marx, as this led to a model for social evolution around the struggle for control of wealth - somewhat akin to Ronfeldt's description of the transition between biform (T+I) and triform (T+I+M) societies.

Applying critical theory to education has been used by academics to attack the present model for education as being primarily based around servicing the needs of the capitalist economy, and thus serves only to reinforce the inequalities that it inevitably generates. However positivists would undoubtedly attack this approach because it proposes a hypothesis and seeks to prove it.

Postmodernism

This approach distinguishes itself from the 'modernist' approaches by rejecting the 'modernist' assumption that there is a single explanation for things, which leads to a natural order. For education, this almost rejects the need for theories of education, since such theories are rooted in modernist approaches! This approach does overlap with my TIMN world-view, because it is by nature multi-layered, and acknowledges that 'people and organisations can play several and sometimes conflicting roles', in keeping with the different types of organisations and the interfaces between them.

Postmodernism bears a great deal of similarity to the network (+N) principles described by Ronfeldt, with researchers seeking to act more as nodes in a network, unpicking their assumptions and sharing data that reflects their local situation, with the ideal that the value of the network increases not simply with the number of nodes but the number of connections between them.

Future reading: Steven Johnson: Future Perfect 

Summary

My adoption to the T-I-M-N framework in my outlook leads me to cross several of the boundaries of research philosophy, but perhaps I identify more strongly than I realised with the post-modernist approach for rejecting the established order with its demanded polarisation of political outlooks.

References:

• Hobson, J. (2012). The problems with educating a quadriform society. Thinking and learning in the digital age [blog] 7 August. Available at: <http://jimmyhob.blogspot.co.uk/2012/08/the-problems-with-educating-quadriform.html>
• Meadows, M., 2004. Measures of examiner job satisfaction as predictors of intention to continue examining. Available at: https://cerp.aqa.org.uk/research-library/measures-examiner-job-satisfaction-predictors-intention-continue-examining [Accessed December 8, 2013].
• Ronfeldt, D. (1996). Tribes, Institutions, Markets, Networks – A Framework about Societal Evolution. Available at: <http://www.rand.org/pubs/papers/P7967.html>[Accessed September 2012]
• Ronfeldt, D. (2012a). TIMN in 20 minutes: a video presentation. Visions from two theories [blog] 23 May. Available at: <http://twotheories.blogspot.co.uk/2012_05_01_archive.html> [Accessed March 2013]
• Ronfeldt, D. (2012b). Q’s & A’s about “TIMN in 20 minutes” (4th of 7): creation of a new sector? Visions from two theories [blog] 29 August. Available at: <http://twotheories.blogspot.co.uk/2012/10/qs-as-about-timn-in-20-minutes-4th-of-7_29.html> [Accessed March 2013]
• Ronfeldt, D. (2013). Further points about “tribes” (T) — plus a new proposition about TIMN as a whole. Visions from two theories [blog] 10 April. Available at: <http://twotheories.blogspot.co.uk/2013/04/further-points-about-tribes-t-plus-new.html> [Accessed March 2013]
• Tremain, K., 2011. Carry on examining: what predicts examiners’ intentions to continue examining? Available at: https://cerp.aqa.org.uk/research-library/carry-examining-what-predicts-examiners-intentions-continue-examining [Accessed December 7, 2013].
• Tremain, K., 2012. Carry on examining: further investigation. Available at: https://cerp.aqa.org.uk/research-library/carry-examining-further-investigation [Accessed December 7, 2013].

Time for a numbers game?

Image: freeimages

Following on from my last blog post, I'm re-treading the sequence of reading from our Research Methods module to get my bearings again, and I'm coming back to the question of qualitative vs quantitative research. While I strongly identified with the Action Research methodology on my last project, it's worth deliberately opening up my mind to new possibilities, especially as there will be strong interest in some kind of numerical data from colleagues and external auditors if we are questioned on our approach.

So before I start to choose which quantitative disciplines I might wish to draw on, I'll look at the key aspects of quantitative research, consider those that appeal to me, and those I wish to avoid.


Concern with theory

Relating my findings to theory will be helpful to ensure some kind of tethers to related work, but there's a danger of getting obsessed with reproducibility and control here. When you're moving into the realm of on-demand learning, you can't guarantee learning outcomes, nor indeed that learners will even access the materials or activities that you produce for them. Newby (2010, p.96) acknowledge the limitations for educational researchers trying to identify pattern and control influences, as they are only able to view a small part of the overall education system. I would prefer to think in terms of Praxis (Wheeler, 2013), which requires practitioners to consider how closely their practice overlaps with the theories they identify with.

Concern with proof

Here lies one of the real problems for educational research - although I understand that establishing proof would give greater peace of mind, the complexity and ambiguity of the situation makes this extremely difficult:

• The situation I face will not be the same as another practitioner does, even if our verbal descriptions of it seem similar to the untrained eye
• The next situation that I (and the learners) face will not be the same as this one, even if it's 'just another e-marking system'
• The time needed to establish proof would be completely at odds with the time pressures for the project, where the learning is 'on-demand'.

The best that I can hope for is to show that using theories to guide my design leads to a dependable business outcome, and that particular methods or techniques are better suited to my situation.

Identification of variables

This is one of the key aspects of quantitative theory that I see as helpful. Although my control over most independent variables involved will be limited to say the least, it will definitely be helpful to at least make some systematic efforts to identify variables in the design of materials that may be having an effect, and to measure any dependent variables which are of interest. Our particular concerns would be the performance of examiners, and intention to continue based on their experiences. Attempting to correlate these with participation in the different aspects of the support might yield useful insights into which components have succeeded, but this would have to be linked to effective practice in design.

Simply saying that an approach should be abandoned because it doesn't seem to have an effect in this situation would be potentially misleading without some understanding as to why. Creswell (2009, p.49) refers to confounding variables (e.g. discriminatory attitudes) that can come into play, which I have had some experience of when trying to introduce online learning methods in the past. Participants who are negative about the use of the tools go to great lengths to discredit them when given the opportunity to do so, whilst the majority of participants actually acknowledge a positive effect.

Conclusion

This project will benefit from the use of some quantitative approaches to analysing data about examiner performance and intention to continue, but these will need to be paired effectively with qualitative methods to understand what dependent variables relating to the choice and design of approaches might be influencing the outcomes. My next blog post will focus on the type(s) of quantitative research methods might be useful, followed by a look at rational design approaches for the learning provision.

References:

• Creswell, J. W. (2009). Research Design: qualitative, quantitative and mixed methods approaches. (3rd edition) Sage.
• Newby, P. (2010). Research Methods for Education. Pearson Education Limited.
• Wheeler, S. (2013). Praxis makes perfect. Learning with e's [blog] 31 October. Available at: <http://steve-wheeler.blogspot.co.uk/2013/10/praxis-makes-perfect.html>

Selection of research method

Draft section of my reflective assignment, based on the blog post ‘A little more action (research) please!’

I have been forced to challenge my own pre-disposition towards quantitative research methods, which was influenced by my physical sciences background. I identified this bias at an early stage in my writing, and found the distinction with qualitative research better defined in my mind by reading the comparisons made by Creswell (2009, Ch.7) and Newby (2010, Ch.3). I also realised that my previous experiences lacked any real involvement in the formal planning of research; my previous projects has always been funded without me having to submit research proposals myself. I decided that qualitative approaches seemed better suited to my context, but it took some time to fully challenge my unconscious habits. I was able to identify possible sources of bias towards theoretical models that I had used (Salmon, 2004).

However it took some additional reading (and re-reading) to fully isolate my unconscious assumptions. After extensive reading about how to create both quantitative and qualitative research proposals, I believed that I had created a set of reasonable questions for qualitative research. My initial research questions were phrased as ‘What effect does...’ and ‘How does...affect...’ Only by revisiting some of the initial reading did I notice that, despite my initial conclusions that qualitative research would be the best approach, that I had automatically designed my questions in a directive way, that would lead to bias towards theory rather than interpreting participants responses from a neutral standpoint.

After some more reading, I was able to present a much more complete and reasoned overview of my research proposal, showing a great deal more thought. This revised research plan fitted much more closely with the principles of action research (Creswell, 2009; Newby, 2010 pp.623-4; O’Brien, 1998), and represents a successful change in my thinking about research.

‘Re-learning means to abolish some toxic assumptions’ (Leonhard, 2013)

References:

• Creswell, J. W. (2009). Research Design: qualitative, quantitative and mixed methods approaches. (3rd edition) Sage.
• Leonhard, G. (2013). Beyond the obvious: re-defining the meaning of learning in a networked society. (video online) Available at: https://www.annotag.tv/learningtechnologies/play/18320
• Newby, P. (2010). Research Methods for Education. Pearson Education Limited.
• O'Brien, R. (1998). An Overview of the Methodological Approach of Action Research. (online) Available at: http://www.web.ca/~robrien/papers/xx%20ar%20final.htm (Accessed March 2013)
• Salmon, G. (2004). e-Moderating: The Key to Teaching and Learning Online. 2nd ed. London: Routledge-Falmer.

Reliability

Is 'reliability' the right word to use for qualitative research? Cohen et al. (2007, Ch.10) suggest a variety of different words that should be used to distinguish the variable-controlled nature of quantitative research from the variable-embracing nature of qualitative research. Points addressing reliability that are present in my research include repeating the method of data collection and analysis across several groups (related to triangulation, below) and using anonymous questionnaires to encourage honesty amongst participants (p.209).

Early on in my research plan I discussed some of the principles laid out by Newby (2010, pp.120-3) for ensuring some degree of reliability in qualitative research, particularly the concept of triangulation. In the case of my project, the point about getting information form multiple sources is the best source of reliability. The questionnaire has been answered by participants from different cohorts of the same program, so we can gain some idea of how reliable (or not) the results are by comparing different cohort responses. Although the research is qualitative in nature, the reliability can be commented on by comparing the number of responses with the actual number of participants for each cohort.

References

• Cohen, L., Manion, L., Morrison, K. & Bell, R. (2007). Research methods in education. Routledge Limited.
• Newby, P. (2010). Research Methods for Education. Pearson Education Limited.

Data analysis

Although I set out to do a qualitative research project, I did reserve the right to quantify my results to some extent. I've found it helpful when making sense of my data to do a quick finger-test of whether comments were positive, neutral or negative, and summarise them by question and group. These can then be totalled and presented graphically, which will be very helpful when sharing back with participants. The idea of sharing with participants as a first audience seems particularly prudent to avoid accusations of 'intellectual imperialism' (Newby, 2010, p.48). This will also act as an important feedback exercise to help practitioners see the reaction of participants, and to give me more direct contact with both parties to better understand how to inductively expand the research.

There is still a lot more work to do on full data analysis at this stage, but deadlines are looming!

References:

• Newby, P. (2010). Research Methods for Education. Pearson Education Limited.

Collecting the data

I've been a bit lax in following the last stages of the pathway, so I thought I had better get some thoughts down before they fade away. 

What challenges do you face and what successes do you have?

Challenges:

• Some of the experiences I am asking people to recount are in some cases from over 2 years ago.
• The return rate for the questionnaire has been around 50% of all possible participants - partly this is due to it being a busy time of year for staff.

Successes:

• I have received a good number of responses, with an excellent level of detail
• Responses have yielded insights into how to improve the processes in question

What unexpected experiences do you have?

The use of an online questionnaire has left me feeling somewhat disembodied - most of the methods described in our reading concerned face-to-face methods. However they have been very effective, and in keeping with the idea that online learning should be anytime, anywhere...

How does your experience of data collection reflect, or conflict with, the principles and theories encountered in your reading?

I realise from reading through Newby (2010, pp.332-3) that I didn't pilot the questionnaire as rigorously as I should have done.  My testing was limited to sending the link to one participant who I asked about trialling it beforehand. Fortunately I had made significant revisions to my questions early on in the project that kept the research on track.

References:

• Newby, P. (2010). Research Methods for Education. Pearson Education Limited.

The nature of academic publishing

Image: forwardcom via freeimages

Building up towards my reflective assignment, I thought it might help to draft this in sections, according to particular areas of interest throughout the module. First stop is publishing...

One of the assumptions that I am interested in challenging is the area of academic publishing. We tend to regard the world of academic journals as ‘an open and public forum’ (Colquhoon, 2006) that is created for the common good, but a growing number of people are beginning to question if this holds true, especially in the age of open web content. Wheeler (2011a, b, c) has written a number of blog posts criticising the traditional system of academic publishing, noting that:

‘It’s a hierarchy that rarely changes.’

Furthermore, the hierarchical nature of the publishing world is often self-defeating in terms of securing the widest possible audience:

‘...many of the top, elite journals we are expected to publish in are in fact read by a very small percentage of the community the research is intended to reach.’

From considering my own worldview, I have noted the restrictions that hierarchical institutions place on the education system, particularly when combined with the solutions offered by market organisations (Hobson, 2012; Ronfeldt, 2012b). Potential disruptive alternatives may yet be found in the emerging network sector predicted by Ronfeldt (2012a), or the peer-to-peer ideology (Bauwens, 2009), although sustainable business models are yet to emerge. It should be noted that there are dangers inherent in the misuse of data for either solution. Newby (2010) warns of ‘intellectual imperialism’ when researchers are able to claim sole ownership of research data, discounting the rights of those involved in the study. At the other extreme of open web publishing, Lanier (2006a, b; 2012) warns of the dangers of ‘digital Maoism’, whereby all ownership and authorship of published data is lost, and transferred to control of the crowd.

References:

• Bauwens, M. (2009). Open Process academic publishing: towards the peer production of academic peer review. P2P Foundation's blog [blog] 15 September. Available at: <http://blog.p2pfoundation.net/open-process-academic-publishing-towards-the-peer-production-of-academic-peer-review/2009/09/15>
• Colquhoon, D. (2006). Research Methods in Education Contexts. University of Hull
• Hobson, J. (2012). The problems with educating a quadriform society. Thinking and learning in the digital age [blog] 7 August. Available at: <http://jimmyhob.blogspot.co.uk/2012/08/the-problems-with-educating-quadriform.html>
• Lanier, J. (2006a). Digital Maoism: The hazards of the new online collectivism. Edge.org [online] 30 May. Available at: <http://www.edge.org/3rd_culture/lanier06/lanier06_index.html>
• Lanier, J. (2006b). Beware the online collective. Edge.org [online] 25 December. Available at: <http://www.edge.org/3rd_culture/lanier06/lanier06.2_index.html>
• Lanier, J. (2012). Are we at the beginning of the rise of post human machine intelligence? [Video online] Available at: <http://dpcloud.co/v3/enterprise/learningtech/index.php?option=com_dpcloud&v=play&p=7117&u=learningtech&cat=Keynotes&Itemid=2>
• Newby, P. (2010). Research Methods for Education. Pearson Education Limited.
• Ronfeldt, D. (2012a). TIMN in 20 minutes: a video presentation. Visions from two theories [blog] 23 May. Available at: <http://twotheories.blogspot.co.uk/2012_05_01_archive.html>
• Ronfledt, D. (2012b). Q’s & A’s about “TIMN in 20 minutes” (4th of 7): creation of a new sector? Visions from two theories [blog] 29 August. Available at: <http://twotheories.blogspot.co.uk/2012/10/qs-as-about-timn-in-20-minutes-4th-of-7_29.html>
• Wheeler, S. (2011a). Sharp practice. Learning with e's [blog] 26 September. Available at: <http://steve-wheeler.blogspot.co.uk/2011/09/sharp-practice.html>
• Wheeler, S. (2011b). The open case. Learning with e's [blog] 2 October. Available at: <http://steve-wheeler.blogspot.co.uk/2011/10/open-case.html>
• Wheeler, S. (2011c). When the dam breaks… Learning with e's [blog] 18 October. Available at: <http://steve-wheeler.blogspot.co.uk/2011/10/when-dam-breaks.html>

Validity

There is a danger of intertwining the concepts of reliability and validity, so I'm attempting to address them in separate posts.  Cohen, Manion & Morrison (2013, pp. 177-99) devote a great deal of attention to the concept of validity, drawing on a variety of sources to lend weight and richness to the discussion.  Several of the points (p.180) raised are applicable to my research data:

• The natural setting is the principle source of data
• Context-boundedness and 'thick description'
• Data are socially situated
• The researcher is part of the researched world
• The researcher - rather than the research tool - is the key instrument of research
• The data are descriptive
• There is a concern for processes rather than simply with outcomes (inherent in my choice of action research)
• Data are analysed inductively rather than using a priori categories
• Data are presented in terms of the respondents rather than researchers

There are some points to beware of that might undermine the validity of my research - I will also list how these can be addressed:

• Reactivity (internal validity) - I am aiming to improve the processes involved in the learning programs; this should not affect the data for previous cohorts, although the current cohort could potentially be affected by knowing that they are being observed.  This also touches on the issue of Researcher bias mentioned by the authors
• Concensual validity (external) - since I am undertaking this research as a learning experience, will 'competent others' dismiss my findings due to my inexperience, or because they think I'm simply forcing the data to fit so that I pass my assessment?

Triangulation is dealt with as a means of ensuring validity.  Newby (2010, pp. 121-3) also mentions this technique, although in somewhat less detail.  So to what extent do my research methods yield themselves to this?

• Time triangulation - I have used the same method for a number of cohorts, although this effect may be diminished because I did not carry out the survey for each group immediately following their participation, so earlier groups may not recall their experiences as accurately.
• Theoretical triangulation - since I have avoided basing my questions on one particular theory, there is the opportunity to compare the results from the point of view of competing theories for social and online learning.
• Investigator triangulation - the data are recorded electronically, so potentially other researchers could give their own interpretations.
• Methodological triangulation - the same method has been used on different groups, so I can easily compare the results of each group to consider how well the results support conclusions for each group.

General points that have contributed to validity on this project:

• Choosing an appropriate methodology for answering the research question - action research.  This has ensured that the focus is on processes - not outcomes, which I might be interested in unfairly interpreting!
• Selecting appropriate instrumentation - using online questionnaires allowed the data to be gathered according to the time needs of respondents, and allowing them access to reminders (their forum postings) whilst responding to questions.

Limitations here include:

• Sampling - by not requiring participation in the survey, I sacrificed control over sample sizes, which could potentially limit the validity (and reliability) of results.  However I considered that sensitivity to participants' wishes was of greater importance in this case, since some people are uncomfortable with the use of online forums in the first place.

Overall I believe I have sufficiently addressed issues of validity, but there are clearly many others that I have not encountered yet, including the points concerning data interpretation.

References

• Cohen, L., Manion, L. & Morrison, K. (2007) Research Methods in Education.
• Newby, P. (2010). Research Methods for Education. Pearson Education Limited.

Revisiting ethics

Some more thoughts on the ethics of my project, based on the principles outline by Newby (2010, pp.47-9)

Moral position - None of the groups involved in the study will be treated differently, since the actual participation in learning programs has already taken place. However the responses will help to guide myself and my colleagues to ensure that the use of forums is not a detriment to future learning experiences, and ensure that problems experienced by individuals can be addressed.

Honesty - I am not aiming to fit the data to a particular theory or framework from the outset, although I will have to be careful to avoid the temptation to do this. In order to provide real justification for this, such results would have to be reliable and reproducible across several data sets, which is not likely to be possible across such small samples. By choosing action research I am committing to developing understanding of the online environment for practitioners & future participants alike.

Responsibility - All information shared is within the same organisation, and responses are anonymous, with the option to avoid being quoted in written reports, so nobody should be disadvantaged as a result of this study. There are no questions that I would expect to reveal anything sensitive, however this is something to bear in mind for future studies.

Sharing - I have expressed my intent to share results with participants, and with the wider organisation. What I will need to consider is the appropriate format to write up my results for others to appreciate. I'm sure most colleagues won't get much benefit from an academic-style paper! Nor should I be disseminating detailed results outside the organisation, although there is probably some benefit in talking about the overall process on a wider platform - this blog to be precise!

Freedom - Participants have had freedom to answer the questionnaire or not, and have also been given the option to have their responses omitted from written reports. I have not based the study on their actual forum entries, and I would be inclined to avoid doing so at such an early stage in the use of forums. People are apprehensive enough about engaging with the medium without the idea of their every word being used in evidence!

Community - This research project has been discussed with colleagues in Learning & Development to ensure that the nature of questions asked is appropriate, and that participants can benefit from participating.

Attribution - This research project is not making use of anyone else's work (but then I would say that, wouldn't I?); I will however be referencing authors whose work has influenced my approach.

Standards - As mentioned previously, there are certain issues to consider about potential audiences for sharing. Whilst this blog, my research plan and my reflections are written for an academic audience, I will need to present my results in such a way that non-academic colleagues can benefit from the results. To this end I will be looking for visual ways to present the overall trends of my findings, and different ways of explaining what I believe to be the most relevant theoretical background.

Breaking down the different issues involved in ethics this way has been far more useful to me than having a code of practice laid out in any format. These questions have to be asked time and time again, for every context, to keep them from being ignored. Hopefully they won't just stay on the pages of my Wiki for assessment purposes...

References:

• Newby, P. (2010). Research Methods for Education. Pearson Education Limited.

Evaluating online communities

Thoughts on Ke & Hoadley (2009)

 

Important point to consider: do we expect online learning communities (OLCs) to appear spontaneously or through design? Study inherently seems to favour studies of those that are well-defined in terms of support & structure, so either well-designed or evolved.

Taxonomy of online learning community evaluations

Recognises that this is a divergent research area, attempts to categorise studies in terms of four key components:

1. Evaluation purpose
Notable distinction between proving and improving purposes – convincing organisations that the community has a value at all, versus looking for ways of systematically enhancing the interactions within it. In my case the interactions don't yet amount to a community, so the emphasis is on making interactions sustainable and of benefit to participants.

2. Evaluation approach
Approaches were sometimes summative, usually for proving, or formative for improving, sometimes with elements of both of these. There are also the participatory and responsive approaches – a choice of whether to include participant evaluation or not. Oliver (2000) and Patton (1997) are cited as primary references here. My approach will be based entirely on participant responses, with a view to formative evaluation of the interactions.

3. Measures for evaluation
Outcome vs process measures. The outcome view looks at the community as a static system, evaluating the raw technical set-up of the environment and the learning outcomes. Process evaluation takes an in-depth look at the factors that facilitate or impede learning within the system. My study will need to focus on the process, with a possibility for pairing this up with outcome evaluations from colleagues.

4. Evaluation techniques
The authors seem to use the term objective in place of quantitative; they also refer to qualitative and mixed-method approaches. An important distinction between the two main forms is made – objective approaches deliberately remove context from the data, focusing on what could be directly comparable between other studies. Qualitative studies allowed for more direct insights for the learning processes within a community. I will be focusing on qualitative approaches, although there is potential for identifying the best factors to use in future studies for building up an objective measure of communities in future.

Conclusions

This gives a very good critical analysis of the factors at play in evaluating OLCs, and can serve as a guide point relating to the higher level discussions of Newby (2010), Cresswell (2009) and Colquhoon (2006). The authors also point out a good number of shortcomings in current research. Partly these are due to the constraints of researchers performing their studies for their own purposes, rather than to sit conveniently into the wider body of research. They also point out that the offline interactions between participants play a large role in the actual learning process, and these are very difficult to find any record of. There are also no studies that show how a community has evolved over time.

Long term goals of the researchers are to establish a framework for understanding OLCs, possibly towards a central theory. They make no reference to the five stage model identified by Salmon (2003); they do refer to phases of community development (Palloff and Pratt, 1999). It will be useful to pursue this systematic approach for research to inform my own understanding, and note if any similarities or contradictions with Salmon’s model emerge, which has previously been central to my understanding of online learning interactions.

Reference:

• Colquhoon, D. (2006). Research Methods in Education Contexts. University of Hull.
• Creswell, J. W. (2009). Research Design: qualitative, quantitative and mixed methods approaches. (3rd edition) Sage.
• Ke, F. and Hoadley, C. (2009). Evaluating Online Learning Communities. Educational Technology Research and Development, 57(4), pp.487-510.
• Newby, P. (2010). Research Methods for Education. Pearson Education Limited.
• Salmon, G., 2003. E-Moderating: The Key to Teaching and Learning Online. 2nd ed. London: Routledge-Falmer.

Reviewing the proposal

Considering points raised by Newby (2010, Ch. 2 & 14) to refine my research proposal has given me a mixture of outcomes. Some of these confirm that my basic methodology is correct, whilst some aspects might need tweaking.

Why this research?

My primary aim in this research is gaining a qualification – however I do need to relate the research to my work context, which has been a source of tension since in future I will need to contribute to projects that further divisional goals more directly.

Why am I interested in this area?

We are aiming to move towards a blended training model, with online and face-to-face components.  The use of forums to encourage interactions among participants has been running for some time, with uncertain results.

Why do I think that it is important?

The building of learning communities with increase responsibility for their own learning will become important to sustain learning across the organisation. Our current resourcing will not enable us to meet the continuously growing demand.

Will I be able to write enough to cover my task?

I have chosen this learning program because I know that there is the potential for enough research data to be gathered for a number of cohorts, and I can combine research on the participants with benefit for practitioners.

What experience do I have of this area?

I have made use of forums in my own study and introduced the idea to colleagues through an explorative example. This led to the introduction of forums in the context that I am researching, and I have been involved in an advisory and technical capacity since then.

Who else thinks this is worth exploring?

There is a slight difference of opinion amongst my colleagues as to the value of exploring this area further. Some of my colleagues however have expressed an interest in gaining insights into this area, and even those who don’t fully support it at least acknowledge their frustrations that the use of forums is problematic.

Creating the context for the proposal

Will it deliver on our personal goals?

Although it might meet the requirements of a modular assessment, it may not have a high enough ‘impact potential’ for further progression in some respects. The project should therefore be regarded as a stepping stone for future research efforts – my key learning points will be how successful the approach is in educating practitioners and giving further insights into a learning program. Once my methodology has been established I will then look for higher impact applications.

Does the topic have a political dimension?

Online education in general is an area of intense interest, along with the tools and media for achieving success. Political commentators and politicians may well have opinions on the area, but they are likely more interested in conventional learning matters at this time.

How much do we know about the area?

As stated previously, I have been involved with the use of online discussion for learning in a variety of contexts, both academic and work related. My initial literature review has uncovered some very useful information about how online communities can be investigated methodically, hence my approach to this project.

Identifying the research boundaries

Total amount of time available

In this case the time available is dictated by the time constraints of the assessment task, and my personal time for writing up my results.

What help can we expect from others?

The work does need to be primarily my own due to the assessment criteria, and the data collection should be fairly straightforward due to sending out questionnaires by e-mail. I do need to consider how much time it might take participants to answer my questionnaires, given that we will be approaching a time of higher workload.

Do we need specialist help?

No outside help will be necessary in this case since the data collection and interpretation is of my own devising, and I have sufficient access to journal articles at this time.

What if things don’t go according to plan?

I believe I have sufficient safety in terms of preserving data and the actual collection. My concern is with the possible response rate of participants within the time frame. For this reason I had already opted to research a program that has had multiple cohorts, and also the interactions have been digitally recorded due to the nature of the medium.

Assessing ethical issues

This has already been covered in the blog post on principles to some extent. As a summary, I will be carrying out my research within my workplace and sharing those results with colleagues. One potential issue could come from the sharing of participants’ responses to a wider audience, from a forum that was set up as a confidential area. For this reason I will need to ask colleagues about how to use their responses. I will treat responses anonymously by default, possibly with an option for their responses to not be quoted in reports. All postings on forums will remain confidential.

Preparing a case for a research programme

This research programme is for assessment purposes, but setting out my thinking will be of use in future, especially if I want to keep research as a positive influence on our evaluation strategy.

Getting an idea for research

From the sources of stimulus I have identified several different strands to my research idea & motivation:

1.   Literature

My review of the literature has revealed that there are a number of different theories cited by researchers, in some cases not including those that I have used as a theoretical basis for my own learning. Because of this I am wary of using quantitative methodology to tie results to a particular theory

2.   Policy

Ongoing interest in the field of online education interests me, particularly from the point of view of keeping practitioner education a key issue. Research that is carried out purely from a top-down institutional perspective is unlikely to produced sustainable benefits for the education system or society.

3.   Experience

Following on from previous points, my experience of online education is that it is most powerful as a constructivist environment, to support communities of inquiry and learning by doing.

Potential to finish research project

For this assessed research project I am confident that I can deliver the required results, and show that my methodology for design and data collection has been adequate. Questions about the impact of my research will need to be addressed for future efforts, since online learning projects will be focussed on areas of greatest business interest.

Arguing the case for the research

(a) Can the topic convince our judges?
The project should sufficiently convince my academic judges, although from a work perspective I will need to convince colleagues that the project has sufficient merit as a developmental tool for future work, and show that my approach to sharing results and insights can benefit the wider team.

(b) Is it feasible for me to undertake it?
I believe that this research project is well within my capabilities, and can be completed satisfactorily within the given time frame.

(i)  The field of enquiry
Research in this area is nothing particularly new, since there has been significant research into the field by academic researchers. Neither should this be considered as a theoretical study, although the context will always be different from others. The real interest here is how policies of using online education methods are succeeding against our objectives, and how the use of these approaches compares between academic origins and the actual implementation in the workplace.

(ii) The research issue / question
Related to the issues from previous questions, my key questions for research are based around the experiences of people participating in the use of new types of media, and ensuring that practitioner approaches can match the desire for business results.

Making sense of what we have

My interpretation of results may well draw on theory, although this will not be the primary lens. My main focus will be on judging whether activities have met objectives and how they can be modified for improvements, and also to identify what course of action should not be taken. My approach to truth here is realising how others perceive it, and using this to inform practice.

References

• Newby, Peter (2010).  Research Methods for Education Chapter 2 p.31-65

Creating the research questions

So how hard can it be to ask a question? Depends on how important the question is really! Newby (2010, Ch. 14) helpfully breaks down the different types of research and the types of questions that we might associate with them, to ensure that our data stays true to the purpose that we set out with. In particular, the Action Research school of though appeals to me, because it serves to educate the practitioners themselves. The method embraces uncertainty, and has an interesting parallel with the concept of Constructive Alignment (Biggs & Tang, 2007, Ch. 4), in that we can gain insight into whether or not participants are actually learning what we profess to be teaching them. For courses with emphasis on people management skills, and such a broad range of applications, this could be invaluable for determining success and continued development.

Creswell (2009, Ch. 7) helps to narrow down the research question strategy further for qualitative and quantitative approaches. In particular, he gives advice about the choice of language to ensure that our research questions serve their intended purpose, and ensure that answers aren't biased towards (or against) theories through a poor choice of words. If our purpose is to avoid referencing theories directly, this could help to ensure that we gain richer insights that aren't clouded by arguments about theories or hypotheses.

After an initial attempt at devising research questions where I realised I did the complete opposite of what Creswell suggests, I have refined my research questions to the following:

Primary:

• How would participants describe their experiences of using online interactions to support a face-to-face training event?

Secondary:

• Describe your experience of socialising with other participants who you interacted with online before the training event
• Describe your experience of preparing for a face-to-face event where online interaction was required, relative to an event with no prior interaction
• Describe your experience of participating in online interactions around course related content after the face-to-face event

References:

• Biggs, J. & Tang, C. (2007). Teaching for Quality Learning at University. 3rd Ed. Maidenhead: Open University Press.
• Creswell, J. W. (2009). Research Design: qualitative, quantitative and mixed methods approaches. (3rd edition) Sage.
• Newby, P. (2010). Research Methods for Education. Pearson.

Thoughts on defining research purpose

Trying to define my research purpose effectively has required extensive reading and critical thought to make sure that my purpose is properly defined

Chapter 5: The Introduction

The Research Problem – Creswell gives some good advice that relates general writing skills to the research context. We are normally trying to convey a very complex topic, so it is important that we consider how we are going to ‘hook’ our readers into the subject, and make sure we don’t bombard them with too much information too soon. There is also the need to make sure that our work is seen as a scholarly endeavour by outlining our links to the literature.

 

Studies Addressing the Problem – Using an inverted triangle seems like a good idea for the process of zeroing in on our research problem, using the wider body of literature to provide the background. We need to acknowledge that there are limitations in the literature we draw on (otherwise there wouldn’t be anything to research) and instead look for ways that the lessons learned or methods employed could inform our own research.

Deficiencies in Past Literature – Whilst avoiding outright condemnation of the existing literature, it is important to point out where it does not (or cannot) address our particular area. For instance, whilst I am planning to use Ke & Hoadley (2009) as a primary reference point for methods, it will be useful to explore the different contexts of the studies they included in their review - how many of the studies are based in the workplace? Referring to studies of learning in the workplace could help to draw links – thinking of the inverted triangle here.

Significance of a Study for Audiences – This does have some good points about writing for particular audiences, and trying to make the study appeal to a wide variety of people. Some of my original reservations remain around the default ‘market’ mentality!  Interestingly one of the sidebars mentions organisational mentalities, and may be worth following up in its own right.

Summary – The five part introduction model should serve well in conjunction with critical thought about my world view and the literature that I will use. The writing exercises will help me to set out my thinking ahead of writing the actual paper, and hopefully draw in some useful feedback. I will make sure that I give feedback to my peers first as an incentive for them to reciprocate, and to strengthen connections amongst the group.

 

Chapter 6: The Purpose Statement

 

Creswell states that this is the most important statement in the entire study – it effectively sets out the means by which others are actually going to be assessing the worth of your research.

 

Significance and Meaning of a Purpose Statement – The purpose statement is deliberately unpicked from the separate points of research problem (the wider issues) and research questions (the fine details of the study).

 

A Qualitative Purpose Statement – Careful use of language is required here. Using the draft script to produce some initial ideas and then going back through the text as a preliminary round of feedback sounds more sensible for learning than trying to construct one from the example words given. That way I can reflect on the differences between how I normally think internally, and the way I externalise this in my writing.

 

A Quantitative Purpose Statement – Although I consider that my study will be more qualitative in nature, it is worth considering the different way that a quantitative purpose statement should be written, so that I know the difference! Considering the variables that may affect an outcome and systematically investigating them is a hallmark of a quantitative study. Although I need to consider what factors may affect communities, I will not try to quantify them at this stage. Theory is implied as being more central to a quantitative study than a qualitative one.

 

A Mixed Method Purpose Statement – Understanding the previous two types of purpose is vital for properly introducing a mixed methods study, as the reader has to be able to distinguish between them through your writing. For success, there should be a central theory, and a single qualitative phenomenon under discussion.

 

Summary – Universal techniques underpin all of the types of study. Action words must be used for outlining the purpose, whilst keeping to non-directional language (i.e. avoid trying to prove a hypothesis through interfering with data acquisition). Mentioning the strategy for the enquiry, along with the participants and site to provide context, helps a reader further discern the relevance for them.

 

Influences on the selection of a research issue (Creswell)

Research agenda – Ultimately everything needs to be paid for! Reading through some of Creswell’s assumptions on the last unit led me to challenge the assumption that research has to be targeted at the greatest number of people. This actually fits more with a market philosophy, whereas I would rather that my research fitted into the network paradigm, and also be carried out for on-going practitioner development. My organisation wants more learning to go online, but there are likely to be many different perceptions of what does and does not work. I may actually be going against accepted wisdom with my desire to research this area!

 

Research issue – From my context, online communication actually becomes a potential measure for how successful a traditional learning event has been, as opposed to one where online interactions have not been present. The Research Problem can then be further defined as outlining effective ways of evaluating training success beyond the event, fitting in with the research agenda more closely.

 

Research question – Can online learning interactions increase the effectiveness of training programs, and when are they most likely to succeed?

 

Learning from a research agenda - This section gives some very good insights on how the research agenda can shift, and acknowledges the limitations of academic journals in our learning and research.

Determining a research issue - Generation of ideas, particularly questions, is dependent on having a rich and diverse environment.  In this way we are more likely to see alternative perspectives that allow us to challenge agendas and assumptions.

Kick-starting the search for a research issue - Whilst this is not the first research project that I have conducted, I do think that I am better regarding myself as being a ‘new’ researcher as I did not undertake any of this kind of learning before my research project – everything was already funded before I started! Within my department there is not always formal research, although my colleagues did use the Delphi research method for determining the best approach to performance & talent management. I also have the benefit of having colleagues who work in educational research to connect with.

 

Scoping the research problem - There are some interesting thoughts here. The reference to Mallik and McGowan about workplace learning is potentially very helpful, although some care will be needed for drawing similarities. How feasible any research will be in my area will sometimes depend on people’s willingness to provide feedback, which is not always guaranteed as people are busy in the workplace. I will have to be careful to make use of different methodologies, as I will have to continue researching towards my dissertation.

References:

• Creswell, J. W. (2009). Research Design: qualitative, quantitative and mixed methods approaches. (3rd edition) Sage.
• Ke, F. and Hoadley, C. (2009). Evaluating Online Learning Communities. Educational Technology Research and Development, 57(4), pp.487-510.
• Newby, P. (2010). Research Methods for Education. Pearson.