Five questions

Image: freeimages

I'm currently working through some questions for framing qualitative research (Mason, 2002), which I hope will help me to better frame my research.

The social 'reality': Your Ontological Perspective

What is the nature of the phenomena, or entities, or 'social reality' that I wish to investigate?

Elements from Table 1.1 that appeal:

• People, social actors, motivations, identities, cultural or social constructions
• Experiences, development, behaviours, interactions, social processes
• Institutions, markets, societies, organisation, connectedness, multiple realities or versions + tribes, networks

These are more relevant to my overall world-view, so I will generate an additional set specific to the current project:

• People, understandings, perceptions,attitudes, thought
• Experiences, development, actions, interactions, situations, rules
• The 'material', groups, organisation

Knowledge and Evidence: Your Epistemological Position

What might represent knowledge or evidence of the entities or social 'reality' that I wish to investigate?

Experiences of users will allow me to link my design approach to how people perceive the materials and support framework - affects my ability to rationally design or re-design materials and approaches in response to user feedback.

The research would benefit from some objective measures of performance to show that knowledge about systems and procedures has successfully transferred, rather than asking for user ratings - allows for genuine accountability in our training approach.

Your Broad Research Area

What topic, or broad substantive area, is the research concerned with?

This research topic is concerned with understanding what experiences examiners have when using online resources and support to adopt new software for marking and standardisation, and any different procedures that must be adopted. My focus is on interpreting results in such a way as to allow continuous improvement of the design of such materials, and to understand how much effect the strategy has on learning, as evidenced through performance. I do not have control over the precise methods for measuring performance, but am able to access such data.

Your Intellectual Puzzle and Your Research Questions

What is the intellectual puzzle?

How do I show that my learning intervention has had the desired impact, and how can I rationally design better approaches in future, or re-design to off-set any shortcomings?

What do I wish to explain or explore?

I wish to explain how the support approaches link to performance and attitudes among the examiners, and explore how to improve both in the future.

What type of puzzle is it?

This is a causal / predictive puzzle.

Your Research Questions

What are my research questions?

For gaining insights from examiners taking part in the live pilot I'm thinking along the lines of:

• Describe your experiences of using the software to carry out standardisation of marking
• Describe your experiences of using the software to record and submit your marks
• Describe your experiences of communicating with your senior examiner during the marking period

For improving on my online materials:

• Describe your experience of using the online resources to prepare you for marking
• Describe your experience of using the online resources and printable materials during the marking period

More of these to come - from previous experience I find that it's best to mull over these and look at what I've written down with a fresh perspective later.

Your Aims and Purpose

What is the purpose of my research? What am I doing it for?

The purpose of this research is to ensure the timely and accurate delivery of high quality marking for candidates sitting national examinations. The research will be done on behalf of my employer, to benefit examiners whilst delivering the marking and help to maintain motivation through consideration of the support offered. By achieving these objectives it is also expected that examiner performance will improve, which will be judged through the assessments from senior examiners and staff. The timeliness and accuracy of results could also be interrogated relative to expected deadlines and the number of examiners stopped or re-marked.

References

• Mason, J., 2002. Qualitative researching, London: SAGE Publications.

Architecture revisited

I'm going to take another look at the Learning Architect approach for my project, but this time from the perspective of staff offering support to examiner. Now this covers a wide variety of job roles, so I'll try to cover as many angles as I can, and think of how we might make best use of the different areas. This post will probably come across as a little scattered at the time of writing - it's primarily intended for some reflection after the event!

From the top down:

Experiential
Staff will naturally be taking part in performance appraisals so it will be worth considering how their involvement with the online marking will be judged - relating this to our cornerstone behaviours. Making sure that good ideas are credited will help to embed good behaviours.

On-demand
Producing performance support materials that relate to specific roles - these could be as simple as one-sheet reference guides or checklists for particular tasks that can be printed out. These will include the materials that are provided for examiners so that staff can advise examiners who are under pressure, and could also be expanded to include common problems that are encountered by the team leading the initiative.

Non-formal
Staff will have access to the same rapid e-learning as examiners via the LMS, although perhaps there may be a case for adding in some modules that relate to specific roles? This could also be covered in mini-workshops for staff - though possibly this is a continuous blur into on-job training. The main focus should be on discussing particular problems, with the knowledge transfer aspect left to e-learning to be looked at before the sessions.
Other possibilities are webinars to cover discussion of emergent issues, although this is probably more likely to be done on an individual basis and difficult to capture.

Formal
Classroom courses will undoubtedly be offered in some form - although these might actually be better described as mini-workshops. The opportunity for collaboration is limited, and the focus is likely to be on knowledge transfer, which is better left to the e-learning modules. Whether these are regarded as formal self-study or rapid is a debate for elsewhere!
Our main nod to formal learning should be to ensure that business goals are made explicit, and that staff know how their role relates to achieving them. Assessment will be based on accomplishing objectives, and nothing more!

From the bottom up:

Experiential
Encouraging personal reflection or reflection with others within or across work teams will help to ensure that lessons are learned well. This will be dependent on engaging effectively with line managers.

On-demand
The use of forums and wikis is one area where we might be able to advance the information sharing between colleagues, although there would probably need to be some moderation of comments and content to ensure accuracy, and many colleagues would probably prefer to stick to more formal channels that they are used to monitoring. The use of a wiki may be worth pursuing for future, but effective guidance would have to be in place, and this would have to take a back seat until other top-down measures are in place.

Non-formal
Nothing here for now

Formal

Nothing here for now

Overall, I believe the need for bottom-up learning is less immediate in this context - the tasks and goals are generally quite fixed, rather than fluid. Our key focus for bottom-up learning should centre around making tacit knowledge transfer more readily to explicit knowledge. This will stem primarily from the staff leading the project, but careful consideration of affected parties will help us know where best to direct our efforts.

Re-examining my worldview

Image: ariss via freeimages

Research Philosophy

How do I identify with four of the main worldview areas, as discussed by Newby?

Scientism and Positivism

This school of thought demands predictable cause and effect - not easy to establish where people are concerned. Once people realise that others are able to predict their behaviour in any way, they tend to change their behaviour to avoid manipulation! It may be beneficial to look for patterns if you have a critical interest, such as a business issue, although this may actually overlap with more humanistic traditions (see below) because you will be seeking a model that is 'fit for purpose' rather than trying to establish an objective truth.

This is the case for the research carried out by Meadows (2004) and Tremain (2011, 2012), where the critical factors affecting examiner retention and job satisfaction needed to be identified. Establishing an all-encompassing theory or truth (positivism) may be too ambitious - and would be clouded by personal interests - identifying and monitoring the relevant factors was achieved through careful selection of questions and interpretation of data.

Future reading: Discussion of the work of Karl Popper (Newby, Ch.3)

Humanism, Phenomenology and Existentialism

Whist positivism demands an objective external truth independent of human influence, humanism treats 'truth' as a social construction, where one culture's truth may not be another's. This ties well with my understanding of social evolution (Hobson, 2012; Ronfeldt, 1996, 2012a,b) whereby societies develop distinct cultures of varying complexity through the addition of different forms of organisation. Conflicts arise between or even within societies due to disagreements about how society should be ordered (or not) by hierarchical institutions and free market policies. Ronfeldt (2013) acknowledges that even the term 'tribe' in his T-I-M-N framework is frequently contested by others, and progresses the model by engaging with and incorporating these disagreements into his model.

Phenomenology focuses on individual and collective experiences to form a rational basis for future action, by probing the differences between 'perceived' and 'experienced' world. Methods include description observation, reporting and reflection.

Existentialism centres around seeking to understand the world from a personal perspective, driven by conviction and desire. Methods could extend to asking participants for other forms of 'data' such as pictures, videos and stories of their lives - anything that conveys their viewpoint in richer detail than a questionnaire could. Existentialism has been applied to curriculum design, by designing a curriculum that centres around self-discovery.

Critical theory

This theory is concerned with political beliefs, particularly those that are left-of-centre and seeking to change society by making people aware of their circumstances in order to liberate them. Research carried out within a critical theory framework seeks to observe and expose individual facts which can then be combined to form an argument for change.

Aside: I am particularly interested in the references to Karl Marx, as this led to a model for social evolution around the struggle for control of wealth - somewhat akin to Ronfeldt's description of the transition between biform (T+I) and triform (T+I+M) societies.

Applying critical theory to education has been used by academics to attack the present model for education as being primarily based around servicing the needs of the capitalist economy, and thus serves only to reinforce the inequalities that it inevitably generates. However positivists would undoubtedly attack this approach because it proposes a hypothesis and seeks to prove it.

Postmodernism

This approach distinguishes itself from the 'modernist' approaches by rejecting the 'modernist' assumption that there is a single explanation for things, which leads to a natural order. For education, this almost rejects the need for theories of education, since such theories are rooted in modernist approaches! This approach does overlap with my TIMN world-view, because it is by nature multi-layered, and acknowledges that 'people and organisations can play several and sometimes conflicting roles', in keeping with the different types of organisations and the interfaces between them.

Postmodernism bears a great deal of similarity to the network (+N) principles described by Ronfeldt, with researchers seeking to act more as nodes in a network, unpicking their assumptions and sharing data that reflects their local situation, with the ideal that the value of the network increases not simply with the number of nodes but the number of connections between them.

Future reading: Steven Johnson: Future Perfect 

Summary

My adoption to the T-I-M-N framework in my outlook leads me to cross several of the boundaries of research philosophy, but perhaps I identify more strongly than I realised with the post-modernist approach for rejecting the established order with its demanded polarisation of political outlooks.

References:

• Hobson, J. (2012). The problems with educating a quadriform society. Thinking and learning in the digital age [blog] 7 August. Available at: <http://jimmyhob.blogspot.co.uk/2012/08/the-problems-with-educating-quadriform.html>
• Meadows, M., 2004. Measures of examiner job satisfaction as predictors of intention to continue examining. Available at: https://cerp.aqa.org.uk/research-library/measures-examiner-job-satisfaction-predictors-intention-continue-examining [Accessed December 8, 2013].
• Ronfeldt, D. (1996). Tribes, Institutions, Markets, Networks – A Framework about Societal Evolution. Available at: <http://www.rand.org/pubs/papers/P7967.html>[Accessed September 2012]
• Ronfeldt, D. (2012a). TIMN in 20 minutes: a video presentation. Visions from two theories [blog] 23 May. Available at: <http://twotheories.blogspot.co.uk/2012_05_01_archive.html> [Accessed March 2013]
• Ronfeldt, D. (2012b). Q’s & A’s about “TIMN in 20 minutes” (4th of 7): creation of a new sector? Visions from two theories [blog] 29 August. Available at: <http://twotheories.blogspot.co.uk/2012/10/qs-as-about-timn-in-20-minutes-4th-of-7_29.html> [Accessed March 2013]
• Ronfeldt, D. (2013). Further points about “tribes” (T) — plus a new proposition about TIMN as a whole. Visions from two theories [blog] 10 April. Available at: <http://twotheories.blogspot.co.uk/2013/04/further-points-about-tribes-t-plus-new.html> [Accessed March 2013]
• Tremain, K., 2011. Carry on examining: what predicts examiners’ intentions to continue examining? Available at: https://cerp.aqa.org.uk/research-library/carry-examining-what-predicts-examiners-intentions-continue-examining [Accessed December 7, 2013].
• Tremain, K., 2012. Carry on examining: further investigation. Available at: https://cerp.aqa.org.uk/research-library/carry-examining-further-investigation [Accessed December 7, 2013].