Appendix 1
Activity 1
Analysis of research methodology,
AR2 on research methodology :
The collaborative editable document started by Andy Roberts and created collaboratively:
Action Research? - Navel gazing unworthy of the title research
(Note title can be attributed to AR - well known LF)
(Not AR - well known researcher then?)
(Nor AR the methodology!)
(Not AR the university!)
Who benefits from the Action Research?
1) The researcher. Practioner researchers in particular can benefit from
improvements in their practice.
2) Others around them who may then be drawn into to looking at their own
practice more reflectively may benefit.
3) Participants can benefit. Their voices heard, their experience of
changes given value. Ideally they may move towards co-researcher status.
(This can include pupils/students/clients whose opinions would not
normally be heard)
4) There is goverment money for action research in schools via the Network
Learning movement. This money can help schools fund new initiatives, pay
for staff training etc. so organisations may benefit.
5) Future pupils
6) The workplace benefits because of encouragement given to researcher, helps to create open dialogue and lessen 'us and them' scenarios
What are the advantages of AR over other methodologies?
1) Low requirement for resources.
- As well as being cheap, this also means in theory that a large number of
practioner/researchers may be able to make a greater improvement on the
industry as a whole than a few large scale research projects conducted by
external professionals for the same cost.
but can be very time consuming which is also a resource!
2) It's a powerful technique for consciously improving own practice If
chosen by researcher and not imposed it is usually a subject of great
interest to them and in which they are deeply involved.
3) It relies mostly on qualitative data which gives a richer picture of
complex situations again referring to the above - qualitative data can
take more time to analyse.
4) The spiral nature of the research means that this complex situation is
constantly re-visited and reassesed.
5) It deals with 'real life' situations/problems and attempts improvement
rather than study of them.
6) Less intrusive - because the researcher is known by and working amongst
the people being researched rather than observing them as an outsider.
Could be more prone to bias because of this.
7) By focussing on small groups or individuals, the researcher can gain a
deeper insight into the experience from others point of view than they
would by collecting large volumes of more superficial data from surveys.
8) It allows the researcher to critically reflect on actions in a non-threatening way as reflection tends to be on a personal level and changes implemented are worked on at a manageable level.
And the disadvantages...over other methodologies
1) Results are not taken seriously by anybody except the researcher. Less
support offered too.
2) Researchers can only look at things where changes are within their
locus of control. This means that researchers who have little or no
control in their workplace are heavily disadvantaged. So whereas a TA
might usefully evaluate some aspect of the school reading policy and
report back to the rest of the staff on their findings they may not be in
a position to effect changes in that policy.
3) It may be seen as never having an endpoint, with constant ‘review and
refine’ occurring.
4) It relies mostly on qualitative data which cannot be measured.
5) Action researchers usually have full time jobs as well.
6) It's impossible to eliminate personal bias in the action researchers'
interpretations.
Ethical issues - disadvantages
Who may be harmed?
1)" It's a powerful technique for consciously improving own practice"
Exactly that. It is a powerful technique. It is possible that it could be
used by the powerful to silence those at the bottom of the heap. This is
why there has to be the constant emphasis on making room for the voices of
all affected by the changes implemented. Researchers need to be open to
questioning their own assumptions and to others questioning them. There
needs to be a close examination of the governing variables.
2) People who just want to get on with their lives without having some
busybody with a clipboard watching them and putting them off.
Hmm - I think this applies to any form of research that involves people
rather than paper trails and might even apply less to AR where at least
you should have some chance to give your views or decide not to be
involved. So does that needs to be added under advantages then...?
3)There is often some group or individual that may be disenfranchised when routine is looked at and perceived as threatened. Necessary to be ultra-sensitive to format of questions and presentation of data.
The features of action research in the CRASP model developed by
Zuber-Skeritt (1992).
Action research is a:
Critical collaborative enquiry by
Reflective practitioners being
Accountable and making the results of their enquiry public,
Self-evaluating their practice and engaged in
Participative problem solving and continuing professional development. Did
you mean to leave out the capital S???????????
Worthy sounding
Ordinary ideas
Randomly thrown together in a
Disorganised way intended to
Sound impressive!
Zuber-Skerrit, O. (1992). Action Research in Higher Education: Examples &
Reflections. London: Kogan Page Ltd.
as cited by
Regina Hatten, Donna Knapp and Ruth Salonga, 1997, 2000 Action Research:
Comparison with the Concepts of ‘The Reflective Practitioner'
I tried to edit the doc but it wouldn't let me.
It did you know :-)
I'm just wondering how best to use the ideas in here - do they belong in
the methodolgy section somehow?
: you might use some of it in the justification
A conversation in the Beehive led me to confirm with my LF the correct place for my discussion of AE literature -
Does this mean I am right to include both action research literature and topic lit in this review? Several well meaning peer reviewers think not. I was happily exploring Dick and McNiff in some depth untill it was pointed out to me tonight that this review is supposed to be only on the research topic.
Some guidance would be much appreciated - thanks
(posted to my LF)
Notes on literature read
Quantitative research
Cook and Reichard (1979)
Defined differences between qualitative and quantitative research but then concluded that these underpinning world view differences present a false dichotomy. People could include elements quantitative research as part of an inductive approach, a process orientated view could be added to a study with an underpinning positivistic world view
It is more helpful, then, to look at the difference as being between the use of numbers or statistics in quantitative and concepts or categories in qualitative.
Paton (CH1-2) identifies 3 methods of qualitative data collection, the interview (open, not just a read questionnaire), direct observation (includes use of video), analysis of written documents (includes words and visual data)
These lead to 3 kinds of data, quotes, descriptions and extracts. These produce 1 final product, a narrative description of a complex situation. This description may include charts and diagrams.
Strengths of qualitative include the depth of detailed data, openness to emergent phenomena and may give a deeper understanding of the world view of participants. Weaknesses include the small scale of samples makes it hard to generalise, uniqueness of situations makes it difficult to do systematic comparisons. Also the skills of the researcher assume a more obvious role in the quality of the study.
Paton identifies 10 themes of qualitative research. Many of these resonate for the action researcher too. Paton describes a flexible process of allowing almost all the aspects of the research to emerge from the data, of categorising and re-categorising data as new insights are gained, of immersion followed by withdrawal and reflection. He also recommends the use of multiple data collection methods including quantitative so long as they add to the overall picture of the situation. All of these may apply in an action research study.
The need to explain sampling - why and how was it done. Can be emergent, may be different for the action researcher. In my study there is a need for willing participants. I will therefore have to explain my choice of participants in some detail.
Data Collection
Observations.
Descriptions should be concrete, no inferences, no vague terms. Use simple language and avoid quasi- scientific terminology. Use details, even if they seem irrelevant at the time. Get down the obvious but do not include evaluations or generalisations in the body of the observation - may be in the margin, especially feelings, insights etc. These can have value later but don't belong in the actual observation. Take still photos and record audio descriptions at time if possible, or use video. Describe everything - if nothing is happening describe physical surroundings. Notes written up later should be reflective rather than actual observations.
Interviews
The best interview questions may come out of observations
Note on Grounded Theory from Dick
Dick on grounded theory and the treatment of literature as another form of data.
Notes from Robson re data analysis
Robson considers at some length the different kinds of data analysis available to the qualitative researcher but makes almost no mention of the place of data analysis is Action Research beyond stating that one phase involves the application of research techniques to the situation.
Notes
Counters to 4 'Myths' in action research
positivism not ruled out by approach
use of quantitative methods in data analysis valid
No need for a fixed research design
easier because of issues of access /control and data collection/analysis
McNiff -- great quote re teacher attitudes to research - might be important
Teachers as educational theorists: transforming epistemological hegemonies
A paper presented at the British Educational Research Association 2005 Annual Conference at the University of Glamorgan on the 16th September 2005
One set of reasons could be in relation to how practitioners have come to believe popular discourses that they are not capable of doing research and therefore frequently adopt a defensive attitude that regards research as above their heads and an esoteric and rather useless activity, something that other people do to them as objects of study. By developing their own discourses of derision (Ball 1990), about the importance and significance of research and theory, teachers often protect their own ontological identity as practitioners
When I wrote my research proposal I thought I might need to justify the use of both quantitative as well as qualitative data. I had intended to explore the work of Mann and Stuart (2000) and their belief that the internet provided an opportunity for the development of a new research paradigm. As I have read more deeply about action research however it has become clearer to me that quantitative data has an accepted place within the action research framework and can be used to provide triangulation when it is used in conjunction with qualitative data.
Activity 3 Pilot exhibition together with review
Discussions
Chat with other members of Beehive 10/11/05
Linda Hartley[10:33:26 pm]: seriously considering giving up on my topic, can't get any response from people - beginging to thing it was a rubbbish idea
Eve Thirkle[10:39:06 pm]: How is everyone's work coming along?
Linda Hartley[10:40:32 pm]: mine isn't - no movement this week at all. I've had one log sheet - and people have started hiding from me again. Nothing back from the people I'd asked to help on-line either
Eve Thirkle[10:41:00 pm]: Are you doing your pilot this week then?
Linda Hartley[10:41:03 pm]: so nothing to put in my pilot exhibition which is supposed to be a week on Monday
Eve Thirkle[10:41:17 pm]: Ah - data collection this week ...
Linda Hartley[10:41:52 pm]: No last week but - no much response yet. Should have analysed the data this week
Linda Hartley[10:42:03 pm]: but nothing to analyse
Eve Thirkle[10:42:07 pm]: oops
Eve Thirkle[10:42:43 pm]: - so at least you can say that you need to look at this lag in response and allow for it in the main feedback?
Linda Hartley[10:43:20 pm]: Lag? No these are people who are now saying 'sorry - I don't have time after-all'
Eve Thirkle[10:43:33 pm]: ARGGHHHH!
Eve Thirkle[10:44:24 pm]: So - is your log sheet too complex? or did they not know exactly what was involved? or did they say 'yes' to keep you happy ...
Eve Thirkle[10:45:25 pm]: BTW - is it Ok to put an abbreviated summary of our threeway conversation on 'change of topic' into AR2?
Linda Hartley[10:45:38 pm]: sure no probs for me
Eve Thirkle[10:46:18 pm]: well - you can analyse this data of non-cooperation ... and use it somehow ...?
Andy Roberts[10:46:29 pm]: Fine with me, Eve
Andy Roberts[10:46:43 pm]: Good use of private community first.
Linda Hartley[10:47:27 pm]: I need to have a big re-think
Eve Thirkle[10:47:59 pm]: Definitely! Good idea having the beehive - BTW - what are we doing re conversations in there that are relevant to the evidence of online conversations - modify and remove any personal references ..?
Linda Hartley[10:48:59 pm]: I've mostly just quoted them
Eve Thirkle[10:49:16 pm]: Linda - you're looking at resources - can you look at yourself as a resource? How does what you do help with resources ...?
Linda Hartley[10:49:29 pm]: feel free to attribute mine
Linda Hartley[10:49:39 pm]: it doesn't
Linda Hartley[10:50:56 pm]: they use me like a resource rather than a team member, the resources I've created on-line are really not used by them at all. Trying to involve them was a daft idea
Andy Roberts[10:51:01 pm]: Sounds like you might be pushed further away from teh school Linda
Linda Hartley[10:51:42 pm]: but no one else is interested either - except a mental health nurse tutor in Wales that I met on Flickr
Linda Hartley[10:52:48 pm]: the other responses I got on Flickr haven't replied to my e-mails
Eve Thirkle[10:53:44 pm]: so ... 'how do you use this?' is what I'm thinking ... you've said that your research design will be flexible and emergent ... how can you utilise the lack of responses as you review this pilot as part of your first cycle - playing advocate here BTW ...
Andy Roberts[10:53:48 pm]: How about the use of Furls by teachers
Eve Thirkle[10:55:21 pm]: Can any of your people in your research network be of help to you?
Linda Hartley[10:56:05 pm]: the pilot data collection was to form part of the planning and it's a very good job I tried it out otherwise I really would be stuck
Linda Hartley[10:57:50 pm]: schools are busy places at this time of year - everyone is near breaking point already and determined not to take on anything else.
Activity 4
LJ extracts /reflections/43 Things
From my LJ:
reflection on chat with Eve and Andy :
What?
Only one log sheet has been returned and no photos. I have had no response on-line at all. My pilot exhibition is supposed to be next Friday.
So what?
I have no new data to exhibit.
Now what?
need to work out what I have got to put in my exhibition :
Use the questions for headings on the board
What is the purpose of the exhibition?
1. The idea of the main workplace exhibition is to plan a 'change strategy' and maximise impact in the workplace. Therefore the pilot needs to start that process. I need to pilot my display ideas to check impact.
2. The data collection method used to collect the first bit of data has been piloted and although there was only small amount of data it is still seen as valid in a qualitative study. This will allow S and any other participants to respond to my interpretation of his response.
3. I need to trial my feedback collection for the main exhibition.
What?
Printed and backed to go on display board.
1 screen shot of links page
4 screen shots of Classroom Displays Archive
2 screen shots of the Hansel and Gretel page
3 Screen shots of Practical Literacy
The urls
2 quotes from literature
2 quotes about H&G
So what?
• photo of log-sheets
• 2 findings diagrams
Now What?
• single sentence explanations: blog with comments, wiki,
• some quotes about web 2.0 - connectivism? Wils?
• screen shots of teacher's blogs with comments
• screen shots of behaviour management wiki pages plus explanation
Diagram of action research spiral
Short paragraph of explanation of process
Screen shot On-line pilot Exhibition with explanation of comments space.
Something with the urls on it for people to take away.
DO I GIVE A PRESENTATION?
Feedback data collection
Interviews with S, P and L on day
Evaluation style feedback sheets to leave on table
Informal observations - notes in LJ and on audio
Conversations/ focus group.
Take photos of exhibition
Photos of people looking at it
On-line exhibition
3 linked web pages based on the 'what' questions - content much the same as physical one but with working links to resources, blog posts, etc.
Also Slide show with audio of same material
links to web notes page
The blog page acts as a portal to them and a place to give feedback
Ideas for pilot
Criteria
• needs to be very small - enough to fit on a blackboard or 2/3 web pages
• a way of trying out both data collection methods and presentations methods
The main thing I need to trial is the observation techniques
1. the photos
2. the log sheet
Can I therefore ask for a couple volunteers to try these out, looking only at Classroom Displays group on Flickr, and then present the results prior to doing it with people face to face?
I could manipulate the photos in photoshop and use them as part of the display
Do a short talk about the classroom displays archive on Flickr for face to face.
Make a short video about it - with audio for the web
Explain the pilot study's purpose.
Display common themes from the logs both about the experience of working on-line and the reactions to the resource.
Use some quotes about the importance of observation in qualitative research. (Mapping the territory)
Use some screen shots of the group
Use this as a way of tightening up log sheets.
Use the interviews as a way of rehearsing for the AE
Take a few stills as a way of recording exhibition
On-line Space
Got to use something with a blog and a wiki, ideally with tagging facility. Perfect if also allows RSS
Could mean ELEG
Use Andy's DAR. net blog
use Ultraversity wiki pages
use this blog also for the pilot - use a password protected page so limiting access
On-line audience
• Who are they?
• Where are they?
• How can they be encouraged to give feedback?
• Can I interview some of them?
• Collect stats?
• Use an on-line questionnaire?
Physical Space
• marketing of the exhibition/creating interest
• the audience
• the use of visual displays
• the presentation of the information
• my presentation?
• the ambience
• the catering
Audience/potential participants
people I know personally:
teachers in school
TAs
Learning mentors
teachers in the NEW network
people I know who teach
people i know who want to teach
LEA advisors
other head teachers
children in my school
People from on-line interactions
people on usenet teacher list
people on Flickr already
people from other on-line lists (Ray?)
people who are married to people I know on-line
teachers who blog
People who I don't know
teaching students at St Martins
the chap from St Martins on ELLG
education students at Lancaster Uni
students at Charlotte Mason
students at other institutions
Feedback sheet
what are 'change strategies'?
Approaches (strategies) to bring about change. For example, addressing managerial issues such as delegation, or professional development that could aim to upskill individuals or highlight individuals beliefs and values and how they might support or work against change.
This seems to mostly apply to managers or people in a position of power with the ability to implement change. The only literature I've been able to find that relates to it is aimed at management. The stuff talks about systemic organisational change (more Senge - groan) and how management can facilitate it.
Thing is my research is aimed at improving my practice not that of the audience of my pilot. They just share a work context with me not a job role. I'm not aiming to change their practice. I suppose it's possible that as McNiff suggests being around someone who is using reflective practice might influence others in the same context but that's hardly the sort of 'big bang' effect. I just don't see even the main exhibition having that sort of impact.
I would just skirt round this but it is one of the Learning Outcomes.
This means, I think that in my evaluation of the pilot exhibition I have to consider how effective it has been as a change strategy. SO I suppose I just say "Not at all" and then justify why it's unreasonable to expect it would be....
(from a posting to FirstClass )
Reflection on lack of formal feedback
From my Meta-Learning journal:
Well, I did my pilot exhibtion and they came and they ate my cake and they looked like they were listening and they clapped at the end. And I asked some questions, and they asked some questions. And only now do I realise, a week later, that I never got any answers.
So I have no feed back - none, as in, not any. I can cross structured focus groups off my list of data collection methods, and evaluation sheets too. Oh, and individual interviews of both a structured and unstructured nature, and informal conversations as no-one likes to be quoted when they are just chatting, so what's left?
Nothing much.
Conclusions?
Two main ones so far
1. I am rubbish at this - this seems to be S's favoured position. He thinks I must have sounded like an expert and they were intimidated into silence by me.
Does he realise these people are mostly teachers and I am a TA? They are not intimidated by me..... I think. I mean, but maybe he's right. How the hell should I know how I sounded? They won't tell me and I'm way too embarrassed to ever let anyone hear the audio recording.
It didn't feel natural standing up in front of them telling them stuff. It doesn't fit my prefered pedagogy. I'd rather have got people doing something real but that wasn't possible.
Pesonally I incline more to the belief that I bored them into a sort of trance.
2. I was talking to the wrong people. Somewhere - out there there are people who might be interested in what I'm doing. But not in my workplace - which is a kicker when this is a workplace based degree.
I left work having been close to tears all day. I knew this. I knew they weren't interested, so why does it hurt so much
Comments:
Posted by E @ 12/05/2005 12:02 PM PST
Haven't got much energy to write much - don't know if you watch the 'X' factor but Chico, who was voted off, said somethiing quite profound on Saturday - along the lines of :
Some people cry when something's finished but I rejoice because it's happened -
sending you some hugs (((Linda)))and hope you can find something to rejoice in...
Posted by Richard Millwood @ 11/29/2005 11:55 AM PST
I am quite certain (in a kind of religious way!) that you are not rubbish. I have spoken publicly, as a teacher, as a lecturer, as a governor, as a keynote speaker many, many times in my life. Always I have been uncertain about impact and rarely has anyone given me helpful feedback. So what's left as an evaluation? Well your own view is no bad thing. Also, have you seen any change in subsequent days, in conversations or actions on the part of your colleagues? It may be that you need to keep your eyes peeled for signs for some time to come! I do wish you all the best in this - impact is often a slow burn, but I know it wasn't a wreck.
Posted by Viv @ 11/28/2005 11:59 PM PST
Poor you Linda - don't be too downhearted. I fear that i may suffer the same fate next term. I don't know what year three entails or how exactly i will'exhibit' but I'm not looking forward to it. My colleagues are 'Dr's' and 'Prof's.' and a degree to them is yesterday's news - like GCSE's are to me.
I have no answers, sorry- but at the end of the day you've tried and maybe you can use your 'negative - ish' experiences to your advantage
somehow in the write-up?
Chin up:-)
Activity 1
Analysis of research methodology,
AR2 on research methodology :
The collaborative editable document started by Andy Roberts and created collaboratively:
Action Research? - Navel gazing unworthy of the title research
(Note title can be attributed to AR - well known LF)
(Not AR - well known researcher then?)
(Nor AR the methodology!)
(Not AR the university!)
Who benefits from the Action Research?
1) The researcher. Practioner researchers in particular can benefit from
improvements in their practice.
2) Others around them who may then be drawn into to looking at their own
practice more reflectively may benefit.
3) Participants can benefit. Their voices heard, their experience of
changes given value. Ideally they may move towards co-researcher status.
(This can include pupils/students/clients whose opinions would not
normally be heard)
4) There is goverment money for action research in schools via the Network
Learning movement. This money can help schools fund new initiatives, pay
for staff training etc. so organisations may benefit.
5) Future pupils
6) The workplace benefits because of encouragement given to researcher, helps to create open dialogue and lessen 'us and them' scenarios
What are the advantages of AR over other methodologies?
1) Low requirement for resources.
- As well as being cheap, this also means in theory that a large number of
practioner/researchers may be able to make a greater improvement on the
industry as a whole than a few large scale research projects conducted by
external professionals for the same cost.
but can be very time consuming which is also a resource!
2) It's a powerful technique for consciously improving own practice If
chosen by researcher and not imposed it is usually a subject of great
interest to them and in which they are deeply involved.
3) It relies mostly on qualitative data which gives a richer picture of
complex situations again referring to the above - qualitative data can
take more time to analyse.
4) The spiral nature of the research means that this complex situation is
constantly re-visited and reassesed.
5) It deals with 'real life' situations/problems and attempts improvement
rather than study of them.
6) Less intrusive - because the researcher is known by and working amongst
the people being researched rather than observing them as an outsider.
Could be more prone to bias because of this.
7) By focussing on small groups or individuals, the researcher can gain a
deeper insight into the experience from others point of view than they
would by collecting large volumes of more superficial data from surveys.
8) It allows the researcher to critically reflect on actions in a non-threatening way as reflection tends to be on a personal level and changes implemented are worked on at a manageable level.
And the disadvantages...over other methodologies
1) Results are not taken seriously by anybody except the researcher. Less
support offered too.
2) Researchers can only look at things where changes are within their
locus of control. This means that researchers who have little or no
control in their workplace are heavily disadvantaged. So whereas a TA
might usefully evaluate some aspect of the school reading policy and
report back to the rest of the staff on their findings they may not be in
a position to effect changes in that policy.
3) It may be seen as never having an endpoint, with constant ‘review and
refine’ occurring.
4) It relies mostly on qualitative data which cannot be measured.
5) Action researchers usually have full time jobs as well.
6) It's impossible to eliminate personal bias in the action researchers'
interpretations.
Ethical issues - disadvantages
Who may be harmed?
1)" It's a powerful technique for consciously improving own practice"
Exactly that. It is a powerful technique. It is possible that it could be
used by the powerful to silence those at the bottom of the heap. This is
why there has to be the constant emphasis on making room for the voices of
all affected by the changes implemented. Researchers need to be open to
questioning their own assumptions and to others questioning them. There
needs to be a close examination of the governing variables.
2) People who just want to get on with their lives without having some
busybody with a clipboard watching them and putting them off.
Hmm - I think this applies to any form of research that involves people
rather than paper trails and might even apply less to AR where at least
you should have some chance to give your views or decide not to be
involved. So does that needs to be added under advantages then...?
3)There is often some group or individual that may be disenfranchised when routine is looked at and perceived as threatened. Necessary to be ultra-sensitive to format of questions and presentation of data.
The features of action research in the CRASP model developed by
Zuber-Skeritt (1992).
Action research is a:
Critical collaborative enquiry by
Reflective practitioners being
Accountable and making the results of their enquiry public,
Self-evaluating their practice and engaged in
Participative problem solving and continuing professional development. Did
you mean to leave out the capital S???????????
Worthy sounding
Ordinary ideas
Randomly thrown together in a
Disorganised way intended to
Sound impressive!
Zuber-Skerrit, O. (1992). Action Research in Higher Education: Examples &
Reflections. London: Kogan Page Ltd.
as cited by
Regina Hatten, Donna Knapp and Ruth Salonga, 1997, 2000 Action Research:
Comparison with the Concepts of ‘The Reflective Practitioner'
I tried to edit the doc but it wouldn't let me.
It did you know :-)
I'm just wondering how best to use the ideas in here - do they belong in
the methodolgy section somehow?
: you might use some of it in the justification
A conversation in the Beehive led me to confirm with my LF the correct place for my discussion of AE literature -
Does this mean I am right to include both action research literature and topic lit in this review? Several well meaning peer reviewers think not. I was happily exploring Dick and McNiff in some depth untill it was pointed out to me tonight that this review is supposed to be only on the research topic.
Some guidance would be much appreciated - thanks
(posted to my LF)
Notes on literature read
Quantitative research
Cook and Reichard (1979)
Defined differences between qualitative and quantitative research but then concluded that these underpinning world view differences present a false dichotomy. People could include elements quantitative research as part of an inductive approach, a process orientated view could be added to a study with an underpinning positivistic world view
It is more helpful, then, to look at the difference as being between the use of numbers or statistics in quantitative and concepts or categories in qualitative.
Paton (CH1-2) identifies 3 methods of qualitative data collection, the interview (open, not just a read questionnaire), direct observation (includes use of video), analysis of written documents (includes words and visual data)
These lead to 3 kinds of data, quotes, descriptions and extracts. These produce 1 final product, a narrative description of a complex situation. This description may include charts and diagrams.
Strengths of qualitative include the depth of detailed data, openness to emergent phenomena and may give a deeper understanding of the world view of participants. Weaknesses include the small scale of samples makes it hard to generalise, uniqueness of situations makes it difficult to do systematic comparisons. Also the skills of the researcher assume a more obvious role in the quality of the study.
Paton identifies 10 themes of qualitative research. Many of these resonate for the action researcher too. Paton describes a flexible process of allowing almost all the aspects of the research to emerge from the data, of categorising and re-categorising data as new insights are gained, of immersion followed by withdrawal and reflection. He also recommends the use of multiple data collection methods including quantitative so long as they add to the overall picture of the situation. All of these may apply in an action research study.
The need to explain sampling - why and how was it done. Can be emergent, may be different for the action researcher. In my study there is a need for willing participants. I will therefore have to explain my choice of participants in some detail.
Data Collection
Observations.
Descriptions should be concrete, no inferences, no vague terms. Use simple language and avoid quasi- scientific terminology. Use details, even if they seem irrelevant at the time. Get down the obvious but do not include evaluations or generalisations in the body of the observation - may be in the margin, especially feelings, insights etc. These can have value later but don't belong in the actual observation. Take still photos and record audio descriptions at time if possible, or use video. Describe everything - if nothing is happening describe physical surroundings. Notes written up later should be reflective rather than actual observations.
Interviews
The best interview questions may come out of observations
Note on Grounded Theory from Dick
Dick on grounded theory and the treatment of literature as another form of data.
Notes from Robson re data analysis
Robson considers at some length the different kinds of data analysis available to the qualitative researcher but makes almost no mention of the place of data analysis is Action Research beyond stating that one phase involves the application of research techniques to the situation.
Notes
Counters to 4 'Myths' in action research
positivism not ruled out by approach
use of quantitative methods in data analysis valid
No need for a fixed research design
easier because of issues of access /control and data collection/analysis
McNiff -- great quote re teacher attitudes to research - might be important
Teachers as educational theorists: transforming epistemological hegemonies
A paper presented at the British Educational Research Association 2005 Annual Conference at the University of Glamorgan on the 16th September 2005
One set of reasons could be in relation to how practitioners have come to believe popular discourses that they are not capable of doing research and therefore frequently adopt a defensive attitude that regards research as above their heads and an esoteric and rather useless activity, something that other people do to them as objects of study. By developing their own discourses of derision (Ball 1990), about the importance and significance of research and theory, teachers often protect their own ontological identity as practitioners
When I wrote my research proposal I thought I might need to justify the use of both quantitative as well as qualitative data. I had intended to explore the work of Mann and Stuart (2000) and their belief that the internet provided an opportunity for the development of a new research paradigm. As I have read more deeply about action research however it has become clearer to me that quantitative data has an accepted place within the action research framework and can be used to provide triangulation when it is used in conjunction with qualitative data.
Activity 3 Pilot exhibition together with review
Discussions
Chat with other members of Beehive 10/11/05
Linda Hartley[10:33:26 pm]: seriously considering giving up on my topic, can't get any response from people - beginging to thing it was a rubbbish idea
Eve Thirkle[10:39:06 pm]: How is everyone's work coming along?
Linda Hartley[10:40:32 pm]: mine isn't - no movement this week at all. I've had one log sheet - and people have started hiding from me again. Nothing back from the people I'd asked to help on-line either
Eve Thirkle[10:41:00 pm]: Are you doing your pilot this week then?
Linda Hartley[10:41:03 pm]: so nothing to put in my pilot exhibition which is supposed to be a week on Monday
Eve Thirkle[10:41:17 pm]: Ah - data collection this week ...
Linda Hartley[10:41:52 pm]: No last week but - no much response yet. Should have analysed the data this week
Linda Hartley[10:42:03 pm]: but nothing to analyse
Eve Thirkle[10:42:07 pm]: oops
Eve Thirkle[10:42:43 pm]: - so at least you can say that you need to look at this lag in response and allow for it in the main feedback?
Linda Hartley[10:43:20 pm]: Lag? No these are people who are now saying 'sorry - I don't have time after-all'
Eve Thirkle[10:43:33 pm]: ARGGHHHH!
Eve Thirkle[10:44:24 pm]: So - is your log sheet too complex? or did they not know exactly what was involved? or did they say 'yes' to keep you happy ...
Eve Thirkle[10:45:25 pm]: BTW - is it Ok to put an abbreviated summary of our threeway conversation on 'change of topic' into AR2?
Linda Hartley[10:45:38 pm]: sure no probs for me
Eve Thirkle[10:46:18 pm]: well - you can analyse this data of non-cooperation ... and use it somehow ...?
Andy Roberts[10:46:29 pm]: Fine with me, Eve
Andy Roberts[10:46:43 pm]: Good use of private community first.
Linda Hartley[10:47:27 pm]: I need to have a big re-think
Eve Thirkle[10:47:59 pm]: Definitely! Good idea having the beehive - BTW - what are we doing re conversations in there that are relevant to the evidence of online conversations - modify and remove any personal references ..?
Linda Hartley[10:48:59 pm]: I've mostly just quoted them
Eve Thirkle[10:49:16 pm]: Linda - you're looking at resources - can you look at yourself as a resource? How does what you do help with resources ...?
Linda Hartley[10:49:29 pm]: feel free to attribute mine
Linda Hartley[10:49:39 pm]: it doesn't
Linda Hartley[10:50:56 pm]: they use me like a resource rather than a team member, the resources I've created on-line are really not used by them at all. Trying to involve them was a daft idea
Andy Roberts[10:51:01 pm]: Sounds like you might be pushed further away from teh school Linda
Linda Hartley[10:51:42 pm]: but no one else is interested either - except a mental health nurse tutor in Wales that I met on Flickr
Linda Hartley[10:52:48 pm]: the other responses I got on Flickr haven't replied to my e-mails
Eve Thirkle[10:53:44 pm]: so ... 'how do you use this?' is what I'm thinking ... you've said that your research design will be flexible and emergent ... how can you utilise the lack of responses as you review this pilot as part of your first cycle - playing advocate here BTW ...
Andy Roberts[10:53:48 pm]: How about the use of Furls by teachers
Eve Thirkle[10:55:21 pm]: Can any of your people in your research network be of help to you?
Linda Hartley[10:56:05 pm]: the pilot data collection was to form part of the planning and it's a very good job I tried it out otherwise I really would be stuck
Linda Hartley[10:57:50 pm]: schools are busy places at this time of year - everyone is near breaking point already and determined not to take on anything else.
Activity 4
LJ extracts /reflections/43 Things
From my LJ:
reflection on chat with Eve and Andy :
What?
Only one log sheet has been returned and no photos. I have had no response on-line at all. My pilot exhibition is supposed to be next Friday.
So what?
I have no new data to exhibit.
Now what?
need to work out what I have got to put in my exhibition :
Use the questions for headings on the board
What is the purpose of the exhibition?
1. The idea of the main workplace exhibition is to plan a 'change strategy' and maximise impact in the workplace. Therefore the pilot needs to start that process. I need to pilot my display ideas to check impact.
2. The data collection method used to collect the first bit of data has been piloted and although there was only small amount of data it is still seen as valid in a qualitative study. This will allow S and any other participants to respond to my interpretation of his response.
3. I need to trial my feedback collection for the main exhibition.
What?
Printed and backed to go on display board.
1 screen shot of links page
4 screen shots of Classroom Displays Archive
2 screen shots of the Hansel and Gretel page
3 Screen shots of Practical Literacy
The urls
2 quotes from literature
2 quotes about H&G
So what?
• photo of log-sheets
• 2 findings diagrams
Now What?
• single sentence explanations: blog with comments, wiki,
• some quotes about web 2.0 - connectivism? Wils?
• screen shots of teacher's blogs with comments
• screen shots of behaviour management wiki pages plus explanation
Diagram of action research spiral
Short paragraph of explanation of process
Screen shot On-line pilot Exhibition with explanation of comments space.
Something with the urls on it for people to take away.
DO I GIVE A PRESENTATION?
Feedback data collection
Interviews with S, P and L on day
Evaluation style feedback sheets to leave on table
Informal observations - notes in LJ and on audio
Conversations/ focus group.
Take photos of exhibition
Photos of people looking at it
On-line exhibition
3 linked web pages based on the 'what' questions - content much the same as physical one but with working links to resources, blog posts, etc.
Also Slide show with audio of same material
links to web notes page
The blog page acts as a portal to them and a place to give feedback
Ideas for pilot
Criteria
• needs to be very small - enough to fit on a blackboard or 2/3 web pages
• a way of trying out both data collection methods and presentations methods
The main thing I need to trial is the observation techniques
1. the photos
2. the log sheet
Can I therefore ask for a couple volunteers to try these out, looking only at Classroom Displays group on Flickr, and then present the results prior to doing it with people face to face?
I could manipulate the photos in photoshop and use them as part of the display
Do a short talk about the classroom displays archive on Flickr for face to face.
Make a short video about it - with audio for the web
Explain the pilot study's purpose.
Display common themes from the logs both about the experience of working on-line and the reactions to the resource.
Use some quotes about the importance of observation in qualitative research. (Mapping the territory)
Use some screen shots of the group
Use this as a way of tightening up log sheets.
Use the interviews as a way of rehearsing for the AE
Take a few stills as a way of recording exhibition
On-line Space
Got to use something with a blog and a wiki, ideally with tagging facility. Perfect if also allows RSS
Could mean ELEG
Use Andy's DAR. net blog
use Ultraversity wiki pages
use this blog also for the pilot - use a password protected page so limiting access
On-line audience
• Who are they?
• Where are they?
• How can they be encouraged to give feedback?
• Can I interview some of them?
• Collect stats?
• Use an on-line questionnaire?
Physical Space
• marketing of the exhibition/creating interest
• the audience
• the use of visual displays
• the presentation of the information
• my presentation?
• the ambience
• the catering
Audience/potential participants
people I know personally:
teachers in school
TAs
Learning mentors
teachers in the NEW network
people I know who teach
people i know who want to teach
LEA advisors
other head teachers
children in my school
People from on-line interactions
people on usenet teacher list
people on Flickr already
people from other on-line lists (Ray?)
people who are married to people I know on-line
teachers who blog
People who I don't know
teaching students at St Martins
the chap from St Martins on ELLG
education students at Lancaster Uni
students at Charlotte Mason
students at other institutions
Feedback sheet
what are 'change strategies'?
Approaches (strategies) to bring about change. For example, addressing managerial issues such as delegation, or professional development that could aim to upskill individuals or highlight individuals beliefs and values and how they might support or work against change.
This seems to mostly apply to managers or people in a position of power with the ability to implement change. The only literature I've been able to find that relates to it is aimed at management. The stuff talks about systemic organisational change (more Senge - groan) and how management can facilitate it.
Thing is my research is aimed at improving my practice not that of the audience of my pilot. They just share a work context with me not a job role. I'm not aiming to change their practice. I suppose it's possible that as McNiff suggests being around someone who is using reflective practice might influence others in the same context but that's hardly the sort of 'big bang' effect. I just don't see even the main exhibition having that sort of impact.
I would just skirt round this but it is one of the Learning Outcomes.
This means, I think that in my evaluation of the pilot exhibition I have to consider how effective it has been as a change strategy. SO I suppose I just say "Not at all" and then justify why it's unreasonable to expect it would be....
(from a posting to FirstClass )
Reflection on lack of formal feedback
From my Meta-Learning journal:
Well, I did my pilot exhibtion and they came and they ate my cake and they looked like they were listening and they clapped at the end. And I asked some questions, and they asked some questions. And only now do I realise, a week later, that I never got any answers.
So I have no feed back - none, as in, not any. I can cross structured focus groups off my list of data collection methods, and evaluation sheets too. Oh, and individual interviews of both a structured and unstructured nature, and informal conversations as no-one likes to be quoted when they are just chatting, so what's left?
Nothing much.
Conclusions?
Two main ones so far
1. I am rubbish at this - this seems to be S's favoured position. He thinks I must have sounded like an expert and they were intimidated into silence by me.
Does he realise these people are mostly teachers and I am a TA? They are not intimidated by me..... I think. I mean, but maybe he's right. How the hell should I know how I sounded? They won't tell me and I'm way too embarrassed to ever let anyone hear the audio recording.
It didn't feel natural standing up in front of them telling them stuff. It doesn't fit my prefered pedagogy. I'd rather have got people doing something real but that wasn't possible.
Pesonally I incline more to the belief that I bored them into a sort of trance.
2. I was talking to the wrong people. Somewhere - out there there are people who might be interested in what I'm doing. But not in my workplace - which is a kicker when this is a workplace based degree.
I left work having been close to tears all day. I knew this. I knew they weren't interested, so why does it hurt so much
Comments:
Posted by E @ 12/05/2005 12:02 PM PST
Haven't got much energy to write much - don't know if you watch the 'X' factor but Chico, who was voted off, said somethiing quite profound on Saturday - along the lines of :
Some people cry when something's finished but I rejoice because it's happened -
sending you some hugs (((Linda)))and hope you can find something to rejoice in...
Posted by Richard Millwood @ 11/29/2005 11:55 AM PST
I am quite certain (in a kind of religious way!) that you are not rubbish. I have spoken publicly, as a teacher, as a lecturer, as a governor, as a keynote speaker many, many times in my life. Always I have been uncertain about impact and rarely has anyone given me helpful feedback. So what's left as an evaluation? Well your own view is no bad thing. Also, have you seen any change in subsequent days, in conversations or actions on the part of your colleagues? It may be that you need to keep your eyes peeled for signs for some time to come! I do wish you all the best in this - impact is often a slow burn, but I know it wasn't a wreck.
Posted by Viv @ 11/28/2005 11:59 PM PST
Poor you Linda - don't be too downhearted. I fear that i may suffer the same fate next term. I don't know what year three entails or how exactly i will'exhibit' but I'm not looking forward to it. My colleagues are 'Dr's' and 'Prof's.' and a degree to them is yesterday's news - like GCSE's are to me.
I have no answers, sorry- but at the end of the day you've tried and maybe you can use your 'negative - ish' experiences to your advantage
somehow in the write-up?
Chin up:-)