Research Reflection
I am in the early stages of "implementing" my research, and I find that I am questioning just about everything about my project. Since the areas of this project are so multiple, I think I'll take them one at a time based on categories I determine on the fly.
Purpose/Inquiry
I was reviewing MacNeally's chapter on surveys, and she devotes some attention to the role of purpose in surveys. She mentions that purpose affects the content of questions as well as their design. She also places particular emphasis on having a good sense of the survey's purpose and determining if that purposes "will do more harm than good in the organization" (152). Purpose is so important because it has a ripple affect throughout the choice and implementation of methodology as well as methods. (Surveys being just one method.) In my earlier work done with the help of Creswell, I defined my purpose this way:
"The purpose of this three-phase, sequential mixed method study is to clarify the role of reflection within the Freshman Composition program at
Key words in my purpose statement include clarify, exploration, test, and probe. The question I am wrestling with now is how I maintain a "mixed" methodological perspective and if that is possible. Do I put on my interpretive hat, then take it off to put on my scientist hat, and then switch into my interpretive hat once again? Is that possible? I just read Thomas Schwandt's "Constructivist, Interpretivist Approaches to Human Inquiry" in the Handbook of Qualitative Research and I need to think in what ways I fit within the wide array of interpretive and qualitative methodological perspectives. I'm messy right now, and I can't say that I have a coherent perspective that I inhabit. I think I can fit Schwandt's broad goal of these approaches as "understanding the complex world of lived experience from the point of view of those who live it" (118). Right now I sense that developing this coherent methodological perspective may take more time and a lot more reading and experience researching. My goal in the next few weeks is to keep reading and see if I can't refine and define this perspective as best I can. The contradiction I am feeling right now is the social scientific nature of surveys and the interpretive nature of qualitative inquiry in general. I suppose these survey results are going to need interpretation!
The second feature of purpose I am dealing with is the ethical dimension MacNeally mentions. Will the survey and its purpose do more harm to the organization than good? Although my intentions behind the research are innocuous, the place the Composition Program is in positions my survey as having potential harmful implications. Since the program is only one year into a transition phase to a new curriculum, and the current and previous Composition directors have differing visions for the courses, the situation seems like it could turn into a struggle between old and new ways of teaching the course. Perhaps not. And this struggle may happen whether or not I did my research study. However, in the potential debate on curriculum for FYC, I don't want my study to be ammunition for one side or the other. Ammunition may be the word. What matters, I think, is not so much what my results are as how I present my results. I would rather my study provide information that assists the program as they may debate future curriculum changes. My study should not come down on one side or the other. In other words, I must protect myself from any bias I may have or may present. Since I am a believer in the value of reflection, protecting from this bias will be particularly difficult. My results may inevitably be seen as tainted. BUT there I am going down a path that says we must disregard our subjectivities—oh no, that's not what qualitative research is about. So how to deal with my subjectivities and bias? How do I keep from seeing what I want to see? Hmm… I don't know. I think part of what I am struggling with in my design of the survey is dealing with this question of my bias, so more on that in a bit. My final thought on this ethical question is this: would the program be better off if I didn't do my study? If I had done my study a year ago, I think the answer would be clearly the program would be better off. At this point, as the new director phases reflection out of the curriculum and the issue of reflection becomes debated, I think my study takes on a different level of significance. I don't know. I'm probably putting too much significance on the study, and in the broad perspective of the program it represents a small activity.
Survey Questions
My most immediate concern is finalizing my survey. I received so many excellent peer responses to my first draft that I feel overwhelmed. Thank you to my classmates! I have a list of questions, so I'll list them and then muse of possible answers.
1) What will I use as my lead question.
The lead question I've heard is important just as an engaging opener in an essay. Right now, though, my questions are pretty utility-based on the goals of the survey. I don't have any particular question that is supposed to be interest-generating. Diane, I think, suggested a question on TOPIC in general. I'm worried though about each question. My original draft had 20 questions, and I am working to hone it down to 15 (as Becky suggested). Each question is important, so I don't want to throw in just any old question just to get their interest. I suppose I can examine what I have to see which one is the most interesting.
2) Question order?
MacNeally stresses the importance of questions moving in a logical order and the survey having the sense of one question leading into the next. I haven't really looked at this sequence question carefully, but I will in doing this next draft.
3) Recruiting/Sampling
I am worried about recruiting. I'm not yet on the instructor listserv, so I haven't even begun to recruit them. How many of these instructors will respond? I'm thinking about the rhetorical task of recruiting and how I will generate enough interest in potential participants to get them to do the survey. And if I only have results from those who have the gumption to respond, then am I getting a tainted sampling? Is a sampling of the willing inherently flawed? How do I get around that? I should probably have a bit of instructor demographic data questions so that I can say (hopefully) that I cover a broad enough and representative enough sampling. Now that I think about it, the same question of the "sampling of the willing" holds true for my survey of students. One of my goals for my annotated bibliography reading is to find out more about survey sampling and if there is anything written on "only-willing-participant" sampling.
4) Question bias
Diane, I think, was the one to notice that I was voicing all my questions in a positive way toward reflection. My general approach was to voice an opinion (as the participant might say it) and see if they agree or don't agree with the opinion/belief. I think this is a valid approach. But what is the effect on the participant of phrasing the belief with a positive or negative valence? I don't know? With a couple of questions I see that I can get the opinion part into the likert scale itself, and that seems more valid, but in other places I don't think I can and I don't think I want to. For instance, rather than saying, "My attitude toward doing Writer's Reviews was generally favorable?—strongly agree to strongly disagree." I've rephrased that question: "My attitude toward doing Writer's Review is generally—Strongly favorable to strongly unfavorable." The opinion is in the scale. It seems to me that this approach is better, but I wonder if I might run into problems as a valid likert scale if I use unusual terms. Probably. So I think I will have a mix of question types and I should probably be sure to have some negatively phrased questions as well. I think I only have one right now.
5) Are behavior questions important to the survey purpose?
I'm looking closely at those three or four behavior questions. Do they matter? Does it matter if they write their reflection directly into TOPIC or do it on their computer first and then copy and paste it in? Probably not for this survey. That could be another survey maybe. The issue of time spent and drafts may have some relevance though.
I have other more minute questions that I won't go into here since I have been going on an on. The last concern I have is the actual implementation of the survey. It seems clear to me that I won't administer the survey by March 9th, since that is the due date for Writer's Review 1.2 and the day before Spring Break. No the right timeing. It would be nice, but I think I need to have a big "survey recruiting rush" right when everyone gets back from Spring Break on 3/19. The week to get surveys done is 3/19-3/23. That's my target. That means that I need to get my survey pretty much done this week, preferably early this week. I think it needs to be in place BEFORE Spring Break, so that it can be available after Spring Break. I won't be able to do anything during Spring Break. Depending upon how well I can get the word out about the survey, I may have to push for 3/26-3/30 as my survey week. I don't know that it will matter that much, but ideally I'd like 3/26-3/30 to analyze the survey results and set up the follow up interviews. I have a feeling that getting these interviews set up will take a bit of time. These details of logistics will be very important in the timing of my research, so it will be interesting to see how it turns out. My present goal is to revise the survey and create the instructor parallel version. Then, one more revision and I enter it into the survey tool that
No comments:
Post a Comment