Some Insight

I've been reading two books recently that have offered some insight into resolving the metaphysical debate and crisis I have been feeling regarding methodology. The two books are:

Newman, Isadore and Carolyn R. Benz. Qualitative-Quantitative Research Methodology: Exploring the Interactive Continuum. Carbondale: Southern Illinois Press, 1998.

and

Seale, Clive. The Quality of Qualitative Research. London: Sage Publications, 1999.

Clive seems to know just what I have been feeling regarding landing on "the right" methodological perspective for research as well as the quandry and tension I have felt attempting to resolve modernist (quantitative) and postmodernist (quanlitative) world views.

He opens by summarizing a research project by Denzin and reveals some of its gaps and inadequacies. Special attention is paid to critical analysis and political analysis that seems to characterize post-modern research. Ultimately, he points out these modes of inquiry become as foundational as modernist view points. He says, "we are left with the view that such work may be a useful source of ideas, but cannot be proposed as a wholly adequate successor to more scientific conceptions of social research" (7). He points out though that a "return to modernist assumptions seems impossible" (7). Clive then goes on to point out that his book takes this tension--the exact one I have been feeling--as the starting point for his book.

I am reading the book with interest, and I find that I sympathize with his position. He describes his own stance toward research and the resolution of this tension: "I am in favor of a fallibilistic approach to research, within a 'subtle realist' orientation, that does not give up on scientific amis as conventionally conceived, but also draws on the insights of postscientific conceptions of social research" (x). He seems to take the stance that no ONE right way or approach to research and knowledge is possible, and that we can benefit from all of them. I'm still reading.

The second book by Newman and Benz is short and excellent also. I need to think about their thesis and ask others what they think, but this quote basically sums up their thinking: "We take the position that the two philosophies [quant and qual] are neither mutually exclusive (i.e., one need not totally commit to either one or the other) nor interchangeable (i.e., one cannot merge methodologies with no concern for underlying assumptions)" (xi). Just hearing this viewpoint of a more expansive view toward the two approaches is helpful for me to hear. But they go on: "Rather, we present them as interactive places on a methodological and philosophical continuum based on the philosophy of science" (xi). The see inquiry as starting from a RESEARCH QUESTION. The question is more important than determining the methodology. In this continuum, the researcher applies qualitative approaches to inquiry to generate a theory. Then he or she applies quantitative approaches to test and generalize (if possible?) the theory. I'm oversimplifiying their view of the continuum. What I have to ponder is it seems to maintain a scientific approach (i.e. positivistic or postpositivistic) view.

I'll close with one statement that struck me from Clive's book: "These all involve a moral commitment to use research activity to aid intersubjective understandings" (15). This statement struck me because is does two things: 1) it acknowledges that "knowledge" is of a postmodern form, an interpretation, but 2) it also highlights that we share interpretations, "intersubjective" understandings of reality that as I.A. Richards pointed out form something substantial as if real. These intersubjective realities can be studied--no?

More Methodological Musings

I'm still balking at entering into a "methodology" that I believe in and that fits with this particular research question/subject. "Entering into" is the phrase I can grasp at this point because it seems that I have to declare my methodology as if it were like placing my foot into the proper size shoe. I almost wish it were a matter of recognition rather than declaration; I'd rather discover what I already know and think that rhetorically adjust my philosophical stance to fit the research situation. But it seems that research is a matter of rhetoric too, and I must undergo a good deal of invention before I am ready to deliver.

Creswell in his book Research Design says this about designing a research project: "the proposal developer needs to consider three framework elements: philosophical assumptions about what constitutes knowledge claims; general procedures or research called strategies of inquiry; and detailed procedures of data collection, analysis, and writing, called methods" (3). Methodologies and methods—that's what he is talking about and their relationship. What you believe makes for "knowledge claims" shapes everything! That means that you need a firm grounding in what you think is the nature of reality and how WE know reality. Here is where I get stumped. This question is a big—I mean BIG—one to tackle. Let me step back and see if perhaps I can't find a way to approach the question of knowledge claims by allowing that there are degrees of knowledge or kinds of knowledge. There isn't just ONE kind of valid knowledge claim. That makes me breath easier, but then I question what the heck "knowledge" is anyway. What does it mean "to know?" Does what it means "to know" change depending upon our situation and framework of experience? Do we "know" in different ways? And are these ways mutually exclusive?

I've been thinking about this question of methodology a lot, and I have two writers who I have turned to to help me in this dilemma. The first is from S. Zuboff's The Age of the Smart Machine. In the discussion of workers knowledge in a saw mill, she brings up the notion of a skill as "action-centered" where the "capacity 'to know' has been lodged in sentience and displayed in action." She goes on: "Certain knowledge was conveyed through the immediacy of their sensory experience. Instead of Descarte's 'I think therefore I am,' these workers might say, 'I see, I touch, I smell, I hear; therefore, I know.' …belief was a seamless extension of sensory experience." This kind of action-centered knowledge is often tacit and doesn't require that the knowledge be made explicit. The workers referred to the "artistry" of their skill and how they had a form of "felt sense" (my words) that constituted their knowledge. They couldn't say exactly how or what they know—they just knew it. This form of knowledge is an excellent example of the classical term of "techne," but I also think it stands for our everyday experience of the senses. We can "know" the temperature in a room is hot because we feel it—we can also measure it. We can quantify it. Close observation will provide us some "truth" about what we experience—right? For me, and most of the modern world, this form of perceiving "cause-effect" relationships within reality is a matter of experience and fact. We depend upon "knowing" these cause-effect relationships within our world in order to do just about everything we do. We "know" a lot this way.

Zuboff, though, describes another form of knowledge that confronted these saw mill workers when they began running their machine processes via a computer rather than by physical efforts. The automation run by the computer replaced their direct sensory experience with a symbolic representation of reality: "Immediate physical responses must be replaced by an abstract thought process in which options are considered, and choices are made and then translated into the terms of the information system." She says that this new form of knowledge required inferential and procedural reasoning where the worker has to "know" what the symbols within the information system mean in relation to each other PLUS what they mean to the outside world. She calls this an "intellective skill," (what Clare interestingly called "epistome"). She says that the workers struggled with these two starkly different forms of knowledge.

Zuboff makes this extremely interesting comment: "In a symbolic medium, meaning is not a given value; rather it must be constructed." She mentions that the linkage between symbol and experience (sign and signified) must be established over time, but that eventually this linkage becomes so tight that the original problematic (or unlikely) nature of the linkage becomes essentially invisible.

What Zuboff describes with "intellective skill" is a form of knowledge contained within symbols. Rather than direct knowledge of "reality" via the senses, this form of knowledge is many layers abstracted from "reality"—it is only a reference or stand in for reality. When I say the temperature is 32 degrees, the words "32 degrees" are a symbolic representation of the freezing temperature outside. By touching the word, I don't feel the cold. Not even by listening or looking at the word (except in our imagination).

Knowledge claims within a "symbolic medium" are different from knowledge claims within a physical environment. I suppose the key question is how far you go with what constitutes a "symbol," but common sense I think can guide us not to go overboard. Of course, all experience is "interpreted" by the consciousness of each individual, but the immediacy of sensory connections to "reality" provide a form of knowledge that is very powerful.

This idea of Zuboff about the linkage of symbol to reality brings me to my second author—I. A. Richards. Richards in his The Philosophy of Rhetoric presents the general principle that "Language and though are not—need I say?—one and the same" (13). No one-to-one relationship exists between language and thought. If I were to say "freezing," the word would not always generate the same thought for all people. No one-to-one relationship exists between language and thought. He calls this notion of an equation between language and thought the "Proper Meaning Superstition": "the common belief…that a word has a meaning of its own (ideally, only one) independent of and controlling its use and the purpose for which is should be uttered" (11). He mentions that this belief is a superstition when we forget "that the stability of the meaning of a word comes from the constancy of the contexts that give it its meaning" (11). I like that Richards acknowledges that though the linkage between sign and signified (word and reality, language and thought) is arbitrary and contrived there are strong stabilities of meaning for many words. He compares the stability of meaning between a word like knife and a word like mass. It seems that we can study these stabilities and instabilities of meanings.

Significantly, Richards states: "At present it is still Thought which is most accessible to study and accessible largely through Language" (13). Isn't that what I am studying—people's thoughts, the evolution of their thinking. I am studying reflective thinking by examining the linguistic acts of writers—the external, discursive acts.

Let me go on and mention Richard's important point about meaning. He asks, "How does a word mean?...How does an idea (or an image) mean what it does?" (15). Later he provides a series of questions to examine more deeply the functions of language: "What is the connection between the mind and the world by which events in the mind mean other events in the world? Or How does a thought come to be 'of' whatever it is that it is a thought of? Or What is the relation between a thing and its name?" (28). He is essentially asking, "What is knowledge?" (knowledge essentially being the same thing as "meaning"). He makes a very similar point about knowledge as Zuboff related to the physical world. Noting that we are responsive in many ways to the world, he uses the analogy of a thermometer to point out the nature of our responsiveness. A thermometer responds consistently to the present temperature, with no influence from what has happened in the past. He then explains how we are not responsive in the way a thermometer is responsive: "Do we ever respond to a stimulus in a way which is not influenced by the other things that happened to us when more or less similar stimuli struck us in the past?" (29-30). He goes on:

I can make the same point by denying that we have any sensations. That sounds drastic but it is almost certainly true if rightly understood. A sensation would be something that just was so, on its own, a datum; as such we have none. Instead we have perceptions, responses whose character comes to them from the past as well as the present occasion. A perception is never just of an it; perception takes whatever it perceives as a thing of a certain sort. All thinking from the lowest to the highest—whatever else it may be—is sorting. (30)

It seems that Richards adds a complexity to Zuboff's idea of "action-centered" skill, but I don't think he invalidates in this instance the close connection between sensation and reality. These workers through extensive past experience refine their senses connection to reality. It doesn't invalidate their form of knowledge; it only explains how it comes about.

Richards continues by noting the importance of context for words and their meanings. He refers to meaning as "delegated efficacy" especially through words "whose virtue is to be substitutes exerting the powers of what is not there. They do this as other signs do it, though in more complex fashions, through their contexts" (32). Context, though, is not a cut and dried term. We don't respond to present contexts like a thermometer. The traditional meaning of context is circumstance, situation, instance. But Richards gets at context from "considering those recurrences in nature which statements of causal laws are about" (33). He states the simple definition of a causal law: "a causal law may be taken as saying that, under certain conditions, of two events if one happens the other does" (33). This is common sense. He explains the messiness of cause and effect relationships and how we decide them to suit our purposes: "we distribute the titles of 'cause' and 'effects' as we please" (33). Richard's "causal theorem of meaning" is similar to Burke's "terministic screen" in saying that we select or privilege certain kinds of causal laws and don't select others. Speaking about language, he describes how for a context a word can "take over the duties of parts which can then be omitted from the recurrence. There is thus an abridgement of the context. …When this abridgement happens, what the sign or word—the item with these delegated powers—means is the missing parts of the context" (34). What a complicated way in which words relate to reality (context) and to our thought (meaning/knowledge) of that reality! Sensing the complexity of what he is talking about, Richards offers this simplified summary of his thinking: "It is enough for our purposes to say that what a word means is the missing parts of the contexts from which it draws its delegated efficacy" (35). Context, then,--that complex perception from the past and present of circumstance—is what constitutes the bedrock of meaning, and when we "mean" to each other though language if we are to have any "understanding" we must supply the context behind the words. This is hard to do, and as Richards says the source of most misunderstanding.

I've gotten side-tracked into philosophic explorations of the relationship between reality, language, and thought, but it seems I must have a firm grounding in how I believe these all relate before I can declare a methodology. Based upon my methodology (philosophical viewpoint on the world and the word!), I can THEN see how I will go about making my own "knowledge claims."

It seems that if I seek to find the stability of meaning(s) for words and experience related to reflection that I could use scientific methods—quantitative means like a survey. Can I explore how far consistencies of cause-effect relationships go from a given context? The crux seems to lie in how much consistency we provide to context. If we assert that context is so variable as to have no true consistency or uniformity, then we have no basis to assert any cause-effect relationships. Nothing usefully generalizable is generated. But could we step back and in a more general, big-picture way assert the consistency and uniformity of a context or activity (notice I am integrating the notion of action into context)? For instance, could I in general examine the influence of reflection on writing? Or examine the effect and what is generated when students are asked to reflect? What do they do? Is there any consistency in what is generated or what students do when they reflect as seen in their language.

I'm thinking more and more that I should not shy away from positivistic or post-positivistic "knowledge claims," but perhaps I could triangulate and contextualize some positivistic knowledge claims with qualitative analysis. So for instance I might do a content analysis of a ton of reflective writing pieces and generate some larger generalizations about what students say and do in reflection. I could even do an additional study that sought to link stated aims or goals for revision and actual revisions made by students. I could have two cohorts—one that did consistent reflection in-process and one that did not. Then examine the choices and changes make in revision between the two groups. Is there any difference?

I need to stop now. In a way, I’m left as confused as ever. Here is where I need to assistance of my professors to gain perspective. I think it will be my goal for this May and Summer to land on my methodology and the particular research project I will pursue for my dissertation research.

3/4 Research Reflection

Research Reflection 3/4/07

I am in the early stages of "implementing" my research, and I find that I am questioning just about everything about my project. Since the areas of this project are so multiple, I think I'll take them one at a time based on categories I determine on the fly.

Purpose/Inquiry

I was reviewing MacNeally's chapter on surveys, and she devotes some attention to the role of purpose in surveys. She mentions that purpose affects the content of questions as well as their design. She also places particular emphasis on having a good sense of the survey's purpose and determining if that purposes "will do more harm than good in the organization" (152). Purpose is so important because it has a ripple affect throughout the choice and implementation of methodology as well as methods. (Surveys being just one method.) In my earlier work done with the help of Creswell, I defined my purpose this way:

"The purpose of this three-phase, sequential mixed method study is to clarify the role of reflection within the Freshman Composition program at Texas Tech University. The first phase of the study will be a qualitative exploration of the design intentions behind reflection's place in the curriculum by interviewing present and past Composition Program Administrators. In the second phase, ideas and themes from these interviews will then be developed into a survey so that the intentions and expectations about reflection articulated by the designers of the curriculum can be tested against the attitudes and experiences of those who deliver the curriculum (Classroom and Document Instructors) and those who experience the curriculum (students). In the third phase, qualitative interviews will be used to probe significant results from the survey in more depth."

Key words in my purpose statement include clarify, exploration, test, and probe. The question I am wrestling with now is how I maintain a "mixed" methodological perspective and if that is possible. Do I put on my interpretive hat, then take it off to put on my scientist hat, and then switch into my interpretive hat once again? Is that possible? I just read Thomas Schwandt's "Constructivist, Interpretivist Approaches to Human Inquiry" in the Handbook of Qualitative Research and I need to think in what ways I fit within the wide array of interpretive and qualitative methodological perspectives. I'm messy right now, and I can't say that I have a coherent perspective that I inhabit. I think I can fit Schwandt's broad goal of these approaches as "understanding the complex world of lived experience from the point of view of those who live it" (118). Right now I sense that developing this coherent methodological perspective may take more time and a lot more reading and experience researching. My goal in the next few weeks is to keep reading and see if I can't refine and define this perspective as best I can. The contradiction I am feeling right now is the social scientific nature of surveys and the interpretive nature of qualitative inquiry in general. I suppose these survey results are going to need interpretation!

The second feature of purpose I am dealing with is the ethical dimension MacNeally mentions. Will the survey and its purpose do more harm to the organization than good? Although my intentions behind the research are innocuous, the place the Composition Program is in positions my survey as having potential harmful implications. Since the program is only one year into a transition phase to a new curriculum, and the current and previous Composition directors have differing visions for the courses, the situation seems like it could turn into a struggle between old and new ways of teaching the course. Perhaps not. And this struggle may happen whether or not I did my research study. However, in the potential debate on curriculum for FYC, I don't want my study to be ammunition for one side or the other. Ammunition may be the word. What matters, I think, is not so much what my results are as how I present my results. I would rather my study provide information that assists the program as they may debate future curriculum changes. My study should not come down on one side or the other. In other words, I must protect myself from any bias I may have or may present. Since I am a believer in the value of reflection, protecting from this bias will be particularly difficult. My results may inevitably be seen as tainted. BUT there I am going down a path that says we must disregard our subjectivities—oh no, that's not what qualitative research is about. So how to deal with my subjectivities and bias? How do I keep from seeing what I want to see? Hmm… I don't know. I think part of what I am struggling with in my design of the survey is dealing with this question of my bias, so more on that in a bit. My final thought on this ethical question is this: would the program be better off if I didn't do my study? If I had done my study a year ago, I think the answer would be clearly the program would be better off. At this point, as the new director phases reflection out of the curriculum and the issue of reflection becomes debated, I think my study takes on a different level of significance. I don't know. I'm probably putting too much significance on the study, and in the broad perspective of the program it represents a small activity.

Survey Questions

My most immediate concern is finalizing my survey. I received so many excellent peer responses to my first draft that I feel overwhelmed. Thank you to my classmates! I have a list of questions, so I'll list them and then muse of possible answers.

1) What will I use as my lead question.
The lead question I've heard is important just as an engaging opener in an essay. Right now, though, my questions are pretty utility-based on the goals of the survey. I don't have any particular question that is supposed to be interest-generating. Diane, I think, suggested a question on TOPIC in general. I'm worried though about each question. My original draft had 20 questions, and I am working to hone it down to 15 (as Becky suggested). Each question is important, so I don't want to throw in just any old question just to get their interest. I suppose I can examine what I have to see which one is the most interesting.

2) Question order?
MacNeally stresses the importance of questions moving in a logical order and the survey having the sense of one question leading into the next. I haven't really looked at this sequence question carefully, but I will in doing this next draft.

3) Recruiting/Sampling

I am worried about recruiting. I'm not yet on the instructor listserv, so I haven't even begun to recruit them. How many of these instructors will respond? I'm thinking about the rhetorical task of recruiting and how I will generate enough interest in potential participants to get them to do the survey. And if I only have results from those who have the gumption to respond, then am I getting a tainted sampling? Is a sampling of the willing inherently flawed? How do I get around that? I should probably have a bit of instructor demographic data questions so that I can say (hopefully) that I cover a broad enough and representative enough sampling. Now that I think about it, the same question of the "sampling of the willing" holds true for my survey of students. One of my goals for my annotated bibliography reading is to find out more about survey sampling and if there is anything written on "only-willing-participant" sampling.

4) Question bias

Diane, I think, was the one to notice that I was voicing all my questions in a positive way toward reflection. My general approach was to voice an opinion (as the participant might say it) and see if they agree or don't agree with the opinion/belief. I think this is a valid approach. But what is the effect on the participant of phrasing the belief with a positive or negative valence? I don't know? With a couple of questions I see that I can get the opinion part into the likert scale itself, and that seems more valid, but in other places I don't think I can and I don't think I want to. For instance, rather than saying, "My attitude toward doing Writer's Reviews was generally favorable?—strongly agree to strongly disagree." I've rephrased that question: "My attitude toward doing Writer's Review is generally—Strongly favorable to strongly unfavorable." The opinion is in the scale. It seems to me that this approach is better, but I wonder if I might run into problems as a valid likert scale if I use unusual terms. Probably. So I think I will have a mix of question types and I should probably be sure to have some negatively phrased questions as well. I think I only have one right now.

5) Are behavior questions important to the survey purpose?
I'm looking closely at those three or four behavior questions. Do they matter? Does it matter if they write their reflection directly into TOPIC or do it on their computer first and then copy and paste it in? Probably not for this survey. That could be another survey maybe. The issue of time spent and drafts may have some relevance though.

I have other more minute questions that I won't go into here since I have been going on an on. The last concern I have is the actual implementation of the survey. It seems clear to me that I won't administer the survey by March 9^th, since that is the due date for Writer's Review 1.2 and the day before Spring Break. No the right timeing. It would be nice, but I think I need to have a big "survey recruiting rush" right when everyone gets back from Spring Break on 3/19. The week to get surveys done is 3/19-3/23. That's my target. That means that I need to get my survey pretty much done this week, preferably early this week. I think it needs to be in place BEFORE Spring Break, so that it can be available after Spring Break. I won't be able to do anything during Spring Break. Depending upon how well I can get the word out about the survey, I may have to push for 3/26-3/30 as my survey week. I don't know that it will matter that much, but ideally I'd like 3/26-3/30 to analyze the survey results and set up the follow up interviews. I have a feeling that getting these interviews set up will take a bit of time. These details of logistics will be very important in the timing of my research, so it will be interesting to see how it turns out. My present goal is to revise the survey and create the instructor parallel version. Then, one more revision and I enter it into the survey tool that Chad gave me access to. Ahhh! All this by Wednesday?