Right to believe no right to disbelieve By Prof Dr Sohail Ansari

 And O my people, give full measure and weight in justice and do not deprive the people of their due and do not commit abuse on the earth, spreading corruption. Surat Hūd 

First rule of subordination: everything is my fault

·        Every good subordinate is right to believe that he has no right to disbelieve the statement of a boss no matter how far it stretches credulity.

·        No man goes before his time - unless the boss leaves early. Groucho Marx
First rule of leadership: everything is your fault.-A Bug’s Life

THE CONTEXT OF DESIGN 

Quantitative and qualitative research

Designs are often equated with qualitative and quantitative research methods. Social surveys and experiments are frequently viewed as prime examples of quantitative research and are evaluated against the strengths and weaknesses of statistical, quantitative research methods and analysis.

Case studies, on the other hand, are often seen as prime examples of qualitative research which adopts an interpretive approach to data, studies `things' within their context and considers the subjective meanings that people bring to their situation.

It is erroneous to equate a particular research design with either quantitative or qualitative methods. Yin (1993), a respected authority on case study design, has stressed the irrelevance of the quantitative/ qualitative distinction for case studies.

He points out that: Design type Method of data collection Experiment Questionnaire Interview (structured or loosely structured).

 A point of confusion . . . has been the unfortunate linking between the case study method and certain types of data collection. For example those focusing on qualitative methods, ethnography, or participant observation. People have thought that the case study method required them to embrace these data collection methods . . . On the contrary; the method does not imply any particular form of data collection which can be qualitative or quantitative. (1993: 32) Similarly, Marsh (1982) argues that quantitative surveys can provide information and explanations that are `adequate at the level of meaning'. While recognizing that survey research has not always been good at tapping the subjective dimension of behaviour, she argues that: Making sense of social action . . . is . . . hard and surveys have not traditionally been very good at it. The earliest survey researchers started a tradition . . . of bringing the meaning from outside, either by making use of the researcher's stock of plausible explanations . . . or by bringing it from subsidiary in-depth interviews sprinkling quotes . . . liberally on the raw correlations derived from the survey. Survey research became much more exciting . . . when it began including meaningful dimensions in the study design. [This has been done in] two ways, firstly [by] asking the actor either for her reasons directly, or to supply information about the central values in her life around which we may assume she is orienting her life. [This] involves collecting a sufficiently complete picture of the context in which an actor finds herself that a team of outsiders may read off the meaningful dimensions. (1982: 123±4) Adopting a sceptical approach to explanations. The need for research design stems from a sceptical approach to research and a view that scientific knowledge must always be provisional. The purpose of research design is to reduce the ambiguity of much research evidence. We can always and some evidence consistent with almost any theory. However, we should be sceptical of the evidence, and rather than seeking evidence that is consistent with our theory we should seek evidence that provides a compelling test of the theory. There are two related strategies for doing this: eliminating rival explanations of the evidence and deliberately seeking evidence that could disprove the theory.

Plausible rival hypotheses: 

A fundamental strategy of social research involves evaluating `plausible rival hypotheses'. We need to examine and evaluate alternative ways of explaining a particular phenomenon. This applies regardless of whether the data are quantitative or qualitative; regardless of the particular research design (experimental, cross-sectional, longitudinal or case study); and regardless of the method of data collection (e.g. observation, questionnaire). Our mindset needs to anticipate alternative ways of interpreting findings and to regard any interpretation of these findings as provisional subject to further testing.

The idea of evaluating plausible rival:

 Hypotheses can be illustrated using the example of the correlation between type of school attended and academic achievement. Many parents accept the causal proposition that attendance at fee paying private schools improves a child's academic performance. Schools themselves promote the same notion by prominently advertising their pass rates and comparing them with those of other schools or with national averages. By implication they propose a causal connection:

`Send your child to our school and they will pass (or get grades to gain entry into prestigious institutions, courses).'

The data they provide are consistent with their proposition that these schools produce better results. Causal relationship School type Academic achievement

 

 

Alternative explanation: selectivity on child’s initial ability.

 Child’s ability School type Academic achievement Alternative explanation:

Family resources Parental resources:

·         Facilities in home for study School type Academic achievement.

·         Educational values. Parental valuation of education. Child’s valuation of education School type Academic achievement.

But these data are not compelling. There are at least three other ways of accounting for this correlation without accepting the causal link between school type and achievement.

 There is the selectivity explanation: the more able students may be sent to fee paying private schools in the first place.

There is the family resources explanation: parents who can afford to send their children to fee paying private schools can also afford other help (e.g. books, private tutoring, quiet study space, computers). It is this help rather than the type of school that produces the better performance of private school students.

Finally, there is the family values explanation: parents who value education most are prepared to send their children to fee paying private schools and it is this family emphasis on education, not the schools themselves, that produces the better academic performance.

All these explanations are equally consistent with the observation that private school students do better than government school students. Without collecting further evidence we cannot choose between these explanations and therefore must remain open minded about which one makes most empirical sense. There might also be methodological explanations for the finding that private school students perform better academically. These methodological issues might undermine any argument that a causal connection exists. Are the results due to questionable ways of measuring achievement? From what range and number of schools were the data obtained? On how many cases are the conclusions based? Could the pattern simply be a function of chance? These are all possible alternative explanations for the finding that private school students perform better.

Good research design will anticipate competing explanations before collecting data so that relevant information for evaluating the relative merits of these competing explanations is obtained. In this example of schools and academic achievement, thinking about alternative plausible hypotheses beforehand would lead us to find out:

·         About the parents' financial resources and the study resources available in the home.

·         The parents' and child's attitudes about education

·         The child's academic abilities before entering the school.

The fallacy of affirming the consequent:

 Although evidence may be consistent with an initial proposition it might be equally consistent with a range of alternative propositions.

Too often people do not even think of the alternative hypotheses and simply conclude that since the evidence is consistent with their theory then the theory is true. This form of reasoning commits the logical fallacy of affirming the consequent.

This form of reasoning has the following logical structure: · If A is true then B should follow. · We observe B. · Therefore A is true.

If we apply this logic to the type of school and achievement proposition, the logical structure of the school type and achievement argument becomes clearer.

Initial proposition: · Private schools produce better students than do government schools.

The test: · If A then B If private schools produce better students (A) then their students should get better annual marks than those from government funded schools (B).

 B is true Private school students do achieve better annul marks than government school students (observe B). · Therefore A is true.  Therefore private schools do produce better students (A is true). But as I have already argued, the better performance of private school students might also reflect the effect of other factors. The problem here is that any number of explanations may be correct and the evidence does not help rule out many of these. For the social scientist this level of indeterminacy not determinate; not precisely fixed in extent;indefinite; uncertain) is quite unsatisfactory. In effect we are only in a position to say: · If A [or C, or D, or E, or F, or . . .] then B. · We observe B. · Therefore A [or C, or D, or E, or F, or . . .] is true. Although explanation (A) is still in the running because it is consistent with the observations, we cannot say that it is the most plausible explanation. We need to test our proposition more thoroughly by evaluating the worth of the alternative propositions.

 Falsification: looking for evidence to disprove the theory As well as evaluating and eliminating alternative explanations we should rigorously evaluate our own theories. Rather than asking `What evidence would constitute support for the theory?’ ask `What evidence would convince me that the theory is wrong?' It is not difficult to find evidence consistent with a theory. It is much tougher for a theory to survive the test of people trying to disprove it. Unfortunately some theories are closed systems in which any evidence can be interpreted as support for the theory. Such theories are said to be non-falsiable. (faisable' in Other Languages. British English: feasible /ˈfiːzəbl/ ADJECTIVE. If something is feasible, it can be done, made, or achieved. The committee will decide whether the idea is feasible). Many religions or belief systems can become closed systems whereby all evidence can be accommodated by the theory and

Nothing will change the mind of the true believer. Exchange theory (Homans, 1961; Blau, 1964) is largely non-falsiable. It assumes that we always maximize our gains and avoid costs. But we can see almost anything as a gain. Great sacrifices to care for a disabled relative can be interpreted as a gain (satisfaction of helping) rather than a loss (income, time for self etc.).

We need to frame our propositions and explain our terms in such a way that they are capable of being disproven.

 The provisional (arranged or existing for the present, possibly to be changed later.) nature of support for theories.  Even where the theory is corroborated and has survived attempts to disprove it, the theory remains provisional:  corroboration gives only the comfort that the theory has been tested and survived the test, that even after the most impressive corroborations of predictions it has only achieved the status of `not yet disconfirmed'. This . . . is far from the status of `being true'. (Cook and Campbell, 1979: 20). There always may be an unthought-of explanation. We cannot anticipate or evaluate every possible explanation. The more alternative explanations that have been eliminated and the more we have tried to disprove our theory, the more condense we will have in it, but we should avoid thinking that it is proven. However we can disprove a theory.

The logic of this is: · If theory A is true then B should follow. · B does not follow. · Therefore A is not true. So long as B is a valid test of A the absence of B should make us reject or revise the theory.

In reality, we would not reject a theory simply because a single fact or observation does not support it. Before rejecting a plausible theory we would require multiple disconfirmations using different measures, different samples and different methods of data collection and analysis.

In summary, we should adopt a sceptical approach to explanations. We should anticipate rival interpretations and collect data to enable the winnowing out of the weaker explanations and the identification of which alternative theories make most empirical sense. We also need to ask what data would challenge the explanation and collect data to evaluate the theory from this more demanding perspective.