It may be tempting, when you are studying some belief system, to ask people what they believe—to ask them individually in interviews, or to ask them wholesale in a survey. The temptation should nearly always be resisted.
Let’s say you want to find out whether I believe in Modern Monetary Theory, or animal liberation, or ghosts, or whatever,—and so you just ask me. I think there are three possible situations.
(1) I may have a clear and definite belief on the subject (including possibilities like conscious agnosticism or the belief that the question is wrongly posed), and be happy to express it publicly.
(2) I may hold a belief but be unwilling to express it (whether because it is discreditable, or because I expect most hearers will misunderstand it, or because I want to avoid awkwardness among people who believe the opposite, or for some other reason).
(3) I may be unable to express any clear belief on the matter at all, even if I wanted to—either because I have never given it any consideration, or because I have not yet succeeded in putting my hunches and intuitions into words.
We can look briefly at each of the three possibilities.
In scenario (1), you have every chance of getting a clear and honest answer—but you will not gain much from it. If I have a definite belief I am willing to express, then I will probably have expressed it before. You didn’t need to go to the trouble of interviewing me: you could have looked at things I’ve written, talks I’ve delivered, leaflets I’ve given out in the street, etc., etc. And, if you want to be sure that you are indeed dealing with case (1), your only way of proving it will be precisely to show that I have already expressed the belief in question, one way or another. So you can only rely on my answer if you know it anyway. Asking the question has gained you nothing.
It has at least not lost you anything—which is not necessarily true of case (2), the case in which I am reluctant to profess my belief except to people I trust. I may of course attempt to fob you off with generalities, in this scenario, or simply refuse to answer. But I am afraid the possibility also exists that I will just lie to you. There are some questions, remember, to which ‘I don’t know’ or ‘no comment’ is far from being a respectable response. So in this case you will run the risk of emerging from the interview with a picture of my beliefs that is actually less accurate than the one you began with. You will have formed the impression that I believe some doctrines I in fact reject (albeit only when I am among friends and they have their recording devices switched off).
Your idea of my beliefs will thus have suffered some distortion—but my beliefs themselves will presumably not. In case (3), not even this can be said. Perhaps ‘distortion’ is too negative a word: it would be more neutral to say simply that your questions may have caused me to formulate and adopt some beliefs I had not previously held. It is quite possible that I will feel you have positively helped me, by encouraging me to clarify my thinking in particular areas. Even if your interview technique is less subtle and Socratic than that, I will at least have been prompted to state some opinions that, having stated them, I shall be apt in future to regard as mine. In much the same way, participants in market research surveys perhaps initially find themselves saying ‘cornflakes, Star Wars, the Pyramids, Trump’ for no other reason than because it feels so feeble to keep repeating ‘don’t know’: but, once they have heard themselves say these are their favourites, they may well come to feel that they really are. (Put like that, it sounds narcissistic or shallow; but I don’t think it has to be. I suspect we have less introspective insight into our own beliefs, attitudes, etc., than is sometimes assumed, so that we often have to judge what they are exactly as we would with other people—by observing what we say and do.) But, however grateful I may be to you for prodding me to develop and extend my belief system, for you it is a disastrous result—assuming your intention was to understand my beliefs, rather than to modify them.
If we are seriously interested in a belief system, there is no alternative to studying the actual evidence for it: the things believers say and do of their own accord. Our questions, almost inevitably reflecting our interests and priorities more than theirs, risk obscuring (or missing altogether) the very points that believers themselves find most significant. The space of belief systems is not quartered into neatly identifiable segments that can be mapped out and labelled in advance, before we start investigating any particular belief system: it is devious and many-dimensioned, and the questions believers have felt were worth addressing have been the questions that occurred to them, not to us. Try drawing up a questionnaire that you could put in front of a Methodist, a Maoist, a Mahayanist, a monetarist, and a Moonie, so that their responses would give you the slightest understanding of what those belief systems are about. I don’t think it can be done.
Fortunately, technological advances in recent decades mean that the kind of evidence we need is radically more accessible than at any previous time. Thirty years ago, back numbers of (let’s say) a provincial flying saucer magazine published in another country would only have been available in the biggest academic and specialist libraries, if even there. You might have needed to travel abroad just to peer at them on microfiche. And if you had then realized, just as you boarded the plane for home, that there was another item you should have consulted, you would have been out of luck. The internet puts infinite libraries of this kind of material at the disposal of anyone who cares to look it up. Research projects that would have been difficult, expensive, and time-consuming in the twentieth century, if they had been possible at all, can now be carried out on a laptop computer or mobile phone without leaving the armchair. And the widespread use of social media probably creates, at least in principle, the scope for Mass-Obs. work of unprecedented accuracy and scope: it is now possible to overhear thousands upon thousands of rank-and-file believers as they argue, react to events, assuage one another’s doubts, try to make converts, etc., etc. Taking full advantage of this opportunity will undoubtedly require a good deal of careful methodological thought; but the opportunity is there. In the face of this wealth of irreproachable evidence, continuing to waste believers’ time and our own with interviews and surveys would be simply quixotic.
There is a minor urban legend among students of Russian poetry to the effect that some writer on versification, unable to find an example of an iambic tetrameter with stresses on the third and fourth metrical ictuses but not the first or second, and yet not knowing any rule that said it was impossible, ended up asking Andrei Bely to write one specially—and that the poet obliged with I velosipedíst letít ‘And the cyclist is flying.’ It is thankfully not quite true. Bely did write the line, and he did write it to illustrate the rhythmical point in question; but it was for his own essay ‘An Attempt to Characterize the Russian Iambic Tetrameter’ (in his book Simvolizm, Moscow 1910, pp. 286-330) rather than for somebody else’s research. The procedure would in any case have been an anomalous one: reputable literary critics have always preferred to base their analyses on things their authors have actually written, rather than commissioning new work to back up the analysis. Workers in the field of belief systems would do well to keep to the same scientific standard.