“Don’t you have anything positive to say?”
Well, as it happens I do! It can be found in a new paper by myself, Phoebe Gaston, Nick Huang, and Hanna Muller, “Theories all the way down: remarks on ‘theoretical’ and ‘experimental’ linguistics.”
The question above came from a student in my 2015 LSA Summer Institute class at the University of Chicago. I enjoyed a fun two weeks teaching in a beautiful old lecture hall straight out of central casting, talking about The Psycholinguistics of Grammar. Each class was followed by an open-ended coffee hour with students in one of Hyde Park’s many fine coffee shops. Although I had a great time talking about many different lines of research on many different kinds of linguistic phenomena, some students were disappointed. They came expecting to learn about how experiments can help to resolve questions in grammatical theory. And a recurring message from me was that things generally don’t work out that way.
The questioner, Iria de Dios Flores, later became a collaborator. She asks good questions. Sometimes it takes a while to answer them.*
There is a strange notion that is surprisingly popular among linguists, that we can be classified as theoreticians or as experimentalists. The theoreticians are alternately the high minded folk who are working on Stuff That Matters, or the underappreciated folk who don’t get enough love or funding. The experimentalists get their hands dirty with fancy toys that probably cost more than is worthwhile. The experimentalists could at least justify themselves if they could show that their Large Morpheme Colliders could solve long-standing theoretical questions.
I’m not sure where this distinction comes from, but like many other things it’s probably borrowed from those Sciences That Get More Respect. We’ve all seen those news features where a reporter at CERN in Geneva makes a valiant attempt to explain what a boson is while a bemused Peter Higgs tries to look impressed by folks who have figured out something that he figured out decades ago at a tiny fraction of the cost.
We encounter expectations for something along these lines quite often: in job ads, in graduate school applications, in papers that we review.
And I have to admit that this is one of the reasons why I got into this line of work in the first place. When I first got into linguistics in the early 1990s my starting point was syntax and semantics. There was excitement at the time about recent discoveries that reaction time evidence could reveal the existence of phonologically empty categories, or children’s errors could reveal the true theory of anaphora. And brain recording techniques were starting to emerge. Who knew what truths they might uncover!
Things didn’t exactly work out as expected. Those early hopes were dashed. Diagnosing empty categories using reaction time data isn’t so straightforward. More generally, using timing evidence to arbitrate among theories that explicitly disavow timing predictions isn’t so easy. And the findings about kids and theories of anaphora didn’t pan out as expected, either.
As I gave up on one youthful dream after another, it must have seemed depressing. That’s certainly an impression that I must have given in some of my speaking and writing on the topic.
But that couldn’t have been further from the truth. What happened again and again was that we would start with some question or assumption that we took from traditional high level theories, and then as we dug into experiments on the topic we would discover new questions that we had not been aware of before. Those questions were theoretical, just at a different grain of analysis than where we had started.
Although we would occasionally write papers explaining how this or that experimental evidence was not so informative as claimed about some issue from traditional linguistic theory, the vast majority of our papers were concerned with those other theoretical questions that we had become interested in.
Our new paper weaves together a series of these stories, about how we thought we were working on one thing, and then it turned out that we were actually working on something different, and just as interesting. To us, at least.
There’s a terminological point here that is straightforward. Nobody own the term “theory”. All flavors of linguist are using evidence and reasoning to build generalizable accounts of how the human language system works. We all use empirical evidence, and we all develop theories. The distinction between theoreticians and experimentalists is largely a myth. Sometimes our experiments are so easy that we’re embarrassed to label them as experiments (e.g., “Does that sentence sound better to me if I take out the complementizer?”). Sometimes the experiments take a long, long time, so we get to spend less time thinking about the theoretical questions. But it’s all basically the same thing.
That much should have been obvious ahead of time. Some other things that we have learned (repeatedly) along the way are: the value of embracing discrepant findings, the lessons that come from working through experimental details, and the clarity that comes from working with simple computational models — ideally just a step or two beyond the things that you can do without a model.
The title of our article is a nod to the saying, “Turtles all the way down”, referring to a somewhat far fetched cosmological theory. Interestingly, the Wikipdia page for the saying has sections devoted to 19th century psychologist William James, and to twentieth century linguist Haj Ross. Enjoy!
* I think that Ellen Lau was pushing me on the same question at the same time. Possibly even at the same party in Chicago where Iria was prodding me. I do not like to ignore Ellen’s questions.
Thanks for the thought provoking paper! I think over the years I’ve come to have a slightly different take on the ‘theoretical’ vs ‘experimental’ labels: I suspect they are used simply as imprecise labels for different subfields. That is: theoretical linguists are people whose research questions address grammatical structure at an abstract level, and experimental linguists are people whose research questions involve what we might recognize as ‘performance’ questions (e.g. how is grammatical number coded in short-term or working memory, what retrieval cues are used to reactive dependents at key choice points in processing). My sense is that this is primarily on the ‘S-side:’ this isn’t a distinction I really ever hear about in phonology these days, but perhaps I’m wrong on this point!
So while I don’t think the distinction is a myth, I think it’s more about a slightly imprecise use of the terms than anything else. Of course, I agree 100% that the term theory doesn’t belong to anyone, and we have *plenty* of theories. But I also think it’s fair to say that the theories associated with ‘experimentalists’ (read: psycholinguists) bear on research questions that aren’t every linguist’s cup of tea. The disappointment that some folks might feel when moving from ‘theoretical’ to ‘experimental’ is assuming that their research questions will be straightforwardly testable within a psycholinguistic context, which is not often the case.. as this paper very nicely traces out for us.
I think I agree on all of this, aside from a couple of terminological quibbles. People are certainly working at different grains of analysis, specifying more or less detail in their theories. That is all fine and good. No grain of analysis is privileged, there are useful theories at each. For any phenomenon we simply have to place our bets on which grain of analysis will yield the most useful insights. It’s definitely right that folks who like to self identify as “theoretical” tend to be focused on the insights that come from one specific grain of analysis. And the folks who get labeled as “experimental” (few self-identify as such in my experience) are more commonly working at finer grains of analysis, though that association is certainly weaker. Although I disagree with some of the conclusions drawn from syntactic priming studies, it does seem to be a tool that requires non-trivial experimental effort and does also seem well-suited to addressing questions at an abstract grain of analysis.
But that’s where I find the terms ‘theoretical’ and ‘experimental’ counterproductive, as they set up unrealistic expectations. Everybody has theories, and everybody gets evidence. Some of the evidence is just a bit more of a pain to obtain.
I also squirm at the term “performance” since it tends to be such a grab bag. Sometimes it’s used to distinguish levels of analysis. Sometimes it’s used to distinguish tasks, e.g., the effect of uncertainty in comprehension. Sometimes it’s used to invoke distinct mechanisms. All reasonable distinctions. But things get confusing when they’re munged together.