The Ostwald Test: Historiography for Dummies
These ideas aren’t particularly original to me, but I’ll pretend like they are by giving them a name.
How good (how rigorous, how scholarly, how systematic, how analytical, how serious, how developed…) is the historiography on any particular topic? The sophistication of a research field isn’t necessarily the same thing as being able to tell whether any given historiographical argument is true or not, but there’s a pretty high correlation between the two.
There are so many ways to test this idea of historiographical sophistication – I’ll assume everyone knows to ask whether any non-published, i.e. archival, sources are used (unless you’re only looking at the media or the public sphere). But here are a few really simple and easy rules of thumb that I rely upon. An individual work should, ideally, pass at least two (probably three) of the following criteria, It’s much more damning, however, to find multiple works within a field that fail these tests. That’s a sign of some systemic problems within the field.
Step 1: Look at the subject of a work (a book, an article…) on the topic of interest.
Step 2: Look at the other sources that secondary source uses (e.g. in the footnotes and bibliography).
Step 3: Ask the following four questions:
I. Citations’ Age
Do the sources incorporate recent thinking on the subject? How old is your historiography, i.e. how old are your citations? Is it past its Best Before date? Examples that have jumped out at me over my career:
- Scholars in the 1990s or later still basing their understanding of Vaubanian siegecraft upon Guerlac’s analysis from the 1940s, or Blomfield’s 1930s-era analysis. Not a good sign.
- Scholars in the 1990s or later relying upon hagiographical treatments of the Duke of Marlborough that are 70+ years old. Not a good sign.
- Scholars in the 1990s or later relying upon Wright’s 1930s-era discussion of the customs of siege warfare. Not a good sign.
In short, if no significant reassessment of a field has occurred in 70+ years, that’s a bad sign.
The diagnosticity of the first question can be improved by asking the next question as well:
II. Pages-to-Coverage Ratio
Is the coverage adequately deep or pathetically shallow? For the sources cited in a secondary source (and for the secondary source itself), what is the ratio of page length to number of years/countries/topics covered? Is it 100 pages on one decade of a single country’s ‘life’, or 100 pages covering an entire continent’s 1000-year ‘age’? If the basis for a discussion in the modern literature comes purely from topical “art of war” works that claim to describe a continent’s worth of conduct across three centuries in 25 pages, that’s a bad sign.
Not every source needs to be a monograph, but if there are almost no recent monographs cited on a subject, that’s a bad sign. Either it means nobody has looked at it closely, or perhaps somebody has, but the author you’re reading hasn’t. That’s a bad sign.
We can add a further corollary:
III. Argument Coverage-to-Sources Coverage Ratio
Does the argument match the sources? Compare the geographical and chronological coverage claimed in an author’s argument with what their sources actually cover. You can probably cut their title a little slack. Does someone covering western European siege warfare in the War of the Spanish Succession claim that this equally applies to siegecraft in 1500 Poland? If so, that’s a bad sign.
One final, should-be-obvious, rule of thumb:
IV. Language Correspondence
Is the author bothering to see what his subjects actually thought? Does the language of the citations match the language of the people under study? Do the languages used in the author’s sources coincide with the languages spoken where the events took place? If, for example, you’re trying to explain why the Dutch did something, it would help if you could point to Dutch sources.
Four simple questions that will tell you a lot about the state of your historiography. In active (which I think we can use as a reasonable proxy for rigorous) subfields, a majority of academic works will almost always pass these four tests. I’d really love to see them used (as summary statistics) in book reviews – would tell us a lot.
So try it for yourself. And feel free to recommend any additional tests in the comments.
The Ostwald Test. Kinda like the Turing Test, but with no chance of research funding.
What is this reciprocity of which you speak?
In the debate over cultural vs. practical explanations for military behavior (assuming we need to prefer one over the other), the concept of reciprocity will undoubtedly surface. It’s frequently used to explain why some combatants broke real or apparent ‘rules’ of war, or otherwise violated expected norms of behavior. Often times it’s used to excuse bad behavior: soldiers, for example, may have done something that might be considered ‘bad’ in some circumstances, but they did it only after the other side did it first, or only after the other side’s past actions (or maybe just one action) made it clear that a more acceptable response was too dangerous to their own troops. In other words, their response was not conditioned by culture (e.g. a cultural hatred of the Other), but by pragmatism, maybe a rational desire to avoid casualties, or perhaps punishing the enemy for similarly bad behavior. You also see reciprocity come up a lot as a mechanism when discussing the “laws” of war – that reciprocity was one of the key factors limiting excessive violence. In the early modern context, reciprocity is often alluded to when describing how the post-Thirty Years War period became a period of “limited” war.
Given the frequency with which we mention the concept, I’m surprised how little we seem to understand it, at least in an early modern context. I’m sure it’s quite well covered in the literature on counterinsurgency (the Nazi response to partisan attacks comes quickly to mind), though Roy McCullough’s Coercion, Conversion and Counterinsurgency in Louis XIV’s France is about the only early modern study I know of. If reciprocity really does help explain an era of “limited” war, has anybody explained why reciprocity, which would seem to have a universal logic, apparently broke down from 1560-1648? (Yes, wars of religion and all that, but spell it out, e.g. does reciprocity fail when the stakes are so high that you’re not going to be deterred by whatever reprisal the enemy commits? Or more extreme acts are no longer seen as being so far outside the pale that they merit reprisal?…)
I admit I don’t have a really clear grasp of how exactly it worked on the ground, particularly in a conventional conflict between European states, and particularly how it was supposed to work over time. So here are my initial thoughts (and questions) about how military reciprocity was supposed to work in limiting early modern military excess. Somebody should write a book or article just on this, preferably a case study. Maybe somebody already has, in which case, let me know in the comments. I just hope I don’t have to read too much about MAD.
The System of Reciprocity Read More…
You know you’re a nerd when….
… the happiest you can remember being in months is when you discover that after TWENTY YEARS, Microsoft Excel has finally made it possible to group/outline columns so that the summary column can be on the left and the sub (i.e. grouped) columns will expand to the right. The Data-Group-Setting is to uncheck the default “Summary Column to the Right”.
The Old, Truly Evil, Way
Versus
And the Angels Sing!
I use Excel a bit for research and all the time for school (my fancy grade book), so apparently this assaulted my moral fibre for years. I knew I hated that feature, but I had no idea how much I truly loathed it.
I
AM
ECSTATIC!
Can’t military and diplomatic historians all just get along?
It seems we are fated to be yin and yang. The descent into armed conflict, the domain of military historians, represents the palpable failure of diplomacy, the subject of study for diplomatic historians. Yet ultimately the futility of the military historian’s war-mongering forces an inevitable return to negotiation, and some sort of peace. The circle completes itself.
I guess we need each other after all.
Lesaffer, Randall, ed. The Twelve Years Truce (1609): Peace, Truce, War and Law in the Low Countries at the Turn of the 17th Century. Leiden: Brill Academic Publishers, 2014.
Abstract:
The Twelve Years Truce of 9 April 1609 made a temporary end to the hostilities between Spain and the Northern Netherlands that had lasted for over four decades. The Truce signified a crucial step in the recognition of the Republic of the Northern Netherlands as a sovereign power. As the direct source of inspiration for the 1648 Peace of Munster the Truce is a crucial text in the formation of the early modern law of nations. As few other texts, it reflects the radical changes to the laws of war and peace from around 1600.
The Twelve Years Truce offers a collection of essays by leading specialists on the diplomatic and legal history of the Antwerp Truce of 1609. The first part covers the negotiation process leading up to the Truce. The second part collects essays on the consequences of the Truce on the state of war. In the third part, the consequences of the Truce for the sovereignty of the Northern and Southern Netherlands as well as it wider significance for the changing laws of war and peace of the age are scrutinised.
Table of Contents:
Introduction … 1
Randall Lesaffer
Part 1 Truce and Peace
1 The Twelve Years Truce … 7
Paul Brood
2 Preparing the Ground … 15
Alicia Esteban Estríngana
3 The Act of Cession, the 1598 and 1600 States General in Brussels and the Peace Negotiations during the Dutch Revolt … 48
Bram de Ridder Violet Soen
4 The Anglo-Spanish Peace Treaty of 1604 … 69
Alain Wijffels
Part 2 Truce and War
5 Left ‘Holding the Bag’ … 89
Peter Borschberg
6 The Tactical Military Revolution and Dutch Army Operations during the Era of the Twelve Years Truce (1592–1618) … 121
Olaf van Nimwegen
7 ‘Une oppression insupportable au peuple’ … 152
Tim Piceu
Part 3 Truce and Law
8 The United Provinces … 181
Beatrix C.M. Jacobs
9 How ‘Sovereign’ were the Southern Netherlands under the Archdukes? … 196
Georges Martyn
10 The Early Doctrine of International Law as a Bridge from Antiquity to Modernity and Diplomatic Inviolability in 16th- and 17th-Century European Practice … 210
Carlo Focarelli
11 From Antwerp to Munster (1609/1648) … 233
Randall Lesaffer, Erik-Jan Broers and Johanna Waelkens
12 ‘La dernière ancre de leur finesse’ … 256
Bernd Klesmann
13 The Treaty of London, the Twelve Years Truce and Religious Toleration in Spain and the Netherlands (1598–1621) … 277
Werner Thomas
The Amateur Side of the Hill
Consider this a historian-in-action thought piece, where I speculate on method, and whether I am convinced by an argument (that I haven’t read) based solely off of tea leaves offered by the author. And no, historians never ever make such summary judgments based off the slightest of evidence. Gripping (griping?) stuff.
Apropos a recent post on footnoting and historical evidence, I read an interesting yet odd interview of historian Tom Reilly (author of Cromwell: An Honourable Enemy and Cromwell was Framed) about his crusade to rehabilitate Oliver Cromwell (and his warts). Here’s the blog link – there’s a pop-up ad that you probably want to ignore, and you might want to check/clear your cookies after you’ve left the site if you have lingering doubts.
A self-proclaimed “amateur”, Reilly portrays himself pitted against politicized Irish academics, pedants who snipe at his citations rather than engage with his irrefutable evidence that Cromwell was dishonorably framed for murder. An interesting idea, though admittedly only one recent example in a series of revisionist histories seeking to rehabilitate Bad Guys of History. Unfortunately I don’t have the expertise or knowledge to settle whether Cromwell really ordered (or allowed) hundreds of Irish civilians to be slaughtered at Drogheda (you out there Wayne Lee?). For broad context I’d probably start with The Massacre in History, The Age of Atrocity (pp 251ff discuss Reilly’s claims as it turns out) and Theatres of Violence. Nor have I taken the time to do much research on either Reilly or his books. That doesn’t, however, prevent me from wildly speculating about the ideas at stake. I am particularly intrigued by the broader issues raised in the interview and the publisher’s (Chronos Books) website.
If you’ve read the above-linked interview, you too may have quickly been put on guard after an initial anodyne question about the origins of Reilly’s interest in Cromwell (he lives in Drogheda, site of the most infamous of the Cromwell “massacres”). By the second question the interview quickly descends into an attack on those damn academics: “Describe how difficult it was for an amateur writing in the current formally academic controlled climate.” Hey, that’s my People! As someone who’s complained far too often about the dominance of amateur popular military history in his own research area, I hadn’t really considered there would be people complaining that published non-fiction works were vetted by experts in the field. Score one for the publishers-as-gatekeepers side, I guess.
In response to this question about the difficulties of an academic-controlled publishing climate, the interviewee spent a surprising amount of time complaining about the last thing I would have expected: the difficulty of footnotes. Turns out his press editor insisted he go back and add footnotes for all his sources, requiring him to imitate a variety of footnote formats from other books. My initial thought was that he should justifiably complain if his press didn’t provide him with a style sheet. But more eye-catching was that he interpreted the arcana of footnotes as a painful and unnecessary introduction to the “bewildering world of academia.” Looks like history publishers get criticized no what matter what they do: for not including enough footnotes, or demanding that they include any at all. One more point for the publishers-as-gatekeepers.
The rest of the interview focused mostly on Reilly’s take on the politicized Irish historical community (or at least some of them), and how they were peddling myth as national identity. Reilly certainly wouldn’t be the first to make this charge. At the end of the interview, I was still a bit confused about his specific argument, so I looked for some context by checking Wikipedia and the press website, which has a sensationalist summary of the (self-published) book:
…”amateur historian Tom Reilly again throws down the gauntlet to professional historians everywhere. This book contains original and radical insights.
Breaking the mould of the genre, for the first time ever, Reilly publishes the contemporary documents (usually the preserve of historians) so that the authentic primary source documents can be interpreted by the general reader, without prejudice.”
Odd. In general, I certainly second his desire to see more quoting of sources and less politicization in academic history. But in this case the tone of both interview and publisher website strike me as rather counter-productive. Unless you want to sell controversy of course.
All of which leaves me with numerous questions I’d pursue if I had the time. Many of the questions that immediately come to mind leave me a bit skeptical of Reilly’s case as soon as I voice them. Does the fact that English writers were “much more circumspect” about Cromwellian massacres than Irish authors provide evidence that the Irish authors were wrong, as Reilly apparently believes? More strangely, have historians been jealously hoarding shiny primary sources from the general public, like some fire-breathing dragon? On this I’m pretty certain the answer is “No”: many of you know that it’s those who donate documents to the archives, and the electorate who doesn’t want to pay more taxes to publish these documents, that deserve most of the blame for those documents that are hidden. Every historian I know would love for all such documents to be freely available, and I can’t imagine there are many Irish historians who would protest greater dissemination either. Where Reilly interprets footnotes as an arcane secret language which only a cabal of professional historians and their acolytes can decipher, academic historians see them as quite practical attempts to actually encourage transparency.
More effort would uncover what exactly the academic historians’ critiques of Reilly’s works were, but his rebuttal to one review sounded uncomfortably like one of my second-year methods students who still thinks primary sources present a crystal-clear window to the past:
‘None of this is convincing’ said the Irish Times. ‘This is a painfully bad book’, said Dr Jason McElligott and he followed up with, ‘and it is tempting to suggest that its main use will be to teach students how not to conduct research, assess evidence or write prose.’ I [i.e. Reilly] was stunned. ‘But, the evidence, I cried. Look at the evidence’.
If one of my students were to respond to the Irish Times‘ critique by saying “But look at the evidence,” I’d probably suggest to that student that in order to rebut a criticism that you didn’t assess the evidence properly, you should do more than just reaffirm your contention that the evidence is self-evident – perhaps discuss what types of evidence were available, and evaluate their reliability. For that matter, I’d tell the hypothetical student that the historical evidence for an emotional topic like a massacre can be interpreted “without prejudice” only with great difficulty. I’d even suggest the student consider the possibility that those human beings creating those documents (hidden or not) might themselves have had opinions and prejudices of their own that deserve investigation, frames of reference that might influence what (and how) they remembered events. I have no idea if Reilly does just that in his books, but his alternation between martyrdom and bomb-throwing radical makes me wonder.
As an academic historian, my confusion and frustration about l’affaire Reilly is perhaps best summarized in the following three sentences from the interview:
“There is no need for footnotes, because the documents are there for all to see.”
To play the pedant, I’ll assume Reilly’s unfamiliarity with academic history means he doesn’t realize that even with transcribed documents, you still need some method of indicating their location for other researchers that might want to return to the original, or ask an archivist to make a copy.
“I didn’t want to hide behind a reference/footnote that I have interpreted on behalf of Joe and Josephine Public. I included them so people can interpret them themselves.”
I too have long sought to provide contemporary quotations whenever possible, although here too I hope Reilly recognizes that his statement is rather methodologically naive if taken at face value. It’s impractical for any historian, amateur or academic, to expect the primary sources to ‘speak for themselves’ in a clear voice, particularly when those primary sources are of varying degrees of reliability, when they contradict each other, and since they were often created for different purposes than those which historians use them for. Interpretation unavoidably requires specialized knowledge (open to “amateurs” who take the time to study the issue), and it requires serious effort, as well as an appreciation of all the decisions the historian must make before he/she “understands” the past:
- The selection of which extant documents are worth consulting (and why) requires contextual interpretation.
- Deciding which types of archives and documents are relevant to your research (and the exact framing of your research question), is itself interpretation.
- Numerous aspects of every document – from the genesis of the document to the author’s reliability to paleography to vocabulary to allusions to context to genre conventions – will necessarily require interpretation by those with specialized knowledge.
- Any evidence you extract from those documents will be of no import unless you frame them within a particular argument or historiography, which is again a matter of interpretation.
What’s my point? Ultimately Reilly may be 100% right in every detail and every claim, though it’s quite possible that no conclusive answer is even possible (barring exhumations, and even then). But from what little I’ve read so far, I’m not the least bit surprised that academics have reacted negatively to his work. At first glance, all these impressionistic bits raise flags for this academic historian.
One of the tactics used by more unsavory revisionist historians has been to focus on some small, discrete historical event within a larger historical phenomenon, question the standard historical interpretation by producing some document (or challenge opponents to produce their “smoking gun”), charge that there is a conspiracy to keep the “proof” from the public (often despite acknowledging in passing that there has in fact been a long debate over the matter), and leave the implication that the larger event of which this was a part couldn’t have been all that bad if this specific incident wasn’t nearly as bad as we’ve thought. I have no idea if Reilly is intentionally doing this or not, and freely admit I’m in uber-speculation mode here. I would, however, be interested to hear his take on English activity in Ireland in the 1640s-1650s more broadly, or even on Essex/Mountjoy in Ireland (under Elizabeth), or William of Orange in the 1680s-1690s.
Any thoughts on the Cromwell debate, or the broader issues raised about scholarly vs. popular history?
Update: Just saw that Reilly posted his actual argument on the Trumpet of Sedition blog, which I’ll probably get around to reading sometime, and maybe even retract this blog post, or not.
Spanish mutinies in an earlier age
Adding to Geoffrey Parker’s classic account of mutinies during the Dutch Revolt, a new article from latest issue of Journal of Military History turns back the clock a bit.
Oh yeah? Says who?
In several previous posts I’ve expressed concern about the pressures encouraging historians to soft-pedal or even minimize the evidence supporting their claims in their published works. In our current age of decreasing attention spans and declining academic books sales, there is growing pressure to eliminate the footnotes (or at least relegate them to annoying endnotes) and shorten the lengths of scholarly tomes by eliminating discussion of historiography and cited evidence. Some historians even counsel us to cull the evidence for our claims to a single example. What’s a historian to do?
Do we really need more than one example? Read More…
But how will the children read Script Ohio?
Yet Another Chronicle article on cursive script dying with a flourish – this time from a university rare-book librarian responding to an undergrad researcher who doesn’t “do” cursive. Money quote:
An informal survey of rare-book librarians and archivists indicates that our experience at Illinois is not uncommon. Research on manuscripts from the 17th to the 20th century is no longer possible for most undergraduates at American colleges.
So sign your 8-11 year old up for their summer “Camp Cursive” – and make sure they know what a long-s is, while you’re at it.
This story also gives me an excuse to post the following advertisement that I just came across. From the 1712.12.27 issue of the British Mercury, a newspaper intended for those busy business executives and bureaucrats on-the-go:
Three hundred years ago somebody apparently thought cursive printed documents came with their own tamper-evident seal.
The Eighty Years War just got a whole lot longer
In a comment Averrones pointed out a recent (in historical terms) release of a new publication on the rise of the Dutch fiscal-military state. I like looking beyond the usual 1648 terminus; and if there was ever a case study that required a fiscal-military approach, and an author to do it, this would be it.
Skulking in Holes and Corners 
Recent Comments