Archive | Research RSS for this section

The Google Giveth and the Google Taketh Away

In other words, hopefully you’ve already downloaded all those tasty EMEMH works from Google Books, like I’ve warned. Because some of them are disappearing from Full View, as publishing companies (I’m guessing) pay Google some money to sell print copies on Amazon and elsewhere. (See, I knew my hoarding instincts and general obsessive-compulsiveness would come in handy.)

But all hope is not lost, for if you can still find interest EMEMH PDFs, Google Books has recently decided to include the OCRed text layer with the PDF download as well, which means they are searchable. Just don’t look too closely at the results…

Automating Newspaper Dates, Old Style (to New Style)

If you’ve been skulking over the years, you know I have a sweet spot for Devonthink, a receptacle into which I throw all my files (text, image, PDF…) related to research and teaching. I’ve been modifying my DTPO workflow a bit over the past week, which I’ll discuss in the future.

But right now, I’ll provide a little glimpse into my workflow for processing the metadata of the 20,000 newspaper issues (yes, literally 20,000 files) that I’ve downloaded from various online collections over the years: Google Books, but especially Gale’s 17C-18C Burney and Nicholls newspaper collections. I downloaded all those files the old-fashioned way (rather than scraping them), but just because you have all those PDFs in your DTPO database, that still doesn’t mean that they’re necessarily in the easiest format to use. And maybe you made a minor error, but one that is multiplied by the 20,000 times you made that one little error. So buckle up as I describe the process of converting text strings into dates and then back, with AppleScript. Consider it a case study of problem-solving through algorithms.

The Problem(s)

I have several problems I need to fix at this point, generally falling under the category of “cleaning” (as they say in the biz) the date metadata. Going forward, most of the following modifications won’t be necessary.

First, going back several years I stupidly saved each newspaper issue by recording the first date for each issue. No idea why I didn’t realize that the paper came out on the last of those dates, but it is what it is.

Screen Shot 2014-03-09 at 7.53.14 PM

London Gazette: published on Dec. 13 or Dec. 17?

Secondly, those English newspapers are in the Old Style calendar, which the English stubbornly clung to till mid-century. But since most of those newspapers were reporting on events that occurred on the Continent, where they used New Style dates, some dates need manipulating.

Automation to the Rescue!

To automate this process (because I’m not going to re-date 20,000 newspaper issues manually), I’ve enlisted my programmer-wife (TM) to help me automate the process. She doesn’t know the syntax of AppleScript very well, but since she programs in several other languages, and because most programming languages use the same basic principles, and because there’s this Internet thing, she was able to make some scripts that automate most of what I need. So what do I need?

First, for most of the newspapers I need to add several days to the listed date, to reflect the actual date of publication – in other words, to convert the first date listed in the London Gazette example above (Dec. 13) into the second date (Dec. 17). So I need to take the existing date, listed as text in the format 1702.01.02, convert it from a text string into an actual date, and then add several days to it, in order to convert it to the actual date of publication. How many days exactly?

Well, that’s the thing about History – it’s messy. Most of these newspapers tended to be published on a regular schedule, but not too regular. So you often had triweekly publications (published three times per week), that might be published in Tuesday-Thursday, Thursday-Saturday, and Saturday-Tuesday editions. But if you do the math, that means the Saturday-Tuesday issue covers a four-day range, whereas the other two issues per week only cover a three-day range. Since this is all about approximation and first-pass cleaning, I’ll just assume all the issues are three-day ranges, since those should be two-thirds of the total number of issues. For the rest, I have derivative code that will tweak those dates as needed, e.g. add one more to the resulting date if it’s a Saturday-Tuesday issue, instead of a T-R or R-S issue. If I was really fancy, I’d try to figure out how to convert it to weekday and tell the code to treat any Tuesday publication date as a four-day range (assuming it knows dates before 1900, which has been an issue with computers in the past – Y2k anyone?).

So the basic task is to take a filename of ‘1702.01.02 Flying Post.pdf’, convert the first part of the string as text (the ‘1702.01.02’) into a date by defining the first four characters as a year, the 6th & 7th characters as a month…, then add 2 days to the resulting date, and then rename the file with this new date, converted back from date into a string with the format YYYY.MM.DD. Because I was consistent in that part of my naming convention, the first ten characters will always be the date, and the periods can be used as delimiters if needed. Easy-peasey!

But that’s not all. I also need to then convert that date of publication to New Style by adding 11 days to it (assuming the dates are 1700 or later – before 1700 the OS calendar was 10 days behind the NS calendar). But I want to keep the original OS publication date as well, for citation purposes. So I replace the old OS date on the front of the filename with the new NS date, and append the original date to the end of the filename with an ‘OS’ after it for good measure (and delete the .pdf), and Bob’s your uncle. In testing, it works when you shift from one month to another (e.g. January 27 converts to February 7), and even from year to year. I won’t worry about the occasional leap year (1704, 1708, 1712). Nor will I worry about how some newspapers used Lady Day (March 25) as their year-end, meaning that they went from December 30, 1708 to January 2, 1708, and only caught up to 1709 in late March. Nor does it help that their issue numbers are often wrong.

I’m too lazy to figure out how to make the following AppleScript code format like code in WordPress, but the basics look like this:
–Convert English newspaper Title from OSStartDate to NSEndDate & StartDate OS, +2 for weekday
— Based very loosely off Add Prefix To Names, created by Christian Grunenberg Sat May 15 2004.
— Modified by Liz and Jamel Ostwald May 26 2017.
— Copyright (c) 2004-2014. All rights reserved.
— Based on (c) 2001 Apple, Inc.

tell application id “DNtp”
set this_selection to the selection
if this_selection is {} then error “Please select some contents.”

repeat with this_item in this_selection

set current_name to the name of this_item
set mydate to texts 1 thru ((offset of ” ” in current_name) – 1) of current_name
set myname to texts 11 thru -5 of current_name

set newdate to the current date
set the year of newdate to (texts 1 thru 4 of mydate)
set the month of newdate to (texts 6 thru 7 of mydate)
set the day of newdate to (texts 9 thru 10 of mydate)

set enddate to newdate + (2 * days)
set newdate to newdate + (13 * days)
tell (newdate)
set daystamp to day
set monthstamp to (its month as integer)
set yearstamp to year
end tell

set daystamp to (texts -2 thru -1 of (“0” & daystamp as text))
set monthstamp to (texts -2 thru -1 of (“0” & monthstamp as text))

set formatdate to yearstamp & “.” & monthstamp & “.” & daystamp as text

tell (enddate)
set daystamp2 to day
set monthstamp2 to (its month as integer)
set yearstamp2 to year
end tell

set daystamp2 to (texts -2 thru -1 of (“0” & daystamp2 as text))
set monthstamp2 to (texts -2 thru -1 of (“0” & monthstamp2 as text))

set formatenddate to yearstamp2 & “.” & monthstamp2 & “.” & daystamp2 as text

set new_item_name to formatdate & myname & ” ” & formatenddate & ” OS”
set the name of this_item to new_item_name

end repeat
on error error_message number error_number
if the error_number is not -128 then display alert “DEVONthink Pro” message error_message as warning
end try
end tell

So once I do all those things, I can use a smart group and sort the Spotlight Comment column chronologically to get an accurate sense of the chronological order in which publications discussed events.

This screenshot shows the difference – some of the English newspapers haven’t been converted yet (I’m doing it paper by paper because the papers were often published on different schedules), but here you can see how OS and NS dates were mixed in willy-nilly, say comparing the fixed Flying Boy and Evening Post with the yet-to-be-fixed London Gazette and Daily Courant issues.

DTPO Newspapers redated.png

Of course the reality has to be even more complicated (Because It’s History!), since an English newspaper published on January 1, 1702 OS will publish items from continental newspapers, dating those articles in NS – e.g., a 1702.01.01 OS English newspaper will have an article dated 1702.01.05 NS from a Dutch paper. So when I take notes on a newspaper issue, I’ll have to change the leading NS date of the new note to the date on the article byline, so it will sort chronologically where it belongs. But still.

There’s gotta be a better way

In preparation for a new introductory digital history course that I’ll be teaching in the fall, I’ve been trying to think about how to share my decades of accumulated computer wisdom with my students (says the wise sage, stroking his long white beard). Since my personal experience with computers goes back to the 80s – actually, the late 70s with Oregon Trail on dial-up in the school library – I’m more of a Web 1.0 guy. Other than blogs, I pretty much ignore social media like Facebook and Twitter (not to mention Snapchat, Instagram, Pinterest…), and try to do most of my computer work on a screen larger than 4″. So I guess that makes me a kind of cyber-troglodyte in 2017. But I think that does allow me a much broader perspective of what computers can and can’t do. One thing I have learned to appreciate, for example, is how many incremental workflow improvements are readily available, shortcuts that don’t require writing Python from the terminal line.

As a result, I’ll probably start the course with an overview of the variety of ways computers can help us complete our tasks more quickly and easily, which requires understanding the variety of ways in which we can achieve these efficiencies. After a few minutes of thought (and approval from my “full-stack” computer-programming wife), I came up with this spectrum that suggests the ways in which we can make computers do more of our work for us. Toil, silicon slave, toil!

Computer automation spectrum.png

Automation Spectrum: It’s Only a Model

Undoubtedly others have already expressed this basic idea, but most of the digital humanities/digital history I’ve seen online is much more focused on the extreme right of this spectrum (e.g. the quite useful but slightly intimidating Programming Historian) – this makes sense if you’re trying to distantly read big data across thousands of documents. But I’m not interested in the debate whether ‘real’ digital humanists need to program or not, and in any case I’m focused on undergraduate History majors that often have limited computer skills (mobile apps are just too easy). Therefore I’m happy if I can remind students that there are a large variety of powerful automation features available to people with just a little bit of computer smarts and an Internet connection, things that don’t require learning to speak Javascript or Python fluently. Call it kaizen if you want. The middle of the automation spectrum, in other words.

So I’ll want my students, for example, to think about low-hanging fruit (efficiency fruit?) that they can spend five minutes googling and save themselves hours of mindless labor. As an example, I’m embarrassed to admit that it was only when sketching this spectrum that I realized that I should try to automate one of the most annoying features of my current note-taking system, the need to clean up hundreds of PDFs downloaded from various databases: Google Books, Gale’s newspaper and book databases, etc. If you spend any time downloading early modern primary sources (or scan secondary sources), you know that the standard file format continues to be Adobe Acrobat PDFs. And if you’ve seen the quality of early modern OCR’d text, you know why having the original page images is a good idea.

But you may want, for example, to delete pages from PDFs that include various copyright text – that text will confuse DTPO’s AI and your searches. I’m sure there are more sophisticated ways of doing that, but the spectrum above should prompt you to wonder whether Adobe Acrobat has some kind of script or macro feature that might speed up deleting such pages from 1,000s (literally) of PDF documents that you’ve downloaded over the years. And, lo and behold, Adobe Acrobat does indeed have an automation feature that allows you to carry out the same PDF manipulation again and again. Once you realize “there’s gotta be a better way!”, you only need to figure out what that feature is called in the application in question. For Adobe Acrobat it used to be called batch processing, but in Adobe Acrobat Pro DC such mass manipulations now fall under the Actions moniker. So google ‘Adobe Acrobat Actions’ and you’ll quickly find websites that allow you to download various actions people have created. Which allows you to quickly learn how the feature works, and to modify existing actions. For example, I made this Acrobat Action to add “ps” (primary source) to the Keywords metadata field of every PDF file in the designated folder:

Screenshot 2017-05-10 18.52.17.png

I already copied and tweaked macros and Applescripts that will add Keywords to rich text files in my Devonthink database, but this Adobe solution is ideal after I’ve downloaded hundreds of PDFs from, say, a newspaper database.

Similarly, this next action will delete the last page of every PDF in the designated folder. (I just hardcoded to delete page 4, because I know newspaper X always has 4 pages – I can sort by file size to locate any outliers – and the last page is always the copyright page with the nasty text I want to delete. I can, for example, change the exact page number for each newspaper series, though there’s probably a way to make this a variable that the user can specify with each use):

Screenshot 2017-05-10 18.52.43.png

Computers usually have multiple ways to do any specific task. For us non-programmers, the internet is full of communities of nerds who explain how to automate all sorts of software tasks – forums (fora?) are truly a god-send. But it first requires us to expect more from our computers and our software. For any given software, RTFM (as they say), and then check out the software’s website forum – you’ll be amazed at the stuff you find. Hopefully all that time you save from automation won’t be spent obsessively reading the forum!

Catch up, Post-First-Snow-Flurries 2016 edition

Busy with various projects, including designing a digital history lab.

But I did attend the Joe Guilmartin memorial conference earlier this semester, where the attendees alternated between laughing at our collective recitation of Guilmartin’s many bons mots, and growing contemplative (and perhaps wiping away a stray tear or two) as his former advisees testified to his impact on their academic careers.

My contribution to the proceedings was to open up the conference with a broad think-piece about developing a more precise taxonomy/typology of the levels of war, spurred by JFG’s introduction to the subject long long ago. A few examples of the course materials he handed out in his seminal European Warfare course.

1st page of JFG syllabus, circa 1993 - note the early use of visuals.

1st page of JFG syllabus, circa 1993 – note the early use of visuals.

JFG Definitions and Tactics handout

JFG Definitions and Tactics handout


So here’s the revised “strategy” matrix. There are plans for conference proceedings, wherein I’ll explicate the below chart (and much more), and add a few more levels. So feel free to leave suggestions or comments. especially about those pesky column labels.

Next draft of military techniques (was Strategy Matrix, but I'm having issues with the term "strategy")

Next draft of military operational techniques (was “Strategy Matrix”, but I’m having issues with the term “strategy”). The gray cells are generally more extreme war objectives, often described as “total war.”

I sure do love Lincoln and Washington

Because they give us U.S. faculty on a MWF teaching schedule a full week off in the Spring, and that’s before Spring Break. Which, combined with the two consecutive snow days last Friday and this past Monday, mean I’ve had the time to finish up my siege capitulation chapter (okay, 99% done) that I’ve been working on forever. Literally. I wrote a graduate seminar paper on the subject circa 1994.

Why has it taken so long to finish this chapter with a target length of only 12,000 words? Let me count the ways, leaving aside non-project issues: Read More…

Snowday 2016

Which gives me time to throw a quick blog post up into the Ether – now that we’ve finally gotten our electricity back, after six hours. I’ve been busy with teaching two grading-intensive courses (a senior seminar on the Age of Enlightenment and the Historical Research and Writing course), as well as my European Warfare 1337-1815, but also getting some overdue research done. So I might as well share little bits that are too long to publish in full.

As I’m nearing the completing of my long-delayed siege capitulation chapter, I came across this humorous piece illustrating a lot of the themes I’ve been exploring over the past few years. Without further ado, I bring you another episode in the rarely-boring Country-man and Observator Show, from 1706.

Cm: It pleases me and all good Christian Englishmen, Master, I have a whole Budget full of Victories.

O: What more Victories? New Ones, Roger?

Cm: Yes, Master, all Spike and Span New. Let me see, Master, I’ll lay ’em out before you in Mode and Form. First and foremost I present with the Surrender of Ostend, that’s the Place you wanted to have Taken, and so I hope you are pleas’d for One, especially since it was Taken in less time than you thought for [it defended itself for more than three years during the Eighty Years War].

As soon as my Lord Overkirk began to fling his Bombs on one side, and the English Fleet did the like on the other side of the Town, the French and Spaniards began to Squeak like so many Rats and Wessels between two Fires. Ah, Master, ’tis a sad thing to be Roasted at that rate; and while a Body is turning upon the Spit to be Basted with huge ugly Bombs and stinking Carcasses [an incendiary bomb]; ’tis enough to Fright any Body. I’ll warrant it the poor Frenchmen Drip’d more T—- [Turd] than Tallow; the heat of the Fire shrivel’d their poor thin hunger-starv’d Carcasses.

But there is one thing I observ’d upon the Papers that seems very Chomical, I cou’dn’t forbear Laughing at it: Master Mothe [La Mothe-Houdancourt], the French Governor in the Town, when it was Surrender’d, excused the Bravery of his Men, which he said Was quite lost in Defending a Ravelin; but he did not attribute it to a Natural Cause, but to Witchery and Devildom, and said All his Men were Bewitch’d. Aye, thought I, and so was thy Master Bewitch’d too, when he sent a Mothe [i.e. a moth] to take Care of the Cloathing of such a Town as Ostend. The Notion of Witchery is a poor excuse for Cowardice, and being over-match’d in Bravery and Skill in Martial Affairs.

But, Master, I foresee this Notion of Witchery will spread a great way; Anjou he’s coming Home Bewitch’d and Bedevil’d. Bavaria and Collogn, the two Brothers of Treachery, they are Bewitch’d and Hagg-Riden out of their Country. Prince Eugene he has Bewitch’d poor Vendosme, as I’ll tell you By and By. But the Duke of Marlborough he has Bewitch’d all Flanders, as the Earl of Peterborough has done all Spain. Bless me, Master, was there ever such Witchery, such a parcel of Martial Necromancers ever known at one time in the World? One would think that all the English Forces had been rais’d in Lancashire [the Pendle witches 1612], and were the Legitimate Issue of Teague O Devilly, begotten on Mother Demdyke [one of the Lancashire witches].


But Master, it rejoyces my Heart to see this Witchery as Monsieur Mothe calls it, go on o’the other side of the Water. Dendermond is Bewitch’d already; and a Spell is lay’d upon Newport, that will be actually Bewitch’d in twelve or fourteen Days time; nay, I heard some of our Coffee-House Wizzards say, That before the Campaign is ended Dunkirk will be horribly Bewitch’d. Ha! thought I, will it so? Then the Prophecy that I remember I read, concerning the French King, may come to pass.

Lorrain you Stole, by Fraud you got Burgundy,
Dunkirk you Bought, and you shall Pay for’t one Day.

Obs: He has Pay’d for’t long ago.

Cm: Aye, Master, to a Corrupt Minister that had no Power to Sell it and Receive the Money; and, I think the English Nation were Bewitch’d at that time, that they did not Hovel-Post that Minister. But now, Master, is our time to make the French King pay for Buying Stollen-Goods; and I foresee we shall do it with a Vengeance.

There’s another great piece of Witchery coming on the Stage too: The Earl of Rivers is going to Bewitch a Power of People somewhere or other; and the French King, tho’ he is an Old Wizzard, and has his Familiar, Goody Maintenon, always about him, he can’t tell whether they are going: But they are going to Bewitch some of his People, that’s for certain; because they carry with ’em Mortars, Bombs, Great and Small Guns, and other Instruments of War, the Spells with which the Duke of Marlborough has Bewitch’d so many Towns and People in Flanders. And what is most inconsolable to the French King is, that 12000 of those very Wizzards that Bewitch’d all Flanders are Ship’d off at Ostend, and wait only for a Wind to joyn in this Expedition.

Well, for my part I would give a Pot of October [brew] to see how the French King himself is Bewitch’d at this News. What are become of all his Little Imps that us’d to Creep into the Cabinet of Princes, even thro’ the Little Key-Holes? That when we had an Expedition forwarded, as at this present, could tel him it was to be at Camaret? [Did Tutchin know that Marlborough is traditionally credited with spilling that particular secret?] That when we had sent a Spy to France he was so effectually told of it, that when the Vessel Arrived, his Officers could come to Her side and ask for Lame Puckle? Master, I fancy the French King is Bewitch’d, because his Devils have lost their Power, because our Devils, with whom they held a Correspondance are Exorcis’d, and render’d Incapable of doing us any more Mischief. Tar-box for that.

Obs: Indeed, Honest Country-man, the little Shifts that are made use of by the French King, his Ministers and Generals, to excuse their Bankrupcty of Power, are so very Weak, that none but the Vassals of France, who must have their Eyes put out for Seeing can help Laughing at them. ‘Tis but t’other Day the French Kings Minister at Madrid told the Grandees of Spain, Asembled for that purpose, That his Master would rather call Home his Grandson, the Duke de Anjou, than that such Sacriledges should be Committed in a Catholick Kingdom by Wicked Heretics. This is the specious Pretence for calling Home his Spanish PERKIN; but who can believe the sincerity of the French King in this Point, who has, himself, been the most Sacrilegious Monster that ever Europe Bred? That has spared neither Religious-Houses, nor their Inhabitants, or the Lands Given and Settled to Pious Uses, when it has been his Interest to Seize them; and had he not been Diverted by a War in Eighty Eight, he had Wag’d War upon his Holy-Father, the Pope, at that time.

Monsieur Mothe’s Whim of Witchery is Comical enough. A good Excuse for being Beaten. This strange Reflux of French Valour cannot proceed from any Natural Cause; no, by no means. The French Courage can never be decay’d in its Nature, ’tis some Evil Planet Governs, and the French Troops are certainly Bewitch’d.

I might here assert that ’tis Natural for Men, so much beaten as the French has been of late every where, and in all Engagements, to be Cow’d and Disspirited, that ’tis also Politick for Men when they find the Dice of War run against ’em to leave off Playing at Soldiers. But to attribute the want of French Courage to Witchcraft, is in so many Words telling the World, That the Devil Reigns in the Year 1706, and that his Ancient Alliance with the French King is come to a Period.

Honest-Roger, all this is the Work of the Almighty, he gives Courage and he Disspirits Men; God is against that Wicked Tyrant, that Grizly Oppressor of Mankind, that Bloody Butcher of Protestants, and can he then Prosper? Who can withstand the fixt Resolves of the Eternal Being? Or can Humane Force over-come the Almighty Arm? No, Roger, ’tis Providence, and not Witchcraft has Disspirited the French Forces; and the same Providence may, and will do the same by us, if we don’t own that High Hand by whose Influence we Conquer, and make suitable Returns for such Auxiliaries of Divine Power.

Cm: Now, Master, out of my Budget I pull a Notable Victory, obtained by Prince Eugene over the Duke D’Vandosme. The French Forces in Italy are also Bewitch’d; this Magick Spell fles a great way: I fancy some or other of our Side has got Pandora’s Box, and opens it at every turn when he pleases, and let his Poisons fly to taint the Frenchmen with Cowardice. Master, in short we han’t yet got the Particulars of the Fight in Italy, but of this we are certain, that there has been a Fight and that the French Army is Routed Horse and Foot, and that Vandosme is Mortally Wounded, so that he’ll hardly be in a Condition to Act against the Duke of Marlborough.

And now what wil become of the Something of Orelans that was to have Vandosme’s Post? I fancy Vandosme was a Malicious Fellow, and being Incens’d at the French King, for putting Orleans over his Head, he carried his Bewitch’d Army to Prince Eugene to have ’em Kill’d, and so that Orleans, when he came, might have no Army to Command. Those Frenchmen are Spiteful, let ’em be Bewitch’d or not Bewitch’d.

Obs: That Prince Eugene, the most Neglected of any general, has done the greatest Exploits in War that ever any Age could produce, always out-Number’d, out of time Recruited, Troops Ill Paid, and yet always Victorious. Unrewarded, and yet Faithful to his Trust; and what is yet more Glorious, as Poor now, after being General so many Campaigns, as at first when he held the Truncheon of Honour. Covetousness is the worst Vice a General can have, a Covetous Man can never be so true to his Trust as he that despises Money, and seeks nothing but the Good of his Country in the Service of it; a Covetous Captain is a Rent to a Kingdom, you must purchase his Fidelity at so high a Rate as the Enemy may’nt out-bid you for his Treachery. But Brave Prince Eugene has not sought himself, but the Interest of his Master, and the Common Good of Europe, and has made such a Stand in Italy as future Ages will wonder at.

Cm: Master, they say the Emperor will give him part of the Duke of Bavaria’s Country, and so Reward Fidelity and true Merit with the Forfeitures arising by Treachery.

Obs: If he had all Bavaria I should rejoyce exceedingly. But, Roger, when the account of this Victory comes confirm’d it will prove a Glorious Stroke on Behalf of the Confederates. The Duke of Savoy will by this means joyn the Imperial Forces, the Seige of Turin will be raised, and the Country of Savoy will be cleared from the Vermin that now Infect it; and so, Roger, let us go to Bed with a good Health to Prince Eugene.

God, I love The Observator.

One more bit of background: using witchcraft to (satirically) describe an otherwise-inexplicable military victory was an old trope – see, for example, A Letter from a Trooper in Flanders, to His Comerade: Shewing, that Luxemburg is a Witch, and Deals with the Devil (1695).

And I still haven’t found, what I’m looking for…

Photo of my “drafting space” for this interminable siege capitulation chapter. I mean, I thought my analysis of how garrison capitulations were interpreted was complicated, but sheesh. That’s what I get for trying to be systematic about 49 different sieges (125 if I were greedy), with a few dozen sources on each.

The Ugly Side of Historical Research

The Ugly Side of Historical Research

A. iMac with Scrivener for composing (I’m ambivalent about Scrivener’s sidebar chunks – they get out of control too quickly); Zotero for bibliographing; and Devonthink for the database (sources primary & secondary, searching, metadata, research journal…).
B. Printed (very, very, very) rough draft of chapter sections, with edits.
C. Scraps of paper, one sub-argument (with multiple subpoints) per quarter-sheet. Incorporate into the draft and into the recycling basket you go.
D. Tiny post-it notes with single facts/documents/thoughts/prose to include somewhere. (All the quotes, notes and source summaries are kept in DTPO.)
E. File folders with larger miscellaneous notes on the project. Usually stored long-term in case I ever go back to the topic.

The laptop is usually on the right when I need additional screenspace.
And of course hard copy books occasionally make a cameo as well. Still considering buying a book holder, unless Wayne is going to build a book wheel for me.

I’m usually still thinking through various bits at the drafting stage (“start writing before you think you’re ready to write – or at least by the time it’s already overdue”), so I:

  1. Write down questions and ideas on small sheets of paper – constantly asking myself: “What’s my main point here?” and “Why would anybody care?” and “Who am I arguing with?”
    1. Common questions for this project: What was the process by which A surrendered? How did X, Y and Z respond to the surrender of A? What did people say about the idea of Q? How was the term W used at the time? What was the context in which they made these statements? Are there patterns (geography, chronology, nationality, winning/losing side, relationship to A…) to these responses? How do the answers of the above questions fit into my argument?
  2. Collect these slips of paper at the desk, and start doing some searches in DTPO for the answers:
    1. First skim through documents on the topic I’ve already identified (DTPO topical groups).
      Screenshot 2016-01-08 15.27.36
    2. Then search for relevant people and documents that I’ve already identified (person involved with event, specific source that covers the period/place…). Maybe do a proximity search of some basic terms.
      Screenshot 2016-01-08 15.30.20
    3. Then skim through documents around the date of the event in question, e.g. search all the documents c. 1708.12.07 for commentary on the capitulation of Lille. Damn, I found something else interesting I have to include.
      Screenshot 2016-01-08 15.43.54
  3. Interpret my findings (possibly requiring more thinking-on-paper).
  4. Add my findings (data, conclusions) into the draft in Scrivener.

Still working on a better system: I compose drafts on the computer, but am still old-school enough that I “think” better on paper. There is no substitute for the portability, persistence and spreadability of paper. Just make sure they’re centralized, and ephemeral as well.

Can you tell I’m teaching a research seminar and Historical Research and Writing in a few weeks?