Data visualization during Election Week

I’ve been spending the week, like most of the world, seeped in the U.S. elections held on Tuesday. I’m finding myself more detached than I would have expected, but still. I have also received much stimulation from two lovely online conferences: Bookshelves in the Age of the COVID-19 Pandemic and Papers for the People: A Symposium on the Dime Novel.

My brain has also been buzzing from the questions asked after my “Nightmare Before Christmas” talk last week at the Five College Book History Seminar (which went well, thanks for asking!). The Google Slides are freely accessible, and I’ve uploaded the video to YouTube.

As a result, my brain has been mashing up my two areas of data nerdiness, elections and book history, to come up with data visualizations to represent the transnational, transmedial and cross-class flow of texts from one area of popular culture to another.

During a very productive shower yesterday, I came up with two data visualizations for projects I am working on:

• using a vote-transfer-type diagram to illustrate the move of American texts into penny bloods for a proposal I plan on sending to the Children’s/Young Adult Series Books and Dime Novels division of the Popular Culture Association for its 2021 conference (deadline 16 November!);
• juxtaposing two slick vertical timelines (just a straight vertical line with events branching out either side to allow for a total of four categories of events) and adding a curvy arrow from the left to the right to illustrate when (and which edition of) E.D.E.N. Southworth’s texts travelled from the U.S. to appear in penny weekly numbers U.K. for an upcoming article co-written with Melissa Homestead to be submitted to the Victorian Popular Fictions Journal special issue on piracy and plagiarism.

My goal was to produce the first over the course of the afternoon and to document the process in this post. I did not finish in the time I had hoped for, but after 11 hours working straight, I managed to produce the following diagram.

I had not realized how much work I would need to put in to come up with the numbers I’d need to input into to the diagram. This proof of concept has revealed, unsurprisingly perhaps, that a database designed to store penny bloods is much better at spitting out refined categories for penny bloods than for any of the works that publishers used as source texts.

Overall, this result is very encouraging to me. I am certainly tempted to redesign the part in which I store information about source texts to make it easier to find meaningful groupings at a subnational level. If you don’t care how I built the diagram, you can skip directly to the conclusion to see how I’ve used it in the 18 hours since I’ve created it.

How hard can it be?

I’ve placed the end result at the top, which makes it all seem fairly straightforward. What was I doing for 11 hours? Once I had found what this type of diagram was called, I had to figure out how to group the texts on both sides, both conceptually (what goes together? what does not?) and formally (how do I select this grouping —and only this grouping— in my MySQL database).

Since the right-hand side seemed fairly straightforward in my mind, I started with the left-hand side, got very lost, formalized the right-hand side and finally returned to the left-hand side.

What do you call that?

The first step to finding a way to do a vote-transfer-type diagram was to know how it’s called to then be able to google said name. I did a Facebook post, but also thought to ask Leonardo Carella. He’s a political scientist well versed in data visualization and my “Twitter friend” from last year, when I was covering the British elections in French as a data analyst on my elections blog Prime à l’urne/Primed for the Polls.

As a pleasant surprise, within two minutes, he had replied “Sankey diagram”. I had my googleable term!

I enjoyed the from Data to Viz explanations and was pleased to see I could produce one either with R (which I know) or Python (which I’m learning with Le Wagon‘s free webinars). Google and Leo both offered up SankeyMATIC as a less programming-y option.

So what data do I need, and how can I extract it from my database?

I was looking into the required format for the data in SankeyMATIC when Sylvain Bérubé, a Facebook friend from my partisan politics days, suggested Flourish, which produces really slick diagrams. As I had seen in SankeyMATIC, what I’d need was a dataset in which “each row has a source name, target name and flow value”, as Flourish puts it.

What did this mean in the context I was thinking of? Sources on the left would be American texts and targets on the right would be penny bloods. They must however both be subdivided into categories.

In my research, so far, I’ve been thinking of penny bloods as divided into publishers as well as specific periodicals. This information is recorded in my database (though it involves some tricky JOIN queries).

The right-hand side divisions were thus fairly straightforward in my mind. The left-hand side required a bit more thinking.

How to categorize the American sources

In terms of sources, I’ve been thinking of American texts reprinted in penny bloods as falling roughly into three clearly delineated categories:

1. James Fenimore Cooper, reprinted in serial anthologies in the late 1830s (which I studied with Melissa in 2014 for a conference I couldn’t attend);
2. Boston pamphlet novels reprinted by Edward Lloyd teaming up with George Purkess and William Strange in the mid-1840s (which I addressed in my “Nightmare Before Christmas” talk starting on slide 35);
3. women serial novelists reprinted in penny weeklies in the second half of the 1850s (a slice of which Melissa and I are examining for our article).

These three categories are how I’ve written about American sources for penny bloods, not how I’ve stored the information in my database. I’ve actively avoided any sort of mushy grouping like genre because I first wanted to build a solid foundation based on accurate descriptions of the bibliographic material. Having now worked with this material for a decade, I feel much more confident about interpreting the data.

I first tried to see if all the data available in the database fit into either of these three categories.

Working with the data as organized

As explained on slide 41 of my talk, the information about the sources is stored in a table called hypotexts, in accordance with Gérard Genette’s terminology in Palimpsestes / Palimpsests (slide 40).

This table has the following fields:

• the identifier hypotext_id;
• the self-explanatory title;
• a variety of fields linking different roles to individuals in the authors table:
  • author1_id;
  • author2_id;
  • translator_id;
  • translator2_id;
  • composer_id;
• a field for larger containers named periodical (can be a collection of short stories too);
• a field called place which contains the city in which the text was produced;
• a variety of fields to record temporality;
  • year for books;
  • start_date and end_date for serializations;
  • first_performance_date for plays and songs;
• a theatre_id linked to a table called theatres that provides its name and the city in which it’s located;
• three different possible references for British plays:
  • lord_chamberlain_copy for the volume in which it can be found in the British Library;
  • elta_id to link to the archived (and therefore inaccessible) East London Theatre Archive project (I’ll have to see what to do about this, at least on the Price One Penny interface);
  • vpp_id to link to the Victorian Plays Project;
• references to fiction published in Britain in 1836 or earlier:
  • estc_id for the English Short Title Catalogue that goes up to 1800;
  • british_fiction_id for British Fiction, which covers 1800 to 1836;
• references to other hypotexts:
  • hypotext and relationship for another entry in hypotexts that is a source for this one;
  • container_id for another entry in hypotexts that contains this one;
• the obligatory other_data field for whatever notes I wanna leave to my future self.

The keen observer will have noticed that:

1. I have no straightforward way of identifying American source texts (I need to query for a list of American cities);
2. I do not record source text publishers.

Let’s see how many hypotexts with American cities in the place field I can pull from my database.

Finding American texts

Let’s first identify the list of American cities:

SELECT DISTINCT place FROM hypotexts ORDER BY place;
+--------------------+
| place |
+--------------------+
| NULL |
| Aarau |
| Baltimore |
| Berlin |
| Boston |
| Bruxelles |
| Cologne |
| Colonia |
| Dresden |
| Dublin |
| Edinburgh |
| Frankfurt am Main |
| Frankfurt, Leipzig |
| Hamburg |
| Leipzig |
| London |
| London, Manchester |
| Milano |
| Montpelier |
| New York |
| Paris |
| Philadelphia |
| Pittsburgh |
| Salisbury |
| Stockholm |
| Stuttgart |
| United States |
| Vienna |
| Washington, D.C. |
+--------------------+
29 rows in set (0,01 sec)

There are seven American places. Now how many American texts are there?

SELECT place, COUNT(*) nb_hypotexts
FROM hypotexts
WHERE place = "Baltimore"
OR place = "Boston"
OR place = "New York"
OR place = "Philadelphia"
OR place = "Pittsburgh"
OR place = "United States"
OR place = "Washington, D.C."
GROUP BY place
ORDER BY nb_hypotexts DESC;
+------------------+--------------+
| place | nb_hypotexts |
+------------------+--------------+
| New York | 70 |
| Philadelphia | 47 |
| Boston | 37 |
| Baltimore | 2 |
| Pittsburgh | 1 |
| Washington, D.C. | 1 |
| United States | 1 |
+------------------+--------------+
7 rows in set (0,00 sec)

That’s a total of 159 results. After this point, I got a bit lost. I was documenting as I was going along, so I’ll leave it here, but you can skip over straight to identifying the right-hand side categories, or target categories.

First attempt at splitting American texts into categories

In the table above, we have a rough estimate of our lot of Boston pamphlet novels (37), using place = "Boston". Using author1_id = 190, we find 11 source texts authored by James Fenimore Cooper. So we have 48 texts accounted for out of 159. Does that mean all 111 other American texts fit into the third category of serializations?

We could approximate the serializations from the presence of something either in the periodical field or in the start_date field. How close are those numbers?

SELECT * FROM hypotexts
WHERE

(place = "Baltimore"
OR place = "Boston"
OR place = "New York"
OR place = "Philadelphia"
OR place = "Pittsburgh"
OR place = "United States"
OR place = "Washington, D.C.")

AND periodical IS NOT NULL;

43 rows in set (0,01 sec)

However, these results contain short stories in collections.

SELECT * FROM hypotexts
WHERE

(place = "Baltimore"
OR place = "Boston"
OR place = "New York"
OR place = "Philadelphia"
OR place = "Pittsburgh"
OR place = "United States"
OR place = "Washington, D.C.")

AND start_date IS NOT NULL;

33 rows in set (0,01 sec)

The short answer is thus, no, the serializations category doesn’t account for all 111 other American texts. In fact, the uncategorized group, with more than 70 texts, is the largest by some margin.

What other categories are we missing?

Let’s investigate the American text that fell through the cracks. I suspect we’ll find some translations from the French or the German which then made their way to London’s cheap literature publishers.

SELECT * FROM hypotexts
WHERE

(place = "Baltimore"
OR place = "Boston"
OR place = "New York"
OR place = "Philadelphia"
OR place = "Pittsburgh"
OR place = "United States"
OR place = "Washington, D.C.")

AND author1_id != 190
AND place != "Boston"
AND start_date IS NULL;

81 rows in set (0,01 sec)

There are indeed 13 translations. I also see novels that are associated with the “women serial novelists” category, because some of the reprints were made from the later volume edition rather than from the periodical serialization. There are also the other American novelists, apart from Cooper, who were reprinted in serial anthologies like the Novel Newspaper.

And what if categories were formed by grouping authors instead? Because an author_id number is not very useful in making sense of the data, I’ll have to join the hypotexts table with the authors table to get their names.

SELECT

first_name,
middle_name,
last_name,
woman,
COUNT(hypotext_id) nb_hypotexts

FROM hypotexts
INNER JOIN authors ON author1_id = author_id
WHERE

(place = "Baltimore"
OR place = "Boston"
OR place = "New York"
OR place = "Philadelphia"
OR place = "Pittsburgh"
OR place = "United States"
OR place = "Washington, D.C.")

AND nationality = "American"
GROUP BY author_id
ORDER BY nb_hypotexts DESC;
+------------+-----------------------+---------------+-------+--------------+
| first_name | middle_name | last_name | woman | nb_hypotexts |
+------------+-----------------------+---------------+-------+--------------+
| Emma | Dorothy Eliza Nevitte | Southworth | 1 | 12 |
| Joseph | Holt | Ingraham | 0 | 9 |
| James | Fenimore | Cooper | 0 | 9 |
| William | Gilmore | Simms | 0 | 7 |
| Robert | Montgomery | Bird | 0 | 6 |
| Sylvanus | NULL | Cobb | 0 | 6 |
| Ann | Sophia | Stephens | 1 | 5 |
| Maria | Jane | McIntosh | 1 | 4 |
| Harriet | Beecher | Stowe | 1 | 4 |
| Benjamin | NULL | Barker | 0 | 3 |
| John | NULL | Neal | 0 | 3 |
| Catharine | Maria | Sedgwick | 1 | 3 |
| James | Kirke | Paulding | 0 | 3 |
| Eliza | Ann | Dupuy | 1 | 3 |
| Sarah | Payson | Willis | 1 | 3 |
| Caroline | Lee | Hentz | 1 | 3 |
| Osgood | NULL | Bradbury | 0 | 2 |
| Dennis | NULL | Hannigan | 0 | 2 |
| Henry | P. | Cheever | 0 | 2 |
| Thomas | Dunn | English | 0 | 2 |
| Edward | Zane Carroll | Judson | 0 | 2 |
| John | Pendleton | Kennedy | 0 | 2 |
| Charles | Brockden | Brown | 0 | 2 |
| Emerson | NULL | Bennett | 0 | 2 |
| Harriot | Flora | Curtis | 1 | 1 |
| Reynell | NULL | Coates | 0 | 1 |
| P. | NULL | O'Shaugnessey | NULL | 1 |
| M. | L. | Sweetser | 1 | 1 |
| Justin | NULL | Jones | 0 | 1 |
| Maturin | Murray | Ballou | 0 | 1 |
| Augustus | Julian | Requier | 0 | 1 |
| George | NULL | Lippard | 0 | 1 |
| Caroline | Matilda | Kirkland | 1 | 1 |
| Sarah | Josepha Buell | Hale | 1 | 1 |
| Henry | Wadsworth | Longfellow | 0 | 1 |
| Richard | Henry Jr | Dana | 0 | 1 |
| Edgar | Allan | Poe | 0 | 1 |
| Harriet | Vaughan | Cheney | 1 | 1 |
| Timothy | NULL | Flint | 0 | 1 |
| John | R. | Willis | 0 | 1 |
| Henry | Theodore | Tuckerman | 0 | 1 |
| Theodore | Sedgwick | Fay | 0 | 1 |
| Anna | Lowell | Snelling | 1 | 1 |
| John | Henry | Mancur | 0 | 1 |
| Tom | NULL | Shortfellow | 0 | 1 |
| John | Alexander | McClung | 0 | 1 |
| James | NULL | Sanford | 0 | 1 |
| Susan | Anne Ridley | Sedgwick | 1 | 1 |
| Timothy | Shay | Arthur | 0 | 1 |
| Calvin | Henderson | Wiley | 0 | 1 |
| Oliver | Bell | Bunce | 0 | 1 |
| Henry | Augustus | Wise | 0 | 1 |
| James | A. | Maitland | 0 | 1 |
| Virginia | Frances | Townsend | 1 | 1 |
| Mary | NULL | Langdon | 1 | 1 |
| Alice | Bradley | Haven | 1 | 1 |
| Genevieve | Genevra | Fairfield | 1 | 1 |
| Richard | NULL | Hildreth | 0 | 1 |
+------------+-----------------------+---------------+-------+--------------+
58 rows in set (0,00 sec)

This query prompted me to try a simpler one: how many American source texts by recognizable American authors were written by men and by women? I was quite surprised with the result!

SELECT woman, COUNT(hypotext_id) nb_hypotexts
FROM hypotexts
INNER JOIN authors ON author1_id = author_id
WHERE

(place = "Baltimore"
OR place = "Boston"
OR place = "New York"
OR place = "Philadelphia"
OR place = "Pittsburgh"
OR place = "United States"
OR place = "Washington, D.C.")

AND nationality = "American"
GROUP BY woman
ORDER BY nb_hypotexts DESC;
+-------+--------------+
| woman | nb_hypotexts |
+-------+--------------+
| 0 | 84 |
| 1 | 48 |
| NULL | 1 |
+-------+--------------+

I did not anticipate that women had written nearly double the number of American texts reprinted in penny weekly numbers!

For more clarity, let’s return to the right-hand side, which should be easier.

Categories of penny bloods that reprinted American works

Let’s return to the three initial categories of American works reprinted in penny bloods, which contained assumptions regarding the types of penny bloods in which they would be reprinted:

1. James Fenimore Cooper, reprinted in serial anthologies in the late 1830s;
2. Boston pamphlet novels reprinted by Edward Lloyd teaming up with George Purkess and William Strange in the mid-1840s;
3. women serial novelists reprinted in penny weeklies in the second half of the 1850s.

Conveniently, they each represent one of the three conceptually different bibliographic formats of penny bloods: 2) stand-alone editions identified a publisher or a group of publishers; 3) serializations in a particular periodical; and 1) tales in a larger edition or a periodical that does not function as a miscellany the way other weeklies do.

By taking them one at a time, we can come up with the right-hand side target categories.

Stand-alone editions of American works in penny weekly numbers

Here’s the intricate JOIN request to find how many editions of American works each penny publisher issued.

SELECT

p.first_name,
p.last_name,
COUNT(hypotext_id) nb_hypotexts

FROM hypotexts h
INNER JOIN works USING(hypotext_id)
INNER JOIN editions USING(work_id)
INNER JOIN publishers_imprint USING(edition_id)
INNER JOIN publishers p USING(publisher_id)
WHERE

(place = "Baltimore"
OR place = "Boston"
OR place = "New York"
OR place = "Philadelphia"
OR place = "Pittsburgh"
OR place = "United States"
OR place = "Washington, D.C.")

AND h.relationship IS NULL
GROUP BY publisher_id
ORDER BY nb_hypotexts DESC;
+------------+------------+--------------+
| first_name | last_name | nb_hypotexts |
+------------+------------+--------------+
| George | Purkess | 17 |
| Edward | Lloyd | 17 |
| William | Strange | 5 |
| Frederick | Hextall | 4 |
| George | Peirce | 2 |
| Henry | Lea | 2 |
| Edmund | Appleyard | 1 |
| Richard | Willoughby | 1 |
+------------+------------+--------------+
8 rows in set (0,00 sec)

Though seemingly eight penny publishers issued at least one American work, the first three worked together on this endeavour. The other five could be lumped together in an “Other publishers” category.

Penny magazines that serialized American works

Same exercice, with the periodicals this time.

SELECT

shorttitle periodical,
COUNT(hypotext_id) nb_hypotexts

FROM hypotexts h
INNER JOIN works USING(hypotext_id)
INNER JOIN serializations USING(work_id)
INNER JOIN periodical_series USING(periodical_series_id)
INNER JOIN periodicals p USING(periodical_id)
INNER JOIN titles t

ON p.title_id = t.title_id

WHERE

(place = "Baltimore"
OR place = "Boston"
OR place = "New York"
OR place = "Philadelphia"
OR place = "Pittsburgh"
OR place = "United States"
OR place = "Washington, D.C.")

AND h.relationship IS NULL
GROUP BY serializations.periodical_series_id
ORDER BY nb_hypotexts DESC;
+-------------------------------------+--------------+
| periodical | nb_hypotexts |
+-------------------------------------+--------------+
| The Family Herald | 14 |
| London Journal | 10 |
| The Weekly Magazine | 8 |
| Cassell's Illustrated Family Paper. | 5 |
| The Weekly Magazine | 5 |
| The Guide | 3 |
| The Illustrated Penny Novelist. | 2 |
| The Ladies' Journal | 1 |
| The London Entertaining Magazine | 1 |
| Cleave's Penny Gazette | 1 |
+-------------------------------------+--------------+
10 rows in set (0,00 sec)

This query seemingly returns 10 periodicals which carried American works, but I realized much later that The Weekly Magazine appears twice (because it ran through two series). When the two instances are summed up, it becomes the second most prolific reprinter of American works in penny magazines before 1860 with 13. The London Journal and The Guide also function as a unit (they were both published by George Stiff at the time).

We can thus have four categories for periodicals as targets:

• The Family Herald;
• The Weekly Magazine;
• London Journal and The Guide;
• other periodicals.

Serial anthologies which carried American works

Last but not least are the serial anthologies, which are described in the database as editions containing tales.

SELECT edition_id, COUNT(h.hypotext_id) nb_hypotexts
FROM hypotexts h
INNER JOIN works USING(hypotext_id)
INNER JOIN tales USING (work_id)
INNER JOIN editions USING (edition_id)
WHERE

(place = "Baltimore"
OR place = "Boston"
OR place = "New York"
OR place = "Philadelphia"
OR place = "Pittsburgh"
OR place = "United States"
OR place = "Washington, D.C.")

AND h.relationship IS NULL
GROUP BY edition_id
ORDER BY nb_hypotexts DESC;
+------------+--------------+
| edition_id | nb_hypotexts |
+------------+--------------+
| 431 | 45 |
| 504 | 7 |
| 292 | 2 |
| 369 | 1 |
| 325 | 1 |
+------------+--------------+
5 rows in set (0,00 sec)

As you can see, my joining skills are not good enough to replace edition_id with the title of the serial anthologies, but I can tell you without a shadow of a doubt that edition_id = 431 is the Novel Newspaper. Its whole mission until volume 17 was to reprint American literature (after which is moved to European literature).

Right-hand side sorted!

Well that felt much easier than the left-hand side exploration! To recap, we have seven target text categories.

Target categories	Number of hypotexts
Novel Newspaper	45
Lloyd, Purkess, and Strange	17
Family Herald	14
The Weekly Magazine	13
London Journal and The Guide	13
Other serial anthologies	11
Other publishers	10
Other magazines	10

Furthermore, if Sankey diagrams are produced for each bibliographic format separately, each contains a manageable number of categories without the need to group together the magazines/serial anthologies/publishers that carried less American works into an “Other” category.

Categories of American works, scene 1, take 2

Returning to the list of American authors whose works were reprinted in penny weekly numbers before 1860, I added a JOIN to remove hypotexts to other hypotexts.

SELECT

first_name,
middle_name,
last_name,
woman,
COUNT(hypotext_id) nb_hypotexts

FROM hypotexts
INNER JOIN authors ON author1_id = author_id
INNER JOIN works USING(hypotext_id)
WHERE

(place = "Baltimore"
OR place = "Boston"
OR place = "New York"
OR place = "Philadelphia"
OR place = "Pittsburgh"
OR place = "United States"
OR place = "Washington, D.C.")

AND nationality = "American"
GROUP BY author_id
ORDER BY nb_hypotexts DESC;
+------------+-----------------------+---------------+-------+--------------+
| first_name | middle_name | last_name | woman | nb_hypotexts |
+------------+-----------------------+---------------+-------+--------------+
| Emma | Dorothy Eliza Nevitte | Southworth | 1 | 10 |
| James | Fenimore | Cooper | 0 | 9 |
| Joseph | Holt | Ingraham | 0 | 8 |
| William | Gilmore | Simms | 0 | 7 |
| Sylvanus | NULL | Cobb | 0 | 6 |
| Robert | Montgomery | Bird | 0 | 5 |
| Maria | Jane | McIntosh | 1 | 4 |
| Ann | Sophia | Stephens | 1 | 4 |
| Benjamin | NULL | Barker | 0 | 3 |
| Eliza | Ann | Dupuy | 1 | 3 |
| Sarah | Payson | Willis | 1 | 3 |
| Harriet | Beecher | Stowe | 1 | 3 |
| Caroline | Lee | Hentz | 1 | 3 |
| Catharine | Maria | Sedgwick | 1 | 3 |
| James | Kirke | Paulding | 0 | 3 |
| John | NULL | Neal | 0 | 3 |
| Dennis | NULL | Hannigan | 0 | 2 |
| Henry | P. | Cheever | 0 | 2 |
| Thomas | Dunn | English | 0 | 2 |
| Emerson | NULL | Bennett | 0 | 2 |
| John | Pendleton | Kennedy | 0 | 2 |
| Charles | Brockden | Brown | 0 | 2 |
| Edward | Zane Carroll | Judson | 0 | 2 |
| Osgood | NULL | Bradbury | 0 | 2 |
| John | Henry | Mancur | 0 | 1 |
| Tom | NULL | Shortfellow | 0 | 1 |
| M. | L. | Sweetser | 1 | 1 |
| P. | NULL | O'Shaugnessey | NULL | 1 |
| Justin | NULL | Jones | 0 | 1 |
| Maturin | Murray | Ballou | 0 | 1 |
| Augustus | Julian | Requier | 0 | 1 |
| George | NULL | Lippard | 0 | 1 |
| Timothy | Shay | Arthur | 0 | 1 |
| Calvin | Henderson | Wiley | 0 | 1 |
| James | A. | Maitland | 0 | 1 |
| Virginia | Frances | Townsend | 1 | 1 |
| Mary | NULL | Langdon | 1 | 1 |
| Oliver | Bell | Bunce | 0 | 1 |
| Alice | Bradley | Haven | 1 | 1 |
| Henry | Augustus | Wise | 0 | 1 |
| Genevieve | Genevra | Fairfield | 1 | 1 |
| John | Alexander | McClung | 0 | 1 |
| Caroline | Matilda | Kirkland | 1 | 1 |
| Sarah | Josepha Buell | Hale | 1 | 1 |
| Henry | Wadsworth | Longfellow | 0 | 1 |
| Richard | Henry Jr | Dana | 0 | 1 |
| Edgar | Allan | Poe | 0 | 1 |
| Harriet | Vaughan | Cheney | 1 | 1 |
| Timothy | NULL | Flint | 0 | 1 |
| John | R. | Willis | 0 | 1 |
| Henry | Theodore | Tuckerman | 0 | 1 |
| Theodore | Sedgwick | Fay | 0 | 1 |
| Anna | Lowell | Snelling | 1 | 1 |
| Harriot | Flora | Curtis | 1 | 1 |
| Susan | Anne Ridley | Sedgwick | 1 | 1 |
| Reynell | NULL | Coates | 0 | 1 |
+------------+-----------------------+---------------+-------+--------------+
56 rows in set (0,00 sec)

Note that authors must be grouped, otherwise the largest category will contain only 10 works (E.D.E.N. Southworth).

Authors vs. publishing venue as categories

Joseph Holt Ingraham is an interesting case since he is, if my memory serves me, the only American novelist to have appeared in two different penny blood bibliographic formats: in serial anthologies (4 novels) and in stand-alone editions (5 novels: Captain Kyd appeared in both formats). Indeed, Cooper only appeared in serial anthologies and Southworth in magazines.

Given the fact that authors rarely appear in more than one bibliographic format, the list of 56 American authors can be cut down to a more manageable size by adding another JOIN statement:

INNER JOIN [format] USING (work_id)

in which [format] is either tales, editions, or serializations .

Predictably, Cooper dominates the query for tales in serial anthologies and Southworth, that for serializations .

Robert Montgomery Bird is another who appears like Ingraham in two formats (five times in serial anthologies and once in a stand-alone edition), as did Sylvanus Cobb (five periodical serializations and once in a serial anthology). Henry P. Cheever (once in a serial anthology and another time in a stand-alone edition) also straddled two formats, but is a much more minor figure.

Looking specifically at Cobb’s split, the single novel to appear in a serial anthology was published in Boston by Frederick Gleason while his five periodical serializations came from the New York Ledger. The penny blood categories therefore are linked, at least in his case, not to the author but to the publishing venue.

I maintain that the Boston powerhouse delineated by Ronald Zboray and Mary Zboray in “The novel in the antebellum book market” (2011) is a relevant category of source texts. Another could be called “Southworth and the New York Ledger“, bringing into a single category all her works and those that were published by other authors in the metropolitan miscellany.

I have since realized that I was fretting for no reason between categorizing according to author or publishing venue is moot in a Sankey diagram since there can be more than two columns. I could therefore have to the left American authors, in the middle American publishers and periodicals, and finally to the right the target categories defined above.

The problem with producing such a visualization is, again, that I do not have data regarding publishers in American works. I can therefore at the moment construct such a Sankey diagram with only periodicals in the middle.

Building meaningful categories according to penny blood reprinting format

A short detour first (I told you I worked for 11 hours straight!).

I thought the reason the Novel Newspaper has such a large number of American works was that I had indexed the short fiction that was used to fill the last pages of the last number of a novel. After much wrangling, I’ve found that I have not indexed that many and that only four were actually American works. They can be removed by adding the following constrait to the WHERE statement:

AND (

edition_id != 431
OR
(edition_id = 431 AND (first_page IS NULL OR last_number != 0))

)

Ok, detour over.

I have found that it’s possible to split the remaining source texts for serial anthologies evenly —once Boston and Ingraham works were removed— by grouping the three most popular authors, three male novelists: James Fenimore Cooper, William Gilmore Simms, and Robert Montgomery Bird. Together, they have written 21 source texts for serial anthologies while the rest of the authors have written 22.

In American works reprinted in stand-alone penny number editions, the split between Boston and others is 15 to 11, making the named category greater than the “other” in this format. Southworth and the New York Ledger account for 20 out of the 41 American works serialized in London’s penny weekly magazines before 1860, which also seems fair.

Formalizing the meaningful categories

We have therefore, after much pain, circumscribed three categories of American source texts. Returning to all hypotexts as opposed to just those published in a certain penny blood format, this is the breakdown.

Cooper, Simms, and Bird

The query is fairly straightforward since it only requires selecting the source texts written by specific authors.

SELECT * FROM hypotexts
INNER JOIN works USING(hypotext_id)
WHERE author1_id = 190
OR author1_id = 96
OR author1_id = 3;
24 rows in set (0,01 sec)

Southworth and the New York Ledger

The query is also simple enough, selecting the source texts written by specific author and/or appearing in a specific magazine.

SELECT * FROM hypotexts
INNER JOIN works USING(hypotext_id)
WHERE author1_id = 279
OR periodical = "New York Ledger";
19 rows in set (0,01 sec)

Ingraham, Buntline and Boston in the 1840s

The two categories above are mutually exclusive since neither Cooper, Simms, nor Bird wrote for the New York Ledger. Things get more complicated for Boston because Cooper did write one story for the Boston Notion in 1841. Fortunately, no relevant Boston publications were published in or before 1841, so the Cooper publication can be avoided by putting the decade’s cut-off date at 1841.

As pointed out above, Ingraham straddled a few categories. Like Southworth with the New York Ledger, he published both in the Boston scene and in New York. The Boston category could therefore be enlarged to contain all of his works.

I also added to the mix Edward Zane Carroll Judson, better known by his pseudonym Ned Buntline, even though he only wrote two works that were reprinted in penny numbers before 1860 (one in Boston, the other in New York) because he later became a more important figure of popular American fiction.

The query thus selects all source texts that are either written by these two authors in addition or in Boston between 1841 and 1850 or at an unspecified date (presumably because it is a periodical serialization with a start_date and an end_date ).

SELECT * FROM hypotexts h
INNER JOIN works USING(hypotext_id)
WHERE h.relationship IS NULL
AND (author1_id = 110 OR author1_id = 155)
OR (

place = "Boston" AND
(

(year > 1841 AND year < 1850)
OR year IS NULL

)

);
26 rows in set (0,00 sec)

The query for this category is a (very convoluted) approximation of what I have in mind, which are the publications of the Williams brothers and Gleason. As a reminder, the database does not contain information regarding the publishers of source texts.

Subdividing the remaining American source texts

These three meaningful groupings (or approximations of) nevertheless left a great many American works uncategorized: 61 to be precise. Given the surprise at the end of scene 1 take 1 that left me throwing my hands up in the air, could gender be a good dividing line among the remaining source texts?

Let’s see using a host of negations (!).

SELECT

woman,
COUNT(hypotext_id) nb_hypotexts

FROM hypotexts h
INNER JOIN authors ON author1_id = author_id
INNER JOIN works USING(hypotext_id)
WHERE h.relationship IS NULL
AND

(place = "Baltimore"
OR place = "Boston"
OR place = "New York"
OR place = "Philadelphia"
OR place = "Pittsburgh"
OR place = "United States"
OR place = "Washington, D.C.")

AND

(periodical != "New York Ledger"
OR periodical IS NULL)

AND

!(place = "Boston" AND
(year IS NULL OR (year > 1840 AND year < 1850)))

AND

!(author1_id = 190
OR author1_id = 96
OR author1_id = 3
OR author1_id = 279
OR author1_id = 110
OR author1_id = 155)

GROUP BY woman
ORDER BY nb_hypotexts DESC;
+-------+--------------+
| woman | nb_hypotexts |
+-------+--------------+
| 0 | 32 |
| 1 | 28 |
| NULL | 1 |
+-------+--------------+

That seeming like a reasonable split, I formalized the three remaining categories. The queries would be slightly different, and not only because woman would be set at 0 for men, 1 for women, and NULL for P. O’Shaugnessey (whom I still caress the dream of identifying one day). The list of authors to be excluded does not need to appear in full in the first two cases (or at all in the last!) because they are gendered.

Hence, for men, after the list of American cities and the negation for both the New York Ledger and Boston in the 1840s as a publication place, would come:

AND woman = 0
AND !(

author1_id = 190
OR author1_id = 96
OR author1_id = 3
OR author1_id = 110
OR author1_id = 155

);

For women, the list would be considerably shorter, since only Southworth was reprinted enough in penny numbers before 1860 to warrant having her name and whole body of works attached to a meaningful category:

AND woman = 1 AND !(author1_id = 279);

Anonymous American works or those written by authors of otherwise unknown gender are trickier because the INNER JOIN authors ON author1_id = author_id statement excludes anonymous source texts. Presumably I could be using another type of join to solve the problem, but it was getting late, so I just removed it (it had been inserted to call on the authors table to get the gender of identified authors).

The query to list the remaining American source texts can thus be:

SELECT * FROM hypotexts h
INNER JOIN works USING(hypotext_id)
WHERE h.relationship IS NULL
AND

(place = "Baltimore"
OR place = "Boston"
OR place = "New York"
OR place = "Philadelphia"
OR place = "Pittsburgh"
OR place = "United States"
OR place = "Washington, D.C.")

AND

(periodical != "New York Ledger"
OR periodical IS NULL)

AND

!(place = "Boston" AND
(year IS NULL OR (year > 1840 AND year < 1850)))

AND

(author1_id IS NULL
OR author1_id = 107);

Here is the final tally thus of the number of texts in each source category.

Source categories	Number of hypotexts
Ingraham, Buntline and the Boston scene in the 1840s	27
Cooper, Simms, and Bird	24
Southworth and the New York Ledger	19
Other male novelists	32
Other women novelists	28
Other anonymous works	5

Calculating flow values

Right! We have six source categories and eight target categories. What else do we need? Returning to Flourish‘s instructions, in the dataset “each row has a source name, target name and flow value”. The flow value is, in this case, the number of American works within a specific category that moves into a specific category of penny bloods.

Since it was getting late and I wanted to see the fruit of my labour before hitting the sack, I chose to go for the easy option (to me): working in a spreadsheet. The disadvantage is that my spreadsheet wouldn’t be dynamically related to my database: I would be querying my database and copying-pasting the number of rows into the spreadsheet, an operation that would need to be redone manually if anything regarding the formal definition of the categories, or the data itself changed.

The advantage was no PHP programming, =P

Building the matrix

As you can see on my publicly accessible Google Spreadsheet, I started by constructing a matrix with rows of source categories and columns of target categories. The cell on the first row and first column would contain the number of Ingraham, Buntline and Boston in the 1840s works that appeared in the Novel Newspaper (two, both written by Ingraham).

To construct the query, I would copy-paste first the large amount of text on top spanning multiple columns (designed to fit the particular bibliographic format of the target category), then add the text corresponding to the row, and finally that corresponding to the specific column.

For example:

SELECT * FROM hypotexts h INNER JOIN authors ON author1_id = author_id INNER JOIN works USING (hypotext_id) INNER JOIN tales USING (work_id) WHERE (place = "Baltimore" OR place = "Boston" OR place = "New York" OR place = "Philadelphia" OR place = "Pittsburgh" OR place = "United States" OR place = "Washington, D.C.") AND h.relationship IS NULL AND

((author1_id = 110 OR author1_id = 155) OR (place = "Boston" AND (year IS NULL OR (year > 1841 AND year < 1850))))

edition_id = 431

Finally, add ;, press enter, and let the MySQL magic happen.

A 6 x 8 matrix meant doing this 48 times, while catching up on the day’s results with the podcasts I had not yet listened. The last row required its own top-line first-step statements because of the whole probably-shouldn’t-be-using-an-INNER JOIN-there issue.

Building the list

Building the matrix provided me with all the flow values I needed through an intuitive (at least to me) presentation. However, it was still not the “each row has a source name, target name and flow value”-format I needed. I thus proceeded in a new tab to transform the matrix into the required format.

Since Flourish required me to create an account, I chose to use SankeyMATIC, in which you can just copy-paste data in the required format, which is:

SOURCE [AMOUNT] TARGET

in which AMOUNT is Flourish’s flow value.

I thus just added a column of opening and closing square brackets on either side of the flow values and voilà! I just had to copy-paste my table into the text field in the top left corner of the SankeyMATIC build page to obtain the previously disclosed graph.

+1 for Sankey diagrams

I absolutely love my Sankey diagram: as a narrative and investigative tool, it does all that I hoped it would.

This preliminary work has nevertheless shown that, on the interface front, I will have to seek a more sophisticated package/app to create truly accurate Sankey diagrams because SankeyMATIC assumes each side is equal in numbers. However, the same American work can appear in more than one category, so it’s not ncessarily a one-to-one relationship from source to target.

On the the data front, it is very apparent that I do not know how to adequately subdivided American literature. Speak of data visualization as an investigative tool, large flow values from “other” source categories are suspicious. Clearly they call for more granularity.

Examining the diagram, I see three flows in which an “other” source is greater than the largest named source for a given target category:

• other male novelists in the Novel Newspaper;
  • could perhaps be solved by adding the fourth most reprinted novelist in serial anthologies to the meaningful category, potentially undetermining its meaningfulness;
• other women novelists in the Family Herald;
  • it appears that New York publisher Appleton could be a fruitful category;
• other women novelists in the London Journal and The Guide.

The prominence of Appleton publications in the Family Herald suggests the issue of listing American publishers in the database is pressing. As I pointed out at the end of the “Authors vs. publishing venue as categories” section, the groupings could be less arbitrary (and thus anxiety-inducing) if they did not need to ackwardly bring together the source texts written by an author and those associated with a particular publishing venue, whether periodical or publisher.

The importance of digital bibliographies in adjacent areas

This wonderful solution from the graphical standpoint can however only be implemented if information about source text publishers could be found in the database. More generally, as you might have noted in the “Working with the data as organized” section, some classes of source texts are linked to other digital resources in which they are more fully described.

British works of fiction up to 1836 are exhaustively described in the English Short Title Catalogue and British Fiction (and At the Circulating Library is aiming to bring that all the way to 1900). British drama can be found in the Lord Chamberlain Plays, the Victorian Plays Project and, once upon a time, in the East London Theatre Archive. Price One Penny links to no such resource for American literature.

During the course of my 11 hours of work (did I mention it took me 11 hours?), I was reminded of Lyle H. Wright’s American Fiction 1851–1875: A Contribution Toward a Bibliography. A sort of index to it has been sitting on my computer under the form of a list of microfilmed works with references to his bibliography.

This rediscovery then led me to Wright American Fiction, (yet another) digital humanities project which seems to have lost steam. It cannot replace Wright’s printed work since it only contains the works for which it has had the time and energy to produce electronic editions. I quickly searched for titles in the Edward T. LeBlanc Dime Novel Bibliography project (dimenovels.org) but found only late 19^th-century reprints.

The Wright bibliography seems the best bet to find the publishers of my American source texts but ,without a digital infrastructure, it won’t allow for the larger analysis I’ve always dreamed of: what was the range of texts available to the London penny publishers or penny magazine editors when they were selecting American works to be reprinted.

---

---