Cautionary tales of networks

Mapping historical networks was an event at Cambridge Digital Humanities (CDH) run by John Schaefer. I was kindly given the opportunity to talk to researchers in the humanities about some dangers that can occur when working with network data. The idea that was conveyed is essentially trivial: complex networks are indeed rather complex, and many of the standard methodology that we use to study and draw conclusions about them are not quite principled in the way we would like them to be. This is essentially the borrowed philosophy of Tiago Peixoto.

It’s something that now permeates the community, but it’s important to continually place an emphasis on, especially for those working in theory. After all, it’s not the fault of practitioners that the tools handed to them are not doing exactly what they are advertised to do.

I gave a short talk on community detection, a famous setting for which Bayesian methods are encouraged over metric-based ones. I devided that for this audience it wouold be important to discuss three dangers of working with graph data.

A graph that seemingly has three communities.

1. Visualization is not always insightful. This is because layout algorithms can decide how to cluster your data in a way that is completely independent of the properties you seek to investigate.
2. Adjacency matrices are not always insightful. This is because the rows and columns can be permuted in any way, so picking up on the arrangement of entries is meaningless.
3. Metrics are not always insightful. Approaches like these tend to rely on null models, which are difficult to define in complex networks, and even harder to interpret. Furthermore, they tend to have many free parameters whose choices can drastically impact the conclusions they make about the data.

The same graph with a different layout.

At the same time, I think that it is unfair to dismiss work that uses the above approaches as exploratory tools. In fact, I believe that these approaches should be encouraged precisely for these purposes. Researchers in the digital humanities want tools that help them navigate data, ways of rethinking about their investigations. In those settings, I think that centralities, layout algorithms, and metrics can all prove to be productive heuristics – especially when working with small datasets.

On the other hand, I do believe that it is important to remain transparent about what these methods do not offer, which is scientific insight. Claiming genuine inferences about the underlying data because a particular metric spewed promising numbers under carefully tuned parameters, is dangerous. But it’s the duty of anyone working in this space, especially on the methodological side, to take some opportunities to re-convey these differences. If the gospel is well preached, I suspect that honest and exciting research will be conducted in many more interesting fields.