The slipperiness of significant publication

I’ve written before about trying to establish the significance of a philosophy publication, which mostly becomes an issue at tenure and promotion time. In an earlier post, I argued that citation counts are mostly rubbish for philosophy. Still, there are pressures to provide numerical measures.

In support of a recent tenure case, our department gave the acceptance rates at journals as evidence of their significance.¹ The danger in using such a measure is that, if widely adopted, it would quickly become uninformative.² If scholars get more credit for publishing in journals with low acceptance rates, they will preferentially start to submit to those. The journals will get more submissions but still accept the same number. This will, in turn, drive down the acceptance rates of those journals.

I recently learned that there is a name for this general phenomenon, Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure.”

This shifts it from being this hunch I have to being a well-established phenomenon. I can now invoke it with greater conversational gravitas.

1. We also had both internal and outside letters discussing the prestige of the journals he had published in.
2. There’s already a limit to how informative it is, because some journals are more likely to get a steady stream of poor submissions than others. For example, The Journal of Symbolic Logic is a prestigious specialist journal which might not get unsuitable papers. A less technical journal might receive more crackpot weltanschauung submissions, driving up its total number of submissions and driving down its acceptance rate.