An interesting post from Nathan Jensen, a political scientist at Wash U, on the practicalities of working with non-public datasets (via @Ted_Underwood). Worth a read; here are the two main takeaways:

… theory is even more important when using “big data”. You can only really harness the richness of complicated micro data if you have clear micro theories.

Barriers to entry can create rents for a researcher, but they also make it much more difficult to replicate your results. This means that journal reviewers and grant reviewers can hold this against you, and the ultimate impact of your work might be lower. This isn’t a suggestion. It is a warning.

That second point’s a big one.