An appreciation
My old friend and former colleague Steve Bellovin has an interesting blog at Columbia, where he's a professor of computer science. Steve is one of those guys who has just done stuff for his whole career. As a graduate student he helped invent Usenet, which I credit as being the first computer social network. His time at Bell Labs (which is where our paths first crossed) produced a lot of different things, possibly most famously his work with Bill Cheswick on internet firewalls and security. For his last sabbatical from Columbia, he was the Chief Technologist of the Federal Trade Commission for a year.
Steve's blog is not notable for it's volume, it's notable for its gems -- thoughtful and thought provoking pieces on a wide variety of topics. There's currently a pair of posts worth reading.
Police vs. Spy
The first "What Spies Do" outlines the difference in goals, outcomes, and rules between law enforcement and espionage.
The goal of a police investigation is prosecution and conviciton of malefactors. They need evidence that is legally admissible, they have to disclose their evidence in court (and turn over exculpatory evidence to the defense), and prove guilt beyond a resonable doubt, in the face of opposition by a defense attorney. Information that can't be used in this fashion isn't useful to them; it's quite proper to insist that it not be collected or retained.
As contrasted with espionage
There's no question of admissibility, only reliability. There's no due process, no requirement to disclose anything, no adversarial process in anything like the same fashion. Intelligence agencies virtually never know anything beyond a reasonable doubt—and if they think they do, they'll worry that they've been misled by disinformation.
For that function, long term retention of what appears to be marginal data is appropriate. Intelligence analysts never know where they'll find a little thread of data that helps them put together a story. Finally, on the subject of the morality or amorality of espionage:
Espionage, it turns out, is not against international law. Spying, by all nations, always has been like this and probably always will be. Telling a major country to give up spying is like telling a lion to become a vegetarian.
All in all a fascinating and factual analysis. As Steve points out, because the goals and methods of the law enforcement and data gathering roles are so different, we wind up in trouble when the two roles are mixed.
End-user Encryption Requirements
The other article worth reading is entitled "What PGP Should Look Like" which is a requirements exercise for end-user email encryption. I do want to note that Steve is referring to the classic email encryption methodology invented by Phil Zimmerman, as opposed to all of the various things currently under the umbrella of the PGP trademark.
In fact it doesn't matter. Whether you are talking about classic PGP, S/MIME, and some other approaches, there just aren't any really great approaches to email encryption, at least not without a gateway intervening. They all require that you manage private keys and keep them secure. They all require that you have some method of sharing public keys with your correspondents. They mostly break down if you are at a kiosk or somebody else's computer.
Steve doesn't set out to solve these problems. Instead he does what any good systems engineer does first: comes up with a list of questions that will help anybody tackling this problem converge on the right requirements, and ultimately a decent design.
This is a great little primer on email cryptography AND a great little primer on how to define a problem so it can be solved. Rather than my quoting it for you, I'll let you go and read it.
EDITED 7/25/14 to correct the job he held during his leave.