Building a panopticon: The evolution of the NSA’s XKeyscore
This is an amazing behind-the-scenes look at the architecture of XKeyscore, and how it evolved from an earlier large-scale packet interception system, Narus’ Semantic Traffic Analyzer. XKeyscore is a federated, distributed system, with distributed packet-capture agents running on Linux, built with protocol-specific plugins, which write 3 days of raw packet data, and 30 days of intercept metadata, to local buffer stores. Central queries are then ‘distributed across all of the XKeyscore tap sites, and any results are returned and aggregated’. Dunno about you, but this is pretty much how I would have built something like this, IMO….
(tags: panopticon xkeyscore nsa architecture scalability packet-capture narus sniffing snooping interception lawful-interception li tapping)