-
Black hats steal code-signing keys from software whitelisting anti-malware firm. Pretty audacious
(tags: malware security whitelisting av)
How did I do the Starwars Traceroute?
It is accomplished using many vrfs on 2 Cisco 1841s. For those less technical, VRFs are essentially private routing tables similar to a VPN. When a packet destined to 216.81.59.173 (AKA obiwan.scrye.net) hits my main gateway, I forward it onto the first VRF on the “ASIDE” router on 206.214.254.1. That router then has a specific route for 216.81.59.173 to 206.214.254.6, which resides on a different VRF on the “BSIDE” router. It then has a similar set up which points it at 206.214.254.9 which lives in another VPN on “ASIDE” router. All packets are returned using a default route pointing at the global routing table. This was by design so the packets TTL expiration did not have to return fully through the VRF Maze. I am a consultant to Epik Networks who let me use the Reverse DNS for an unused /24, and I used PowerDNS to update all of the entries through mysql. This took about 30 minutes to figure out how to do it, and about 90 minutes to implement.
(tags: vrfs routing networking hacks star-wars traceroute rdns ip)
Real-time Analytics in Scala [slides, PDF]
some good approximation/streaming algorithms and tips on Scala implementation
(tags: streams algorithms approximation coding scala slides)
‘E?cient Computation of Frequent and Top-k Elements in Data Streams’ [paper, PDF]
The Space-Saving algorithm to compute top-k in a stream. I’ve been asking a variation of this problem as an interview question for a while now, pretty cool to find such a neat solution. Pity neither myself nor anyone I’ve interviewed has come up with it ;)
(tags: space-saving approximation streams stream-processing cep papers pdf algorithms)
-
ASL-licensed open source library of stream-processing/approximation algorithms: count-min sketch, space-saving top-k, cardinality estimation, LogLog, HyperLogLog, MurmurHash, lookup3 hash, Bloom filters, q-digest, stochastic top-k
(tags: algorithms coding streams cep stream-processing approximation probabilistic space-saving top-k cardinality estimation bloom-filters q-digest loglog hyperloglog murmurhash lookup3)
‘Medians and Beyond: New Aggregation Techniques for Sensor Networks’ [paper, PDF]
‘We introduce Quantile Digest or q-digest, a novel data structure which provides provable guarantees on approximation error and maximum resource consumption. In more concrete terms, if the values returned by the sensors are integers in the range [1;n], then using q-digest we can answer quantile queries using message size m within an error of O(log(n)/m). We also outline how we can use q-digest to answer other queries such as range queries, most frequent items and histograms. Another notable property of q-digest is that in addition to the theoretical worst case bound error, the structure carries with itself an estimate of error for this particular query.’
(tags: q-digest algorithms streams approximation histograms median percentiles quantiles)
Russia’s anti-child-porn internet blocklist allegedly being used for general censorship
Allegedly being used to censor political and anti-corruption journalism, and a Russian wikipedia-like site for hosting an article about suicide
(tags: censorship feature-creep russia politics blocklists)
HyperLogLog++: Google’s Take On Engineering HLL
Google and AggregateKnowledge’s improvements to the HyperLogLog cardinality estimation algorithm
(tags: hyperloglog cardinality estimation streaming stream-processing cep)