Our children should be free to choose to study what really excites them, not subtly steered away from certain subjects because teachers believe in and propagate the stereotypes. Last year the IOP published a report “It’s Different for Girls” which demonstrated that essentially half of state coeducational schools did not see a single girl progress to A-level physics. By contrast, the likelihood of girls progressing from single sex schools were two and a half times greater.Amen to this.
‘SBE is an OSI layer 6 representation for encoding and decoding application messages in binary format for low-latency applications.’ Licensed under ASL2, C++ and Java supported.
‘like inetd, but for WebSockets’ — ‘a small command line tool that will wrap an existing command line interface program, and allow it to be accessed via a WebSocket. It provides a quick mechanism for allowing web-applications to interact with existing command line tools.’ Awesome idea. BSD-licensed. (Via Mike Loukides)
a metric storage daemon, exposing both a carbon listener and a simple web service. Its aim is to become a simple, scalable and drop-in replacement for graphite’s backend.Pretty alpha for now, but definitely worth keeping an eye on to potentially replace our burgeoning Carbon fleet…
In this talk Kaushik Srenevasan describes a new, low overhead, full-stack tool (based on the Linux perf profiler and infrastructure built into the Hotspot JVM) we’ve built at Twitter to solve the problem of dynamically profiling and tracing the behavior of applications (including managed runtimes) in production.Looks very interesting. Haven’t watched it yet though
[MMOGs], the [NSA] analyst wrote, “are an opportunity!”. According to the briefing notes, so many different US intelligence agents were conducting operations inside games that a “deconfliction” group was required to ensure they weren’t spying on, or interfering with, each other.
Fantastic wrap-up of the story so far on the pervasive global surveillance story.
The history of the intelligence community, though, reveals a willingness to violate the spirit and the letter of the law, even with oversight. What’s more, the benefits of the domestic-surveillance programs remain unclear. Wyden contends that the N.S.A. could find other ways to get the information it says it needs. Even Olsen, when pressed, suggested that the N.S.A. could make do without the bulk-collection program. “In some cases, it’s a bit of an insurance policy,” he told me. “It’s a way to do what we otherwise could do, but do it a little bit more quickly.” In recent years, Americans have become accustomed to the idea of advertisers gathering wide swaths of information about their private transactions. The N.S.A.’s collecting of data looks a lot like what Facebook does, but it is fundamentally different. It inverts the crucial legal principle of probable cause: the government may not seize or inspect private property or information without evidence of a crime. The N.S.A. contends that it needs haystacks in order to find the terrorist needle. Its definition of a haystack is expanding; there are indications that, under the auspices of the “business records” provision of the Patriot Act, the intelligence community is now trying to assemble databases of financial transactions and cell-phone location information. Feinstein maintains that data collection is not surveillance. But it is no longer clear if there is a distinction.
Sherlock’s record is spotty at best when it comes to engagement. Setting aside the 80,680 people who were ignored by the minister, he was hostile and counter productive to debate from the beginning, going so far as to threaten to pull out of a public debate because a campaigner against the ['Irish SOPA'] SI would be in attendance. His habit of blocking people online who publicly ask him tough yet legitimate questions has earned him the nickname “Sherblock”.
Most utilities don’t want smart metering. In fact they seem to have used the wrong dictionary. It is difficult to find anything smart about the UK deployment, until you realise that the utilities use smart in the sense of “it hurts”. They consider they have a perfectly adequate business model which has no need for new technology. In many Government meetings, their reluctant support seems to be a veneer for the hope that it will all end in disaster, letting them go back to the world they know, of inflated bills and demands for money with menaces. [...] Even when smart meters are deployed, there is no evidence that any utility will use the resulting data to transform their business, rather than persecute the consumer. At a recent US conference a senior executive for a US utility which had deployed smart meters, stated that their main benefit was “to give them more evidence to blame the customer”. That’s a good description of the attitude displayed by our utilities.
Similar to ACID properties, if you partially provide properties it means the user has to _still_ consider in their application that the property doesn’t exist, because sometimes it doesn’t. In you’re fsync example, if fsync is relaxed and there are no replicas, you cannot consider the database durable, just like you can’t consider Redis a CP system. It can’t be counted on for guarantees to be delivered. This is why I say these systems are hard for users to reason about. Systems that partially offer guarantees require in-depth knowledge of the nuances to properly use the tool. Systems that explicitly make the trade-offs in the designs are easier to reason about because it is more obvious and _predictable_.
Good blog post about EVE’s algorithm to load-balance a 3D map of star systems
a nice pattern for unit tests which need deterministic time behaviour. Trying to think up a really nice API for this….
Simon McGarr on Ireland’s looming data-protection train-crash.
Last week, during the debate of his proposals to increase fees for making a Freedom of Information request, Brendan Howlin was asked how one of his amendments would affect citizens looking for data from the State’s electronic databases. His reply was to cheerfully admit he didn’t even understand the question. “I have no idea what an SQL code is. Does anyone know what an SQL code is?” Unlike the minister, it probably isn’t your job to know that SQL is the computer language that underpins the data industry. The amendment he had originally proposed would have effectively allowed civil servants to pretend that their computer files were made of paper when deciding whether a request was reasonable. His answer showed how the Government could have proposed such an absurd idea in the first place. Like it or not – fair or not – these are not the signals a country that wanted to build a long-term data industry would choose to send out. They are the sort of signals that Ireland used to send out about Financial Regulation. I think it’s agreed, that approach didn’t work out so well.
good blog post writing up the ‘flock -n -c’ trick to ensure single-concurrent-process locking for cron jobs
Good article on road safety and visual perception, for both cyclists and drivers.
a modern HTTP benchmarking tool capable of generating significant load when run on a single multi-core CPU. It combines a multithreaded design with scalable event notification systems such as epoll and kqueue. An optional LuaJIT script can perform HTTP request generation, response processing, and custom reporting.Written in C, ASL2 licensed.
Based on a working paper from University of Toronto researcher Laurina Zhang
Comparing album sales of four major labels before and after the removal of DRM reveals that digital music revenue increases by 10% when restrictions are removed. The effect goes up to 30% for long tail content, while top-selling albums show no significant jump. The findings suggest that dropping technical restrictions can benefit both artists and the major labels.more details: http://inside.rotman.utoronto.ca/laurinazhang/files/2013/11/laurina_zhang_jmp_nov4.pdf , “Intellectual Property Strategy and the Long Tail: Evidence from the Recorded Music Industry”, Laurina Zhang, November 4, 2013
The English bulldog has come to symbolize all that is wrong with the dog fancy and not without good reason; they suffer from almost every possible disease. A 2004 survey by the Kennel Club found that they die at the median age of 6.25 years (n=180). There really is no such thing as a healthy bulldog. The bulldog’s monstrous proportions makes them virtually incapable of mating or birthing without medical intervention.(via Bryan)
Samy Kamkar strikes again. ‘Using a Parrot AR.Drone 2, a Raspberry Pi, a USB battery, an Alfa AWUS036H wireless transmitter, aircrack-ng, node-ar-drone, node.js, and my SkyJack software, I developed a drone that flies around, seeks the wireless signal of any other drone in the area, forcefully disconnects the wireless connection of the true owner of the target drone, then authenticates with the target drone pretending to be its owner, then feeds commands to it and all other possessed zombie drones at my will.’
Good article about emergent behaviour from networked malware: ‘The metabot, therefore, is viral. You get followed because of who follows you. This tendency explains the strange geographical cluster among San Diego high school students. Perhaps one of those kids was being followed by a really popular account (like @Interscope records, perhaps, which follows hundreds of thousands of people), and through that link, the bot stumbled into this little circle of San Diego teens. All of this activity would have remained under the radar, of course, all part of the silent non-human web. Except something went awry. For some reason, Olivia got stuck in a weird loop, and the metabot kept spawning spambots that chose to follow her over and over, relentlessly. Maybe once the metabot reached the San Diego kids, a bug kicked in. Instead of negative feedback keeping her (and everyone else) from being followed too often, we got runaway positive feedback. The bots followed her because other bots followed her. And on and on. Which is, perhaps a kind of reasoning that we can understand: It’s the core logic of fame and celebrity itself. Attention flows to Snooki because attention flowed to Snooki. Attention flows to Olivia because attention flowed to Olivia. Olivia and her friends weren’t wrong when they thought she’d become suddenly famous. Her audience just wasn’t human.’
> reorg Ok, you reorganize all zero of your direct reports. Way to stay out of trouble, Hoss. Perhaps you’d like to coin an acronym?
y’know, for kids. now that would improve the slightly boring, functional helmet my middle kid wears…
Wow, I didn’t know about this. Great idea.
Need a flexible format to record, export, and analyze network performance data? Well, that’s exactly what the HTTP Archive format (HAR) is designed to do! Even better, did you know that Chrome DevTools supports it? In this episode we’ll take a deep dive into the format (as you’ll see, its very simple), and explore the many different ways it can help you capture and analyze your sites performance. Join Ilya Grigorik and Peter Lubbers to find out how to capture HAR network traces in Chrome, visualize the data via an online tool, share the reports with your clients and coworkers, automate the logging and capture of HAR data for your build scripts, and even adapt it to server-side analysis use cases
this is absolutely fantastic. Thanks flood.io!
it might seem that current efforts to identify and track potential terrorists would be approached with caution. Yet the federal government’s main terrorist watch list has grown to at least 700,000 people, with little scrutiny over how the determinations are made or the impact on those marked with the terrorist label. “If you’ve done the paperwork correctly, then you can effectively enter someone onto the watch list,” said Anya Bernstein, an associate professor at the SUNY Buffalo Law School and author of “The Hidden Costs of Terrorist Watch Lists,” published by the Buffalo Law Review in May. “There’s no indication that agencies undertake any kind of regular retrospective review to assess how good they are at predicting the conduct they’re targeting.”
a demo of Doug Lea’s latest concurrent data structure in Java 8
lulz. (via John Handelaar)
A nice worked-through Docker example
Really stupid — Facebook infers a “like” for a site when you send a reference to a URL on that site. Obviously broken behaviour. (via http://www.forbes.com/sites/anthonykosner/2013/01/21/facebook-is-recycling-your-likes-to-promote-stories-youve-never-seen-to-all-your-friends/ )
Newegg, an online retailer that has made a name for itself fighting the non-practicing patent holders sometimes called “patent trolls,” sits on the losing end of a lawsuit tonight. An eight-person jury came back shortly after 7:00pm and found that the company infringed all four asserted claims of a patent owned by TQP Development, a company owned by patent enforcement expert Erich Spangenberg.“patent enforcement expert”. That’s one way to put it. This is insanity.
pretty strong argument. However, I think shlibs still have an advantage in that their pages are easier to share…
“We’ve heard a good bit in this courtroom about public key encryption,” said Albright. “Are you familiar with that? “Yes, I am,” said Diffie, in what surely qualified as the biggest understatement of the trial. “And how is it that you’re familiar with public key encryption?” “I invented it.”(via burritojustice)
Yahoo!’s streaming machine learning platform, built on Storm, implementing:
As a library, SAMOA contains state-of-the-art implementations of algorithms for distributed machine learning on streams. The first alpha release allows classification and clustering. For classification, we implemented a Vertical Hoeffding Tree (VHT), a distributed streaming version of decision trees tailored for sparse data (e.g., text). For clustering, we included a distributed algorithm based on CluStream. The library also includes meta-algorithms such as bagging.
yay (via Tony Finch)
The jury found that Agence France-Presse and Getty Images willfully violated the Copyright Act when they used photos Daniel Morel took in his native Haiti after the 2010 earthquake that killed more than 250,000 people, Morel’s lawyer, Joseph Baio, said
beautiful German boardgame, suitable for playing with kids — an adult moves a tealight candle around the board, while kids take turns moving gnomes around in the shadows behind tall “trees”. recommended by JK
Freemen wackiness in Cork.
The house of one member of the group was raided by gardaí last week, but it is not thought that any arrests were made, according to an eyewitness. Gardaí broke down the front door of the house. The group, which appears to be part of the Freemen of the Land movement, which does not recognise the State, has attempted to hold ‘trials’ in Ballyphehane Community Centre. It attempted to summon HSE staff, gardaí, social workers, solicitors and others to appear to be tried by a self-selected jury earlier this month. The group handed out documents purporting to be a summons to HSE staff and garda stations, demanding that named people attend a trial by ‘éire court’ on Tuesday 5 November at 9am “to stand trial for their acts of terrorism against mothers, their offspring and others in our community”, according to the group’s literature. This week the group has begun posting about UCC, saying the college is “a private for profit corporation, and a business partner of and partly owned by Pfizers and Bank of Ireland”. The group suggest that UCC bases its “authority” on Maritime Law. UCC has yet to respond to the group’s allegations.
I’m trying to avoid doing this in order to avoid more power consumption and unpopular hardware in the house — but if necessary, this is a good up-to-date homebuild design
Interesting article around using mmap’d files from Java using RandomAccessFile.getChannel().map(), which allows them to be accessed directly as a ByteBuffer. together with Atomic variable lazySet() operations, this provides pretty excellent performance results on low-latency writes to disk. See also: http://psy-lob-saw.blogspot.ie/2012/12/atomiclazyset-is-performance-win-for.html
a realtime processing engine, built on a persistent queue and a set of workers. ‘The main goal is data availability and persistency. We created grape for those who cannot afford losing data’. It does this by allowing infinite expansion of the pending queue in Elliptics, their Dynamo-like horizontally-scaled storage backend.
‘remember, there is no axe murderer. probably’
MITM attacks via BGP route hijacking now relatively commonplace on the internet, with 60 cases observed so far this year by Renesys
Steven J. Murdoch presents some interesting results indicating that the EURion constellation may have been obsoleted:
Recent printers, scanners and image manipulation software identify images of currency, will not process the image and display an error message linking to www.rulesforuse.org. The detection algorithm is not disclosed, however it is possible to test sample images as to whether they are identified as currency. This webpage shows an initial analysis of the algorithm’s properties, based on results from the automated generation and testing of images. [...] Initially it was thought that the “Eurion constellation” was used to identify banknotes in the newly deployed software based system, since this has been confirmed to be the technique used by colour photocopiers, and was both necessary and sufficient to prevent an item being duplicated using the photocopier tested. However further investigation showed that the detection performed by software is different from the system used in colour photocopiers, and the Eurion constellation is neither necessary nor sufficent, and in fact it probably is not even a factor.
a simple-to-use, extensible, text-based data workflow tool that organizes command execution around data and its dependencies. Data processing steps are defined along with their inputs and outputs and Drake automatically resolves their dependencies. [...] Drake is similar to GNU Make, but designed especially for data workflow management. It has HDFS [and S3] support, allows multiple inputs and outputs, and includes a host of features designed to help you bring sanity to your otherwise chaotic data processing workflows.Via Nelson. Looks interesting, although I’d like to see more features around retries, single-executor locking, parallelism, alerting/metrics, and unattended cron-like operation — those are always the hard part when I wind up coding up a data pump.
good Redshift tips
A rather sordid tale of IP acquisition and exploitation, from the sounds of it
A very worrying quote from Una Dillon of the Irish Payment Services Organisation in regard to the Loyaltybuild incident:
“I wouldn’t be overly concerned if one of my cards was caught up in this,” Dillon says. “Even in the worst-case scenario – one in which my card was used fraudulently – my card provider will refund me everything that is taken”.This reflects a deep lack of understanding of (a) how identity fraud works, and (b) how card-fraud refunds in Ireland appear to work. (a): Direct misuse of credit card data is not always the result. Fraudsters may prefer to instead obtain separate credit through identity theft, ie. using other personal identifying data. (b): Visa debit cards have no credit limit — your bank account can be cleared out in its entirety, and refunds can take a long time. For instance, http://www.askaboutmoney.com/showthread.php?t=174482 describes several cases, including one customer who waited 21 days for a refund. All in all it’s trivialising a major risk for consumers. As I understand it, a separate statement from IPSO recommended that all customers of Loyaltybuild schemes need to monitor their bank accounts daily to keep an eye out for fraud, which is pretty absurd. Not impressive at all.
There is really astonishingly little value in looking at someone’s GitHub projects out of context. For a start, GitHub has no way of customising your profile page, and what is shown by default is the projects with the most stars, and the projects you’ve recently pushed to. That is, GitHub picks your most popular repos and puts those at the top. You have no say about what you consider important, or worthwhile, or interesting, or well-engineered, or valuable. You just get what other people think is useful. Aside from which, GitHub displays a lot of useless stats about how many followers you have, and some completely psychologically manipulative stats about how often you commit and how many days it is since you had a day off. So really, your GitHub profile displays two things: how ‘influential’ you are, and how easily you can be coerced into constantly working. It’s honestly about as relevant to a decent hiring decision as your Klout score.
In this paper, we have analyzed the performance of TCP in of ShenZhou Online, a commercial, mid-sized MMORPG. Our study indicates that, though TCP is full-fledged and robust, simply transmitting game data over TCP could cause unexpected performance problems. This is due to the following distinctive characteristics of game traffic: 1) tiny packets, 2) low packet rate, 3) application-limited traffic generation, and 4) bi-directional traffic. We have shown that because TCP was originally designed for unidirectional and network-limited bulk data transfers, it cannot adapt well to MMORPG traffic. In particular, the window-based congestion control mechanism and the fast retransmit algorithm for loss recovery are ineffective. This suggests that the selective acknowledgement option should be enabled whenever TCP is used, as it significantly enhances the loss recovery process. Furthermore, TCP is overkill, as not every game packet needs to be transmitted reliably and processed in an orderly manner. We have also shown that the degraded network performance did impact users’ willingness to continue a game. Finally, a number of design guidelines have been proposed by exploiting the unique characteristics of game traffic.via Nelson
What is afoot here is a rerun of the Celtic Tiger era “light touch regulation” of financial services. Ireland has again made a Faustian pact whereby we lure employers here on the understanding that they will not subject to too-stringent a regulatory system. As the Loyaltybuild breach has shown, this is a bargain that will probably end badly. And as with the financial services boom, it is making the Germans nervous. Perhaps we will listen to them this time.
Looks very alpha, but one to watch.
A JVM Implementation of the Raft Consensus Protocol
‘ A persistent key-value store for fast storage environments’, ie. BerkeleyDB/LevelDB competitor, from Facebook.
RocksDB builds on LevelDB to be scalable to run on servers with many CPU cores, to efficiently use fast storage, to support IO-bound, in-memory and write-once workloads, and to be flexible to allow for innovation. We benchmarked LevelDB and found that it was unsuitable for our server workloads. Thebenchmark results look awesome at first sight, but we quickly realized that those results were for a database whose size was smaller than the size of RAM on the test machine – where the entire database could fit in the OS page cache. When we performed the same benchmarks on a database that was at least 5 times larger than main memory, the performance results were dismal. By contrast, we’ve published the RocksDB benchmark results for server side workloads on Flash. We also measured the performance of LevelDB on these server-workload benchmarks and found that RocksDB solidly outperforms LevelDB for these IO bound workloads. We found that LevelDB’s single-threaded compaction process was insufficient to drive server workloads. We saw frequent write-stalls with LevelDB that caused 99-percentile latency to be tremendously large. We found that mmap-ing a file into the OS cache introduced performance bottlenecks for reads. We could not make LevelDB consume all the IOs offered by the underlying Flash storage.Lots of good discussion at https://news.ycombinator.com/item?id=6736900 too.
Colm McCarthaigh has open sourced Infima, ‘a library for managing service-level fault isolation using Amazon Route 53′.
Infima provides a Lattice container framework that allows you to categorize each endpoint along one or more fault-isolation dimensions such as availability-zone, software implementation, underlying datastore or any other common point of dependency endpoints may share. Infima also introduces a new ShuffleShard sharding type that can exponentially increase the endpoint-level isolation between customer/object access patterns or any other identifier you choose to shard on. Both Infima Lattices and ShuffleShards can also be automatically expressed in Route 53 DNS failover configurations using AnswerSet and RubberTree.
The LatencyUtils package includes useful utilities for tracking latencies. Especially in common in-process recording scenarios, which can exhibit significant coordinated omission sensitivity without proper handling.
slides from Ilya Grigorik’s tutorial on the topic at O’Reilly’s Velocity conference. lots of good data and tips for internet protocol optimization
tl;dr: ‘a lot to like’.
The grand design and originality thus of ‘Modernising Copyright’ thus is the injection of targeted flexibility into the legal framework – this is no mere echo of the Hargreaves Report in the UK, which backed away from Fair Use out of fear at the uncertainty it would necessarily entail. If the Report’s authors have their way, contested uses in Ireland will first be examined to see if they fit the exceptions spelled out in the EUCD, or checked against the innovation exception if they are derivative works/adaptations. Only if they have fallen at those two fences, will the fair use test be their last chance saloon.
‘It can’t just be Big Data, it has to be Fast Data: Reactor 1.0 goes GA’:
Reactor provides the necessary abstractions to build high-throughput, low-latency–what we now call “fast data”–applications that absolutely must work with thousands, tens of thousands, or even millions of concurrent requests per second. Modern JVM applications must be built on a solid foundation of asynchronous and reactive components that efficiently manage the execution of a very large number of tasks on a very small number of system threads. Reactor is specifically designed to help you build these kinds of applications without getting in your way or forcing you to work within an opinionated pattern.Featuring the LMAX Disruptor ringbuffer, the JavaChronicle fast persistent message-passing queue, Groovy closures, and Netty 4.0. This looks very handy indeed….
According to Backblaze’s data, 80% of drives last 4 years, and the median lifespan is projected to be 6 years
This is a beauty. I wonder if they can ship to Ireland?
To tell our story for this Kickstarter project, we really have to start in Christmas of 1959. Like many young scientists of the time, I received a Gilbert Chemistry set. This chemistry set provided me hours of great fun and learning as well as laying the foundation for my future as a research chemist. As I became an adult I wanted to share these types of experiences with my daughter, my nephews and nieces, and friends. But soon I became aware real chemistry sets were no longer available. Without real chemistry sets and opportunities for students to learn and explore, where would our future chemists come from? So …. I set out on a mission.
Nice deck covering HyperLogLog and its origins, plus a slide at the end covering the Flajolet/Wegman Adaptive Sampling algorithm (“how do you count the number of elements which appear only once in stream using constant size memory?”)
“In reality, [tacos and flautas] aren’t bad meals,” the report argues. “The error that many of us Mexicans [Gustavo note: and gabachos] commit is including these types of dishes in our regular diet without an appropriate balance of them and falling into excessively eating them; accompanied by a lack of physical activity, it creates bad eating habits.” The good docs go on to note that people can eat tacos and flautas without negatively affecting their health, but “the key resides in controlling the quantity and frequency of eating these types of meals.” They also make the point that overall, tacos and flautas have less grease than doughnuts, french fries and even some health bars, although they didn’t specify which brands in the latter. In a subsequent blog post, the scientists go on to describe flautas as an “energy food” due to their composition, and conclude by recommending that a healthy diet can include three tacos al pastor or four flautas per order, “controlling the frequency of intake.” So have at it, boyos, but in moderation. And I can already hear the skeptics: What about tacos de chicharrones? Why not focus on carne asada? Did they take into consideration chiles de mordida? Did they factor in horchata? And whither the burrito variable?
‘what Jeff Dean and team have been up to at Google’. Reducing request latency in a network SOA architecture using backup requests, etc., via Ilya Grigorik
It’s common for even the best programmers to make simple mistakes. And commonly, a refactoring which seems safe can leave behind code which will never do what’s intended. We’re used to getting help from the compiler, but it doesn’t do much beyond static type checking. Using error-prone to augment the compiler’s static analysis, you can catch more mistakes before they cost you time, or end up as bugs in production. We use error-prone in Google’s Java build system to eliminate classes of serious bugs from entering our code, and we’ve open-sourced it, so you can too!
This is really disappointing; many meats labelled as “Irish” are anything but. The only trustworthy mark is the Bord Bia “Origin Ireland” stamp — I’ll be avoiding any products without this in future.
Under European labelling law, country of origin is mandatory for beef, fish, olive oil, honey and fresh fruit and vegetables. Next month the EU will make it law to specify country of origin for the meat of pigs, chicken, sheep and goats, with a lead-in time of anywhere up to three years for food companies to comply. The pork rule, however, will only apply to fresh pork and not to processed meat, so consumers still won’t get a country-of-origin label on rashers, sausages or ham. In the meantime, the Bord Bia Origin-Ireland stamp is a guarantee that your Irish breakfast ingredients are indeed Irish.
TheStory.ie will, in all likelihood, cease all FOI requests. And we will not seek funding from the public to support an immoral, cynical, unjustified and probably illegal FOI fee regime. We will not pay for information that the public already pays for. We will not support a system that perpetuates an outrageous infringement of citizen rights. The legislation was gutted in 2003 and it is being gutted again. More generally the number of requests from journalists from all news organisations in Ireland will fall as a result of these amendments, and the resulting efforts to shine a light on the administration of the State will certainly deteriorate. And secrecy will prevail.
Some decent tips in here, mainly EC2-focussed
really fascinating, from Ethan Zuckerman:
The photo of workers making Guy Fawkes masks is something of a Rorschach test. If you’re primed to see the exploitative nature of global capitalism when you see people making a plastic mask, it’s there in the image. if you’re looking for the global spread of a protest movement, it’s there too, with a Brazilian factory making a local knock-off of a global icon to cash in on a national protest. Because the internet is a copying machine, it’s very bad at context. It’s easier to encounter the image of masks being manufactured devoid of accompanying details than it is to find the story behind the images. And given our tendency to ignore information in languages we don’t read, it’s easy to see how the masks come detached from their accompanying story. For me, the image is more powerful with context behind it. It’s possible to reflect on the irony of a Hollywood prop becoming an activist trope, the tensions between mass-production and anonymity and the individuality of one’s identity and grievance, the tensions between local and global, Warner Bros and Condal, intellectual property and piracy, all in the same image.
For creative groups, we aim to offer easy access to a rich and varied selection of textures, colours and shapes. Members are also be able to participate in creativity workshops facilitated by fully trained professional artists either in-house or on your own premises. We intend to be the first choice of teachers, early childhood educators and arts animators in the community. For businesses, ReCreate reduces the costs of moving on end-of-line materials. We are a professional, credible and reliable partner organisation and our aim is to divert approximately 115 metric tonnes of clean materials from landfill annually. All collections are free of charge.
When I get my hands on a 3-D printer, this will be high up my list of things to fabricate: a replica of a 20-million year old hominid skull.
With over 40 digitized fossils in their collection, you can explore 3D renders of fossils representing prehistoric animals, human ancestors, and even ancient tools. Captured using Autodesk software, an SLR camera, and often the original specimen (rather than a cast replica), these renderings bring us closer than most will ever get to holding ancient artifacts. And if you’ve got an additive manufacturing device at your disposal, you can even download Sketchfab plans to generate your own.
‘A Tiny Seasonal Department Store’, featuring the amazing cakes of Wildflour Bakery among others, at 5 Dame Lane, D2.
The tiny department store will be a wonderful seasonal gathering of Makers & Brothers favourite local and international brands. The Others in this project are a carefully considered bunch of partners from the worlds of flowers, food, fashion, beauty, homeware, gifts and more. Makers & Brothers & Others, the tiny department store, promises to be a unique, exciting and engaging retail environment. A place to explore, a seasonal store alive with wonder and served by experts. Kindly hosted by the Fumbally Exchange.
Good commentary on the recent CRC report’s recommendations. See also http://ipkitten.blogspot.ie/2013/10/modernising-irish-copyright-katseries-1.html
+1 to all of this, but especially #5 (polar area diagrams).
Presto has become a major interactive system for the company’s data warehouse. It is deployed in multiple geographical regions and we have successfully scaled a single cluster to 1,000 nodes. The system is actively used by over a thousand employees,who run more than 30,000 queries processing one petabyte daily. Presto is 10x better than Hive/MapReduce in terms of CPU efficiency and latency for most queries at Facebook. It currently supports a large subset of ANSI SQL, including joins, left/right outer joins, subqueries,and most of the common aggregate and scalar functions, including approximate distinct counts (using HyperLogLog) and approximate percentiles (based on quantile digest). The main restrictions at this stage are a size limitation on the join tables and cardinality of unique keys/groups. The system also lacks the ability to write output data back to tables (currently query results are streamed to the client).
DNA tests show that many pills labeled as healing herbs are little more than powdered rice and weeds. [...] Among their findings were bottles of echinacea supplements, used by millions of Americans to prevent and treat colds, that contained ground up bitter weed, Parthenium hysterophorus, an invasive plant found in India and Australia that has been linked to rashes, nausea and flatulence.
Scryer is a new system that allows us to provision the right number of AWS instances needed to handle the traffic of our customers. But Scryer is different from Amazon Auto Scaling (AAS), which reacts to real-time metrics and adjusts instance counts accordingly. Rather, Scryer predicts what the needs will be prior to the time of need and provisions the instances based on those predictions.
We have known about [the dental health benefits of xylitol in chewing gum] for a surprisingly long time. In the 1980s, a high-quality, randomized trial in Finland found that children who chewed xylitol-sweetened gum had as much as 60 percent fewer cavities compared with children who didn’t. A 1989-93 randomized study of children around age 10 in Belize showed an even greater benefit; chewing xylitol-sweetened gum decreased the risk of cavities by up to 70 percent, and a follow-up study showed that the benefit lasted for up to five years.
The packet capture shown in these new NSA slides shows internal database replication traffic for the anti-hacking system I worked on for over two years. Specifically, it shows a database recording a user login.This kind of confirms my theory that the majority of interesting traffic for the NSA/GCHQ MUSCULAR sniffing system would have been inter-DC replication. Was, since it sounds like that stuff’s all changing now to use end-to-end crypto…
‘This article will use NettoSphere, a framework build on top of the popular Netty Framework and Atmosphere with support of WebSockets, Server Side Events and Long-Polling. NettoSphere allows [async JVM framework] Atmosphere’s applications to run on top of the Netty Framework.’
This looks really nice — it’s quite similar to something I was hacking on a while back. Only problem is that it’s AGPL-licensed… ‘Pushpin makes it easy to create HTTP long-polling and streaming services using any web stack as the backend. It’s compatible with any framework, whether Django, Rails, ASP, or even PHP. Pushpin works as a reverse proxy, sitting in front of your server application and managing all of the open client connections.’
‘A recent ruling by the European Court of Human Rights (ECHR) has called into question [...] the liability of media organisations for online comment.’ Delfi, a news website in Estonia, found liable for a user’s comments by the ECHR
Bookmarking particularly for the 3 “favourite DevOps patterns”:
“Make sure we have environments available early in the Development process”; enforce a policy that the code and environment are tested together, even at the earliest stages of the project; “Wake up developers up at 2 a.m. when they break things”; and “Create reusable deployment procedures”.
The problem in a nutshell is that for an uncomfortable amount of the year the demand outstrips what the system can comfortably supply. In the graph below you’ll see the red line (demand for water) matches and regularly exceeds the blue line (what’s produced).
Circa 1800, the Cocktail was a “hair of the dog” morning drink that tamed spirits with water, sugar and bitters (patent medicine). The late 19th Century expanded the use of the word “cocktail” to encompass just about any mixed drink. Since then, the Old Fashioned—literally, the old-fashioned way of making a cocktail—has been our contemporary expression of the original drink. During the 20th Century, various bad ideas encrusted the Old Fashioned. Here we will strip off those barnacles to expose the amazingly simple and sublime drink beneath.thanks to Ben for this one…
“We assess that Miranda is knowingly carrying material [...] the disclosure or threat of disclosure is designed to influence a government, and is made for the purpose of promoting a political or ideological cause. This therefore falls within the definition of terrorism.”
One of the most important results in distributed systems theory was published in April 1985 by Fischer, Lynch and Patterson. Their short paper ‘Impossibility of Distributed Consensus with One Faulty Process’, which eventually won the Dijkstra award given to the most influential papers in distributed computing, definitively placed an upper bound on what it is possible to achieve with distributed processes in an asynchronous environment. This particular result, known as the ‘FLP result’, settled a dispute that had been ongoing in distributed systems for the previous five to ten years. The problem of consensus – that is, getting a distributed network of processors to agree on a common value – was known to be solvable in a synchronous setting, where processes could proceed in simultaneous steps. In particular, the synchronous solution was resilient to faults, where processors crash and take no further part in the computation. Informally, synchronous models allow failures to be detected by waiting one entire step length for a reply from a processor, and presuming that it has crashed if no reply is received. This kind of failure detection is impossible in an asynchronous setting, where there are no bounds on the amount of time a processor might take to complete its work and then respond with a message. Therefore it’s not possible to say whether a processor has crashed or is simply taking a long time to respond. The FLP result shows that in an asynchronous setting, where only one processor might crash, there is no distributed algorithm that solves the consensus problem.
Terrible internet ad-spam recast as machine-learning spam
’37-year-old patriot discovers “weird” trick to end slavery to the Bayesian monopoly. Discover the underground trick she used to slash her empirical risk by 75% in less than 30 days… before they shut her down. Click here to watch the shocking video! Get the Shocking Free Report!’
These companies and their technologies are built on data, and the data is us. If we are to have any faith in the Internet, we have to trust them to protect it. That’s a relationship dynamic that will become only more intertwined as the Internet finds its way into more aspects of our daily existences, from phones that talk to us to cars that drive themselves. The US’s surveillance programs threaten to destroy that trust permanently. America’s tech companies must stand up to this pervasive and corrosive surveillance system. They must ask that difficult question: “Is it worth it?”
‘a service discovery and orchestration tool that is decentralized, highly available, and fault tolerant. Serf runs on every major platform: Linux, Mac OS X, and Windows. It is extremely lightweight: it uses 5 to 10 MB of resident memory and primarily communicates using infrequent UDP messages [and an] efficient gossip protocol.’
Skew is prevalent in many data sources such as IP traffic streams.To continually summarize the distribution of such data, a high-biased set of quantiles (e.g., 50th, 90th and 99th percentiles) with finer error guarantees at higher ranks (e.g., errors of 5, 1 and 0.1 percent, respectively) is more useful than uniformly distributed quantiles (e.g., 25th, 50th and 75th percentiles) with uniform error guarantees. In this paper, we address the following two prob-lems. First, can we compute quantiles with finer error guarantees for the higher ranks of the data distribution effectively, using less space and computation time than computing all quantiles uniformly at the finest error? Second, if specific quantiles and their error bounds are requested a priori, can the necessary space usage and computation time be reduced? We answer both questions in the affirmative by formalizing them as the “high-biased” quantiles and the “targeted” quantiles problems, respectively, and presenting algorithms with provable guarantees, that perform significantly better than previously known solutions for these problems. We implemented our algorithms in the Gigascope data stream management system, and evaluated alternate approaches for maintaining the relevant summary structures.Our experimental results on real and synthetic IP data streams complement our theoretical analyses, and highlight the importance of lightweight, non-blocking implementations when maintaining summary structures over high-speed data streams.
Implemented as a timer-histogram storage system in http://armon.github.io/statsite/ .
A C reimplementation of Etsy’s statsd, with some interesting memory optimizations.
Statsite is designed to be both highly performant, and very flexible. To achieve this, it implements the stats collection and aggregation in pure C, using libev to be extremely fast. This allows it to handle hundreds of connections, and millions of metrics. After each flush interval expires, statsite performs a fork/exec to start a new stream handler invoking a specified application. Statsite then streams the aggregated metrics over stdin to the application, which is free to handle the metrics as it sees fit. This allows statsite to aggregate metrics and then ship metrics to any number of sinks (Graphite, SQL databases, etc). There is an included Python script that ships metrics to graphite.
if Linnane’s and Cronin’s are anything to go by, these will be worth a visit
A fax machine called my #twilio voice number, this is how @twilio transcribed it…. http://pic.twitter.com/RYh19Pg2pGThis is amazing. Machine talking to machine, with hilarious results
Founded by Silent Circle and Lavabit. this is promising….
To bring the world our unique end-to-end encrypted protocol and architecture that is the ‘next-generation’ of private and secure email. As founding partners of The Dark Mail Alliance, both Silent Circle and Lavabit will work to bring other members into the alliance, assist them in implementing the new protocol and jointly work to proliferate the worlds first end-to-end encrypted ‘Email 3.0′ throughout the world’s email providers. Our goal is to open source the protocol and architecture and help others implement this new technology to address privacy concerns against surveillance and back door threats of any kind.
A rather dark short story about little girls, peer pressure, and childhood. no fun for this dad of 3 girls :( (via Tatu Saloranta)
A Histogram that supports recording and analyzing sampled data value counts across a configurable integer value range with configurable value precision within the range. Value precision is expressed as the number of significant digits in the value recording, and provides control over value quantization behavior across the value range and the subsequent value resolution at any given level.
John Allspaw with an interesting post on the Knight Capital disaster
This is exactly what you do NOT want to read about embedded systems controlling acceleration in your car:
The Camry electronic throttle control system code was found to have 11,000 global variables. Barr described the code as “spaghetti.” Using the Cyclomatic Complexity metric, 67 functions were rated untestable (meaning they scored more than 50). The throttle angle function scored more than 100 (unmaintainable). Toyota loosely followed the widely adopted MISRA-C coding rules but Barr’s group found 80,000 rule violations. Toyota’s own internal standards make use of only 11 MISRA-C rules, and five of those were violated in the actual code. MISRA-C:1998, in effect when the code was originally written, has 93 required and 34 advisory rules. Toyota nailed six of them. Barr also discovered inadequate and untracked peer code reviews and the absence of any bug-tracking system at Toyota.On top of this, there was no error-correcting RAM in use; stack-killing recursive code; a quoted 94% stack usage; risks of unintentional RTOS task shutdown; buffer overflows; unsafe casting; race conditions; unchecked error code return values; and a trivial watchdog timer check. Crappy, unsafe coding.
The sounds were not, however, caused by ghosts but by a group of three or four men at least to some degree professionally trained, the FBI now believes, in tunneling: a close-knit and highly disciplined team, perhaps from the construction industry, perhaps even a disgruntled public works crew who decided to put their knowledge of the city’s underside to more lucrative work. After all, Rehder explained, their route into the bank was as much brute-force excavation as it was a retracing of the region’s buried waterways, accessing the neighborhood by way of the city’s complicated storm-sewer network, itself built along old creek beds that no longer appear on city maps. As LAPD lieutenant Doug Collisson, one of the men present on the day of the tunnel’s discovery, explained to the Los Angeles Times back in 1987, the crew behind the burglary “would have had to require some knowledge of soil composition and technical engineering. … The way the shaft itself was constructed, it was obviously well-researched and extremely sophisticated.” Rehder actually goes further, remarking that when Detective Dennis Pagenkopp “showed crime scene photos of the core bit holes” produced by the burglars’ boring upward into the vault “to guys who were in the concrete-coring business, they whistled with professional admiration.”
The Copyright Review Committee report has been published. Headline recommendations:
Ensure the right of free speech is a central element of the new copyright regime, including in the areas of parody and satire; Legalise legitimate forms of copying by introducing an explicit and broadly defined “Fair Use” policy. Ensure the extent of copyright ownership is balanced against the public good; Design a system which is clear to all parties, including end users; Design an enforcement mechanism which is easy to understand, transparent and accessible to all parties; Target penalties at those who infringe on copyright rather than on third parties such as intermediaries; Future-proof the new regime by basing it on applicable principles rather than rules relevant to today’s technology only; Make it easy for end-users to identify and engage with owners of copyright material.Here’s hoping Sean Sherlock now does what he said he’d do, and acts on these recommendations.
Not just a Storm success story. Interesting slides indicating where a startup *stopped* using Storm as realtime wasn’t useful to their customers
I like the impromptu docking station hack
This is very well-put.
In its early days, there was a lot of talk about the “natural laws of the Internet” and how it would empower the masses, upend traditional power blocks, and spread freedom throughout the world. The international nature of the Internet made a mockery of national laws. Anonymity was easy. Censorship was impossible. Police were clueless about cybercrime. And bigger changes were inevitable. Digital cash would undermine national sovereignty. Citizen journalism would undermine the media, corporate PR, and political parties. Easy copying would destroy the traditional movie and music industries. Web marketing would allow even the smallest companies to compete against corporate giants. It really would be a new world order. Unfortunately, as we know, that’s not how it worked out. Instead, we have seen the rise of the feudal Internet: Feudal security consolidates power in the hands of the few. These companies [like Google, Apple, Microsoft, Facebook etc.] act in their own self-interest. They use their relationship with us to increase their profits, sometimes at our expense. They act arbitrarily. They make mistakes. They’re deliberately changing social norms. Medieval feudalism gave the lords vast powers over the landless peasants; we’re seeing the same thing on the Internet.
Cyber criminals are planting chips in electric irons and kettles to launch spam [jm: actually, malware] attacks, reports in Russia suggest. State-owned channel Rossiya 24 even showed footage of a technician opening up an iron included in a batch of Chinese imports to find a “spy chip” with what he called “a little microphone”. Its correspondent said the hidden devices were mostly being used to spread viruses, by connecting to any computer within a 200m (656ft) radius which were using unprotected Wi-Fi networks. Other products found to have rogue components reportedly included mobile phones and car dashboard cameras.
Planetary Resources is a company with a sky-high (some might claim “pie in the sky”) goal: to find and mine asteroids for useful minerals and other compounds. It is also a company that uses Linux and lots of free software. So two of the engineers from Planetary Resources, Ray Ramadorai and Marc Allen, gave a presentation at LinuxCon North America to describe how and why the company uses FOSS—along with a bit about what it is trying to do overall.
a pretty stupid Mail.app IMAP bug hoses Fastmail:
Yes you read that right. It’s copying all the email from the Junk Folder back into the Junk Folder again!. This is legal IMAP, so our server proceeds to create a new copy of each message in the folder. It then expunges the old copies of the messages, but it’s happening so often that the current UID on that folder is up to over 3 million. It was just over 2 million a few days ago when I first emailed the user to alert them to the situation, so it’s grown by another million since. The only way I can think this escaped QA was that they used a server which (like gmail) automatically suppresses duplicates for all their testing, because this is a massively bad problem.
One of those analyses showed that when a human was behind the wheel, Google’s cars accelerated and braked significantly more sharply than they did when piloting themselves. Another showed that the cars’ software was much better at maintaining a safe distance from the vehicle ahead than the human drivers were. “We’re spending less time in near-collision states,” said Urmson. “Our car is driving more smoothly and more safely than our trained professional drivers.”
interesting new data structure, pending addition in Java 8. Basically an array of arrays which presents the API of a single List.
An ordered collection of elements. Elements can be added, but not removed. Goes through a building phase, during which elements can be added, and a traversal phase, during which elements can be traversed in order but no further modifications are possible.
Direct Democracy Ireland, the party linked to Freemen-on-the-land and the Christian Solidarity Party, is having a bit of a bumpy ride with party governance it sounds like
Michael Hayden, former NSA and CIA boss, who famously argued that the only people complaining about NSA surveillance were internet shut-ins who couldn’t get laid, apparently never learned that when you’re in a public place, someone might overhear your phone calls. Entrepreneur and former MoveOn.org director Tom Matzzie just so happened to be on the Acela express train from DC to NY when he (1) spotted Hayden sitting behind him and (2) started overhearing a series of “off the record” phone calls with press about the story of the week: the revelations of the NSA spying on foreign leaders. Matzzie did what any self-respecting American would do: live-tweet the calls.
A tool to manage inter-container dependencies so that continuous delivery with Jenkins and Docker is feasible. Looks very helpful
Looks pretty persuasive, especially considering they hold a patent on the design
[This case] shows the link between the popular and the state. This is tabloid journalism followed by tabloid policing. It’s also completely ignorant. I wrote my article on the Roma after covering the community for a week. I thought, “that’s interesting – there’s a range of phenotypes, ways of looking, that include Roma.” I mentioned two blonde kids by chance. I mentioned that Roma are more likely to speak the language of the country they’re in than Romani, more likely to have the religion of the country they’re in. But they have the basic aspect that is true for all identities – they know each other and other people know them. It’s not like I’m an expert on the Roma. I was covering them for a week and after the second day I knew Roma children had blonde hair and blue eyes. These people who took that kid away knew nothing. And on that basis they abducted a child.
This is what happens when you don’t have strong controls on data protection/data privacy — the US experience.
While [posing as a US-based private investigator] may have gotten the [Vietnam-based gang operating the massive identity fraud site Superget.info] past Experian and/or CourtVentures’ screening process, according to Martin there were other signs that should have alerted Experian to potential fraud associated with the account. For example, Martin said the Secret Service told him that the alleged proprietor of Superget.info had paid Experian for his monthly data access charges using wire transfers sent from Singapore. “The issue in my mind was the fact that this went on for almost a year after Experian did their due diligence and purchased” Court Ventures, Martin said. “Why didn’t they question cash wires coming in every month? Experian portrays themselves as the data-breach experts, and they sell identity theft protection services. How this could go on without them detecting it I don’t know. Our agreement with them was that our information was to be used for fraud prevention and ID verification, and was only to be sold to licensed and credentialed U.S. businesses, not to someone overseas.”via Simon McGarr
“the European Parliament has today sent a clear message that enough is enough. The revelations about NSA interception of SWIFT data make a mockery of the EU’s agreement with the US, through which the bank data of European citizens is delivered to the US anti-terror system (TFTP). What is the purpose of an agreement like this, which was concluded in good faith, if the US authorities are going to circumvent its provisions? “The EU cannot continue to remain silent in the face of these ongoing revelations: it gives the impression we are little more than a lap dog of the US. If we are to have a healthy relationship with the US, based on mutual respect and benefit, EU governments must not be afraid of defending core EU values when they are infringed. EU leaders must finally take a clear and unambiguous stance on the NSA violations at this week’s summit.”
A follow up to the LinkedIn VM-tuning blog post at http://engineering.linkedin.com/performance/optimizing-linux-memory-management-low-latency-high-throughput-databases —
Do not read in to this article too much, especially for trying to understand how the Linux VM or the kernel works. The authors misread the “global spinlock on the zone” source code and the interpretation in the article is dead wrong.
Y! engineer doubles the speed of Storm’s messaging layer by replacing the zeromq implementation with Netty
Service discovery a la Airbnb — Nerve and Synapse: two external daemons that run on each node, Nerve to manage registration in Zookeeper, and Synapse to generate a haproxy configuration file from that, running on each host, allowing connections to all other hosts.
What is near-future late-capitalist dystopian fiction but a world where there is no discernible difference between corporations, nations, sports teams, brands, and celebrities? Adam was partly right in our original email thread. @TwoHeadlines is not generating jokes about current events. It is generating jokes about the future: a very specific future dictated by what a Google algorithm believes is important about humans and our affairs.
‘Welcome to the New York Review of Bots, a professional journal of automated-agent studies. We aspire to the highest standards of rigorous analysis, but will often just post things we liked that a computer made.’
Major outage and $465m of trading loss, caused by staggeringly inept software management: 8 years of incremental bitrot, technical debt, and failure to have correct processes to engage an ops team in incident response. Hopefully this will serve as a lesson that software is more than just coding, at least to one industry
Ha, cool. Skip the OS, write the Riak store natively to the drive. This sounds frankly terrifying ;)
The Seagate Kinetic Open Storage platform eliminates the storage server tier of traditional data center architectures by enabling applications to speak directly to the storage system, thereby reducing expenses associated with the acquisition, deployment, and support of hyperscale storage infrastructures. The platform leverages Seagate’s expertise in hardware and software storage systems integrating an open source API and Ethernet connectivity with Seagate hard drive technology.
Following months of revelations, and on the same day that France heard its citizens’ phone calls were being reportedly recorded en masse by the Americans, the Parliament’s committee gave a resounding thumbs-up to every single amendment proposed by industrious German Green MEP Jan Phillip Albrecht (pictured above).lolz.
the National Council for Curriculum and Assessment are looking for feedback on adding programming to the junior cycle (ie., early secondary school) in Ireland. Add your EUR.02!
‘the most exhaustive study of [multi-core] synchronization to date’