Links for 2014-10-04

  • Ebola vaccine delayed by IP spat

    This is the downside of publicly-funded labs selling patent-licensing rights to private companies:

    Given the urgency, it’s inexplicable that one of the candidate vaccines, developed at the Public Health Agency of Canada (PHAC) in Winnipeg, has yet to go in the first volunteer’s arm, says virologist Heinz Feldmann, who helped develop the vaccine while at PHAC. “It’s a farce; these doses are lying around there while people are dying in Africa,” says Feldmann, who now works at the Rocky Mountain Laboratories of the U.S. National Institute of Allergy and Infectious Diseases (NIAID) in Hamilton, Montana. At the center of the controversy is NewLink Genetics, a small company in Ames, Iowa, that bought a license to the vaccine’s commercialization from the Canadian government in 2010, and is now suddenly caught up in what WHO calls “the most severe acute public health emergency seen in modern times.” Becker and others say the company has been dragging its feet the past 2 months because it is worried about losing control over the development of the vaccine.

    (tags: ip patents drugs ebola canada phac newlink-genetics health epidemics vaccines)

  • sferik/t

    “A command-line power tool for Twitter.” It really is — much better timeline searchability than the “real” Twitter UI, for example

    (tags: twitter ruby github cli tools unix search)

  • Ebola: While Big Pharma Slept

    We’ve had almost 40 years to develop, test and stockpile an Ebola vaccine. That has not happened because big pharma has been entirely focused on shareholder value and profits over safety and survival from a deadly virus. For the better part of Ebola’s 38 years ? big pharma has been asleep. The question ahead is what virus or superbug will wake them up?

    (tags: pharma ebola ip patents health drugs africa research)

Posted in Uncategorized | Comments closed

Links for 2014-10-02

Posted in Uncategorized | Comments closed

Links for 2014-10-01

Posted in Uncategorized | Comments closed

Links for 2014-09-30

Posted in Uncategorized | Comments closed

Links for 2014-09-29

  • Prototype

    Prototype is a brand new festival of play and interaction. This is your chance to experience the world from a new perspective with removable camera eyes, to jostle and joust to a Bach soundtrack whilst trying to disarm an opponent, to throw shapes as you figure out who got an invite to the silent disco, to duel with foam pool noodles, and play chase in the dark with flashlights. A unique festival that incites new types of social interaction, involving technology and the city, Prototype is a series of performances, workshops, talks, and games that spill across the city, alongside an adult playground in the heart of Temple Bar.
    Project Arts Centre, 17-18 October. looks nifty

    (tags: prototype festivals dublin technology make vr gaming)

  • Confessions of a former internet troll – Vox

    I want to tell you about when violent campaigns against harmless bloggers weren’t any halfway decent troll’s idea of a good time — even the then-malicious would’ve found it too easy to be fun. When the punches went up, not down. Before the best players quit or went criminal or were changed by too long a time being angry. When there was cruelty, yes, and palpable strains of sexism and racism and every kind of phobia, sure, but when these things had the character of adolescents pushing the boundaries of cheap shock, disagreeable like that but not criminal. Not because that time was defensible — it wasn’t, not really — but because it was calmer and the rage wasn’t there yet. Because trolling still meant getting a rise for a laugh, not making helpless people fear for their lives because they’re threatening some Redditor’s self-proclaimed monopoly on reason. I want to tell you about it because I want to make sense of how it is now and why it changed.

    (tags: vox trolls blogging gamergate 4chan weev history teenagers)

Posted in Uncategorized | Comments closed

Links for 2014-09-27

  • La Maison des Amis

    Paul Hickey’s gite near Toulouse, available for rent! ‘a beautifully converted barn on 5 acres, wonderfully located in the French countryside. 4 Bedrooms, sleeps 2-10, Large Pool, Tennis Court, Large Trampoline, Broadband Internet, 30 Mins Toulouse/Albi, 65 Mins Carcassonne, 90 Mins Rodez’

    (tags: ex-iona gites france holidays vacation-rentals vacation hotels)

  • waxpancake on Ello

    The Ello founders are positioning it as an alternative to other social networks — they won’t sell your data or show you ads. “You are not the product.” If they were independently-funded and run as some sort of co-op, bootstrapped until profitable, maybe that’s plausible. Hard, but possible. But VCs don’t give money out of goodwill, and taking VC funding — even seed funding — creates outside pressures that shape the inevitable direction of a company.

    (tags: advertising money vc ello waxy funding series-a)

  • Inviso: Visualizing Hadoop Performance

    With the increasing size and complexity of Hadoop deployments, being able to locate and understand performance is key to running an efficient platform.  Inviso provides a convenient view of the inner workings of jobs and platform.  By simply overlaying a new view on existing infrastructure, Inviso can operate inside any Hadoop environment with a small footprint and provide easy access and insight.  
    This sounds pretty useful.

    (tags: inviso netflix hadoop emr performance ops tools)

  • The End of Linux

    ‘Linux is becoming the thing that we adopted Linux to get away from.’ Great post on the horrible complexity of systemd. It reminds me of nothing more than mid-90s AIX, which I had the displeasure of opsing for a while — the Linux distros have taken a very wrong turn here.

    (tags: linux unix complexity compatibility ops rant systemd bloat aix)

Posted in Uncategorized | Comments closed

Links for 2014-09-24

Posted in Uncategorized | Comments closed

Links for 2014-09-23

  • Avoiding Chef-Suck with Auto Scaling Groups – forty9ten

    Some common problems which arise using Chef with ASGs in EC2, and how these guys avoided it — they stopped using Chef for service provisioning, and instead baked AMIs when a new version was released. ASGs using pre-baked AMIs definitely works well so this makes good sense IMO.

    (tags: infrastructure chef ops asg auto-scaling ec2 provisioning deployment)

  • Introducing Groups.io

    Mark “ONEList” Fletcher’s back, and he’s reinventing the email group! awesome.

    email groups (the modern version of mailing lists) have stagnated over the past decade. Yahoo Groups and Google Groups both exude the dank air of benign neglect. Google Groups hasn’t been updated in years, and some of Yahoo’s recent changes have actually made Yahoo Groups worse! And yet, millions of people put up with this uncertainty and neglect, because email groups are still one of the best ways to communicate with groups of people. And I have a plan to make them even better. So today I’m launching Groups.io in beta, to bring email groups into the 21st Century. At launch, we have many features that those other services don’t have, including: Integration with other services, including: Github, Google Hangouts, Dropbox, Instagram, Facebook Pages, and the ability to import Feeds into your groups. Businesses and organizations can have their own private groups on their own subdomain. Better archive organization, using hashtags. Many more email delivery options. The ability to mute threads or hashtags. Fully searchable archives, including searching within attachments. One other feature that Groups.io has that Yahoo and Google don’t, is a business model that’s not based on showing ads to you. Public groups are completely free on Groups.io. Private groups and organizations are very reasonably priced.

    (tags: email groups communication discussion mailing-lists groups.io yahoo google google-groups yahoo-groups)

Posted in Uncategorized | Comments closed

Links for 2014-09-22

  • The Open Source Software Engagement Award

    SFU announces award for students who demonstrate excellence in contributing to an Open Source project

    (tags: sfu awards students open-source oss universities funding)

  • DublinDashboard

    ‘provides citizens, public sector workers and companies with real-time information, time-series indicator data, and interactive maps about all aspects of the city. It enables users to gain detailed, up to date intelligence about the city that aids everyday decision making and fosters evidence-informed analysis.’

    (tags: dublin dashboards maps geodata time-series open-data ireland)

  • mcrouter: A memcached protocol router for scaling memcached deployments

    New from Facebook engineering:

    Last year, at the Data@Scale event and at the USENIX Networked Systems Design and Implementation conference , we spoke about turning caches into distributed systems using software we developed called mcrouter (pronounced “mick-router”). Mcrouter is a memcached protocol router that is used at Facebook to handle all traffic to, from, and between thousands of cache servers across dozens of clusters distributed in our data centers around the world. It is proven at massive scale — at peak, mcrouter handles close to 5 billion requests per second. Mcrouter was also proven to work as a standalone binary in an Amazon Web Services setup when Instagram used it last year before fully transitioning to Facebook’s infrastructure. Today, we are excited to announce that we are releasing mcrouter’s code under an open-source BSD license. We believe it will help many sites scale more easily by leveraging Facebook’s knowledge about large-scale systems in an easy-to-understand and easy-to-deploy package.
    This is pretty crazy — basically turns a memcached cluster into a much more usable clustered-storage system, with features like shadowing production traffic, cold cache warmup, online reconfiguration, automatic failover, prefix-based routing, replicated pools, etc. Lots of good features.

    (tags: facebook scaling cache proxy memcache open-source clustering distcomp storage)

  • DIRECT MARKETING – A GENERAL GUIDE FOR DATA CONTROLLERS

    In particular:

    Where you have obtained contact details in the context of the sale of a product or service, you may only use these details for direct marketing by electronic mail if the following conditions are met: the product or service you are marketing is of a kind similar to that which you sold to the customer at the time you obtained their contact details At the time you collected the details, you gave the customer the opportunity to object, in an easy manner and without charge, to their use for marketing purposes Each time you send a marketing message, you give the customer the right to object to receipt of further messages The sale of the product or service occurred not more than twelve months prior to the sending of the electronic marketing communication or, where applicable, the contact details were used for the sending of an electronic marketing communication in that twelve month period.

    (tags: email spam regulations ireland law dpc marketing direct-marketing)

Posted in Uncategorized | Comments closed

Links for 2014-09-21

Posted in Uncategorized | Comments closed

Links for 2014-09-19

Posted in Uncategorized | Comments closed

Links for 2014-09-18

  • 75% of domestic violence victims in US shelters were spied on by their abusers using spyware

    via Mikko

  • Alex Payne — Thoughts On Five Years of Emerging Languages

    One could read the success of Go as an indictment of contemporary PLT, but I prefer to see it as a reminder of just how much language tooling matters. Perhaps even more critical, Go’s lean syntax, selective semantics, and cautiously-chosen feature set demonstrate the importance of a strong editorial voice in a language’s design and evolution. Having co-authored a book on Scala, it’s been painful to see systems programmers in my community express frustration with the ambitious hybrid language. I’ve watched them abandon ship and swim back to the familiar shores of Java, or alternately into the uncharted waters of Clojure, Go, and Rust. A pity, but not entirely surprising if we’re being honest with ourselves. Unlike Go, Scala has struggled with tooling from its inception. More than that, Scala has had a growing editorial problem. Every shop I know that’s been successful with Scala has limited itself to some subset of the language. Meanwhile, in pursuit of enterprise developers, its surface area has expanded in seemingly every direction. The folks behind Scala have, thankfully, taken notice: upcoming releases are promised to focus on simplicity, clarity, and better tooling.

    (tags: scala go coding languages)

Posted in Uncategorized | Comments closed

Links for 2014-09-17

  • Texas Judge References ‘The Big Lebowski’

    “The First Amendment of the U.S. Constitution is similarly suspicious of prior restraints,” wrote Justice Lehrmann in the decision highlighting a cornerstone that has “been reaffirmed time and again by the Supreme Court, this Court, Texas courts of appeals, legal treatises, and even popular culture.” That last reference to popular culture contained an interesting footnote citing none other than Walter Sobchak, a character in ['The Big Lebowski'].

    (tags: lebowski movies coen-brothers prior-restraint law supreme-court walter-sobchak funny)

  • on using JSON as a config file format

    Ben Hughes on twitter: “JSON is fine for config files, if you don’t want to comment your config file. Which is a way of saying, it isn’t fine for config files.”

    (tags: ben-hughes funny json file-formats config-files configuration software coding)

  • Understanding weak isolation is a serious problem

    Peter Bailis complaining about the horrors of modern transactional databases and their unserializability, which noone seems to be paying attention to: ‘As you’re probably aware, there’s an ongoing and often lively debate between transactional adherents and more recent “NoSQL” upstarts about related issues of usability, data corruption, and performance. But, in contrast, many of these transactional inherents and the research community as a whole have effectively ignored weak isolation — even in a single server setting and despite the fact that literally millions of businesses today depend on weak isolation and that many of these isolation levels have been around for almost three decades.’ ‘Despite the ubiquity of weak isolation, I haven’t found a database architect, researcher, or user who’s been able to offer an explanation of when, and, probably more importantly, why isolation models such as Read Committed are sufficient for correct execution. It’s reasonably well known that these weak isolation models represent “ACID in practice,” but I don’t think we have any real understanding of how so many applications are seemingly (!?) okay running under them. (If you haven’t seen these models before, they’re a little weird. For example, Read Committed isolation generally prevents users from reading uncommitted or non-final writes but allows a number of bad things to happen, like lost updates during concurrent read-modify-write operations. Why is this apparently okay for many applications?)’

    (tags: acid consistency databases peter-bailis transactional corruption serializability isolation reliability)

  • “Left-Right: A Concurrency Control Technique with Wait-Free Population Oblivious Reads” [pdf]

    ‘In this paper, we describe a generic concurrency control technique with Blocking write operations and Wait-Free Population Oblivious read operations, which we named the Left-Right technique. It is of particular interest for real-time applications with dedicated Reader threads, due to its wait-free property that gives strong latency guarantees and, in addition, there is no need for automatic Garbage Collection. The Left-Right pattern can be applied to any data structure, allowing concurrent access to it similarly to a Reader-Writer lock, but in a non-blocking manner for reads. We present several variations of the Left-Right technique, with different versioning mechanisms and state machines. In addition, we constructed an optimistic approach that can reduce synchronization for reads.’ See also http://concurrencyfreaks.blogspot.ie/2013/12/left-right-concurrency-control.html for java implementation code.

    (tags: left-right concurrency multithreading wait-free blocking realtime gc latency reader-writer locking synchronization java)

  • Russell91/sshrc

    ‘bring your .bashrc, .vimrc, etc. with you when you ssh’. A really nice implementation of this idea (much nicer than my own version!)

    (tags: hacks productivity ssh remote shell sh bash via:johnke home-directory unix)

  • Troubleshooting Production JVMs with jcmd

    remotely trigger GCs, finalization, heap dumps etc. Handy

    (tags: jvm jcmd debugging ops java gc heap troubleshooting)

Posted in Uncategorized | Comments closed

Links for 2014-09-16

Posted in Uncategorized | Comments closed

Links for 2014-09-15

  • The State of ZFS on Linux

    Linux users familiar with other filesystems or ZFS users from other platforms will often ask whether ZFS on Linux (ZoL) is “stable”. The short answer is yes, depending on your definition of stable. The term stable itself is somewhat ambiguous.
    Oh dear. that’s not a good start. Good reference page, though

    (tags: zfs linux filesystems ops solaris)

  • Screen time: Steve Jobs was a low tech parent

    “This is rule No. 1: There are no screens in the bedroom. Period. Ever.”

    (tags: screen-time kids children tv mobile technology life rules parenting)

  • CausalImpact: A new open-source package for estimating causal effects in time series

    How can we measure the number of additional clicks or sales that an AdWords campaign generated? How can we estimate the impact of a new feature on app downloads? How do we compare the effectiveness of publicity across countries? In principle, all of these questions can be answered through causal inference. In practice, estimating a causal effect accurately is hard, especially when a randomised experiment is not available. One approach we’ve been developing at Google is based on Bayesian structural time-series models. We use these models to construct a synthetic control — what would have happened to our outcome metric in the absence of the intervention. This approach makes it possible to estimate the causal effect that can be attributed to the intervention, as well as its evolution over time. We’ve been testing and applying structural time-series models for some time at Google. For example, we’ve used them to better understand the effectiveness of advertising campaigns and work out their return on investment. We’ve also applied the models to settings where a randomised experiment was available, to check how similar our effect estimates would have been without an experimental control. Today, we’re excited to announce the release of CausalImpact, an open-source R package that makes causal analyses simple and fast. With its release, all of our advertisers and users will be able to use the same powerful methods for estimating causal effects that we’ve been using ourselves. Our main motivation behind creating the package has been to find a better way of measuring the impact of ad campaigns on outcomes. However, the CausalImpact package could be used for many other applications involving causal inference. Examples include problems found in economics, epidemiology, or the political and social sciences.

    (tags: causal-inference r google time-series models bayes adwords advertising statistics estimation metrics)

  • Top 10 Historic Sites in Ireland and Northern Ireland — National Geographic

    Shamefully, I haven’t visited most of these!

    (tags: history neolithic ireland northern-ireland national-geographic tourism places)

  • Software patents are crumbling, thanks to the Supreme Court

    Now a series of decisions from lower courts is starting to bring the ruling’s practical consequences into focus. And the results have been ugly for fans of software patents. By my count there have been 11 court rulings on the patentability of software since the Supreme Court’s decision — including six that were decided this month.  Every single one of them has led to the patent being invalidated. This doesn’t necessarily mean that all software patents are in danger — these are mostly patents that are particularly vulnerable to challenge under the new Alice precedent. But it does mean that the pendulum of patent law is now clearly swinging in an anti-patent direction. Every time a patent gets invalidated, it strengthens the bargaining position of every defendant facing a lawsuit from a patent troll.

    (tags: patents law alice swpats software supreme-court patent-trolls)

  • Riding with the Stars: Passenger Privacy in the NYC Taxicab Dataset

    A practical demo of “differential privacy” — allowing public data dumps to happen without leaking privacy, using Laplace noise addition

    (tags: differential-privacy privacy leaks public-data open-data data nyc taxis laplace noise randomness)

Posted in Uncategorized | Comments closed

Links for 2014-09-14

  • Platform Game

    I’m ambivalent about Microsoft acquiring Mojang. Will they Embrace and Extend Minecraft as they’ve done with other categories? Let’s hope not. On the other hand, some adult supervision and a Plugin API would be welcome. Mojang have the financial resources but lack the will and focus needed to publish and support a Plugin API. Perhaps Mojang themselves don’t realise just how important their little game has become.

    (tags: minecraft platforms games plugins mojang microsoft)

Posted in Uncategorized | Comments closed

Links for 2014-09-11

Posted in Uncategorized | Comments closed

Links for 2014-09-10

  • Apple: Untrustable

    Today, Apple announced their “Most Personal Device Ever”. They also announced Apple Pay (the only mentions of “security” and “privacy” in today’s event), and are rolling out health tracking and home automation in iOS 8. Given their feckless track record [with cloud-service security], would you really trust Apple with (even more of) your digital life?

    (tags: icloud apple fail security hacks privacy)

  • Not Safe For Not Working On

    Excellent post from Dan Kaminsky on concrete actions that cloud service providers like Apple and Google need to start taking.

    *It’s time to ban Password1*: [...] Defenders are using simple rules like “doesn’t have an uppercase letter” and “not enough punctuation” to block passwords while attackers are just straight up analyzing password dumps and figuring out the most likely passwords to attempt in any scenario.  Attackers are just way ahead.  That has to change.  Defenders have password dumps too now.  It’s time we start outright blocking passwords common enough that they can be online brute forced, and it’s time we admit we know what they are. [...] *People use communication technologies for sexy times. Deal with it*: Just like browsers have porn mode for the personal consumption of private imagery, cell phones have applications that are significantly less likely to lead to anyone else but your special friends seeing your special bits. I personally advise Wickr, an instant messaging firm that develops secure software for iPhone and Android. What’s important about Wickr here isn’t just the deep crypto they’ve implemented, though it’s useful too. What’s important in this context is that with this code there’s just a lot fewer places to steal your data from. Photos and other content sent in Wickr don’t get backed up to your desktop, don’t get saved in any cloud, and by default get removed from your friend’s phone after an amount of time you control. Wickr is of course not the only company supporting what’s called “ephemeral messaging”; SnapChat also dramatically reduces the exposure of your private imagery. [...]
    via Leonard.

    (tags: icloud apple privacy security via:lhl snapchat wickr dan-kaminsky cloud-services backup)

  • Inside Apple’s Live Event Stream Failure, And Why It Happened: It Wasn’t A Capacity Issue

    The bottom line with this event is that the encoding, translation, JavaScript code, the video player, the call to S3 single storage location and the millisecond refreshes all didn’t work properly together and was the root cause of Apple’s failed attempt to make the live stream work without any problems. So while it would be easy to say it was a CDN capacity issue, which was my initial thought considering how many events are taking place today and this week, it does not appear that a lack of capacity played any part in the event not working properly. Apple simply didn’t provision and plan for the event properly.

    (tags: cdn streaming apple fail scaling s3 akamai caching)

Posted in Uncategorized | Comments closed

Links for 2014-09-09

Posted in Uncategorized | Comments closed

Links for 2014-09-08

Posted in Uncategorized | Comments closed

Links for 2014-09-06

  • ‘The very first release of Gmail simply used spamassassin on the backend’

    Excellent. Confirming what I’d heard from a few other sources, too ;) This is a well-written history of the anti-spam war so far, from Mike Hearn, writing with the Google/Gmail point of view:

    Brief note about my background, to establish credentials: I worked at Google for about 7.5 years. For about 4.5 of those I worked on the Gmail abuse team, which is very tightly linked with the spam team (they use the same software, share the same on-call rotations etc).
    Reading this kind of stuff is awesome for me, since it’s a nice picture of a fun problem to work on — the Gmail team took the right ideas about how to fight spam, and scaled them up to the 10s-of-millions DAU mark. Nicely done. The second half is some interesting musings on end-to-end encrypted communications and how it would deal with spam. Worth a read…

    (tags: gmail google spam anti-spam filtering spamassassin history)

  • The FBI Finally Says How It ‘Legally’ Pinpointed Silk Road’s Server

    The answer, according to a new filing by the case’s prosecution, is far more mundane: The FBI claims to have found the server’s location without the NSA’s help, simply by fiddling with the Silk Road’s login page until it leaked its true location.

    (tags: fbi nsa silk-road tor opsec dread-pirate-roberts wired)

Posted in Uncategorized | Comments closed

Links for 2014-09-05

Posted in Uncategorized | Comments closed

Links for 2014-09-04

  • Visualizing Garbage Collection Algorithms

    Great dataviz with animated GIFs

    (tags: algorithms gc memory visualization garbage-collection dataviz refcounting mark-and-sweep)

  • Standard Markdown

    John Gruber’s canonical description of Markdown’s syntax does not specify the syntax unambiguously. In the absence of a spec, early implementers consulted the original Markdown.pl code to resolve these ambiguities. But Markdown.pl was quite buggy, and gave manifestly bad results in many cases, so it was not a satisfactory replacement for a spec. Because there is no unambiguous spec, implementations have diverged considerably. As a result, users are often surprised to find that a document that renders one way on one system (say, a GitHub wiki) renders differently on another (say, converting to docbook using Pandoc). To make matters worse, because nothing in Markdown counts as a “syntax error,” the divergence often isn’t discovered right away. There’s no standard test suite for Markdown; the unofficial MDTest is the closest thing we have. The only way to resolve Markdown ambiguities and inconsistencies is Babelmark, which compares the output of 20+ implementations of Markdown against each other to see if a consensus emerges. We propose a standard, unambiguous syntax specification for Markdown, along with a suite of comprehensive tests to validate Markdown implementations against this specification. We believe this is necessary, even essential, for the future of Markdown.

    (tags: writing markdown specs standards text formats html)

  • Postcodes at last but random numbers don’t address efficiency

    Karlin Lillington assembles a fine collection of quotes from various sources panning the new Eircode system:

    Critics say the opportunity has been missed to use Ireland’s clean-slate status to produce a technologically innovative postcode system that would be at the cutting edge globally; similar to the competitive leap that was provided when the State switched to a digital phone network in the 1980s, well ahead of most of the world. Instead, say organisations such as the Freight Transport Association of Ireland (FTAI), the proposed seven-digit format of scrambled letters and numbers is almost useless for a business sector that should most benefit from a proper postcode system: transport and delivery companies, from international giants like FedEx and UPS down to local courier, delivery and service supplier firms. Because each postcode will reveal the exact address of a home or business, privacy advocates are concerned that online use of postcodes could link many types of internet activity, including potentially sensitive online searches, to a specific household or business.

    (tags: eircode government fail ireland postcodes location ftai random)

Posted in Uncategorized | Comments closed

Links for 2014-09-03

Posted in Uncategorized | Comments closed

Links for 2014-09-02

  • Nix: The Purely Functional Package Manager

    ‘a powerful package manager for Linux and other Unix systems that makes package management reliable and reproducible. It provides atomic upgrades and rollbacks, side-by-side installation of multiple versions of a package, multi-user package management and easy setup of build environments. ‘ Basically, this is a third-party open source reimplementation of Amazon’s (excellent) internal packaging system, using symlinks to versioned package directories to ensure atomicity and the ability to roll back. This is definitely the *right* way to build packages — I know what tool I’ll be pushing for, next time this question comes up. See also nixos.org for a Linux distro built on Nix.

    (tags: ops linux devops unix packaging distros nix nixos atomic upgrades rollback versioning)

Posted in Uncategorized | Comments closed

Links for 2014-09-01

  • Facebook’s drop-in replacement for std::vector

    Fixes some low-hanging fruit, performance-wise. ‘Simply replacing std::vector with folly::fbvector (after having included the folly/FBVector.h header file) will improve the performance of your C++ code using vectors with common coding patterns. The improvements are always non-negative, almost always measurable, frequently significant, sometimes dramatic, and occasionally spectacular.’ (via Tony Finch)

    (tags: c++ facebook performance algorithms vectors via:fanf optimization)

  • Applying cardiac alarm management techniques to your on-call

    An ops-focused take on a recent story about alarm fatigue, and how a Boston hospital dealt with it. When I was in Amazon, many of the teams in our division had a target to reduce false positive pages, with a definite monetary value attached to it, since many teams had “time off in lieu” payments for out-of-hours pages to the on-call staff. As a result, reducing false-positive pages was reasonably high priority and we dealt with this problem very proactively, with a well-developed sense of how to do so. It’s interesting to see how the outside world is only just starting to look into its amelioration. (Another benefit of a TOIL policy ;)

    (tags: ops monitoring sysadmin alerts alarms nagios alarm-fatigue false-positives pages)

  • “Invertible Bloom Lookup Tables” [paper]

    ‘We present a version of the Bloom filter data structure that supports not only the insertion, deletion, and lookup of key-value pairs, but also allows a complete listing of the pairs it contains with high probability, as long the number of key- value pairs is below a designed threshold. Our structure allows the number of key-value pairs to greatly exceed this threshold during normal operation. Exceeding the threshold simply temporarily prevents content listing and reduces the probability of a successful lookup. If entries are later deleted to return the structure below the threshold, everything again functions appropriately. We also show that simple variations of our structure are robust to certain standard errors, such as the deletion of a key without a corresponding insertion or the insertion of two distinct values for a key. The properties of our structure make it suitable for several applications, including database and networking applications that we highlight.’

    (tags: iblt bloom-filters data-structures performance algorithms coding papers probabilistic)

  • Some UX Dark Patterns now illegal in the EU

    The EU’s new consumer rights law bans certain dark patterns related to e-commerce across Europe. The “sneak into basket” pattern is now illegal. Full stop, end of story. You cannot create a situation where additional items and services are added by default. [...] Hidden costs are now illegal, whether that’s an undeclared subscription, extra shipping charges, or extra items. [....] Forced continuity, when imposed on the user as a form of bait-and-switch, has been banned. Just the other day a web designer mentioned to me that he had only just discovered he had been charged for four years of annual membership dues in a “theme club”, having bought what he thought was a one-off theme. Since he lives in Europe, he may be able to claim all of this money back. All he needs to do is prove that the website did not inform him that the purchase included a membership with recurring payments.

    (tags: design europe law ecommerce ux dark-patterns scams ryanair selling online consumer consumer-rights bait-and-switch)

  • Girl Not Against Fluoride

    The CDC (Centre for Disease Control) lists water fluoridation as one of the ten great public health achievements of the 20th Century. Today, Dublin City Council will vote on whether to remove fluoride from our water supply, and when they do, it will not be because the CDC or the WHO have changed their mind about fluoridation, or because new and compelling information makes it the only choice. It will be because people who believe in angel healing, homeopathy, and chemtrails, have somehow gained the ability to influence public policy.

    (tags: dcc dublin law flouride science zenbuffy homeopathy woo health teeth)

  • Revisiting How We Put Together Linux Systems

    Building a running OS out of layered btrfs filesystems. This sounds awesome.

    Instantiating a new system or OS container (which is exactly the same in this scheme) just consists of creating a new appropriately named root sub-volume. Completely naturally you can share one vendor OS copy in one specific version with a multitude of container instances. Everything is double-buffered (or actually, n-fold-buffered), because usr, runtime, framework, app sub-volumes can exist in multiple versions. Of course, by default the execution logic should always pick the newest release of each sub-volume, but it is up to the user keep multiple versions around, and possibly execute older versions, if he desires to do so. In fact, like on ChromeOS this could even be handled automatically: if a system fails to boot with a newer snapshot, the boot loader can automatically revert back to an older version of the OS.
    (via Tony Finch)

    (tags: via:fanf linux docker btrfs filesystems unionfs copy-on-write os hacking unix)

Posted in Uncategorized | Comments closed

Links for 2014-08-29

Posted in Uncategorized | Comments closed

Links for 2014-08-28

Posted in Uncategorized | Comments closed

Links for 2014-08-27

  • Apache Kafka 0.8 basic training

    This is a pretty voluminous and authoritative presentation about getting started with Kafka; wish this was around when we started using it for 0.7. (We use our own homegrown realtime system nowadays, due to better partitioning, monitoring and operability.)

    (tags: storm kafka presentations documentation ops)

  • Wiki Loves Monuments

    Wiki Loves Monuments is an international photo contest, organised by Wikimedia [...]. This year, the Wikimedia Ireland Community are running the competition for the very first time in Ireland. The contest is inspired by the successful 2010 pilot in the Netherlands which resulted in 12,500 freely licensed images uploaded to Wikimedia Commons. It has grown substantially since its inception; in 2013 369,589 photographs were submitted by 11,943 participants from over 50 countries. Cultural heritage is an important part of the knowledge that Wikipedia collects and disseminates. An image is worth a thousand words, in any language and local enthusiasts can (re)discover the cultural, historical, or scientific significance of their neighbourhood. The Irish contest, focussing on Ireland’s national monuments, runs from August 23 – September 30. Follow our step-by-step guide to find out how you can take part.

    (tags: wikipedia wikimedia images monuments history ireland contests creative-commons licensing)

  • “CryptoPhone” claims to detect IMSI catchers in operation

    To show what the CryptoPhone can do that less expensive competitors cannot, he points me to a map that he and his customers have created, indicating 17 different phony cell towers known as “interceptors,” detected by the CryptoPhone 500 around the United States during the month of July alone.  Interceptors look to a typical phone like an ordinary tower.  Once the phone connects with the interceptor, a variety of “over-the-air” attacks become possible, from eavesdropping on calls and texts to pushing spyware to the device. “Interceptor use in the U.S. is much higher than people had anticipated,” Goldsmith says.  “One of our customers took a road trip from Florida to North Carolina and he found 8 different interceptors on that trip.  We even found one at South Point Casino in Las Vegas.”

    (tags: imsi-catchers security cryptophone phones mobile 3g 4g eavesdropping surveillance)

Posted in Uncategorized | Comments closed

Links for 2014-08-26

Posted in Uncategorized | Comments closed

Links for 2014-08-25

Posted in Uncategorized | Comments closed

Links for 2014-08-22

Posted in Uncategorized | Comments closed

Links for 2014-08-21

Posted in Uncategorized | Comments closed

Links for 2014-08-20

Posted in Uncategorized | Comments closed

Links for 2014-08-19

  • Nyms Identity Directory

    The way that [problems with the PGP bootstrapping] are supposed to be resolved is with an authentication model called the Web of Trust where users sign keys of other users after verifying that they are who they say they are. In theory, if some due diligence is applied in signing other people’s keys and a sufficient number of people participate you’ll be able to follow a short chain of signatures from people you already know and trust to new untrusted keys you download from a key server. In practice this has never worked out very well as it burdens users with the task of manually finding people to sign their keys and even experts find the Web of Trust model difficult to reason about. This also reveals the social graph of certain communities which may place users at risk for their associations. Such signatures also reveal metadata about times and thus places for meetings for key signings. The Nyms Identity Directory is a replacement for all of this. Keyservers are replaced with an identity directory that gives users full control over publication of their key information and web of trust is replaced with a distributed network of trusted notaries which validate user keys with an email verification protocol.

    (tags: web-of-trust directories nyms privacy crypto identity trust pgp gpg security via:ioerror keyservers notaries)

  • Frogsort

    Frogsort as an exam question (via qwghlm)

    (tags: via:qwghlm frogsort sorting big-o algorithms funny comics smbc)

  • Punished for Being Poor: Big Data in the Justice System

    This is awful. Totally the wrong tool for the job — a false positive rate which is miniscule for something like spam filtering, could translate to a really horrible outcome for a human life.

    Currently, over 20 states use data-crunching risk-assessment programs for sentencing decisions, usually consisting of proprietary software whose exact methods are unknown, to determine which individuals are most likely to re-offend. The Senate and House are also considering similar tools for federal sentencing. These data programs look at a variety of factors, many of them relatively static, like criminal and employment history, age, gender, education, finances, family background, and residence. Indiana, for example, uses the LSI-R, the legality of which was upheld by the state’s supreme court in 2010. Other states use a model called COMPAS, which uses many of the same variables as LSI-R and even includes high school grades. Others are currently considering the practice as a way to reduce the number of inmates and ensure public safety. (Many more states use or endorse similar assessments when sentencing sex offenders, and the programs have been used in parole hearings for years.) Even the American Law Institute has embraced the practice, adding it to the Model Penal Code, attesting to the tool’s legitimacy.
    (via stroan)

    (tags: via:stroan statistics false-positives big-data law law-enforcement penal-code risk sentencing)

Posted in Uncategorized | Comments closed

Links for 2014-08-18

  • Microservices – Not a free lunch! – High Scalability

    Some good reasons not to adopt microservices blindly. Testability and distributed-systems complexity are my biggest fears

    (tags: microservices soa devops architecture testing distcomp)

  • Richard Clayton – Failing at Microservices

    Solid warts-and-all confessional blogpost about a team failing to implement a microservices architecture. I’d put most of the blame on insufficient infrastructure to support them (at a code level), inter-personal team problems, and inexperience with large-scale complex multi-service production deployment and the work it was going to require

    (tags: microservices devops collaboration architecture fail team deployment soa)

  • Box Tech Blog » A Tale of Postmortems

    How Box introduced COE-style dev/ops outage postmortems, and got them working. This PIE metric sounds really useful to head off the dreaded “it’ll all have to come out missus” action item:

    The picture was getting clearer, and we decided to look into individual postmortems and action items and see what was missing. As it was, action items were wasting away with no owners. Digging deeper, we noticed that many action items entailed massive refactorings or vague requirements like “make system X better” (i.e. tasks that realistically were unlikely to be addressed). At a higher level, postmortem discussions often devolved into theoretical debates without a clear outcome. We needed a way to lower and focus the postmortem bar and a better way to categorize our action items and our technical debt. Out of this need, PIE (“Probability of recurrence * Impact of recurrence * Ease of addressing”) was born. By ranking each factor from 1 (“low”) to 5 (“high”), PIE provided us with two critical improvements: 1. A way to police our postmortems discussions. I.e. a low probability, low impact, hard to implement solution was unlikely to get prioritized and was better suited to a discussion outside the context of the postmortem. Using this ranking helped deflect almost all theoretical discussions. 2. A straightforward way to prioritize our action items. What’s better is that once we embraced PIE, we also applied it to existing tech debt work. This was critical because we could now prioritize postmortem action items alongside existing work. Postmortem action items became part of normal operations just like any other high-priority work.

    (tags: postmortems action-items outages ops devops pie metrics ranking refactoring prioritisation tech-debt)

  • NTP’s days are numbered for consumer devices

    An accurate clock is required to negotiate SSL/TLS, so clock sync is important for internet-of-things usage. but:

    Unfortunately for us, the traditional and most widespread method for clock synchronisation (NTP) has been caught up in a DDoS issue which has recently caused some ISPs to start blocking all NTP communication. [....] Because the DDoS attacks are so widespread, and the lack of obvious commercial pressure to fix the issue, it’s possible that the days of using NTP as a mechanism for setting clocks may well be numbered. Luckily for us there is a small but growing project that replaces it. tlsdate was started by Jacob Appelbaum of the Tor project in 2012, making use of the SSL handshake in order to extract time from a remote server, and its usage is on the rise. [....] Since we started encountering these problems, we’ve incorporated tlsdate into an over-the-air update, and have successfully started using this in situations where NTP is blocked.

    (tags: tlsdate ntp clocks time sync iot via:gwire ddos isps internet protocols security)

  • Cloudwash – Creating the Technical Prototype

    This is a lovely demo of integrating modern IoT connectivity functionality (remote app control, etc.) with a washing machine using Bergcloud’s hardware and backend, and a little logic-analyzer reverse engineering.

    (tags: arduino diy washing-machines iot bergcloud hacking reversing logic-analyzers hardware)

  • Systemd: Harbinger of the Linux apocalypse

    While there are many defensible aspects of Systemd, other aspects boggle the mind. Not the least of these was that, as of a few months ago, trying to debug the kernel from the boot line would cause the system to crash. This was because of Systemd’s voracious logging and the fact that Systemd responds to the “debug” flag on the kernel boot line — a flag meant for the kernel, not anything else. That, straight up, is a bug. However, the Systemd developers didn’t see it that way and actively fought with those experiencing the problem. Add the fact that one of the Systemd developers was banned by Linus Torvalds for poor attitude and bad design and another was responsible for causing significant issues with Linux audio support, but blamed the problem on everything else but his software, and you have a bad situation on your hands. There’s no shortage of egos in the open source development world. There’s no shortage of new ideas and veteran developers and administrators pooh-poohing something new simply because it’s new. But there are also 45 years of history behind Unix and extremely good reasons it’s still flourishing. Tools designed like Systemd do not fit the Linux mold, to their own detriment. Systemd’s design has more in common with Windows than with Unix — down to the binary logging.
    The link re systemd consuming the “debug” kernel boot arg is a canonical example of inflexible coders refusing to fix their own bugs. (via Jason Dixon)

    (tags: systemd linux red-hat egos linus-torvalds unix init booting debugging logging design software via:obfuscurity)

  • Inside a Chinese Bitcoin Mine

    The mining operation resides on an old, repurposed factory floor, and contains 2500 machines hashing away at 230 Gh/s, each. (That’s 230 billion calculations per second, per unit). [...] The operators told me that the power bill of this specific operation is in excess of ¥400,000 per month [..] about $60,000 USD.

    (tags: currency china economics bitcoin power environment green mining datacenters)

  • Moving Big Data into the Cloud with Tsunami UDP – AWS Big Data Blog

    Pretty serious speedup. 81 MB/sec with Tsunami UDP, compared to 9 MB/sec with plain old scp. Probably kills internet performance for everyone else though!

    (tags: tsunami-udp udp scp copying transfers internet long-distance performance speed)

  • The “sidecar” pattern

    Ha, great name. We use this (in the form of Smartstack).

    For what it is worth, we faced a similar challenge in earlier services (mostly due to existing C/C++ applications) and we created what was called a “sidecar”.  By sidecar, what I mean is a second process on each node/instance that did Cloud Service Fabric operations on behalf of the main process (the side-managed process).  Unfortunately those sidecars all went off and created one-offs for their particular service.  In this post, I’ll describe a more general sidecar that doesn’t force users to have these one-offs. Sidenote:  For those not familiar with sidecars, think of the motorcycle sidecar below.  Snoopy would be the main process with Woodstock being the sidecar process.  The main work on the instance would be the motorcycle (say serving your users’ REST requests).  The operational control is the sidecar (say serving health checks and management plane requests of the operational platform).

    (tags: netflix sidecars architecture patterns smartstack netflixoss microservices soa)

Posted in Uncategorized | Comments closed

Links for 2014-08-15

Posted in Uncategorized | Comments closed

Links for 2014-08-14

Posted in Uncategorized | Comments closed

Links for 2014-08-13

Posted in Uncategorized | Comments closed

Links for 2014-08-12

Posted in Uncategorized | Comments closed