Links for 2015-05-26

  • Schedule Recurring AWS Lambda Invocations With The Unreliable Town Clock (UTC)

    The Unreliable Town Clock (UTC) is a new, free, public SNS Topic (Amazon Simple Notification Service) that broadcasts a “chime” message every quarter hour to all subscribers. It can send the chimes to AWS Lambda functions, SQS queues, and email addresses. You can use the chime attributes to run your code every fifteen minutes, or only run your code once an hour (e.g., when minute == “00″) or once a day (e.g., when hour == “00″ and minute == “00″) or any other series of intervals. You can even subscribe a function you only want to run only once at a specific time in the future: Have the function ignore all invocations until it’s after the time it wants. When it is time, it can perform its job, then unsubscribe itself from the SNS Topic.

    (tags: alestic aws lambda cron time clock periodic-tasks recurrence hacks)

Posted in Uncategorized | Comments closed

Links for 2015-05-25

Posted in Uncategorized | Comments closed

Links for 2015-05-24

Posted in Uncategorized | Comments closed

Links for 2015-05-19

Posted in Uncategorized | Comments closed

Links for 2015-05-18

Posted in Uncategorized | Comments closed

Links for 2015-05-17

  • ‘Can People Distinguish Pâté from Dog Food?’


    Considering the similarity of its ingredients, canned dog food could be a suitable and inexpensive substitute for pâté or processed blended meat products such as Spam or liverwurst. However, the social stigma associated with the human consumption of pet food makes an unbiased comparison challenging. To prevent bias, Newman’s Own dog food was prepared with a food processor to have the texture and appearance of a liver mousse. In a double-blind test, subjects were presented with five unlabeled blended meat products, one of which was the prepared dog food. After ranking the samples on the basis of taste, subjects were challenged to identify which of the five was dog food. Although 72% of subjects ranked the dog food as the worst of the five samples in terms of taste (Newell and MacFarlane multiple comparison, P<0.05), subjects were not better than random at correctly identifying the dog food.

    (tags: pate food omgwtf science research dog-food meat economics taste flavour)

  • Redditor runs the secret Python code in Ex Machina

    and finds:

    when you run with python2.7 you get the following: ISBN = 9780199226559 Which is Embodiment and the inner life: Cognition and Consciousness in the Space of Possible Minds. and so now I have a lot more respect for the Director.

    (tags: python movies ex-machina cool books easter-eggs)

  • Metalwoman beer recipe

    via the Dublin Ladies Beer Society ;)

    (tags: metalman metalwoman recipes beer brewing hops dlbs)

Posted in Uncategorized | Comments closed

Links for 2015-05-15

Posted in Uncategorized | Comments closed

Links for 2015-05-14

Posted in Uncategorized | Comments closed

Links for 2015-05-12

  • Input: Fonts for Code

    Non-monospaced coding fonts! I’m all in favour…

    As writing and managing code becomes more complex, today’s sophisticated coding environments are evolving to include everything from breakpoint markers to code folding and syntax highlighting. The typography of code should evolve as well, to explore possibilities beyond one font style, one size, and one character width.

    (tags: input fonts via:its typography code coding font text ide monospace)

  • Apache HTrace

    a Zipkin-compatible distributed-system tracing framework in Java, in the Apache Incubator

    (tags: zipkin tracing trace apache incubator java debugging)

  • Intel speeds up etcd throughput using ADR Xeon-only hardware feature

    To reduce the latency impact of storing to disk, Weaver’s team looked to buffering as a means to absorb the writes and sync them to disk periodically, rather than for each entry. Tradeoffs? They knew memory buffers would help, but there would be potential difficulties with smaller clusters if they violated the stable storage requirement. Instead, they turned to Intel’s silicon architects about features available in the Xeon line. After describing the core problem, they found out this had been solved in other areas with ADR. After some work to prove out a Linux OS supported use for this, they were confident they had a best-of-both-worlds angle. And it worked. As Weaver detailed in his CoreOS Fest discussion, the response time proved stable. ADR can grab a section of memory, persist it to disk and power it back. It can return entries back to disk and restore back to the buffer. ADR provides the ability to make small (<100MB) segments of memory “stable” enough for Raft log entries. It means it does not need battery-backed memory. It can be orchestrated using Linux or Windows OS libraries. ADR allows the capability to define target memory and determine where to recover. It can also be exposed directly into libs for runtimes like Golang. And it uses silicon features that are accessible on current Intel servers.

    (tags: kubernetes coreos adr performance intel raft etcd hardware linux persistence disk storage xeon)

Posted in Uncategorized | Comments closed

Links for 2015-05-11

  • streamtools: a graphical tool for working with streams of data | nytlabs

    Visual programming, Yahoo! Pipes style, back again:

    we have created streamtools – a new, open source project by The New York Times R&D Lab which provides a general purpose, graphical tool for dealing with streams of data. It provides a vocabulary of operations that can be connected together to create live data processing systems without the need for programming or complicated infrastructure. These systems are assembled using a visual interface that affords both immediate understanding and live manipulation of the system.
    via Aman

    (tags: via:akohli streaming data nytimes visual-programming coding)

  • MappedBus

    a Java based low latency, high throughput message bus, built on top of a memory mapped file; inspired by Java Chronicle with the main difference that it’s designed to efficiently support multiple writers – enabling use cases where the order of messages produced by multiple processes are important. MappedBus can be also described as an efficient IPC mechanism which enable several Java programs to communicate by exchanging messages.

    (tags: ipc java jvm mappedbus low-latency mmap message-bus data-structures queue message-passing)

Posted in Uncategorized | Comments closed

Links for 2015-05-10

  • Amazon’s Drone Delivery Patent Just Feels Like Trolling At This Point

    Oh dear, Amazon.

    These aren’t actual technologies yet. [...] All of which underscores that Amazon might never ever ever ever actually implement delivery drones. The patent paperwork was filed nearly a year after Amazon’s splashy drone program reveal on 60 Minutes. At the time we called it revolutionary marketing because, you know, delivery drones are technical and logistical madness, not to mention that commercial drone use is illegal right now. Although, in fairness the FAA did just relax some rules so that Amazon could test drones. At this point it feels like Amazon is just trolling. It’s trolling us with public relations BS about its future drones, and it’s trolling future competitors — Google is also apparently working on this — so that if somebody ever somehow does anything relating to drone delivery, Amazon can sue them. If I’m wrong, I’ll deliver my apology via Airmail.

    (tags: amazon trolling patents uspto delivery drones uavs competition faa)

  • Red Hat on rkt vs Docker

    This is like watching a train-wreck in slow motion on Groundhog Day. We, in the broader Linux and open source community, have been down this path multiple times over the past fifteen years, specifically with package formats. While there needs to be room for experimentation, having two incompatible specs driven by two startups trying to differentiate and in direct competition is *not* a good thing. It would be better for the community and for everyone who depends on our collective efforts if CoreOS and Docker collaborated on a standardized common spec, image format, and distribution protocol. To this end, we at Red Hat will continue to contribute to both initiatives with the goal of driving convergence.

    (tags: rkt docker appc coreos red-hat dpkg rpm linux packaging collaboration open-source)

Posted in Uncategorized | Comments closed

Links for 2015-05-09

Posted in Uncategorized | Comments closed

Links for 2015-05-08

Posted in Uncategorized | Comments closed

Links for 2015-05-07

  • KillBiller

    Excellent mobile-phone plan comparison site for the Irish market, using apps which you install and which analyse your call history, data usage, etc. over the past month to compute the optimal plan based on your usage. Pretty amazing results in my case! The only downside is the privacy policy, which allows the company to resell your usage data (anonymised, and in aggregate) — I’d really prefer if this wasn’t the case :(

    (tags: mobile-phones shopping tesco emobile 3g 4g ireland plans comparison-shopping killbiller via:its)

  • Family in No poster Says YES to Marriage Equality | Amnesty International

    Beyond the politics, the risks of stock photo usage are pretty evident too:

    “In 2014, as a young family, we did a photo shoot with a photographer friend to get some nice shots for the family album. No money was exchanged – we got nice photos for free, they got nice images for their portfolio. As part of this agreement, we agreed to let them upload them to a stock photo album. We knew that these were available for purchase and we gave permission. Perhaps, naïvely, we imagined that on the off chance that any was ever selected, it might be for a small magazine or website. To confirm, we have not received any money for the photo – then or now, and nor do we expect any. We were surprised and upset to see that the photo was being used as part of a campaign with which we do not agree. We completely support same-sex marriage, and we believe that same-sex couples’ should of course be able to adopt, as we believe that they are equally able to provide children with much-needed love and care. To suggest otherwise is offensive to us, and to many others.”

    (tags: ssm ireland politics amnesty stock-photos ip rights photos campaigns ads)

  • Lambda: Bees with Frickin’ Laser Beams

    a HTTP testing tool in AWS Lambda. nice enough, but still a toy…

    (tags: lambda aws node javascript hacks http load-testing)

Posted in Uncategorized | Comments closed

Links for 2015-05-06

Posted in Uncategorized | Comments closed

Links for 2015-05-05

  • Smarter testing Java code with Spock Framework

    hmm, looks quite nice as a potential next-gen JUnit replacement for unit tests

    (tags: java testing bdd tests junit unit-tests spock via:trishagee)

  • Tots To Travel

    ‘Baby Friendly Holidays | Child, Toddler & Family Villas | France | Spain | Portugal | Italy’. Joe swears by it, will give it a go next year

    (tags: holidays vacation travel europe kids children via:joe)

  • How the NSA Converts Spoken Words Into Searchable Text – The Intercept

    This hits the nail on the head, IMO:

    To Phillip Rogaway, a professor of computer science at the University of California, Davis, keyword-search is probably the “least of our problems.” In an email to The Intercept, Rogaway warned that “When the NSA identifies someone as ‘interesting’ based on contemporary NLP methods, it might be that there is no human-understandable explanation as to why beyond: ‘his corpus of discourse resembles those of others whom we thought interesting’; or the conceptual opposite: ‘his discourse looks or sounds different from most people’s.’ If the algorithms NSA computers use to identify threats are too complex for humans to understand, it will be impossible to understand the contours of the surveillance apparatus by which one is judged.  All that people will be able to do is to try your best to behave just like everyone else.”

    (tags: privacy security gchq nsa surveillance machine-learning liberty future speech nlp pattern-analysis cs)

  • awslabs/aws-lambda-redshift-loader

    Load data into Redshift from S3 buckets using a pre-canned Lambda function. Looks like it may be a good example of production-quality Lambda

    (tags: lambda aws ec2 redshift s3 loaders etl pipeline)

  • Call me maybe: Aerospike

    ‘Aerospike offers phenomenal latencies and throughput — but in terms of data safety, its strongest guarantees are similar to Cassandra or Riak in Last-Write-Wins mode. It may be a safe store for immutable data, but updates to a record can be silently discarded in the event of network disruption. Because Aerospike’s timeouts are so aggressive–on the order of milliseconds — even small network hiccups are sufficient to trigger data loss. If you are an Aerospike user, you should not expect “immediate”, “read-committed”, or “ACID consistency”; their marketing material quietly assumes you have a magical network, and I assure you this is not the case. It’s certainly not true in cloud environments, and even well-managed physical datacenters can experience horrible network failures.’

    (tags: aerospike outages cap testing jepsen aphyr databases storage reliability)

Posted in Uncategorized | Comments closed

Links for 2015-05-04

Posted in Uncategorized | Comments closed

Links for 2015-05-02

  • In the privacy of your own home

    I didn’t know about this:

    Last spring, as 41,000 runners made their way through the streets of Dublin in the city’s Women’s Mini Marathon, an unassuming redheaded man by the name of Candid Wueest stood on the sidelines with a scanner. He had built it in a couple of hours with $75 worth of parts, and he was using it to surreptitiously pick up data from activity trackers worn on the runners’ wrists. During the race, Wueest managed to collect personal info from 563 racers, including their names, addresses, and passwords, as well as the unique IDs of the devices they were carrying.

    (tags: dublin candid-wueest privacy data marathon running iot activity-trackers)

Posted in Uncategorized | Comments closed

Links for 2015-05-01

Posted in Uncategorized | Comments closed

Links for 2015-04-30

Posted in Uncategorized | Comments closed

Links for 2015-04-29

  • “certificate verification failed” errors due to crappy Verisign certs and overzealous curl policies

    Seth Vargo is correct. Its not the bit length of the key which is at issue, its the signature algorithm. The entire keychain for the key is signed with SHA1withRSA: At issue is that the root verisign key has been marked as weak because of SHA1 and taken out of the curl bundle which is widely popular, and this issue will continue to cause more and more issues going forwards as that bundle makes it way into shipping o/s distributions and aws certification verification breaks.
    ‘This is still happening and curl is now failing on my machine causing all sorts of fun issues (including breaking CocoaPods that are using S3 for storage).’ — @jmhodges This may be a contributory factor to the issue @nelson saw: Curl’s ca-certs bundle is also used by Node: and doubtless many other apps and packages. Here’s a mailing list thread discussing the issue: — looks like the curl team aren’t too bothered about it.

    (tags: curl s3 amazon aws ssl tls certs sha1 rsa key-length security cacerts)

  • Cassandra moving to using G1 as the default recommended GC implementation

    This is a big indicator that G1 is ready for primetime. CMS has long been the go-to GC for production usage, but requires careful, complex hand-tuning — if G1 is getting to a stage where it’s just a case of giving it enough RAM, that’d be great. Also, looks like it’ll be the JDK9 default:

    (tags: cassandra tuning ops g1gc cms gc java jvm production performance memory)

  • The Colossal Shop

    ThisIsColossal now have a shop! bookmarking for some lovely gifts

    (tags: art design shop colossal shopping christmas gifts)

Posted in Uncategorized | Comments closed

Links for 2015-04-28

Posted in Uncategorized | Comments closed

Links for 2015-04-27

Posted in Uncategorized | Comments closed

Links for 2015-04-26

  • StackShare

    ‘Discover and discuss the best dev tools and cloud infrastructure services’ — fun!

    (tags: stackshare architecture stack ops software ranking open-source)

  • OWASP KeyBox

    a web-based SSH console that centrally manages administrative access to systems. Web-based administration is combined with management and distribution of user’s public SSH keys. Key management and administration is based on profiles assigned to defined users. Administrators can login using two-factor authentication with FreeOTP or Google Authenticator . From there they can create and manage public SSH keys or connect to their assigned systems through a web-shell. Commands can be shared across shells to make patching easier and eliminate redundant command execution.

    (tags: keybox owasp security ssh tls ssl ops)

  • 32-bit overflow in BitGo js code caused an accidental 85 BTC transaction fee

    Yes, this is a fucking 32-bit integer overflow. Whatever software was used, it calculated the sum of all inputs using 32-bit variables, which overflow at about 20 BTC if signed or 40 BTC if not. The fee was supposed to be 0xC350 = 50,000 satoshis, but it turned out to be 0×2,0000,C350 = 8,589,984,592 satoshis. Captains of the industry. If they were captains of any other industry, like say for example automotive, we’d have people dying in car crashes between two stationary vehicles.

    (tags: bitcoin fail bitgo javascript bugs 32-bit overflow btc)

  • Eight Docker Development Patterns

    good Docker tips

    (tags: tips docker ops deployment)

  • Google Online Security Blog: A Javascript-based DDoS Attack [the Greatfire DDoS] as seen by Safe Browsing

    We hope this report helps to round out the overall facts known about this attack. It also demonstrates that collectively there is a lot of visibility into what happens on the web. At the HTTP level seen by Safe Browsing, we cannot confidently attribute this attack to anyone. However, it makes it clear that hiding such attacks from detailed analysis after the fact is difficult. Had the entire web already moved to encrypted traffic via TLS, such an injection attack would not have been possible. This provides further motivation for transitioning the web to encrypted and integrity-protected communication. Unfortunately, defending against such an attack is not easy for website operators. In this case, the attack Javascript requests web resources sequentially and slowing down responses might have helped with reducing the overall attack traffic. Another hope is that the external visibility of this attack will serve as a deterrent in the future.
    Via Nelson.

    (tags: google security via:nelson ddos javascript tls ssl safe-browsing networking china greatfire)

Posted in Uncategorized | Comments closed

Links for 2015-04-24

Posted in Uncategorized | Comments closed

Links for 2015-04-23

  • attacks using U+202E – RIGHT-TO-LEFT OVERRIDE

    Security implications of in-band signalling strikes again, 43 years after the “Blue Box” hit the mainstream. Jamie McCarthy on Twitter: “.@cmdrtaco – Remember when we had to block the U+202E code point in Slashdot comments to stop siht ekil stnemmoc?” See also — GMail was vulnerable too; and for more inline control chars. has some official recommendations from the Unicode consortium on dealing with bidi override chars.

    (tags: security attacks rlo unicode control-characters codepoints bidi text gmail slashdot sanitization input)

  • Meet the e-voting machine so easy to hack, it will take your breath away | Ars Technica

    The AVS WinVote system — mind-bogglingly shitty security.

    If an election was held using the AVS WinVote, and it wasn’t hacked, it was only because no one tried. The vulnerabilities were so severe, and so trivial to exploit, that anyone with even a modicum of training could have succeeded. They didn’t need to be in the polling place—within a few hundred feet (e.g., in the parking lot) is easy, and within a half mile with a rudimentary antenna built using a Pringles can. Further, there are no logs or other records that would indicate if such a thing ever happened, so if an election was hacked any time in the past, we will never know. I’ve been in the security field for 30 years, and it takes a lot to surprise me. But the VITA report really shocked me—as bad as I thought the problems were likely to be, VITA’s five-page report showed that they were far worse. And the WinVote system was so fragile that it hardly took any effort. While the report does not state how much effort went into the investigation, my estimation based on the description is that it was less than a person week.

    (tags: security voting via:johnke winvote avs shoup wep wifi windows)

Posted in Uncategorized | Comments closed

Links for 2015-04-22

  • ‘Continuous Deployment: The Dirty Details’

    Good slide deck from Etsy’s Mike Brittain regarding their CD setup. Some interesting little-known details: Slide 41: database schema changes are not CD’d — they go out on “Schema change Thursdays”. Slide 44: only the webapp is CD’d — PHP, Apache, memcache components (, support and back-office tools, developer API, gearman async worker queues). The external “services” are not — databases, Solr/JVM search (rolling restarts), photo storage (filters, proxy cache, S3), payments (PCI-DSS, controlled access). They avoid schema changes and breaking changes using an approach they call “non-breaking expansions” — expose new version in a service interface; support multiple versions in the consumer. Example from slides 50-63, based around a database schema migration. Slide 66: “dev flags” (rollout oriented) are promoted to “feature flags” (long lived degradation control). Slide 71: some architectural philosophies: deploying is cheap; releasing is cheap; gathering data should be cheap too; treat first iterations as experiments. Slide 102: “Canary pools”. They have multiple pools of users for testing in production — the staff pool, users who have opted in to see prototypes/beta stuff, 0-100% gradual phased rollout.

    (tags: cd deploy etsy slides migrations database schema ops ci version-control feature-flags)

  • Etsy’s Release Management process

    Good info on how Etsy use their Deployinator tool, end-to-end. Slide 11: git SHA is visible for each env, allowing easy verification of what code is deployed. Slide 14: Code is deployed to “princess” staging env while CI tests are running; no need to wait for unit/CI tests to complete. Slide 23: smoke tests of pre-prod “princess” (complete after 8 mins elapsed). Slide 31: dashboard link for deployed code is posted during deploy; post-release prod smoke tests are run by Jenkins. (short ones! they complete in 42 seconds)

    (tags: deployment etsy deploy deployinator princess staging ops testing devops smoke-tests production jenkins)

  • Makerbot’s Saddest Hour | TechCrunch

    I’ve been speaking to a few people [at Makerbot] who prefer to remain anonymous and most of my contacts there are gone (the head of PR was apparently fired) and don’t want to talk. But the new from inside is troubling. The mass-layoffs are blamed on low revenue and one former employee wrote “Company was failing. Couldn’t pay vendors, had to downsize.” Do I think Makerbot will sink? At this point I don’t know.

    (tags: makerbot 3d-printing startups downsizing layoffs ouch)

  • credstash

    ‘CredStash is a very simple, easy to use credential management and distribution system that uses AWS Key Management System (KMS) for key wrapping and master-key storage, and DynamoDB for credential storage and sharing.’

    (tags: aws credstash python security keys key-management secrets kms)

  • distributed scalability systems coding server-side erlang devops networking reliability)

Posted in Uncategorized | Comments closed

Links for 2015-04-21

Posted in Uncategorized | Comments closed

Links for 2015-04-20

  • FBI admits flaws in hair analysis over decades

    Wow, this is staggering.

    The Justice Department and FBI have formally acknowledged that nearly every examiner in an elite FBI forensic unit gave flawed testimony in almost all trials in which they offered evidence against criminal defendants over more than a two-decade period before 2000. [....] The review confirmed that FBI experts systematically testified to the near-certainty of “matches” of crime-scene hairs to defendants, backing their claims by citing incomplete or misleading statistics drawn from their case work. In reality, there is no accepted research on how often hair from different people may appear the same. Since 2000, the lab has used visual hair comparison to rule out someone as a possible source of hair or in combination with more accurate DNA testing. Warnings about the problem have been mounting. In 2002, the FBI reported that its own DNA testing found that examiners reported false hair matches more than 11 percent of the time.

    (tags: fbi false-positives hair dna biometrics trials justice experts crime forensics inaccuracy csi)

  • The missing MtGox bitcoins

    Most or all of the missing bitcoins were stolen straight out of the MtGox hot wallet over time, beginning in late 2011. As a result, MtGox operated at fractional reserve for years (knowingly or not), and was practically depleted of bitcoins by 2013. A significant number of stolen bitcoins were deposited onto various exchanges, including MtGox itself, and probably sold for cash (which at the bitcoin prices of the day would have been substantially less than the hundreds of millions of dollars they were worth at the time of MtGox’s collapse). MtGox’ bitcoins continuously went missing over time, but at a decreasing pace. Again by the middle of 2013, the curve goes more or less flat, matching the hypothesis that by that time there may not have been any more bitcoins left to lose. The rate of loss otherwise seems unusually smooth and at the same time not strictly relative to any readily available factors such as remaining BTC holdings, transaction volumes or the BTC price. Worth pointing out is that, thanks to having matched up most of the deposit/withdrawal log earlier, we can at this point at least rule out the possibility of any large-scale fake deposits — the bitcoins going into MtGox were real, meaning the discrepancy was likely rather caused by bitcoins leaving MtGox without going through valid withdrawals.

    (tags: mtgox bitcoin security fail currency theft crime btc)

  • Bank of the Underworld – The Atlantic

    Prosecutors analyzed approximately 500 of Liberty Reserve’s biggest accounts, which constituted 44 percent of its business. The government contends that 32 of these accounts were connected to the sale of stolen credit cards and 117 were used by Ponzi-scheme operators. All of this activity flourished, prosecutors said, because Liberty Reserve made no real effort to monitor its users for criminal behavior. What’s more, records showed that one of the company’s top tech experts, Mark Marmilev, who was also arrested, appeared to have promoted Liberty Reserve in chat rooms devoted to Ponzi schemes.
    (via Nelson)

    (tags: scams fraud crime currency the-atlantic liberty-reserve ponzi-schemes costa-rica arthur-budovsky banking anonymity cryptocurrency money-laundering carding)

  • I was a Lampedusa refugee. Here’s my story of fleeing Libya – and surviving

    ‘The boy next to me fell to the floor and for a moment I didn’t know if he had fainted or was dead – then I saw that he was covering his eyes so he didn’t have to see the waves any more. A pregnant woman vomited and started screaming. Below deck, people were shouting that they couldn’t breathe, so the men in charge of the boat went down and started beating them. By the time we saw a rescue helicopter, two days after our boat had left Libya with 250 passengers on board, some people were already dead – flung into the sea by the waves, or suffocated downstairs in the dark.’

    (tags: lampedusa migration asylum europe fortress-europe italy politics immigration libya refugees)

  • Run your own high-end cloud gaming service on EC2

    Using Steam streaming and EC2 g2.2xlarge spot instances — ‘comes out to around $0.52/hr’. That’s pretty compelling IMO

    (tags: aws ec2 gaming games graphics spot-instances hacks windows steam)

  • Running Arbitrary Executables in AWS Lambda

    actually an officially-supported mode. huh

    (tags: lambda aws architecture ops node.js javascript unix linux)

Posted in Uncategorized | Comments closed

Links for 2015-04-18

Posted in Uncategorized | Comments closed

Links for 2015-04-17

Posted in Uncategorized | Comments closed

Links for 2015-04-16

  • Extracting Structured Data From Recipes Using Conditional Random Fields

    nice probabilistic/ML approach to recipe parsing

    (tags: nytimes recipes parsing text nlp machine-learning probabilistic crf++ algorithms feature-extraction)

  • Large-scale cluster management at Google with Borg

    Google’s Borg system is a cluster manager that runs hundreds of thousands of jobs, from many thousands of different applications, across a number of clusters each with up to tens of thousands of machines. It achieves high utilization by combining admission control, efficient task-packing, over-commitment, and machine sharing with process-level performance isolation. It supports high-availability applications with runtime features that minimize fault-recovery time, and scheduling policies that reduce the probability of correlated failures. Borg simplifies life for its users by offering a declarative job specification language, name service integration, real-time job monitoring, and tools to analyze and simulate system behavior. We present a summary of the Borg system architecture and features, important design decisions, a quantitative analysis of some of its policy decisions, and a qualitative examination of lessons learned from a decade of operational experience with it.
    (via Conall)

    (tags: via:conall clustering google papers scale to-read borg cluster-management deployment packing reliability redundancy)

  • Keeping Your Car Safe From Electronic Thieves –

    In a normal scenario, when you walk up to a car with a keyless entry and try the door handle, the car wirelessly calls out for your key so you don’t have to press any buttons to get inside. If the key calls back, the door unlocks. But the keyless system is capable of searching for a key only within a couple of feet. Mr. Danev said that when the teenage girl turned on her device, it amplified the distance that the car can search, which then allowed my car to talk to my key, which happened to be sitting about 50 feet away, on the kitchen counter. And just like that, open sesame.
    What the hell — who designed a system that would auto-unlock based on signal strength alone?!!

    (tags: security fail cars keys signal proximity keyless-entry prius toyota crime amplification power-amplifiers 3db keyless)

  • Closed access means people die

    ‘We’ve paid 100 BILLION USD over the last 10 years to “publish” science and medicine. Ebola is a massive systems failure.’ See also : ‘The conventional wisdom among public health authorities is that the Ebola virus, which killed at least 10,000 people in Liberia, Sierra Leone and Guinea, was a new phenomenon, not seen in West Africa before 2013. [...] But, as the team discovered, that “conventional wisdom” was wrong. In fact, they found a bunch of studies, buried behind research paywalls, that revealed that there was significant evidence of antibodies to the Ebola virus in Liberia and in other nearby nations. There was one from 1982 that noted: “medical personnel in Liberian health centers should be aware of the possibility that they may come across active cases and thus be prepared to avoid nosocomial epidemics.”

    (tags: deaths liberia ebola open-access papers elsevier science medicine reprints)

  • Making Pinterest — Learn to stop using shiny new things and love MySQL

    ‘The third reason people go for shiny is because older tech isn’t advertised as aggressively as newer tech. The younger companies needs to differentiate from the old guard and be bolder, more passionate and promise to fulfill your wildest dreams. But most new tech sales pitches aren’t generally forthright about their many failure modes. In our early days, we fell into this third trap. We had a lot of growing pains as we scaled the architecture. The most vocal and excited database companies kept coming to us saying they’d solve all of our scalability problems. But nobody told us of the virtues of MySQL, probably because MySQL just works, and people know about it.’ It’s true! — I’m still a happy MySQL user for some use cases, particularly read-mostly relational configuration data…

    (tags: mysql storage databases reliability pinterest architecture)

  • Microservices and elastic resource pools with Amazon EC2 Container Service

    interesting approach to working around ECS’ shortcomings — bit specific to Hailo’s microservices arch and IPC mechanism though. aside: I like their version numbering scheme: ISO-8601, YYYYMMDDHHMMSS. keep it simple!

    (tags: versioning microservices hailo aws ec2 ecs docker containers scheduling allocation deployment provisioning qos)

  • Please Kill Me (Eventually) | Motherboard

    There is much that the wise application of technology can do to help us ease off this mortal coil, instead of tormenting ourselves at the natural end of life in a futile, undignified and excruciating attempt to keep it somehow duct-taped on. Train more people in geriatrics, for example. Learn new ways to make life safe, healthy, fun and interesting for the old. Think like a community, a brotherhood, not like atomized competing individuals a few of whom can somehow “beat the system” of the universe. Maybe it is better to examine clearly what we are with a view to understanding and acceptance than it is to try to escape what perhaps should be our inevitable ending.

    (tags: death mortality cryogenics alcor geriatrics life singularity mind-uploading ray-kurzweil)

  • CGA in 1024 Colors – a New Mode: the Illustrated Guide

    awesome hackery. brings me back to my C=64 demo days

    (tags: pc cga graphics hacks art 1024-colours)

Posted in Uncategorized | Comments closed

Links for 2015-04-15

  • Keywhiz

    ‘a secret management and distribution service [from Square] that is now available for everyone. Keywhiz helps us with infrastructure secrets, including TLS certificates and keys, GPG keyrings, symmetric keys, database credentials, API tokens, and SSH keys for external services — and even some non-secrets like TLS trust stores. Automation with Keywhiz allows us to seamlessly distribute and generate the necessary secrets for our services, which provides a consistent and secure environment, and ultimately helps us ship faster. [...] Keywhiz has been extremely useful to Square. It’s supported both widespread internal use of cryptography and a dynamic microservice architecture. Initially, Keywhiz use decoupled many amalgamations of configuration from secret content, which made secrets more secure and configuration more accessible. Over time, improvements have led to engineers not even realizing Keywhiz is there. It just works. Please check it out.’

    (tags: square security ops keys pki key-distribution key-rotation fuse linux deployment secrets keywhiz)

Posted in Uncategorized | Comments closed

Links for 2015-04-14

Posted in Uncategorized | Comments closed

Links for 2015-04-13

  • Amazon Machine Learning

    Upsides of this new AWS service: * great UI and visualisations. * solid choice of metric to evaluate the results. Maybe things moved on since I was working on it, but the use of AUC, false positives and false negatives was pretty new when I was working on it. (er, 10 years ago!) Downsides: * it could do with more support for unsupervised learning algorithms. Supervised learning means you need to provide training data, which in itself can be hard work. My experience with logistic regression in the past is that it requires very accurate training data, too — its tolerance for misclassified training examples is poor. * Also, in my experience, 80% of the hard work of using ML algorithms is writing good tokenisation and feature extraction algorithms. I don’t see any help for that here unfortunately. (probably not that surprising as it requires really detailed knowledge of the input data to know what classes can be abbreviated into a single class, etc.)

    (tags: amazon aws ml machine-learning auc data-science)

  • Rob Pike’s 5 rules of optimization

    these are great. I’ve run into rule #3 (“fancy algorithms are slow when n is small, and n is usually small”) several times…

    (tags: twitter rob-pike via:igrigorik coding rules laws optimization performance algorithms data-structures aphorisms)

  • AWS Lambda Event-Driven Architecture With Amazon SNS

    Any message posted to an SNS topic can trigger the execution of custom code you have written, but you don’t have to maintain any infrastructure to keep that code available to listen for those events and you don’t have to pay for any infrastructure when the code is not being run. This is, in my opinion, the first time that Amazon can truly say that AWS Lambda is event-driven, as we now have a central, independent, event management system (SNS) where any authorized entity can trigger the event (post a message to a topic) and any authorized AWS Lambda function can listen for the event, and neither has to know about the other.

    (tags: aws ec2 lambda sns events cep event-processing coding cloud hacks eric-hammond)

  • Texting at the wheel kills more US teenagers every year than drink-driving

    Texting while behind the wheel has overtaken drink driving as the biggest cause of death among teenagers in America. More than 3,000 teenagers are killed every year in car crashes caused by texting while driving compared to 2,700 from drink driving. The study by Cohen Children’s Medical Center also discovered that 50 per cent of students admit to texting while driving.

    (tags: texting sms us driving car-safety safety drink-driving)

  • China’s Great Cannon

    Conducting such a widespread attack clearly demonstrates the weaponization of the Chinese Internet to co-opt arbitrary computers across the web and outside of China to achieve China’s policy ends.  The repurposing of the devices of unwitting users in foreign jurisdictions for covert attacks in the interests of one country’s national priorities is a dangerous precedent — contrary to international norms and in violation of widespread domestic laws prohibiting the unauthorized use of computing and networked systems.

    (tags: censorship ddos internet security china great-cannon citizen-lab reports web)

  • Sirius: An open end-to-end voice and vision personal assistant and its implications for future warehouse scale computers

    How to build an Intelligent Personal Assistant: ‘Sirius is an open end-to-end standalone speech and vision based intelligent personal assistant (IPA) similar to Apple’s Siri, Google’s Google Now, Microsoft’s Cortana, and Amazon’s Echo. Sirius implements the core functionalities of an IPA including speech recognition, image matching, natural language processing and a question-and-answer system. Sirius is developed by Clarity Lab at the University of Michigan. Sirius is published at the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) 2015.’

    (tags: sirius siri cortana google-now echo ok-google ipa assistants search video audio speech papers clarity nlp wikipedia)

  • Why We Will Not Be Registering easyDNS.SUCKS –

    If you’re not immersed in the naming business you may find the jargon in it hard to understand. The basic upshot is this: the IPC believes that the mechanisms that were enacted to protect trademark holders during the deluge of new TLD rollouts are being gamed by the .SUCKS TLD operator to extort inflated fees from trademark holders.
    (via Nelson)

    (tags: shakedown business internet domains dns easydns dot-sucks scams tlds trademarks ip)

Posted in Uncategorized | Comments closed

Links for 2015-04-12

Posted in Uncategorized | Comments closed

Links for 2015-04-11

Posted in Uncategorized | Comments closed

Links for 2015-04-10

  • Hacked French network exposed its own passwords during TV interview


    (tags: passwords post-its fail tv5monde authentication security tv funny)

  • RADStack – an open source Lambda Architecture built on Druid, Kafka and Samza

    ‘In this paper we presented the RADStack, a collection of complementary technologies that can be used together to power interactive analytic applications. The key pieces of the stack are Kafka, Samza, Hadoop, and Druid. Druid is designed for exploratory analytics and is optimized for low latency data exploration, aggregation, and ingestion, and is well suited for OLAP workflows. Samza and Hadoop complement Druid and add data processing functionality, and Kafka enables high throughput event delivery.’

    (tags: druid samza kafka streaming cep lambda-architecture architecture hadoop big-data olap)

  • outbrain/gruffalo

    an asynchronous Netty based graphite proxy. It protects Graphite from the herds of clients by minimizing context switches and interrupts; by batching and aggregating metrics. Gruffalo also allows you to replicate metrics between Graphite installations for DR scenarios, for example. Gruffalo can easily handle a massive amount of traffic, and thus increase your metrics delivery system availability. At Outbrain, we currently handle over 1700 concurrent connections, and over 2M metrics per minute per instance.

    (tags: graphite backpressure metrics outbrain netty proxies gruffalo ops)

  • Privacy Security Talk in TOG – 22nd April @ 7pm – FREE

    Dublin is lucky enough to have great speakers pass through town on occasion and on Wednesday the 22nd April 2015, Runa A. Sandvik (@runasand) and Per Thorsheim (@thorsheim) have kindly offered to speak in TOG from 7pm. The format for the evening is a general meet and greet, but both speakers have offered to give a presentation on a topic of their choice. Anyone one interested in privacy, security, journalism, Tor and/or has previously attended a CryptoParty would be wise to attend. Doors are from 7pm and bring any projects with you you would like to share with other attendees. This is a free event, open to the public and no need to book. See you Wednesday. Runa A. Sandvik is an independent privacy and security researcher, working at the intersection of technology, law and policy. She contributes to The Tor Project, writes for Forbes, and is a technical advisor to both the Freedom of the Press Foundation and the TrueCrypt Audit project. Per Thorsheim as founder/organizer of, his topic of choice is of course passwords, but in a much bigger context than most people imagine. Passwords, pins, biometrics, 2-factor authentication, security/usability and all the way into surveillance and protecting your health, kids and life itself.

    (tags: privacy security runa-sandvik per-thorsheim passwords tor truecrypt tog via:oisin events dublin)

Posted in Uncategorized | Comments closed

Links for 2015-04-09

Posted in Uncategorized | Comments closed

Links for 2015-04-08

Posted in Uncategorized | Comments closed