-
At Comcast, our applications need convenient, low-latency access to important reference datasets. For example, our XfinityTV websites and apps need to use entertainment-related data to serve almost every API or web request to our datacenters: information like what year Casablanca was released, or how many episodes were in Season 7 of Seinfeld, or when the next episode of the Voice will be airing (and on which channel!). We traditionally managed this information with a combination of relational databases and RESTful web services but yearned for something simpler than the ORM, HTTP client, and cache management code our developers dealt with on a daily basis. As main memory sizes on commodity servers continued to grow, however, we asked ourselves: How can we keep this reference data entirely in RAM, while ensuring it gets updated as needed and is easily accessible to application developers? The Sirius distributed system library is our answer to that question, and we’re happy to announce that we’ve made it available as an open source project. Sirius is written in Scala and uses the Akka actor system under the covers, but is easily usable by any JVM-based language.
Also includes a Paxos implementation with “fast follower” read-only slave replication. ASL2-licensed open source. The only thing I can spot to be worried about is speed of startup; they note that apps need to replay a log at startup to rebuild state, which can be slow if unoptimized in my experience. Update: in a twitter conversation at https://twitter.com/jon_moore/status/459363751893139456 , Jon Moore indicated they haven’t had problems with this even with ‘datasets consuming 10-20GB of heap’, and have ‘benchmarked a 5-node Sirius ingest cluster up to 1k updates/sec write throughput.’ That’s pretty solid!(tags: open-source comcast paxos replication read-only datastores storage memory memcached redis sirius scala akka jvm libraries)
AWS Elastic Beanstalk for Docker
This is pretty amazing. nice work, Beanstalk team. not sure how well it integrates with the rest of AWS though
(tags: aws amazon docker ec2 beanstalk ops containers linux)
TDD is dead. Long live testing
Oh god. I agree with DHH. shoot me now.
Test-first units leads to an overly complex web of intermediary objects and indirection in order to avoid doing anything that’s “slow”. Like hitting the database. Or file IO. Or going through the browser to test the whole system. It’s given birth to some truly horrendous monstrosities of architecture. A dense jungle of service objects, command patterns, and worse. I rarely unit test in the traditional sense of the word, where all dependencies are mocked out, and thousands of tests can close in seconds. It just hasn’t been a useful way of dealing with the testing of Rails applications. I test active record models directly, letting them hit the database, and through the use of fixtures. Then layered on top is currently a set of controller tests, but I’d much rather replace those with even higher level system tests through Capybara or similar. I think that’s the direction we’re heading. Less emphasis on unit tests, because we’re no longer doing test-first as a design practice, and more emphasis on, yes, slow, system tests.
(tags: tdd rails testing unit-tests system-tests integration-testing ruby dhh mocks)
All at sea: global shipping fleet exposed to hacking threat | Reuters
Hackers recently shut down a floating oil rig by tilting it, while another rig was so riddled with computer malware that it took 19 days to make it seaworthy again; Somali pirates help choose their targets by viewing navigational data online, prompting ships to either turn off their navigational devices, or fake the data so it looks like they’re somewhere else; and hackers infiltrated computers connected to the Belgian port of Antwerp, located specific containers, made off with their smuggled drugs and deleted the records.
(via Mikko Hypponen)(tags: via:mikko security hacking oilrigs shipping ships maritime antwerp piracy malware)
Search Results – (Author:Thomas H Mason)
Photographs taken by my great-grandfather, Thomas H. Mason, in the National Library of Ireland’s newly-digitized online collection
(tags: family thomas-h-mason history ireland photography archive nli)
Syria’s lethal Facebook checkpoints
An anonymous tip from a highly reliable source: “There are checkpoints in Syria where your Facebook is checked for affiliation with the rebellious groups or individuals aligned with the rebellion. People are then disappeared or killed if they are found to be connected. Drivers are literally forced to load their Facebook/Twitter accounts and then they are riffled through. It’s happening daily, and has been for a year at least.”
(tags: boing-boing war facebook social-media twitter internet checkpoints syria)