BikeDroid : Warren’s Android app to track DublinBike availability: ‘Use BikeDroid to locate the nearest free bike or stand to you. Get real-time status of all bike stands displayed on a map of your city.’
(tags: bikes dublinbikes apps android mobile)How to block retweets : in Twitter, obvs. This is incredibly handy, and very poorly-documented
(tags: twitter retweets annoying ui)BallotBox.ie Posts Emigrant-Vote Results : FG 63 seats, Labour 51, SF 23, Greens 10, Ind 11, FF 2. interesting to see SF’s strong showing among emigrants — something for electoral reformers to think about ;)
(tags: ireland politics voting e-voting emigration)
Month: February 2011
Dublin Bikes 2 Go! : ‘an [unofficial] mobile web application that the public can use to find ‘Dublin Bikes’ stations and information about bike availability’
(tags: bikes dublin dublinbikes cycling mobile apps iphone android)
Frank Zappa proposed EMusic in 1983 : incredlble — way ahead of his time on this one
(tags: music internet filesharing business p2p emusic mp3)
It’s pretty common for apps to require “configuration” — external files which can contain settings to customise their behaviour. Ideally, apps shouldn’t require configuration, and this is always a good aim. But in some situations, it’s unavoidable.
In the abstract, it may seem attractive to use a fully-fledged programming language as the language to express configuration in. However, I think this is not a good idea. Here are some reasons why configuration files should not be expressed in a programming language (and yes, I include “Ruby without parentheses” in that bucket):
Provability
If a configuration language is Turing-incomplete, configuration files written in it can be validated “offline”, ie. without executing the program it configures. All programming languages are, by definition, Turing-complete, meaning that the program must be executed in full before its configuration can be considered valid.
Offline validation is a useful feature for operational usability, as we’ve found with “spamassassin –lint”.
Security
Some configuration settings may be insecure in certain circumstances; for example, in SpamAssassin, we allow certain classes of settings like whitelist/blacklists to be set in a users ~/.spamassassin/user_prefs file, while disallowing rule definitions (which can cause poor performance if poorly written).
If your configuration file is simply an evaluated chunk of code, it becomes more difficult to protect against an attacker introspecting the interpreter and overriding the security limitations. It’s not impossible, since you can, for instance, use a sandboxed interpreter, but this is typically not particularly easy to implement.
Usability
Here’s a rather hairy configuration file I’ve concocted.
#! /usr/bin/somelanguage !$ app.status load html !c = [] ;c['sources'] = < > ;c['sources'].append( NewConfigurationThingy("foo_bar", baz="flargle")) ;c['builders'] = < > ;c['bots'] = < > !$ app.steps load source, shell ;bf_mc_generic = factory.SomethingFactory( < woo(source.SVN, svnurl="http://example.com/foo/bar"), woo(shell.Configure, command="/bar/baz start"), woo(shell.Test, command="/bar/baz test"), woo(shell.Configure, command="/bar/baz stop") > ); ;b1 = < "name": "mc-fast", "slavename": "mc-fast", "builddir": "mc-fast", "factory": ;bf_mc_generic > ;c['builders'].append(;b1) ;SomethingOrOther = ;c
This isn’t actually entirely concocted from thin air — it’s actually bits of our BuildBot configuration file, from before we switched to using Hudson. I’ve replaced the familiar Python syntax with deliberately-unfamiliar made-up syntax, to emulate the user experience I had attempting to configure BuildBot with no pre-existing Python knowledge. ;)
Compare with this re-stating of the same configuration data in a simplified, “configuration-oriented” imaginary DSL:
add_source NewConfigurationThingy foo_bar baz=flargle
buildfactory bf_mc_generic source.SVN http://example.com/foo/bar
buildfactory bf_mc_generic shell.Configure /bar/baz start
buildfactory bf_mc_generic shell.Test /bar/baz test
buildfactory bf_mc_generic shell.Configure /bar/baz stop
add_builder name=mc-fast slavename=mc-fast
builddir=mc-fast factory=bf_mc_generic
Essentially, I’ve extracted the useful configuration data from the hairy example, discarded the symbology used to indicate types, function calls, data structure construction, and let the configuration domain knowledge imply what’s necessary. Not only is this easier to comprehend for the casual reader, it also reduces the risk of syntax errors, by simply minimising the number of syntactical components.
See Also
The Wikipedia page on DSLs is quite good on the topic, with a succinct list of pros and cons.
This StackOverflow thread has some good comments — I particularly like this point:
When you need your application to be very “configurable” in ways that you cannot imagine today, then what you really need is a plugins system. You need to develop your application in a way that someone else can code a new plugin and hook it into your application in the future.
+1.
This seems to be a controversial topic — as you can see, that page has people on both sides of the issue. Maybe it fundamentally comes down to a matter of taste. Anyway — my $.02.
Update: discussions elsewhere: HackerNews
Another Update, 2012-04-06: Robey Pointer wrote a post called Why Config?, in which he describes a Scala-based configuration language in use at Twitter, which uses Scala’s runtime code evaluation, and a Scala trait, to express configuration succinctly in a Scala source file and load it at runtime. The downside? It’s a Scala source file, executed at runtime, containing configuration. :(
However, this comment in the comments section is worth a read:
At Netli (now part of Akamai) we had a configuration framework very similar in spirit and appearance to Configgy. It was in early 2000-s, we open sourced it since. (http://ncnf.sourceforge.net/). It would provide on-the-fly reload for the C-based programs (the ncnf if a C library). It also had some perks like attribute inheritance and a concept of block references. Most importantly though, it contained a separate schema language and a validator to allow configuration be checked before pushing in production. At Netli we used it to configure 1200 services on over 400 hardware boxes, the configuration becoming about 20+mb in length (assembled from several pieces by the CPP, then M4 templating library).
Naturally, it wasn’t Netli’s first attempt at doing configuration. One of the first attempts failed since it was Turing-complete. That approach was to specify the configuration as a Perl data specification. In a very short time the lure of unused expressiveness of such Turing-complete environment prevailed and people started to write for-loops around data pieces and doing other tricks to remove redundancy from the configuration. It turned out to be a disaster in the end, with configuration becoming unmaintainable and flaky.
One principle I got out out of that exercise is that configuration shall not be Turing-complete. We’ve got burned specifically by that property far too many times. Yet I do agree with you that a validation facility is a must-have, which is something not usually part of the simple text-based frameworks. C-based NCNF had it almost from the very beginning though, and it proved to be a very useful harness.
+1. There’s lots more info on that system at this post at lionet.livejournal.com.
Another Update, 2017-05-09: casio_juarez on Twitter:
Dev: I'll use a declarative language for config this time.
— 0x0DEADA55 (@casio_juarez) May 8, 2017
6 months later: Let's add variables.
12 mos: And conditionals.
18 mos: Fuck.
Also related: The Configuration Complexity Clock.
(Image credit: Turn The Dial by VERY URGENT Photography)
Tom Morris – Request for comment: a ‘Good API’ checklist and committee : Sane suggestions for good HTTP APIs
(tags: apis http rest open-data)Votomatic : Brilliant! “find out which political parties are compatible with you.” The app asks a few questions, you furnish survey-style responses, and it figures out which party is closest in published policy. It works quite well, determining that my optimum is Labour (correct)
(tags: policies politics ireland voting elections surveys)How a Remote Town in Romania Has Become Cybercrime Central | Magazine : the story of Ramnicu Valcea — Romania’s Silicon Valley of phishing
(tags: ramnica-valcea crime romania wired security spam phishing)U.S. Government Shuts Down 84,000 Websites, ‘By Mistake’ | TorrentFreak : DHS/ICE domain seizures suffer a serious false positive problem, resulting in the seizure and shutting down of 84,000 subdomains of a free DNS provider, replacing them with a banner accusing the site of trafficking in child porn. whoops!
(tags: dhs ice censorship internet domains dns seizure false-positives child-porn)Israeli general claims Stuxnet attacks as one of his successes : ‘Haaretz reports [on a] video that was played at a party organized for General Gabi Ashkenazi’s last day on the job. The video contained references to the successes he achieved during his stint as chief of staff, [including] the Stuxnet worm attack on Iran’s uranium enrichment facility at Natanz and and the nuclear reactor at Bushehr.’
(tags: israel iran stuxnet cyberwar via:slashdot malware)
Gerrit, Git and Jenkins : This is the future of code review. Commit directly from your git checkout to the Gerrit code-review system; change is immediately web-visible and enters the review workflow; at the same time, Jenkins checks out the proposed change and runs the test suite; once it’s approved, it automatically gets checked in. Brilliant!
(tags: git coding code-review workflows jenkins gerrit c-i testing automation)
FareBot: Read data from public transit cards with your NFC-equipped Android phone – codebutler : ‘When demonstrating FareBot, many people are surprised to learn that much of the data on their ORCA card is not encrypted or protected. This fact is published by ORCA, but is not commonly known and may be of concern to some people who would rather not broadcast where they’ve been to anyone who can brush against the outside of their wallet. Transit agencies across the board should do a better job explaining to riders how the cards work and what the privacy implications are.’ (via Boing Boing)
(tags: via:boingboing privacy android rfid security transit mobile encryption mifare desfire farebot)Storymap : great UI for a little Dublin oral-history site — just a GMaps mashup with links to YouTube, but it works very well
(tags: dublin ireland storymap stories oral-history people google-maps mashups youtube video)Spotify Second Largest Source Of Revenue In Europe For Labels : wow. the WinAmp guys were right — ‘on a European level, Spotify is the second single largest source of revenue for record labels. This means that 2010 saw dramatic increase in its usage as well as payouts to record labels and artists themselves.’ this via an IFPI report
(tags: ifpi music spotify streaming revenue record-labels europe sweden isps mp3)Zero stroke – Wikipedia, the free encyclopedia : ‘With the price of bread running into billions a loaf the German people […] had to get used to counting in thousands of billions. This, according to some German physicians, brought on a new nervous disease known as “zero stroke,” or “cipher stroke” […] The persons afflicted with the malady are perfectly normal, except “for a desire to write endless rows of ciphers and engage in computations more involved than the most difficult problems in logarithms.”‘ (via Joe Drumgoole)
(tags: germany zero hyperinflation inflation via:jdrumgoole money brain mental-illness)
Fine Gael’s Facebook spam campaign : jesus. Not only do they coin the cramp-inducing neologism “twolicy”, they then have the temerity to suggest that people should “donate” their Facebook status so that FG can spam their social group. awful
(tags: facebook fine-gael twitter social-media twolicy spam)
No Sleep ‘Til Brooklands: A True Story Of Daily Mail Lies (guest post) : how the Daily Mail (UK) works, via b3ta. mind-boggling misuse of one woman’s comments to concoct a story, according to this
(tags: daily-mail journalism libel media newspapers law uk via:b3ta)Using Git to manage a web site : simple, basic demo of a git post-receive hook to auto-check-out every rev committed to a git repository
(tags: git deployment howto via:hackernews)
If you visit the Irish Times at all frequently, you’ll probably have noticed a nifty “wisdom of crowds” feature in the right sidebar: the list of “most read” articles. It’s quite good, since they’re often very interesting articles. Unfortunately, there’s no RSS feed for this feature.
Well, now there is:
Gamasutra – News – Opinion: Minecraft And The Question Of Luck : ‘Notch’s luck was that he came across the idea of doing a first-person fortress building game. His alignment was that the game that he wanted to make was culturally connected to [he PC gamer] tribe. While the game may appear ugly, and its purchase process etc seem naive to many a gaming professional, all of those decisions that Notch made along the road to releasing his game were from the point of view of a particular perspective of what games are, what matters and what were the things that he could trust the tribe to figure out for themselves.’
(tags: tribes viral minecraft gaming analysis games culture gamasutra via:nelson future software marketing)
Spamwiki : good wiki tracking spam operations, their current campaigns, who’s doing it etc.
(tags: wiki spam anti-spam)Spammers Are Now Using Verified By Visa : Visa’s atrociously-designed “security” program is now being used by criminals to process their credit-card payments, allegedly
(tags: verified-by-visa spam visa security)
Michael “Liar’s Poker” Lewis on Ireland’s economic collapse : PDF of the 15-page Vanity Fair article — from interviews I’ve read in advance, this seems pretty good
(tags: michael-lewis vanity-fair articles pdf toread economy ireland disaster collapse)Dublin bikes revisited : Fantastic comparative number crunching on the JC Decaux Dublin Bikes scheme, compared to their other European cities (Brussels, Lyons, Paris, Seville), times of day, busiest stations, rainfall, etc.
(tags: bikes dublin-bikes cycling dublin ireland jc-decaux number-crunching analysis statistics)Wired: how a Toronto statistician cracked the state lottery : ‘The tic-tac-toe lottery was seriously flawed. It took a few hours of studying his tickets and some statistical sleuthing, but he discovered a defect in the game: The visible numbers turned out to reveal essential information about the digits hidden under the latex coating. Nothing needed to be scratched off—the ticket could be cracked if you knew the secret code.’
(tags: toronto hacks money statistics probability wired tic-tac-toe singleton)
Google: Bing Is Cheating, Copying Our Search Results : laaaame, Microsoft
(tags: lame microsoft google search honeypots stings)Java Hangs When Converting 2.2250738585072012e-308 : ie. the same value as the PHP bug. ‘Konstantin [Pressier] reported this problem to Oracle three weeks ago, but is still waiting for a reply.’ good job, Oracle!
(tags: oracle fail security java bugs floating-point)