ramblings about spam, scalability, software development, the web

k-Nearest Neighbor (k-NN) similarity search engine with Amazon Elasticsearch

Well, that’s handy:

Amazon Elasticsearch Service now offers k-Nearest Neighbor (k-NN) search which can enhance search by similarity use cases like product recommendations, fraud detection, and image, video and semantic document retrieval. Built using the lightweight and efficient Non-Metric Space Library (NMSLIB), k-NN enables high scale, low latency nearest neighbor search on billions of documents across thousands of dimensions with the same ease as running any regular Elasticsearch query.  

(tags: elasticsearch aws knn algorithms similarity searching search nmslib)

The history of leaded gasoline is nuts

It is frankly shocking that this was ignored for so long! “The history of leaded gasoline is nuts. Scientists warned it was poison, the factory where it was made was making workers loopy, but GM/Standard Oil enlisted the surgeon general to convince everyone it was safe and rejected alternatives. Massive public harm resulted.” “A Yale physiologist named Yandell Henderson had tested tetraethyl lead as a potential nerve agent during WWI, and when asked his thoughts on putting it into gasoline, he reacted with alarm. ‘Widespread lead poisoning was almost certain to result.’ Later he deemed it the ‘single greatest question in the field of public health that has ever faced the American public.'”

(tags: gasoline petrol lead health poisoning healthcare yandell-henderson)

Numbers Every Programmer Should Know, By Year

interactively explore how Jeff Dean’s “Numbers Every Programmer Should Know” have changed over time (via Kishore Gopalakrishna)

(tags: memory latency hardware history jeff-dean latencies speed performance)

When Bloom filters don’t bloom

A good exploration into modern CPU/memory performance behaviour, and profiling same on Linux using “perf stat -d” and “google-perftools”:

Modern CPUs are really good at sequential memory access when it’s possible to predict memory fetch patterns (see Cache prefetching). Random memory access on the other hand is very costly. Advanced data structures are very interesting, but beware. Modern computers require cache-optimized algorithms. When working with large datasets, not fitting L3, prefer optimizing for reduced number loads, over optimizing the amount of memory used. I guess it’s fair to say that Bloom filters are great, as long as they fit into the L3 cache. The moment this assumption is broken, they are terrible. This is not news, Bloom filters optimize for memory usage, not for memory access. For example, see the Cuckoo Filters paper.

(tags: cloudflare bloom-filters performance data-structures cpu cache l3 hashing perf perftools)

Connectivity at the origins of domain specificity in the cortical face and place networks | PNAS

Wow, this is cool — babies are born with some “pre-wired” visual connectivity networks, specifically for faces and scenes:

Where does knowledge come from? We addressed this classic question using the test cases of the cortical face and scene networks: two well-studied examples of specialized “knowledge” systems in the adult brain. We found that neonates already show domain-specific patterns of functional connectivity between regions that will later develop full-blown face and scene selectivity. Furthermore, the proto face network showed stronger functional connectivity with foveal than with peripheral primary visual cortex, while the proto scene network showed the opposite pattern, revealing that these networks already receive differential visual inputs. Our findings support the hypothesis that innate connectivity precedes the emergence of domain-specific function in cortex, shedding new light on the age-old question of the origins of human knowledge.

(tags: brains vision babies knowledge learning science biology)

Ciarán Murray on Twitter – another Coronavirus thread – estimating the COVID-19 case fatality rate

‘on the basis of what we can learn from the very unfortunate experiment that was the Diamond Princess, the coronavirus is probably at most 13x more lethal than the flu and likely a lot less lethal – probably closer to 5x more lethal (.3% CFR).’

(tags: cfr diseases covid-19 coronavirus medicine)

Coronavirus in China: The most important lessons from China’s Covid-19 response – Vox

Here, again, is where I’ve seen things starting to break down. What I’ve been told is if you think you’ve been exposed and have a fever, call your [general practitioner]. We’ve got to be better than that. If we are going to use our GPs — do they have an emergency line where you can get through? Do they know what to do? In China, they have set up a giant network of fever hospitals. In some areas, a team can go to you and swab you and have an answer for you in four to seven hours. But you’ve got to be set up — speed is everything.

(tags: covid19 coronavirus china hospitals healthcare medicine pandemics)

Amazon’s Principal Tenets

Principal Engineers in Amazon are expected to model these tenets:

Exemplary Practitioner; Technically Fearless; Balanced and Pragmatic; Illuminate and Clarify; Flexible in Approach; Respect What Came Before; Learn, Educate, and Advocate; Have Resounding Impact

(tags: amazon values tenets work principal-engineers engineering coding)

Sketchfab Launches Public Domain Dedication for 3D Cultural Heritage

This is awesome!

We are pleased to announce that cultural organisations using Sketchfab can now dedicate their 3D scans and models to the Public Domain using the Creative Commons (CC) 0 Public Domain Dedication. This newly supported dedication allows museums and similar organisations to share their 3D data more openly, adding amazing 3D models to the Public Domain, many for the first time. This update also makes it even easier for 3D creators to download and reuse, re-imagine, and remix incredible ancient and modern artifacts, objects, and scenes. We are equally proud to make this announcement in collaboration with 27 cultural organisations from 13 different countries. We are especially happy to welcome the Smithsonian Institution to Sketchfab as part of this initiative. The Smithsonian has uploaded their first official 3D models to Sketchfab as part of their newly launched open access program.

(tags: opensource education licensing creative-commons sketchfab 3d-printing 3d models public-domain museums art history objects smithsonian)

Factcheck: What is the carbon footprint of streaming video on Netflix?

Some decent numbers on datacenter usage and debunking some scaremongering figures:

Drawing on analysis at the International Energy Agency (IEA) and other credible sources, we expose the flawed assumptions in one widely reported estimate of the emissions from watching 30 minutes of Netflix. These exaggerate the actual climate impact by 30- to 60-times.

(tags: datacenters internet streaming netflix carbon climate-change iea energy emissions video)

The Green Cloud: How Climate-Friendly Is Your Cloud Provider?

good summary of the state of hosting at the big clouds. Topline recommendations: * choose 100% sustainable locations for new Cloud instances (e.g. on Azure, Google Cloud or the sustainable AWS regions of Dublin, Frankfurt, Oregon or Canada) and transition existing VMs there as soon as possible. * Enterprises should inform their datacenter providers that they require secure, sustainably-powered compute resources and insist on a delivery commitment. * Google and Azure are doing significantly better than AWS in delivering transparency and sustainability.

(tags: datacenters aws azure google hosting climate-change sustainability offsetting)

New Electric Ireland

A company in Arklow is teaching people how to convert petrol/diesel cars to run as EVs on a budget — including their own split-screen VW bus, running on a Tesla drivetrain, and a BMW 7 series running as a hybrid from a Lexus RX450h

(tags: cars driving tesla vw conversions vehicles evs electric)

2019/2020 Season – Health Protection Surveillance Centre

The HSE’s official reports for influenza monitoring for this ‘flu season. Impressive

(tags: hse monitoring diseases outbreaks flu influenza health medicine ireland)

So you think you’re about to be in a pandemic?

Some concrete advice if COVID-19 hits (which seems very likely), around: 1. Reducing our risk of being infected; 2. Reducing the chance we will run out of essential foods and goods.

(tags: planning pandemic covid-19 stockpiling health medicine diseases epidemic)

Why Some COVID-19 Cases Are Worse than Others

For reasons that aren’t entirely clear, some people—especially the elderly and sick—may have dysfunctional immune systems that fail to keep the response to particular pathogens in check. This could cause an uncontrolled immune response, triggering an overproduction of immune cells and their signaling molecules and leading to a cytokine storm often associated with a flood of immune cells into the lung. “That’s when you end up with a lot of these really severe inflammatory disease conditions like pneumonia, shortness of breath, inflammation of the airway, and so forth.”

(tags: covid-19 health diseases immune-system medicine)

Sprouts

Sprouts is a paper-and-pencil game that can be enjoyed simply by both adults and children. Yet it also can be analyzed for its significant mathematical properties. It was invented by mathematicians John Horton Conway (*that* Conway) and Michael S. Paterson at Cambridge University in the early 1960s./blockquote>

(tags: games paper sprouts kids fun)

SEAI grant changes for domestic photovoltaic solar panel installs in Ireland

The big changes are that they are reducing grant coverage for large installs and require a BER C rating to qualify

(tags: seai grants ireland funding pv solar power home)

Green Software Lab

Lots of research on making sustainable, power-friendly software at a coding level

(tags: coding sustainability power green code optimization papers research)

Methane as a greenhouse gas

Detailed thread from Zeke Hausfather on methane and its effects as a GHG; particularly on its short lifespan in the atmosphere

(tags: ch4 methane ghgs atmosphere climate-change science)

The creator of the YOLO object-detection algorithms has stopped computer vision research due to ethical concerns

‘The creator of the YOLO algorithms, which (along with SSD) set much of the path of modern object detection, has stopped doing any computer vision research due to ethical concerns.’

(tags: computer-vision vision algorithms military ethics yolo research)

BLAKE3

‘an Extremely Fast, Parallel Cryptographic Hash’: BLAKE3’s authors published a benchmark on an Intel Cascade Lake-SP 8275CL processor showing it to be 5x faster than BLAKE2 and 15x faster than SHA3-256.

(tags: blake3 blake hashing hashes algorithms speed performance optimization sha)

The Ars Technica semi-scientific guide to Wi-Fi Access Point placement

top tips

(tags: wifi ops networking ieee802.11 wireless ars-technica tips advice radio)

Heavy-duty electric truck driver ditches diesel

This is significant news — sounds like electrified heavy trucks are well on the way:

When it is time to accelerate, the eCascadia’s single gear provides immediate power. That matters during all-too-frequent slowdowns on California freeways. When traffic begins moving, Williams keeps pace. In a diesel, he would be left behind, working through gears to get up to speed. “I would have the space of two diesel trucks in front of me. By the time I get going, five cars jump in front of me,” he said. “With this truck, I can stay right with the cars rather than being dropped back every time we stop and go.” Even with the truck restricted to 60 mph, Williams said he can shave 15 minutes off the drive from the ports to Chino in heavy traffic. With advanced driver assistance systems (ADAS) like adaptive cruise control, which keeps his truck at a set distance from the vehicle in front of him, and automatic emergency braking, Williams also drives a safer truck.

(tags: trucks lorries vehicles evs electric transport future diesel climate-change)

A biotech firm made a smallpox-like virus on purpose. Nobody seems to care – Bulletin of the Atomic Scientists

The loosely regulated market for synthetic DNA, the normalization of synthetic orthopoxvirus research, and a large number of capable facilities and researchers creates an environment in which a rogue state, unscrupulous company, reckless scientist, or terrorist group could potentially reintroduce one of the worst microbial scourges in human history. Unless world bodies, national governments, and scientific organizations put in place stronger safeguards on synthetic virus research, the next press release touting a new breakthrough in synthetic biology might announce that an unknown scientist in an obscure lab has successfully resurrected the smallpox virus.

(tags: smallpox weapons scary diseases biological-weapons dna viruses)

Stop Using Encrypted Email

This is very persuasive and I’d have to agree.

Email is unsafe and cannot be made safe. The tools we have today to encrypt email are badly flawed. Even if those flaws were fixed, email would remain unsafe. Its problems cannot plausibly be mitigated.

The least interesting problems with encrypted email have to do with PGP. […] But that’s a whole other argument. Even after we replace PGP, encrypted email will remain unsafe. Here’s why: If messages can be sent in plaintext, they will be sent in plaintext. Metadata is as important as content, and email leaks it. Every archived message will eventually leak. Every long term secret will eventually leak.

(tags: cryptography security email pgp smtp flaws metadata crypto)

How Hindu supremacists are tearing India apart | World news | The Guardian

This sounds pretty terrifying.

For seven decades, India has been held together by its constitution, which promises equality to all. But Narendra Modi’s BJP is remaking the nation into one where some people count as more Indian than others.

(tags: bjp abvp rss india fascism hindutva politics asia)

How to 3D Print Your Own Lithophane

Lithophanes are essentially embossed photos generated by a 3D printer. The print results don’t show much at first, but shine some light through one and you’ll be amazed by the details.

(tags: lithophanes 3d-printing 3d objects photos cool)

An Indian politician is using deepfake technology in an election campaign

A deepfake of the president of India’s ruling Bharatiya Janata Party (BJP), Manoj Tiwari, went viral on WhatsApp in the country earlier this month, ahead of legislative assembly elections in Delhi, according to Vice. It’s the first time a political party anywhere has used a deepfake for campaigning purposes. In the original video Tiwari speaks in English, criticizing his political opponent Arvind Kejriwal and encouraging voters to vote for the BJP. The second video has been manipulated using deepfake technology so his mouth moves convincingly as he speaks in Haryanvi, the Hindi dialect spoken by the target voters for the BJP.

(tags: deepfakes video fakes campaigning politics india bjp)

Kashmiri cops filed a case against people using VPN and social media

The police in the Jammu and Kashmir region have issued an “FIR” against individuals using social media via a VPN; “army personnel were allegedly checking phones to see if the owners had any VPN apps installed.” Critics have regularly argued that this partial access to the internet still counts as an internet shutdown.

(tags: vpns future grim-meathook-future politics censorship india kashmir social-media)

The Heartland Lobby

A joint investigation from CORRECTIV and Frontal21 reveals how the American Heartland Institute is supporting climate change deniers in Germany with the goal of undermining climate protection measures:

Throughout the next half hour, Taylor shares the inner workings of his disinformation toolbox. He believes that Mathias, the PR agent sitting opposite him, wants to help his clients funnel cash into the intricate network of climate change deniers. Taylor explains how he is able to raise awareness of topics in exchange for money, how people can make tax-deductible donations anonymously through a U.S. foundation, and how the Institute’s publications mimic the tone of the New York Times so obscure ideas are taken more seriously. He detailed how he intends to make a young YouTuber from Germany the star of climate denier, and how he works closely with German partners whose ideas are consistently cited by the AfD in the Bundestag. Then a few weeks later, Taylor will send an offer in writing. It is something like a strategy document for a PR campaign in Germany: A campaign that the public will not recognize for what it really is, making it even more effective. The goal: No more prohibitive climate laws. Diesel instead of electric cars, energy from coal instead of wind turbines, industry growth instead of environmental protection. 

(tags: heartland-institute germany lobbying astroturfing misinformation disinformation climate-change climate-denial)

Amazon EBS Multi-Attach now available on Provisioned IOPS io1 volumes

Attach multiple EC2 instances to the same EBS volume. Now that is pretty cool

(tags: ebs ec2 filesystems networking ops)

excellent letter to the editor of the Farmer’s Journal regarding the IFA’s climate-denialist stance

in full:

Dr Donal Murphy-Bokern M.Agr.Sc. (NUI), Kroge-Ehrendorf, Germany Dear Sir: I’ve been involved in reseach on diet, sustainable agriculture and climate change for 25 years. Having followed the public debate across Europe in that time, I can only describe the current debate about diet and greenhouse gas emissions in Ireland as hysterical. This hysteria started a year ago with the then Irish Farmers Association’s president appearing to refer to the EAT Lancet Commission, which includes highly respected nutritionists from the Harvard School of Medicine, as “quacks masquerading as nutrition experts”. This was followed by his condemnation of the Taoiseach for answering a question about his carbon footprint by stating an intention to moderate his consumption of red meat. No vegan-led campaign could have better drawn public attention to the links between diet and environment than the IFA’s boorish and ignorant reflex reactions. The hysteria goes on. Now, just a year later, the IFA’s chosen greenhouse gas “guru” reports that methane from farming should be treated differently to CO,, raising hopes of a get-out-of-jail card for cattle and sheep. Self-description as a guru does not invite the confidence of scientific peers and Dr Mitloehner’s presentation, published by the IFA, reveals why he is as controversial as is widely reported. Methane’s short-lived nature does not lead to the public policy outcomes that he implies it should with climate acquittal for ruminant production. He reduced discussion about the impact of livestock to one currency, which is carbon, and then misrepresented the valuation of that currency. Despite being a native of Germany, where most land not suitable for arable crops is under forest, he argued that marginal land in Ireland cannot be used for anything other than for keeping cattle and sheep. But what was most striking about the IFA’s guru is how he worked the audience using rhetorical tricks more associated with demagogic politicians than science. This science denial included using the strawman fallacy, raising and then countering several bogus opposing arguments. Listening to him, one could be forgiven for believing that vegans have been protesting on the streets of Dublin threatening to interfere with the nation’s food supplies. He used the classical conspiracy theory complete with a collective name for the conspirators: “destructors”. He then drew on popular images of Ireland (“green and lush” and “happy cows”) to ingratiate himself with the audience while making wild and poorly informed assumptions about the scope for carbon sequestration on Irish grassland, displaying a poor understanding of basic soil science. The IFA’s stated purpose was the rebalancing of the public debate. Hosting a controversial US scientist who refers to those with views different to those of the IFA on these matters as “destructors” is hardly a promising way forward. The IFA seems to continue to take pride in caring little for the concerns and expectations of the wider society upon which the real long-term interests of its members ultimately depend. Their faux-militancy might go down well with some members, but it now risks presenting Irish farmers as environmental and social pariahs.

(tags: letters farmers-journal farming ifa ireland climate-change climate-denialism)

Shazam’s audio search algorithm

‘a combinatorially-hashed time-frequency constellation analysis of the audio’ [pdf] (via papers we love)

(tags: music shazam search audio algorithms papers pdf via:papers-we-love)

Cheap PC hardware watchdog

Nelson bought a super-cheap, super-simple AliExpress thingy:

It looks like a USB device, but the USB is only for power. The main I/O are two pairs of wires: one that connects to your hard drive activity LED, one that connects to your hardware reset switch. Yes, it’s that dumb. Basically it just watches the LED and if it hasn’t flashed in awhile (no idea how long, maybe a minute?) it sends a reset to the motherboard.

(tags: via:nelson watchdogs hardware gadgets reliability usb)

See how climate change has impacted the world since your childhood

Fantastic (albeit terrifying) dataviz work from Oz’s ABC News

(tags: australia environment visualization climate climate-change future dataviz abc terrifying)

News media article tended to focus on e-cigarette risks, rather than potential benefits

This has implications for cigarette smokers trying to quit the habit:

News media may influence public perceptions and attitudes about electronic cigarettes (e-cigarettes), which may influence product use and attitudes about their regulation. The purpose of this study is to describe trends in US news coverage of e-cigarettes during a period of evolving regulation, science, and trends in the use of e-cigarettes. [….] Across years, articles more frequently mentioned e-cigarette risks (70%) than potential benefits (37.3%).

(tags: media news smoking cigarettes vapes e-cigarettes news-media)

Radbot

This seems very clever — replace traditional central heating radiator thermostatic regulation valves (TRVs) with “Radbot” TRVs, for energy efficiency: ‘Extensive testing of Radbot in both controlled laboratory conditions and field trials have demonstrated it is possible to save up to 30% of your heating energy per radiator. 4-5 Radbots installed in the average sized house can save up to 30% of your energy bill.’ The Radbot detects your presence, and turns down rads in unoccupied rooms, turning them up again when you return.

(tags: radbot trvs radiators heating house home gadgets energy)

12 Signs You’re Working in a Feature Factory

I’ve used the term *Feature Factory *at a couple conference talks over the past two years. I started using the term when a software developer friend complained that he was “just sitting in the factory, cranking out features, and sending them down the line.”

(tags: features product-management agile teams work management product companies prioritization planning)

The false promise of “renewable natural gas”

RNG [renewable natural gas] can, depending on feedstock and circumstances, be low or even zero-carbon. Utilities argue that ramping up the production of RNG and blending it with normal natural gas in pipelines can reduce [greenhouse gases] faster and cheaper than electrifying buildings. By pursuing electrification, they say, regulators are pushing unnecessary cost hikes onto consumers. It would be nice for the utilities if this were true. But it’s not. RNG is not as low-carbon as the industry claims and its local air and water impacts are concentrated in vulnerable communities. Even if it were low-carbon and equitable, there simply isn’t enough of it to substitute for more than a small fraction of natural gas. And even if it were low-carbon, equitable, and abundant, it still wouldn’t be an excuse to expand natural gas infrastructure or slow electrification. It isn’t a close call. The research is clear: Especially in a temperate climate like California, RNG is not a viable alternative for decarbonizing buildings. It is a desperate bid by natural gas utilities to delay their inevitable decline. Policymakers would be foolish to fall for it.

(tags: decarbonization carbon climate-change rng renewables natural-gas pollution environment)

Opinion: Why has the State invested €70m in a private company to look at our genetic data?

In the UK, the publicly-funded 100,000 Genomes Project is attempting to sequence 100,000 genomes from 85,000 NHS patients. It is a private company, owned by the Department of Health and Social Care, that partners with industry and has transparent policies in place on ethics, access to the genetic data and engagement with patients and the public. Ireland too has decided to invest in genomic medicine. Rather than ensure that this investment is in a manner that best serves the Irish public, €73.5 million was given to Genomic Medicine Ireland (GMI), a company owned by the Chinese pharmaceutical company WuXi with zero public ownership, to sequence the genomes of 400,000 Irish people. This investment has serious legal and ethical concerns that are likely to negatively impact genomic research in Ireland.

(tags: ireland genomics genomes medicine health future china wuxi gmi)

An app can be a home-cooked meal

Fantastic — this iOS user cloned the best bits of Tapstack, an app which I similarly mourned when it was shut down last year. Unfortunately his version is iOS-only, and quite closed for his family — by contrast, we just moved to a private Telegram group

(tags: programming mobile coding family friends social-networking tapstack apps)

Wikipedia turned to WebAssembly to provide patent-free video

‘Wikipedia turned to WebAssembly as a <video> polyfill because video codec patents are a pain for folks committed to fully open source stacks: ‘ogv.js implements Ogg Vorbis/Opus/Theora audio & WebM VP8/VP9/AV1 video. https://github.com/brion/ogv.js/’

(tags: ogv.js ogv webassembly wasm wikipedia polyfills standards video patents)

Why People Say ‘Up the RA’ – VICE

tl;dr: young people.

The difference between young people and their parents’ relationship with Irish Republicanism appears even more pronounced when studying the Irish establishment media, which has failed to acknowledge the widespread understanding that Republican slogans have been denuded of militaristic connotations by most people who use them. In March of last year, as Irish meme-lords continued to post a zesty mixture of IRA, Republican and Gerry Adams memes ad nauseum (some even appearing on Sinn Fein’s official social media pages), Mary Lou McDonald was being slated in the Irish press for saying “tiocfaidh ár lá” during a speech at a party conference.

(tags: republicanism ireland ira history sinn-fein memes vice slogans)

The Truth Behind The Theory That Control Was Inspired By The SCP Foundation

Yep! it was indeed:

“I just had this warm fuzzy feeling throughout the game, seeing the cultural influence of something I’ve spent eight years of my life kind of doing as a hobby,” Pierce said. “I think in fairness, they clearly had the inspiration [from us], but they took it in their own direction. They did something with it that we could not do in a thousand years.”

(tags: scp scp-wiki wikis collaboration art writing horror science-fiction control games)

How the CIA used Crypto AG encryption devices to spy on countries for decades – Washington Post

The Crypto AG story returns to the headlines once more:

The operation, known first by the code name “Thesaurus” and later “Rubicon,” ranks among the most audacious in CIA history. “It was the intelligence coup of the century,” the CIA report concludes. “Foreign governments were paying good money to the U.S. and West Germany for the privilege of having their most secret communications read by at least two (and possibly as many as five or six) foreign countries.”

During the sensitive Anglo-Irish negotiations of 1985, the NSA’s British counterpart, GCHQ, was able to decipher the coded diplomatic traffic being sent between the Irish embassy in London and the Irish Foreign Ministry in Dublin. It was reported in the Irish press that Dublin had purchased a cryptographic system from Crypto AG worth more than a million Irish pounds. It was also reported that the NSA routinely monitored and deciphered the Irish diplomatic messages.

(tags: cryptography us nsa gchq crypto-ag surveillance cia spying spies)

The sustainable fashion conversation is based on bad statistics and misinformation – Vox

I pulled all of these statistics and other common “facts” from reputable sources. McKinsey. The United Nations. The Ellen MacArthur Foundation. The World Bank. International labor unions. Advocacy organizations. And these facts have been cited by publications like the Wall Street Journal and the New York Times. Not all of these highly respected experts could be wrong. Could they? It turns out they could. Because only one out of the dozen or so most commonly cited facts about the fashion industry’s huge footprint is based on any sort of science, data collection, or peer-reviewed research. The rest are based on gut feelings, broken links, marketing, and something someone said in 2003.

(tags: bad-data data facts factoids misinformation fashion fast-fashion climate-change)

the CO2 footprint of email is greatly exaggerated

If you care about the environmental impact of tech, worrying about email is not the place to spend your time and energy. Worry instead about the big tech companies accelerating the extraction of fossil fuels, when we need to keep them in the ground. [….] Worry instead about consulting companies you admire doing the same, and helping the same oil and gas companies, but keeping quiet about doing so. Worry about how blase we are about flying when it makes up a significant chunk of company emissions in many tech consultancies and enterprise sales teams.

(tags: climate-change email factoids misinformation carbon)

Circllhist

‘A Log-Linear Histogram Data Structure for IT Infrastructure Monitoring, Heinrich Hartmann, Theo Schlossnagle, (Submitted on 17 Jan 2020). The circllhist histogram is a fast and memory efficient data structure for summarizing large numbers of latency measurements. It is particularly suited for applications in IT infrastructure monitoring, and provides nano-second data insertion, full mergeability, accurate approximation of quantiles with a-priori bounds on the relative error. Open-source implementations are available for C/lua/python/Go/Java/JavaScript.’ The paper compares it against ‘alternative data-structures which are employed in practice for aggregated quantile calculations: Prometheus Histograms, t-digest, [Gil Tene’s] HDR Histograms, and DDSketches’

(tags: histograms aggregation quantiles percentiles measurement graphs data-structures summaries latency monitoring approximation papers)

Cubism.js

A minimalist dashboard style using horizon charts:

Horizon charts reduce vertical space without losing resolution. Larger values are overplotted in successively darker colors, while negative values are offset to descend from the top. As you increase the number of colors, you reduce the required vertical space […] . By combining position and color, horizon charts improve perception: position is highly effective at discriminating small changes, while color differentiates large changes. To further increase data density, Cubism favors per-pixel metrics where each pixel encodes a distinct point in time. Cubism also includes thoughtful default colors by Cynthia Brewer.

(tags: charts javascript visualization d3 charting graphs horizon-charts ui monitoring)

How can data centers use 100% renewable electricity?

The first step has been to offset. This is followed by matching usage with like-for-like energy purchases somewhere. The final stage is direct consumption of locally generated renewables, either in real time or stored from recent generation. So the next time you see a tech company announcing a huge renewables project, you should look to see exactly what that mean and where that energy will really go. New renewables are good, but whether that energy is actually powering the company operations directly is another question.

(tags: datacenters renewables energy power climate-change green offsetting)

Critical Bluetooth Vulnerability in Android (CVE-2020-0022) – BlueFrag

On Android 8.0 to 9.0, a remote attacker within proximity can silently execute arbitrary code with the privileges of the Bluetooth daemon as long as Bluetooth is enabled. No user interaction is required and only the Bluetooth MAC address of the target devices has to be known. For some devices, the Bluetooth MAC address can be deduced from the WiFi MAC address. This vulnerability can lead to theft of personal data and could potentially be used to spread malware (Short-Distance Worm). On Android 10, this vulnerability is not exploitable for technical reasons and only results in a crash of the Bluetooth daemon.

(tags: bluetooth android security exploits worms)

Tesla Remotely Removes Autopilot Features From Customer’s Used Tesla Without Any Notice [Updated]

wow, this is scummy

(tags: tesla scams downgrades remote-upgrade cars driving)

Advancing safe deployment practices in Azure

MS describe their Azure rollout strategy — canary regions -> pilot phase with hardware diversity -> early regions, building up to larger regions -> full deployment.

(tags: deployment azure microsoft rollout ops safety outages)

Twitter Likes Deleter

Now that there are creepy companies tracking every tweet you’ve “liked”, here’s a Glitch app which will revert those Likes en masse

(tags: likes twitter social-media snooping via:anildash privacy)

Green Investing / Finance

a few links for European green users, courtesy of Phil Sturgeon

(tags: via:philsturgeon investing investment finance green climate money)

3D Printer Bed Leveling Guide

good advice for 3D printer users

(tags: levelling 3d-printing make tips advice filament)

Here Are the Most Common Airbnb Scams Worldwide

Oh so many scams :(

(tags: airbnb scams housing holidays vacation rental fraud vacation-rentals)

Health-Records Company Pushed Opioids to Doctors in Secret Deal – Bloomberg

This is APPALLING. holy crap.

To doctors opening patients’ electronic records across the U.S., the alert would have looked innocuous enough. A pop-up would appear, asking about a patient’s level of pain. Then, a drop-down menu would list treatments ranging from a referral to a pain specialist to a prescription for an opioid painkiller. Click a button, and the program would create a treatment plan. From 2016 to spring 2019, the alert went off about 230 million times. The tool existed thanks to a secret deal. Its maker, a software company called Practice Fusion, was paid by a major opioid manufacturer to design it in an effort to boost prescriptions for addictive pain pills — even though overdose deaths had almost tripled during the prior 15 years, creating a public-health disaster. The software was used by tens of thousands of doctors’ offices.

(tags: healthcare capitalism opioids health-records pain painkillers addiction practice-fusion)

How to Actually Personally Fight Climate Change – Erika Reinhardt

These are concrete, practical suggestions that it’s possible for a normal person to achieve — do them!

Mitigating the climate crisis is top of mind for many people. But it’s such a complex issue that it can be hard to distinguish between data-backed improvements and feel-good distractions. This is your action list with lots of context along the way on why not just how so you can soon be an emissions-fighting climate superhero. If you want to get started by just running through and checking off the easy items, start here.

(tags: climate-change green-living future climate carbon tips advice todo)

Climate Change Could Force Millions of Americans to Flee the Coast. AI Predicts Where They’ll Go

By the end of the century, sea level rise could force 13 million people to move away from the U.S. coasts. But it’s not just the coasts that will be affected—so will the places where those migrants end up. In a study published last week in PLOS One, researchers used artificial intelligence to predict where those places are. The findings could have huge value to people not only living on the coast, but the communities that may deal with an influx of climate refugees inland over the coming century. “Our findings indicate that everybody should care about sea-level rise, whether they live on the coast or not,” Bistra Dilkina, a Computer Science Assistant Professor at the University of Southern California who led the study, said in a statement.

(tags: climate-change migration papers climate ai future refugees)

Online Laser Cutting & Engraving

Ponoko provides laser cutting & engraving services to turn your designs into custom products. You select from 99+ beautiful materials, download our design template, add your design to it, then upload it to get an instant online quote to make your design real. Pricing starts from $1. You can make 1 or 100,000. And your designs are made & delivered as fast as same day.

(tags: diy printing 3d 3d-printing cnc laser-cutting engraving making maker)

Why cancer-spotting AI needs to be handled with care

“There’s this idea in society that finding more cancers is always better, but it’s not always true,” Adewole Adamson, a dermatologist and assistant professor at Dell Medical School, tells The Verge. “The goal is finding more cancers that are actually going to kill people.” But the problem is “there’s no gold standard for what constitutes cancer.” As studies have found, you can show the same early-stage lesions to a group of doctors and get completely different answers about whether it’s cancer. And even if they do agree that that’s what a lesion shows — and their diagnoses are right — there’s no way of knowing whether that cancer is a threat to someone’s life. This leads to overdiagnosis, says Adamson: “Calling things cancer that, if you didn’t go looking for them, wouldn’t harm people over their lifetime.” As soon as you do call something cancer, it triggers a chain of medical intervention that can be painful, costly, and life-changing. In the case of breast cancer, that might mean radiation treatments, chemotherapy, the removal of tissue from the breast (a lumpectomy), or the removal of one or both breasts entirely (a mastectomy). These aren’t decisions to be rushed. Overdiagnosis, he says, “is a problem for a lot of different cancers; for prostate, melanoma, breast cancer, thyroid. And if AI systems become better and better at finding smaller and smaller lesions you will manufacture a lot of pseudo-patients who have a ‘disease’ that won’t actually kill them.”

(tags: overdiagnosis health medicine cancer computer-vision automation ai google diagnosis)

UTC or GTFO

a laptop sticker to live by

(tags: utc gtfo time coding funny stickers laptop)

REvil Ransomware

Kevin Beaumont is calling this ‘totally out of control’; ‘the quiet cover up by companies paying ransoms is creating advanced attackers operating at a skill and capability which are going to be very difficult to defend against’:

We’ve seen 150 000 unique infections in the past 5 months. And a total of 148 samples together demanding more than 38 million dollars. Some of the attacks are on a huge scale, encrypting over 3000 unique systems in one attack. Some of these attacks where discussed in the news, but many companies remained silent. Keep in mind we have a limited visibility of all samples; we only extract samples from pastebin. For the infection traffic we don’t have visibility on samples that disable the C2 traffic. Next to this not every sample hits all of the c2 domains. All statistics shown in this blog are a subset of the total scale. The actual problem is even bigger than we can measure. [….] With the rise of more mature and big malicious business relaying on ransomware it is apparent that infosec plays crucial role. The most important step we as a security industry is secure offsite backups that are not removable from the network or using privileges acquired within the network. After that we can spend time actually securing our networks.

(tags: revil ransomware security malware ransoms via:gossi)

Climate Strike Software License

The key bit:

The Software may not be used in applications and services that are used for or aid in the exploration, extraction, refinement, processing, or transportation of fossil fuels. The Software may not be used by companies that rely on fossil fuel extraction as their primary means of revenue. This includes but is not limited to the companies listed at https://climatestrike.software/blacklist

(tags: climate activism climate-change fossil-fuels energy open-source oss licensing)

Food types by CO2 footprint

You want to reduce the carbon footprint of your food? Focus on what you eat, not whether your food is local:

For most foods – and particularly the largest emitters – most GHG emissions result from land use change (shown in green), and from processes at the farm stage (brown). Farm-stage emissions include processes such as the application of fertilizers – both organic (“manure management”) and synthetic; and enteric fermentation (the production of methane in the stomachs of cattle). Combined, land use and farm-stage emissions account for more than 80% of the footprint for most foods. Transport is a small contributor to emissions. For most food products, it accounts for less than 10%, and it’s much smaller for the largest GHG emitters. In beef from beef herds, it’s 0.5%. Not just transport, but all processes in the supply chain after the food left the farm – processing, transport, retail and packaging – mostly account for a small share of emissions.

(tags: co2 food data farming carbon emissions climate-change methane transport locavores)

Use ALB auth to add user authentication

AWS now allows services to be secured using ‘Cognito User Pool (comes with a built-in user database and supports user federation (Google, Facebook, SAML, OICD, …)’, or OpenID Connect (OICD) which ‘integrates with any OICD-compliant identity provider.’

(tags: aws alb authentication load-balancers openid google saml auth)

Making a green internet with the Green Web Foundation

The tech sector is responsible for 2% to 4% of global emissions today. That’s less than all automobile transport, but roughly comparable to the global emissions of all shipping, or aviation. [….] The problem is that even as our electricity grids transition to more sustainable sources of energy, by dropping coal in favour of renewables, for instance, this doesn’t automatically mean we’re getting a much greener internet. That’s partly because the internet, while distributed around the world, is not evenly distributed. If you were to look at a map of all the major infrastructures of the internet, you’d see that it clusters around a number of geographic features. The reason behind this is that there is a cost, both in time and money, to move data around the world, and even though that cost dropped over time, the rate that we generate and use data for processing has grown faster than this cost has dropped. This creates incentives to increase the amount of infrastructure in a few places, rather than distribute it evenly. So, where we’ve previously seen data centres built in places with good access to fossil fuel energy, and in a regulatory environment that favours established fossil fuel industries over renewables, you’ll often see even more internet infrastructure being built, often using the same kinds of ‘grey’ power mixes. The best example of this is the Data Centre Alley in North Virginia, USA. Here, the county of Loudoun boasts that 70% of the world’s internet traffic passes through its digital infrastructure. With 13.5 million square feet of data centres in use, and another 4.5 million planned or developed, it’s the largest concentration of infrastructure in the world. Most of the power needed for this data centre comes from a single company, Dominion Energy, which runs a particularly dirty energy mix, with most of its energy coming from fracked gas, coal and nuclear power. Less than 5% comes from renewables, and this figure will barely pass 10% by 2030.

(tags: green climate-change datacenters energy power renewables north-virginia internet carbon)

“Cyber Rambo”: How a US Army vet aided the right-wing coup in Bolivia

Twitter’s turning out to be a shitfest of a platform:

Julián Macías Tovar, a social media coordinator for the Spanish left-wing party Podemos, analyzed the data from the hashtags and found that thousands of accounts were created in the days before the election and spiked after Bolivian military leaders called on Morales to resign. According to his data, 48,000 accounts were created in a matter of just four days to amplify the hashtags. Tovar also discovered that a single account contributed more than 13,000 retweets to the hashtags. The account belonged to US Army veteran Luis Suarez, who automated his account to retweet posts with the hashtags using a custom app. Data scientist Rubén Rodríguez Casañ similarly found that Suarez’s account was able to retweet as many as 69 posts in a single second.

(tags: twitter hashtags bolivia propaganda botnets bots coups)

Steve Bannon on shitposting

Steve Bannon to Michael Lewis: “The Democrats don’t matter. The real opposition is the media. And the way to deal with them is to flood the zone with shit.” This is actually a remarkable comment, as it puts on paper what has been the Tory/Republican tactic — snowing the media under with bullshit, so they lose track of the important stuff and start rattling on about trivial shitposts like Big Ben bonging or whatever.

(tags: shitposting distraction tactics steve-bannon trump tories politics misinformation disinformation)

BurntSushi/xsv

a command line program for indexing, slicing, analyzing, splitting and joining CSV files. Commands should be simple, fast and composable: Simple tasks should be easy. Performance trade offs should be exposed in the CLI interface. Composition should not come at the expense of performance.

(tags: rust csv cli tools data xsv command-line unix)

A Review of Netflix’s Metaflow

Metaflow looks nice, and used by $work’s data scientists

(tags: metaflow data-science data batch architecture)

XGBoost

‘an optimized distributed gradient boosting library designed to be highly efficient, flexible and portable. It implements machine learning algorithms under the Gradient Boosting framework. XGBoost provides a parallel tree boosting (also known as GBDT, GBM) that solve many data science problems in a fast and accurate way. The same code runs on major distributed environment (Hadoop, SGE, MPI) and can solve problems beyond billions of examples.’

(tags: python xgboost gradient-boosting ml machine-learning mpi)

Historic S3 data corruption due to a fault load balancer

This came up in a discussion of using hashes for end-to-end data resiliency on the og-aws slack. Turns out AWS support staff wrote it up at the time:

We’ve isolated this issue to a single load balancer that was brought into service at 10:55pm PDT on Friday, 6/20 [2008].  It was taken out of service at 11am PDT Sunday, 6/22.  While it was in service it handled a small fraction of Amazon S3’s total requests in the US.  Intermittently, under load, it was corrupting single bytes in the byte stream.  When the requests reached Amazon S3, if the Content-MD5 header was specified, Amazon S3 returned an error indicating the object did not match the MD5 supplied.  When no MD5 is specified, we are unable to determine if transmission errors occurred, and Amazon S3 must assume that the object has been correctly transmitted. Based on our investigation with both internal and external customers, the small amount of traffic received by this particular load balancer, and the intermittent nature of the above issue on this one load balancer, this appears to have impacted a very small portion of PUTs during this time frame. One of the things we’ll do is improve our logging of requests with MD5s, so that we can look for anomalies in their 400 error rates.  Doing this will allow us to provide more proactive notification on potential transmission issues in the future, for customers who use MD5s and those who do not. In addition to taking the actions noted above, we encourage all of our customers to take advantage of mechanisms designed to protect their applications from incorrect data transmission.  For all PUT requests, Amazon S3 computes its own MD5, stores it with the object, and then returns the computed MD5 as part of the PUT response code in the ETag.  By validating the ETag returned in the response, customers can verify that Amazon S3 received the correct bytes even if the Content MD5 header wasn’t specified in the PUT request.  Because network transmission errors can occur at any point between the customer and Amazon S3, we recommend that all customers use the Content-MD5 header and/or validate the ETag returned on a PUT request to ensure that the object was correctly transmitted.  This is a best practice that we’ll emphasize more heavily in our documentation to help customers build applications that can handle this situation.

(tags: aws s3 outages postmortems load-balancing data-corruption corruption failure md5 hashing hashes)

Expert reaction to World Health Organisation Q&A on e-cigarettes

It does seem that scaremongering about vaping is hurting efforts to get people off cigarettes:

“Practically all the factual statements in it are wrong. There is no evidence that vaping is ‘highly addictive’ – less than 1% of non-smokers become regular vapers.  Vaping does not lead young people to smoking – smoking among young people is at all time low.  There is no evidence that vaping increases risk of heart disease or that could have any effect at all on bystanders’ health. The US outbreak of lung injuries is due to contaminants in illegal marijuana cartridges and has nothing to do with nicotine vaping. There is clear evidence that e-cigarettes help smokers quit. “The authors of this document should take responsibility for using blatant misinformation to prevent smokers from switching to a much less risky alternative.”

(tags: cigarettes smoking vaping addiction health medicine scaremongering who cancer)

The No Code Movement

‘No code is the best way to write secure and reliable applications. Write nothing; deploy nowhere.’

(tags: coding no nocode funny true)

Star-Tree Index: Powering Fast Aggregations on Pinot | LinkedIn Engineering

An interesting new indexing technique for multi-dimensional data set queries, where you can predefine the _order_ of query dimensions:

With such huge improvements for both latency and throughput, the Star-Tree index only costs about 12% extra storage space compared to data without indexing techniques and 6% extra compared to data with inverted index.

(tags: star-tree sql querying search pinot linkedin algorithms databases indexing indexes)

Boing Boing is 20 (or 33) years old today.

Wow. happy birthday from this happy mutant

(tags: boing-boing blogs history 1990s zines)

People + AI Guidebook

“Designing human-centered AI products”. Some good UX recommendations when working with AI smarts behind the scenes

(tags: ux ai design google graphics tips machine-learning)

Revealed: betting firms use schools data on 28m UK children

So much for “strict privacy rules”:

Betting companies have been given access to an educational database containing names, ages and addresses of 28 million children and students in one of the biggest breaches of government data. They have used it to help increase the proportion of young people who gamble online. It contains details of children age 14 and above in state schools, private schools and colleges in England, Wales and Northern Ireland.

(tags: privacy data-protection children kids schools uk betting gambling)

Experiencing WSL as a Linux Veteran

Sounds like they’ve done a great job at integration

(tags: linux windows wsl desktop unix cli)

Microsoft announces it will be carbon negative by 2030

This is *amazing* news, and really puts it up to the other big tech companies, particularly Google and Amazon: carbon negative by 2030, more responsibility for Scope 3 emissions, 100% renewables by 2025, and a $1billion fund for climate tech.

(tags: climate-change microsoft good-news carbon tech)

Snowboy Hotword Detection

Open-source, Apache-license hotword detection library for homebrew IoT: ‘Snowboy is an highly customizable hotword detection engine that is embedded real-time and is always listening (even when off-line) compatible with Raspberry Pi, (Ubuntu) Linux, and Mac OS X. Currently, Snowboy supports: all versions of Raspberry Pi (with Raspbian based on Debian Jessie 8.0) 64bit Mac OS X 64bit Ubuntu (12.04 and 14.04) iOS Android with ARMv7 CPUs Pine 64 with Debian Jessie 8.5 (3.10.102) Intel Edison with Ubilinux (Debian Wheezy 7.8)’

(tags: audio iot hardware hotwords speech-recognition speech devices)

Facebook Ad Library Showed Just How Unreliable Facebook’s Security System For Elections Is

On Dec. 10, just two days before the United Kingdom went to the polls, some 74,000 political advertisements vanished from Facebook’s Ad Library, a website that serves as an archive of political and issue ads run on the platform. [….] Facebook has said it will not fact-check political ads or restrict the ability for campaigns to target people. Instead, it said it will provide transparency with tools like the Ad Library, the Ad Library report, and the Ad Library API, so the public, researchers, and journalists can monitor how elections play out on the platform. But that only works to the degree that those tools operate properly. It was only the news media’s reporting that brought the issue out into the open. “The fact that they could have an outage like this that went up to the day before an election, and they didn’t really publicly communicate,” Laura Edelson, a computer scientist at NYU whose work involves using the API, told BuzzFeed News, “that’s just not how you treat a security system. That’s what this is — this is a security system for elections.”

(tags: facebook ads politics uk-politics transparency microtargeting social-media)

How is computer programming different today than 20 years ago?

Some good answers:

A desktop software now means a web page bundled with a browser. You are not officially considered a programmer anymore until you attend a $2K conference and share a selfie from there. Code must run behind at least three levels of virtualization now. Code that runs on bare metal is unnecessarily performant. Running your code locally is something you rarely do. A tutorial isn’t really helpful if it’s not a video recording that takes orders of magnitude longer to understand than its text. Mobile devices can now show regular web pages, so no need to create a separate WAP page on a separate subdomain anymore. We create mobile pages on separate subdomains instead. We run programs on graphics cards now. Since we have much faster CPUs now, numerical calculations are done in Python which is much slower than Fortran. So numerical calculations basically take the same amount of time as they did 20 years ago. Storing passwords in plaintext is now frowned upon, but we do it anyway.

(tags: evolution dev programming humour coding lols fortran history)

Record/Replay testing in Sorbet

I do like record/replay tests. +1

(tags: sorbet testing record-replay-testing unit-tests tests)

The Center Blows Itself Up: Care and Spite in the ‘Brexit Election’

The center of British politics has become a smoldering pit. The country is now being governed by a hard-right government placed in power by its oldest citizens, in the face of the active hatred of its increasingly socialist-inclined youth. It’s fairly clear that for the Johnson team, Brexit was never anything but an electoral strategy, and that they don’t have the slightest idea how to translate it into economic prosperity. (It is an unacknowledged irony of the current situation that the people most likely to profit from the Brexit process are, precisely, lawyers—and, probably secondarily, accountants. For everyone else, it’s hard to imagine a scenario where they will improve their current situation, and quite easy to imagine Johnson being remembered as one of the most disastrous prime ministers in British history.)

(tags: labour brexit uk politics tories boris-johnson jeremy-corbyn centrism)

Scary twitter thread on the new US fascist/”radical catholic” crossover movement

The Groyper movement, “Deus Vult”, Nick Fuentes, and “America First”: “two weeks ago […] employees at the Planned Parenthood clinic in Dover, Delaware, watched stunned as protesters who usually gather outside the clinic performed the Nazi salute as patients were going in.” grim.

(tags: groyper fascism america us-politics catholicism religion nazis)

ecotrip

Figure out how to get from A to B with estimates of CO2 emissions involved — ‘Ecotrip is a platform that offers a solution to planes’ large CO2 emission by showing routes from user’s origin point to their desired destination, suggesting alternative means of transport ordered by their level of sustainability.’ works great! Pity it’s almost impossible to get anywhere from Ireland without flying :(

(tags: travel green climate-change co2 emissions maps sustainability via:cat)

How to Run Your ESP8266 for Years on a Battery

Using the hardware’s deepSleep feature. neato

(tags: arduino esp8266 nodemcu power batteries)

The “Smoky Cokey” cocktail tastes exactly how it sounds, and that’s a good thing

Via Ben — here’s a bunch of critics going nuts about a fecking whiskey and coke. ‘The ‘Smoky Cokey’ […] is a Coca-Cola that tastes smoky thanks to the addition of peaty scotch. Specifically, the drink as it was explained to me requires Lagavulin 16, a scotch that sells for around $75 a bottle in the United States. My god the critics go overboard with this: ‘the rich smoke and intensity of Lagavulin is beautifully countered by the sweet vanilla and gentle spices of the cola’ ‘I actually like to make mine with Mexican Coke when I have one handy, because I find the extra sweetness works even better with the fairly aggressive peatiness of something like a Lagavulin’ ‘There’s something about the sherry notes present in the Lagavulin that makes it really sing when drowned in high fructose corn syrup’ My theory is that this is a marketing wheeze devised by Diageo PR to garner a bit of clickbait….

(tags: diageo marketing morkeshing smoky-cokey cocktails trolls whiskey coke lagavulin)

“One of our office chairs turns off monitors”

Crappy unshielded display cables are prone to electrostatic discharges from gas-lift office chairs… “we have also seen this issue connected to gas lift office chairs. When people stand or sit on gas lift chairs, they can generate an EMI spike which is picked up on the video cables, causing a loss of sync. If you have users complaining about displays randomly flickering it could actually be connected to people sitting on gas lift chairs. Again swapping video cables, especially for ones with magnetic ferrite ring on the cable, can eliminate this problem.”

(tags: chairs furniture funny hardware emi esd monitors twitter video)

Disinformation For Hire: How A New Breed Of PR Firms Is Selling Lies Online

If disinformation in 2016 was characterized by Macedonian spammers pushing pro-Trump fake news and Russian trolls running rampant on platforms, 2020 is shaping up to be the year communications pros for hire provide sophisticated online propaganda operations to anyone willing to pay. Around the globe, politicians, parties, governments, and other clients hire what is known in the industry as “black PR” firms to spread lies and manipulate online discourse. A BuzzFeed News review — which looked at account takedowns by platforms that deactivated and investigations by security and research firms — found that since 2011, at least 27 online information operations have been partially or wholly attributed to PR or marketing firms. Of those, 19 occurred in 2019 alone.

(tags: disinformation china propaganda pr disinfo social-media marketing)

How to monitor Golden signals in Kubernetes

Most of this doc is Kubernetes specific, but this “golden signals” idea is interesting; basically, the four metrics of requests per second, average request latency, CPU usage on service fleet, errors per second. I would modify by adding the P99 or P99.9 request latency, and representing errors per second as a proportion of that period’s request-per-second figure.

(tags: kubernetes monitoring sysdig golden-data k8s golden-signals metrics latency errors)

Serving 100µs reads with 100% availability · Segment Blog

Distributing read-only snapshotted SQLite databases to shared volumes works! nifty hack

(tags: architecture databases performance sqlite segment ops docker)

Ironies of automation

Wow, this is a great paper recommendation from Adrian Colyer – ‘Ironies of automation’, Bainbridge, Automatica, Vol. 19, No. 6, 1983.

In an automated system, two roles are left to humans: monitoring that the automated system is operating correctly, and taking over control if it isn’t. An operator that doesn’t routinely operate the system will have atrophied skills if ever called on to take over. Unfortunately, physical skills deteriorate when they are not used, particularly the refinements of gain and timing. This means that a formerly experienced operator who has been monitoring an automated process may now be an inexeperienced one. Not only are the operator’s skills declining, but the situations when the operator will be called upon are by their very nature the most demanding ones where something is deemed to be going wrong. Thus what we really need in such a situation is a more, not a lesser skilled operator! To generate successful strategies for unusual situtations, an operator also needs good understanding of the process under control, and the current state of the system. The former understanding develops most effectively through use and feedback (which the operator may no longer be getting the regular opportunity for), the latter takes some time to assimilate.

(tags: via:allspaw automation software reliability debugging ops design failsafe failure human-interfaces ui ux outages)

Bellingcat’s Online Investigation Toolkit – Google Docs

‘Welcome to Bellingcat’s freely available online open source investigation toolkit […] The list includes satellite and mapping services, tools for verifying photos and videos, websites to archive web pages, and much more. The list is long, and may seem daunting. There are guides at the end of the document, highlighting the methods and use of these tools in further detail.’ (via Damien)

(tags: bellingcat osint mapping archival search image-search geo-search web fact-checking)

Modin: Speed up your Pandas workflows by changing a single line of code

The modin.pandas DataFrame is an extremely light-weight parallel DataFrame. Modin transparently distributes the data and computation so that all you need to do is continue using the pandas API as you were before installing Modin. Unlike other parallel DataFrame systems, Modin is an extremely light-weight, robust DataFrame. Because it is so light-weight, Modin provides speed-ups of up to 4x on a laptop with 4 physical cores. We have focused heavily on bridging the solutions between DataFrames for small data (e.g. pandas) and large data. Often data scientists require different tools for doing the same thing on different sizes of data. The DataFrame solutions that exist for 1KB do not scale to 1TB+, and the overheads of the solutions for 1TB+ are too costly for datasets in the 1KB range. With Modin, because of its light-weight, robust, and scalable nature, you get a fast DataFrame at small and large data. With preliminary cluster and out of core support, Modin is a DataFrame library with great single-node performance and high scalability in a cluster.

(tags: data parallel python pandas dataframes modin data-science)

IAmA: Reddit’s Own Vacuum Repair Tech

some top tips on what to look for in a vacuum cleaner. Bottom line: bagless and stick vacuums are not the best

(tags: reddit vacuum-cleaners shopping tips ama hoovers)

Buckle Up Twitter

Listen up bitches, it’s time to learn incorrect things about someone you’ve never heard of:

I am thinking of the response to February’s “Beau Brummell invented toxic masculinity” episode, in which the 19th-century English fancy man Beau Brummell, as infamous a dandy as one can be, was “taken down” in a grueling thread which neatly encapsulated all the worst qualities of Buckle Up Twitter: bewilderingly irate, laden with a combination of baroque linguistic flourishes and performatively subversive swearing, assumption of complete ignorance on the part of the audience, fondness for the word “gaslighting,” a powerful youth pastor-like eagerness to “meet people where they are,” high likelihood that it will be retweeted by people who refer to themselves as “Scolds” in their twitter bios, strong urge to lay the blame for the ills of the 21st century firmly at the foot of a basically random actor or event, total erasure of most things that have ever happened.

(tags: twitter threads bores social-media funny)

Facial recognition for the public: Yandex

not such much via, as from, Nelson:

You can use Yandex Image Search right now as a pretty good facial recognition system for anyone who has labelled photos on the Web. I believe this is the first generally accessible facial recognition system with a large database. Yandex isn’t designed for this purpose. The trick is to upload photos cropped to a face and it’ll work more or less to find similar faces.

(tags: privacy face-recognition yandex search similarity images web)

How “special register groups” invaded computer dictionaries for decades

For some reason, a 1960 definition of [a computer’s] “central processing unit” included “special register groups”, an obscure feature from the Honeywell 800 mainframe. This definition was copied and changed for decades, even though it doesn’t make sense. It appears that once something appears in an authoritative glossary, people will reuse it for decades, and obsolete terms may never die out.

(tags: computer computing language history etymology mainframe honeywell cpu dictionaries)

massive Travelex outage

The holiday money exchange site has been offline for the past 7 days, reportedly due to a ransomware infection, with 5GB of PII data exfiltrated

(tags: travelex fail security exploits ransomware malware outages)

SHA-1 is a Shambles – First Chosen-Prefix Collision on SHA-1 and Application to the PGP Web of Trust

Abstract: The SHA-1 hash function was designed in 1995 and has been widely used during two decades. A theoretical collision attack was first proposed in 2004 [WYY05], but due to its high complexity it was only implemented in practice in 2017, using a large GPU cluster [SBK+17]. More recently, an almost practical chosen-prefix collision attack against SHA-1 has been proposed [LP19]. This more powerful attack allows to build colliding messages with two arbitrary prefixes, which is much more threatening for real protocols. In this paper, we report the first practical implementation of this attack, and its impact on real-world security with a PGP/GnuPG impersonation attack. We managed to significantly reduce the complexity of collisions attack against SHA-1: on an Nvidia GTX 970, identical-prefix collisions can now be computed with a complexity of 261.2261.2 rather than 264.7264.7, and chosen-prefix collisions with a complexity of 263.4263.4 rather than 267.1267.1. When renting cheap GPUs, this translates to a cost of 11k US\$ for a collision, and 45k US\$ for a chosen-prefix collision, within the means of academic researchers. Our actual attack required two months of computations using 900 Nvidia GTX 1060 GPUs (we paid 75k US\$ because GPU prices were higher, and we wasted some time preparing the attack). Therefore, the same attacks that have been practical on MD5 since 2009 are now practical on SHA-1. In particular, chosen-prefix collisions can break signature schemes and handshake security in secure channel protocols (TLS, SSH). We strongly advise to remove SHA-1 from those type of applications as soon as possible. We exemplify our cryptanalysis by creating a pair of PGP/GnuPG keys with different identities, but colliding SHA-1 certificates. A SHA-1 certification of the first key can therefore be transferred to the second key, leading to a forgery. This proves that SHA-1 signatures now offers virtually no security in practice. The legacy branch of GnuPG still uses SHA-1 by default for identity certifications, but after notifying the authors, the modern branch now rejects SHA-1 signatures (the issue is tracked as CVE-2019-14855).

(tags: via:fanf security sha sha-1 crypto hashes hashing pgp gpg collisions)

Algorithms interviews: theory vs. practice

Good critique of the current practice of using algorithm questions during tech interviews from Dan Luu

At this point, we’ve gone through a few decades of programming interview fads, each one of which looks ridiculous in retrospect. Either we’ve finally found the real secret to interviewing effectively and have reasoned our way past whatever roadblocks were causing everybody in the past to use obviously bogus fad interview techniques, or we’re in the middle of another fad, one which will seem equally ridiculous to people looking back a decade or two from now. Without knowing anything about the effectiveness of interviews, at a meta level, since the way people get interview techniques is the same (crib the high-level technique from the most prestigious company around), I think it would be pretty surprising if this wasn’t a fad. I would be less surprised to discover that current techniques were not a fad if people were doing or referring to empirical research or had independently discovered what works.

(tags: interviews interviewing hiring tech software jobs fads algorithms dan-luu)

Testing in Production: How we combined tests with monitoring

The Guardian Digital team’s write-up on their “test in prod” setup — post-release monitoring through running integration test suites. We do the same in Swrve, calling our suites the “canary tests”, and it works really well for us.

(tags: testing monitoring ops devops the-guardian prod production releases)

Power Line Adapter noise interference

oh dear, I use this model….

About 3 weeks ago our neighbour installed power line adapters. The PLAs in question were branded TP-Link [….] How did I know that my neighbour had installed these? Well, the 50MHz band was immediately submerged under a wall of radio noise. Much tinkering with the Noise Blanker settings on the Icom IC-7300 allowed me to separate out two distinct types of noise – 1st a sound like a chicken clucking which was there 24 hours per day and – 2nd a wideband swoosh of white noise of varying strength which happened at certain times.

(tags: noise rf wifi powerline networking home hardware radio)

City maps from tourists’ feelings

This is fascinating, and potentially quite useful — although the great loft I stayed in in Antwerp is marked in a decidedly yellowish region :) (via Nelson)

The aim of this project is to map tourists’ perceptions of different urban areas through data retrieved from vacation rental platform Airbnb. After their stay, Airbnb guests score their feeling about the neighbourhood using a star-based rating system. The aggregated rating of each Airbnb listing is publicly accessible, and given the widespread expansion of this platform, a large amount of data is available for the most visited cities. When overlaid on a map of the city, the data reveals interesting geographic patterns and exposes subjective perceptions on safety, upkeep or convenience. — Beñat Arregi

(tags: airbnb dataviz maps mapping via:nelson data tourism europe vacations holidays)

Home Automation Bargain Alerts thread at boards.ie

in case I need to fill my house with IOT tat

(tags: iot tat home-automation home gadgets bargains boards)

New Left Review – Mike Davis: Who Will Build the Ark?

Mike Davis’ 2010 essay predicting a failure of climate change mitigation – then rebutting himself in the second half

(tags: mike-davis climate-change climate politics)

Prof John Byrne: the man who turned Ireland into a tech world power

TK Whitaker may be known as the man who made modern Ireland, but the highly respected civil servant wasn’t the only person who helped make the State what it is today. For those who wonder how Ireland came to excel both at luring the biggest and best tech companies to set up here and at producing a good few homegrown tech heroes, a great deal of credit must go to Prof John Byrne, the man who helped kickstart a revolution.

(tags: tcd software ireland work history computer-science)

Alice Goldfuss clarifies JK Rowling’s “dress however you please” anti-trans comments

This was a really educational thread for me — demonstrating how these phrases are a symptom, not support

JK Rowling hates trans people, but I want to talk a little bit about the *way* she hates them so you can recognize it in the wild. She says “Dress however you please” and “Call yourself whatever you like” At first glance it sounds supportive, but it isn’t It’s disengaging She is taking a very real and concrete issue (the acceptance of trans people as humans with full rights and respect in society), minimizing it to some surface level features (appearance and names), and then abdicating any responsibility She is purposefully mischaracterizing trans people as adults playing dress-up and then claiming to be fine with that…at a distance The language she uses is similar to language used to minimize gay people “You can kiss whoever you want behind closed doors!” It’s also similar to the old favorite “I don’t care if you’re black, white, or purple!” Purple people don’t exist, but now they’ve minimized the issue of racism and swept it away while claiming to be supportive All of these phrases add up to the same message: “I support you, as long as you don’t change my experiences or inconvenience me in any way.” And that’s not actual support

(tags: trans rights jk-rowling gender acceptance racism)

Xor Filters: Faster and Smaller Than Bloom Filters

A new immutable probabilistic set data structure, derived from Bloomier Filters, by Daniel Lemire and Thomas Mueller Graf. Lots of sample implementations, looks very useful!

(tags: algorithms coding performance bloom-filters xor-filters data-structures)

Room to Breathe: My Quest to Clean Up My Home’s Filthy Air

The air quality in your home is probably terrible, if this is anything to go by :O

(tags: air air-quality particulates pm2.5 pm10 health paranoia homes)

J.K. Rowling’s transphobia is a product of British culture

Good explainer on why the UK is so TERFy these days:

Trans-exclusionary radical feminist (TERF) ideology has been helped along in the UK by media under the leadership of Rupert Murdoch and the Times of London for years. Any vague opposition to gender-critical thought in the UK brings accusations of “silencing women” and a splashy feature or op-ed in a British national newspaper. Australian radical feminist Sheila Jeffreys went before the UK Parliament in March 2018 and declared that trans women are “parasites,” language that sounds an awful lot like Donald Trump speaking about immigrants. According to Heron Greenesmith, who studies the modern gender-critical movement as a senior research associate at the social-justice think tank Political Research Associates, gender-critical feminism in the UK grew out of a toxic mix of historical imperialism and the influence of the broader skeptical movement in the early aughts — which was hyperfocused on debunking “junk science” and any idea that considered sociological and historical influence and not just biology. Those who rose to prominence in the movement did so through a lot of “non-tolerant calling-out and attacking people,” Greenesmith said, much like gender-critical feminism. “Anti-trans feminists think they have science on their side. It is bananas how ascientific their rhetoric is, and yet literally they say, ‘Biology isn’t bigotry.’ In fact, biology has been used as bigotry as long as biology has been a thing.”

(tags: feminism politics terfs trans-rights gender biology uk jk-rowling transphobia)

Eco Worriers: Saving the Planet, One Unoptimized Website at a Time

Web sustainability performance checking

(tags: sustainability green web http optimization performance ad-blocking gwf)

The #1 bug predictor is not technical, it’s organizational complexity (August Lilleaas’ blog)

Organizational Complexity. Measures number of developers working on the module, number of ex-developers that used to work on the module but no longer does, how big a fraction of the organization as a whole that works or has worked on the module, the distance in the organization between the developer and the decision maker, etc.

(tags: culture management programming organisations bureaucracy bugs quality)

Opinion | Twelve Million Phones, One Dataset, Zero Privacy – The New York Times

‘We identified people at political rallies and protests, in one case from 2016 we saw a Seattle-based Microsoft employee take a day off and visit Amazon’s campus…when we found him on LinkedIn in 2019 he was an Amazon employee. Turns out his phone had given away his job interview. In recent months we’ve spoken with people we’ve found in the data. Their reaction is surprising — a blend of contradictory emotions like outrage and apathy. We don’t have the language to talk about this stuff. And part of the reason is…we adopted this tech so fast. A big takeaway from my experience with this reporting: This is the decade we were brainwashed into surveilling ourselves. In just over 10 years we were sold a future of personalization and convenience and paid for it with little pieces of ourselves that we can never get back.’

(tags: technology privacy surveillance phones mobile location location-tracking tracking geo)