the coming incompatibility nightmare of USB-C cabling
Elasticache now supports sharding
The square root staffing law is a rule of thumb derived from queueing theory, useful for getting an estimate of the capacity you might need to serve an increased amount of traffic.
See also https://www.percona.com/blog/2016/02/05/measuring-docker-cpu-network-overhead/ for the CPU/Network equivalent. The good news is that nowadays it’s virtually 0 when the correct settings are used
Luis Villa: ‘Is the React license elegant? No. Should you be worried about using it? Probably not. If anything, Facebook’s attempt to give users an explicit patent license should probably be seen as a good faith gesture that builds some confidence in their ecosystem. But yeah, don’t use it if your company intends to invest heavily in React and also sue Facebook over unrelated patents. That… would be dumb. :)’
Writeup of their Consul-based service discovery system, a bit similar to smartstack. Good description of the production problems that they saw with Consul too, and also they figured out that strong consistency isn’t actually what you want in a service discovery system ;) HN comments are good too: https://news.ycombinator.com/item?id=12840803
Kalina Bontcheva leads the EU-funded PHEME project working to compute the veracity of social media content. She said reducing the amount of human oversight for Trending heightens the likelihood of failures, and of the algorithm being fooled by people trying to game it. “I think people are always going to try and outsmart these algorithms — we’ve seen this with search engine optimization,” she said. “I’m sure that once in a while there is going to be a very high-profile failure.” Less human oversight means more reliance on the algorithm, which creates a new set of concerns, according to Kate Starbird, an assistant professor at the University of Washington who has been using machine learning and other technology to evaluate the accuracy of rumors and information during events such as the Boston bombings. “[Facebook is] making an assumption that we’re more comfortable with a machine being biased than with a human being biased, because people don’t understand machines as well,” she said.
Crockford chose not to version [the] JSON definition: ‘Probably the boldest design decision I made was to not put a version number on JSON so there is no mechanism for revising it. We are stuck with JSON: whatever it is in its current form, that’s it.’ Yet JSON is defined in at least six different documents.“Boldest”. ffs. :facepalm:
We can’t easily fix the already broken devices, we can’t easily stop more broken devices from being shipped and we can’t easily guarantee that we can fix future devices that end up broken. The only solution I see working at all is to require ISPs to cut people off, and that’s going to involve a great deal of pain. The harsh reality is that this is almost certainly just the tip of the iceberg, and things are going to get much worse before they get any better.
This is shocking:
At the end of the panel on artificial intelligence, a young black woman asked [Sebastian Thrun, CEO of the education startup Udacity, who is best known for founding Google X] whether bias in machine learning “could perpetuate structural inequality at a velocity much greater than perhaps humans can.” She offered the example of criminal justice, where “you have a machine learning tool that can identify criminals, and criminals may disproportionately be black because of other issues that have nothing to do with the intrinsic nature of these people, so the machine learns that black people are criminals, and that’s not necessarily the outcome that I think we want.” In his reply, Thrun made it sound like her concern was one about political correctness, not unconscious bias. “Statistically what the machines do pick up are patterns and sometimes we don’t like these patterns. Sometimes they’re not politically correct,” Thrun said. “When we apply machine learning methods sometimes the truth we learn really surprises us, to be honest, and I think it’s good to have a dialogue about this.”“the truth”! Jesus. We are fucked
Using a proxy, remove “securityQuestion0” and “securityQuestion1” from the post data.Massive facepalm.
amazing architectural-oddities Tumblr (via Present and Correct)
I’ve had the privilege of experiencing a few different management levels (responsibilities? jobs?) at Etsy since I’ve joined. At each stage, I felt like the job of being a manager totally changed. What I did day-to-day changed, what was hard about it changed, how I measured my own success changed, and though I feel like the experiences built on one another, it continues to be an enormous shift in brainpower each time the gig changes a bit. Given how intangible (and often hidden) management work can be, I’ve outlined some highlights of what my work has been like as a manager over the last four years. (Obvious, major caveat: this is just my experience, and there’s lots in here that is unique to this particular work environment, hierarchy, requirements, and challenges!)
Cyril Connolly, the 50s critic and writer, said that the greatest enemy of creativity is the pram in the hall, but I think that was completely wrong. It was the enemy of a certain kind of dilettante life that he aspired to, the man of letters, but for the real novelist the pram in the hall is the greatest ally – it brings you up sharp and you realise what reality is all about. My children were a huge inspiration for me. Watching three young minds creating their separate worlds was a very enriching experience.
holy crap this is going to be a serious problem
may need this, depending on kiddie preferences this year ;)
Excellent post on team voice comms tactics. Many tips here
Love the ‘decade of’ dig at FB and Amazon — ‘we were doing it first’ ;) Great details on how Google have built out and improved their DC networking. Includes a hint that they now use DCTCP (datacenter-optimized TCP congestion control) on their internal hosts….
pretty amazing. full 60FPS, 2560×1600, everything on Epic quality, streaming from Azure, for $2 per hour
oh god this is funny. Louise Mensch and various UKIPpers fall for transparent pisstake involving “taking Article 50 out of the ring binder and shredding it. It now goes straight from 49 to 51” etc.
What makes an image NSFW, according to Yahoo? I explore this question with a clever new visualization techniqueDeep Dream applied to an NSFW classifier. This is a bit NSFW, as it happens
stack trace leads with a symbol called “_thereIsNoSadnessLikeTheDeathOfOptimism”
interesting presentation describing how to architect Airflow ETL setups; see also https://gtoonstra.github.io/etl-with-airflow/principles.html
‘Generally, a very small number of people account for a disproportionately high share of the total number of noise complaints. In 2015, for example, 6,852 of the 8,760 complaints submitted to Ronald Reagan Washington National Airport originated from one residence in the affluent Foxhall neighborhood of northwest Washington, DC. The residents of that particular house called Reagan National to express irritation about aircraft noise an average of almost 19 times per day during 2015.’ Somebody needs help.
good advice on how to ETL
the Women on Web approach is backed by a column in New Scientist: ‘It’s also safer than many other medicines that we are allowed to buy from pharmacies without a prescription, such as Viagra in the UK. So why can’t women get abortion pills from pharmacies and manage the process themselves at home if they choose? It might sound radical but it’s already widespread in countries where abortion is illegal, with women buying the pills from online pharmacies. While some countries, such as Poland, are trying to tighten their already strict abortion laws, the advent of mail-order abortion pills means the law is becoming almost irrelevant.’
This is amazing — what a wonderland! For instance:
Six dollars for: a GSM chipset, a CPU, an LCD screen, a battery, a PCB, a metal housing, a molded silicone watch band, a MicroUSB cable, and a box. And the labor to assemble and test all of that.
The UK’s version of the POD database project was used by the Home Office to track immigrants for various reasons — in other words, exactly the reasons why parents will choose not to provide that data
One “outlier” voter—a 19-year old black Trump supporter—was weighted so heavily that it shifted the whole poll significantly. Stats fail
this is a nice zero-config scaling story — good work Kafka Streams
‘a Mexican cerveza preparada made with beer, lime juice, and assorted sauces, spices, and peppers. It is served in a chilled, salt-rimmed glass. There are numerous variations of this beverage throughout Mexico and Latin America.’
a variation on the mimosa, with an IPA and grapefruit juice — sounds like Brewdog’s Elvis Juice
‘This crisp, refreshing beer and ginger beer cocktail is a Cuban favorite, and it’s super-simple to make.’
via Saladin Ahmed — the scientist who coined the term abandoned it as useless years ago:
The concept of the alpha wolf is well ingrained in the popular wolf literature at least partly because of my book “The Wolf: Ecology and Behavior of an Endangered Species,” written in 1968, published in 1970, republished in paperback in 1981, and currently still in print, despite my numerous pleas to the publisher to stop publishing it. Although most of the book’s info is still accurate, much is outdated. We have learned more about wolves in the last 40 years then in all of previous history. One of the outdated pieces of information is the concept of the alpha wolf. “Alpha” implies competing with others and becoming top dog by winning a contest or battle. However, most wolves who lead packs achieved their position simply by mating and producing pups, which then became their pack. In other words they are merely breeders, or parents, and that’s all we call them today, the “breeding male,” “breeding female,” or “male parent,” “female parent,” or the “adult male” or “adult female.” In the rare packs that include more than one breeding animal, the “dominant breeder” can be called that, and any breeding daughter can be called a “subordinate breeder.”
20km of private monorail owned and operated by the electricity grid, used to monitor and inspect one of the longest 400kV circuits in Europe
ugh, 213ms mean response overhead
Full AirBnB property management service in Dublin
According to the bankruptcy documents, one of the assets listed is ‘a chair worth 2,902,119JPY, or roughly $28,000USD.’
“Chinese internet giant Tencent” have been granted a USPTO patent for drawing a box on a screen.
This is the new fear — that FF/FG will accidentally and stupidly disengage Ireland from the EU as a side effect of trying to keep the UK happy and cross-border trade intact
Score 1 “My management culture is fucked up” point for each of the following: We have an unlimited vacation policy; We don’t do regular 1:1s, but we have open office hours/are super available if anyone wants to chat; We don’t have a process for interviewing, we just hire awesome people when we meet them; We super care about diversity, but we don’t want to lower the bar so we just hire the best person for the job even if it means diversity suffers; We don’t have defined levels and career paths for our employees, we’re a really flat org; We don’t have formal managers for every staff member, everyone just gets their work done; We don’t have, like, HR HR, but our recruiter/office manager/only female employee is super good if you want someone to talk to; We don’t do performance improvement plans for employees that are struggling. We just have a super honest conversation about how they aren’t a good fit and fire them; We would have some hard explaining to do if our salary list accidentally became public.
Great font factoid: ‘The name “Noto” comes from the little squares that show when a font is not supported by a computer. This are often referred to as “tofu”, because of their shape, therefore the font is short for No Tofu.’
Specifically, the following 3 classes of errors were implicated in 92% of the major production outages in this study and could have been caught with simple code review:
Error handlers that ignore errors (or just contain a log statement); error handlers with “TODO” or “FIXME” in the comment; and error handlers that catch an abstract exception type (e.g. Exception or Throwable in Java) and then take drastic action such as aborting the system.(Interestingly, the latter was a particular favourite approach of some misplaced “fail fast”/”crash-only software design” dogma in Amazon. I wasn’t a fan)
An excellent explanation of what is going on in the UK right now. What a nightmare:
Finally there are the self-styled buccaneers of the free-trade seas. Boris Johnson would probably cast himself as Sir Walter Raleigh – polymath, wordsmith, adventurer. That leaves Liam Fox to play Sir Francis Drake, looking for domestic glory in global circumnavigation but seen from abroad as a pirate. This is all myth and fantasy, of course. But parties have always been sustained by internal mythologies, and the task of exiting the EU is so complicated and fraught with danger that fantasy becomes a necessary comfort. As one former minister says of the puritan choristers: “They have spent their lives working towards this dream. Of course they don’t want to accept that it’s a nightmare.” Tory pro-Europeans are in the impossible position of using rational argument against faith. If they counsel compromise on migration or the single market, they are accused of talking Britain down or trying to refight the referendum. They have few reinforcements across the political water. Labour is a shambles. The Lib Dems are puny in parliament. Scotland has its own distinct politics, and in Nicola Sturgeon its own remainian queen with her own independence agenda. The Tories do not speak for all of England, but in the absence of credible opposition they feel as if they do, and will act accordingly. To those millions who did not vote to leave the EU, the message is clear: you are free to pray for whatever you like. Your antique rites will be tolerated. But do not expect your concerns to be represented in the court of Queen Theresa. Be humble instead. Swallow your doubts and take a pew in the reformed national church of Brexit.
Martin Fowler’s take on the 4 kinds of tech debt
This is a great blog post, spot on:
You can’t just go “dudes it’s faster” and jump off a cliff. This shit is basic. Test real production workloads. Have a rollback plan. (Not for *10 days* … try a month or two.)The only thing I’d nitpick on is that it’s all very well to say “buy my book” or “come see me talk at Blahcon”, but a good blog post or webpage would be thousands of times more useful.
Excellent talk. I love this analogy for ML applied to real-world data which affects people:
Treating the world as software promotes fantasies of control. And the best kind of control is control without responsibility. Our unique position as authors of software used by millions gives us power, but we don’t accept that this should make us accountable. We’re programmers—who else is going to write the software that runs the world? To put it plainly, we are surprised that people seem to get mad at us for trying to help. Fortunately we are smart people and have found a way out of this predicament. Instead of relying on algorithms, which we can be accused of manipulating for our benefit, we have turned to machine learning, an ingenious way of disclaiming responsibility for anything. Machine learning is like money laundering for bias. It’s a clean, mathematical apparatus that gives the status quo the aura of logical inevitability. The numbers don’t lie.Particularly apposite today given Y Combinator’s revelation that they use an AI bot to help ‘sift admission applications’, and don’t know what criteria it’s using: https://twitter.com/aprjoy/status/783032128653107200
So assuming the mission continues well, in 2014 the Rosetta Probe will land on Comet 67P/Churyumov-Gerasimenko, where it will measure the comet’s molecular composition. Then it will remain at rest as the comet orbits the sun for hundreds of millions of years. So somewhere in the solar system, where it is safe but hard to reach, a backup sample of human languages is stored, in case we need one.As jwz says: ‘The Rosetta Disc is now safely installed on 67P/Churyumov-Gerasimenko.’
Some tantalising discussion on twitter of an Airflow + AMI + ASG workflow for ML packaging: ‘We build models using Airflow. We deploy new models as AMIs where each AMI is model + scoring code. The AMI is hence a version of code + model at a point in time : #immutable_infrastructure. It’s natural for Airflow to build & deploy the model+code with each Airflow DAG Run corresponding to a versioned AMI. if there’s a problem, we can simply roll back to the previous AMI & identify the problematic model building Dag run. Since we use ASGs, Airflow can execute a rolling deploy of new AMIs. We could also have it do a validation & ASG rollback of the AMI if validation fails. Airflow is being used for reliable Model build+validation+deployment.’
C64C used in Polish auto repair shop to balance driveshafts, working non-stop for over a quarter of a century. I’d like to see a Spectrum do _this_ ;)
‘Build Negroni over ice, top with sparkling wine, stir briefly. They’ll get you messed up quick, but it’s a great way to go.’ aka. the Double Sbagliato. yum
Sorry, Daily Mail article —
Blunders in the use of controversial snooping powers meant 13 people were wrongly arrested last year on suspicion of being paedophiles. Another four individuals had their homes searched by detectives following errors in attempts to access communications data, a watchdog revealed yesterday. Other mistakes also included people unconnected to an investigation being visited by police and delayed welfare checks on vulnerable people including children whose lives were at risk, said the Interception of Communications Commissioner. [….] A large proportion of the errors involved an internet address which was wrongly linked to an individual. Of the 23 serious mistakes, 14 were human errors and the other nine ‘technical system errors’.
“The Secret Barrister” explains a classic case of empty-gesture lawmaking in the UK:
in 2012, the coalition government, in a fit of virtue signalling, announced a bold plan to offer extra protection to victims of stalking, following a rash of reported cases where obsessive nutjobs had slipped through the net. Hence, via the 2012 Act, section 2A was shoved into the Protection from Harassment Act, creating a shiny new offence of stalking. What is stalking, you ask? Well here’s the clever bit. Stalking is…”a course of conduct which amounts to harassment…and [where] the acts or omissions involved are ones associated with stalking“. To inject some colour into the dull circularity of the definition, section 2A(3) provides “examples of acts or omissions associated with stalking”. In other words, you need to prove that the defendant is guilty of both harassment and stalking, in order to convict them of stalking. Therefore, proving stalking is by definition harder for the prosecution than simply proving harassment. And what do you get if you opt for the harder road? What prize awaits the victorious prosecutor who has slogged her way through the additional evidential burden thrust upon her by section 2A? The answer is….nothing. Or at least, nothing more than if you successfully prosecuted for harassment. The maximum sentence in each case is 6 months’ imprisonment. It is the very definition of empty gesture legislating. Section 2A is so very pointlessly pointless that I want urgently to go back in time to the day when then-crime prevention minister Jeremy Browne was hubristically prattling on about what a difference this law is going to make and shove a whoopee pie right up his schnoz. Section 2A does nothing other than create a new offence that is harder to prove than an existing offence that prohibits the same conduct, solely, it seems, to allow for the drawing of an entirely semantic distinction between “harassment” and “stalking”.
European transport group, Transport and Environment, said that the Loughborough study shows that better design “could save hundreds of pedestrian and cyclists’ lives”. It added that the study “finds huge differences in the direct vision – what drivers can see with their own eyes – of best and worst-in-class trucks in all categories, and that ‘low-entry cabs’ like the Mercedes Econic out perform all of today’s best performing vehicles.” A P-Series truck, from truck maker Scania, was rated at the best of its class with zero blind spots — this could go a long way to explaining why the makers of a Road Safety Authority video using another P-Series truck reportedly had to fake blind spots last year. Mandatory extra mirrors has been EU policy to try to reduce collisions with people cycling and walking but researchers point out that blind spots remain on many trucks and improving direct vision may be a better policy than improving indirect vision using mirrors. […] The EU currently has a deadline of 2028 for improved vision in trucks but Transport and Environment said: “Given that better vision cabs are already available on the market and in all market segments (best in class, smarter configurations, low entry vehicles) a 2028 deadline is not justifiable.”
good page on the Universal Scalability Law and how to apply it
‘You probably don’t know much about the Staten Island Ferry Disaster Memorial Museum, which honors the 400 victims who died when a giant octopus attacked the Cornelius G. Kolff, a Staten Island Ferry boat, on Nov. 22, 1963. That isn’t because the event was overshadowed by the assassination of JFK that same day—it’s because, as you may have guessed based on the word “tricks” in the headline, there was no such octopus-induced tragedy.’
great idea — donate old, obsolete iPhone 4/4s phones to a charity which repurposes them for autistic/non-verbal kids
Events of the past week have convinced me that one of the fastest-growing censorship threats on the Internet today comes not from nation-states, but from super-empowered individuals who have been quietly building extremely potent cyber weapons with transnational reach. More than 20 years after Gilmore first coined [his] turn of phrase, his most notable quotable has effectively been inverted — “Censorship can in fact route around the Internet.” The Internet can’t route around censorship when the censorship is all-pervasive and armed with, for all practical purposes, near-infinite reach and capacity.
I love this story — a wealthy couple buy a vineyard in the Languedoc for its theoretically-optimal microclimate for wine-making. Defying what one’s preconceptions would expect (mine included!), the results were fantastic.
In the Languedoc there is a vineyard that teaches us an important lesson about textbook learning and its application to the world. In the early Seventies it was bought by a wealthy couple, who consulted professors Emile Peynaud and Henri Enjalbert, the world’s leading academic oenologist and oenological geologist respectively. Between them these men convinced the couple that their new vineyard had a theoretically ideal microclimate for wine-making. When planted with theoretically ideal vines whose fruits would be processed in the optimal way according to the up-to-date science of oenology, this vineyard had the potential to produce wine to match the great first growths of Bordeaux. The received wisdom that great wine was the product of an inscrutable (and untransferable) tradition was quite mistaken, the professors said: it could be done with hard work and a fanatical attention to detail. The couple, who had no experience of wine-making but much faith in professorial expertise, took a deep breath and went ahead. If life were reliably like novels, their experiment would have been a disaster. In fact Aimé and Véronique Guibert have met with a success so unsullied that it would make a stupefying novel (it has already been the subject of a comatogenic work of non-fiction). The first vintage they declared (in 1978) was described by Gault Millau as ‘Château Lafite du Languedoc’; others have been praised to the heights by the likes of Hugh Johnson and Robert Parker. The wine is now on the list at the Tour d’Argent and the 1986 vintage retails at the vineyard for £65 a bottle. The sole shadow on the lives of these millionaires is cast by the odd hailstorm. No one to whom I have begun recounting the story believes it will end well. Most people are extremely unwilling to grant that faith in textbook knowledge should ever be crowned with success. We have a very strong narrative bias against such stories. It is a bias we forget once our children fall sick or we have to travel in an aeroplane, but so long as we are in storytelling mode we simply deny that systematic textbook reasoning can make headway against whimsy and serendipity. Apart from anything else, it is deeply unfair that it should.
“A lot of people feel that they want to live in a cul-de-sac, they feel like it’s a safer place to be,” Marshall says. “The reality is yes, you’re safer – if you never leave your cul-de-sac. But if you actually move around town like a normal person, your town as a whole is much more dangerous.” This is the opposite of what traffic engineers (and home buyers) have thought for decades. And it’s just the beginning of what we’re now starting to understand about the relative advantages of going back to the way we designed communities a century ago. Marshall and Garrick took the same group of California cities and also examined all their minutely classified street networks for the amount of driving associated with them. On average, they found, people who live in more sparse, tree-like communities drive about 18 percent more than people who live in dense grids. And that’s a conservative calculation.(via Tony Finch)
“A modern standard for event-oriented data”. Avro schema, events have time and type, schema is external and not part of the Avro stream. ‘a modern standard for representing event-oriented data in high-throughput operational systems. It uses existing open standards for schema definition and serialization, but adds semantic meaning and definition to make integration between systems easy, while still being size- and processing-efficient. An Osso event is largely use case agnostic, and can represent a log message, stack trace, metric sample, user action taken, ad display or click, generic HTTP event, or otherwise. Every event has a set of common fields as well as optional key/value attributes that are typically event type-specific.’
Some good slides with tips on running java apps in production in Docker
‘its legacy can be seen in factories, call centres and warehouses today, although new technology has taken the place of Taylor’s instruction cards and stopwatches. Many warehouse workers for companies such as Amazon use handheld devices that give them step-by-step instructions on where to walk and what to pick from the shelves when they get there, all the while measuring their “pick rate” in real time. For Jeremias Prassl, a law professor at Oxford university, the algorithmic management techniques of Uber and Deliveroo are Taylorism 2.0. “Algorithms are providing a degree of control and oversight that even the most hardened Taylorists could never have dreamt of,” he says.’
‘How to cheat at Bridge by breaking the tournament card-dealing random number generator’, via Tony Finch
The purpose of the drill was to see how the data center’s fire suppression system worked. Data centers typically rely on inert gas to protect the equipment in the event of a fire, as the substance does not chemically damage electronics, and the gas only slightly decreases the temperature within the data center. The gas is stored in cylinders, and is released at high velocity out of nozzles uniformly spread across the data center. According to people familiar with the system, the pressure at ING Bank’s data center was higher than expected, and produced a loud sound when rapidly expelled through tiny holes (think about the noise a steam engine releases). The bank monitored the sound and it was very loud, a source familiar with the system told us. “It was as high as their equipment could monitor, over 130dB”. Sound means vibration, and this is what damaged the hard drives. The HDD cases started to vibrate, and the vibration was transmitted to the read/write heads, causing them to go off the data tracks. “The inert gas deployment procedure has severely and surprisingly affected several servers and our storage equipment,” ING said in a press release.
‘All the information you need while traveling including visa requirements, currency, electricity, communication info and more.’
This is pretty awful — an accidental, careless and brutal side effect of marketers passing on sensitive info to one another, without respect for their users’ privacy: ‘I hadn’t realized, however, that when I had entered my information into the pregnancy app, the company would then share it with marketing groups targeting new mothers. Although I logged my miscarriage into the app and stopped using it, that change in status apparently wasn’t passed along. Seven months after my miscarriage, mere weeks before my due date, I came home from work to find a package on my welcome mat. It was a box of baby formula bearing the note: “We may all do it differently, but the joy of parenthood is something we all share.”’
on the list
Beautiful background art from a 2006 anime by Shinji Kimura, as a 10″ x 7″ full-colour hardback art book.
The leading theory that I’ve seen going around is that Google is actually blocking all links in any FeedBurner feed, because it’s a violation of its own terms of service. Seriously. The link-shortener “goo.gl”, run by Google, is blocking all URLs generated by Feedburner, run by Google. pic.twitter.com/IR7wrlv6xj — Great Again Also (@agentdero) September 6, 2016 That’s because Google’s URL shortener’s terms of service bans “URL re-directors” and it appears that the genius engineers at Google have decided that Google-run FeedBurner is nothing more than a URL re-director and killed off everyone’s links without notice or explanation. This despite the fact that they’re the same damn company and that FeedBurner unilaterally moved everyone’s RSS feed to use Goo.gl links in the first place.
Coda Hale’s new gig on how they’re using Docker, AWS, etc. I like this: “Use containers. Not too much. Mostly for packaging.”
great post from Loggly on production usage of regular expressions on shared, multitenant architecture, where a /.*/ can really screw things up. “NFA isn’t a golden ticket” paragraph included
‘we are enhancing the Spot Fleet model with the addition of Auto Scaling. You can now arrange to scale your fleet up and down based on a Amazon CloudWatch metric. The metric can originate from an AWS service such as EC2, Amazon EC2 Container Service, or Amazon Simple Queue Service (SQS). Alternatively, your application can publish a custom metric and you can use it to drive the automated scaling.’
Unfortunately the usual ML problem arises at the end:
One of the current challenges with deep learning is that you need to have a large number of training datasets. To train the model, Makoto spent about three months taking 7,000 pictures of cucumbers sorted by his mother, but it’s probably not enough. “When I did a validation with the test images, the recognition accuracy exceeded 95%. But if you apply the system with real use cases, the accuracy drops down to about 70%. I suspect the neural network model has the issue of “overfitting” (the phenomenon in neural network where the model is trained to fit only to the small training dataset) because of the insufficient number of training images.”In other words, as with ML since we were using it in SpamAssassin, maintaining the training corpus becomes a really big problem. :(
This is a mad story. The insurance company is accusing a guy in NZ of using remote-login software from 400km away to trigger a “print” command to a complicated Heath Robinson setup in order to light a fire to burn down his house