Category Archives: Cyber Issues

FBI signed $15 million contract with Apple vendor, Cellebrite; parent company’s stock soars

Fortune reports: The U.S. government’s announcement Monday that it hacked into the San Bernardino terrorist’s iPhone ended the FBI’s legal feud with Apple. But while many observers thought the incident left both the FBI and Apple looking foolish, there does appear to be a winner emerging from the case.

Shares of Suncorp, a Japanese technology company traded on the Tokyo stock exchange (ticker: 6736), soared 17% on Tuesday following the government’s court declaration that it “successfully accessed the data stored on [Syed] Farook’s iPhone.” In all, Suncorp’s shares have more than doubled in the six weeks since February 16, when Apple published its letter refusing to help the FBI.

Suncorp, which specializes in mobile data transfer as well as equipment for a popular Japanese pinball-like game called pachinko, owns Cellebrite, the Israel-based company that reportedly helped the FBI crack the iPhone.

Apple’s stock, meanwhile, was up just about 2% Tuesday afternoon, despite the fact that it is now free of legal expenses relating to the FBI case as well as the technological burden the government tried to impose.

Suncorp’s shares started rising last month, and really took off after the government said last Wednesday that an “outside party” had demonstrated “a possible method for unlocking” the iPhone. An Israeli newspaper quickly identified the unnamed company as Cellebrite, a government contractor that makes a mobile forensic device for extracting and decoding data from smartphones and tablets. Since then, Suncorp’s stock has risen nearly 40%, while Japan’s Nikkei 225 stock market index has been basically flat, and fell slightly on Tuesday.

The odd thing about the company’s dramatic stock rise is that neither the FBI nor Suncorp has confirmed the company was involved in unlocking the phone. In fact, the FBI has said very little so far about how it might have cracked the iPhone. [Continue reading…]

The Daily Beast reports: The FBI has said practically nothing about the “tool” that helped the FBI get inside the phone, as a U.S. law enforcement official called it in a hastily arranged press conference on Monday evening. Nor would the official say whether investigators might use it again on the dozen or so other iPhones the FBI is reportedly trying to gain access to, or whether the bureau would share the tool with local law enforcement agencies, who are believed to have hundreds of phones just waiting to be cracked.

“I think the best answer I can give you is it’s premature to say anything about our ability to access other phones,” said the official, who discussed the case with reporters on condition of anonymity and said almost nothing about where the FBI will go from here.

But he didn’t have to. Comey’s earlier remarks, coupled with the government’s decision to drop the warrant request, sent a message to other tech companies: Work with us, or don’t. We’ll get what we need without you.

Notably, the U.S. official didn’t say whether the FBI would disclose its newfound technique to Apple, which has a vested interest in protecting the security and privacy of its customers. But Cellebrite, an Israeli company, has been identified in some news accounts as the company that came to the FBI’s rescue. It signed a contract with the bureau worth more than $15 million last week.

In other words: The American government may have used foreign hackers to crack the signature product of America’s top technology company.

But it’s hard to imagine Apple didn’t have some idea what was coming. One of Cellebrite’s other clients is Apple itself. [Continue reading…]

Facebooktwittermail

Google search technique aided N.Y. dam hacker in Iran

The Wall Street Journal reports: An Iranian charged with hacking the computer system that controlled a New York dam used a readily available Google search process to identify the vulnerable system, according to people familiar with the federal investigation.

The process, known as “Google dorking,” isn’t as simple as an ordinary online search. Yet anyone with a computer and Internet access can perform it with a few special techniques. Federal authorities say it is increasingly used by hackers to identify computer vulnerabilities throughout the U.S.

Hamid Firoozi, who was charged Thursday by federal prosecutors, stumbled onto the Bowman Avenue Dam in Rye Brook, N.Y., in 2013 by using the technique to identify an unprotected computer that controlled the dam’s sluice gates and other functions, said people briefed on the investigation. Once he identified the dam, he allegedly hacked his way in using other methods.

“He was just trolling around, and Google-dorked his way onto the dam,” one person familiar with the investigation said.

The search technique has been around for about 10 years, said cybersecurity experts, and is neither illegal nor always malicious. It is primarily used by “white hat hackers,” computer specialists who test an organization’s computer system for vulnerabilities, said Michael Bazzell, a former computer crime investigator for the Federal Bureau of Investigation. [Continue reading…]

Facebooktwittermail

FBI adds two Syrian hackers to its most-wanted list for cybercriminals

The Atlantic reports: In late April 2013, a tweet from the Associated Press claimed that a pair of explosions at the White House had injured President Barack Obama. Markets reacted nearly instantly, sending stocks plunging. But when, a short time later, Press Secretary Jay Carney told reporters there was no explosion, the market quickly righted itself.

The news organization’s Twitter account was hacked, it turned out. A group calling itself the Syrian Electronic Army claimed credit. In only a few minutes, their rogue tweet demonstrated the market-moving power of 140 characters sent from a credible source.

The Syrian Electronic Army has also defaced websites belonging to the U.S. Marines, Harvard University, and Human Rights Watch, as well as websites and Twitter feeds of other major news organizations like the BBC, CNN, and The Washington Post. The group’s members remained anonymous, going by pseudonyms like “The Shadow” and “The Pro.”

But on Tuesday, the Justice Department revealed the identity of three members of the group, charging them with computer hacking and placing two of them on the FBI’s “Cyber’s Most Wanted” list. The FBI is offering a $100,000 bounty for information leading to their arrest. [Continue reading…]

Facebooktwittermail

Is access to our phones a step toward the police wanting access to our minds?

By Nathan Emmerich, Queen’s University Belfast

We use our smartphones so much these days, it almost feels like they have become extensions of ourselves, boosting our capacity to calculate and remember. What might come of this closer union of human and technological device? If police can serve a warrant to search your phone, and we see these devices as extensions of ourselves, how long until investigators one day serve a warrant to search your mind?

This line of thinking was roused by the FBI’s legal efforts to force Apple to help them access an iPhone that belonged to a suspected terrorist – something Apple says would undermine the security of its products. This is one of several similar cases, and part of a larger effort by the FBI and intelligence agencies, to ensure they can access a variety of now common devices.

Continue reading →

Facebooktwittermail

UN rights chief says unlocking gunman’s iPhone could open ‘Pandora’s box’

The New York Times reports: The top human rights official at the United Nations warned the United States authorities on Friday that their efforts to force Apple to unlock an iPhone belonging to a gunman risked helping authoritarian governments and jeopardizing the security of millions around the world.

The remarks by Zeid Ra’ad al-Hussein, the United Nations high commissioner for human rights, came as American investigators continued to press Apple to write software to help them gain access to an iPhone used by one of the gunmen in a shooting in San Bernardino, Calif., in December. Though the F.B.I. says it is a one-time request, Apple and others have raised concerns that the case could set a precedent and could force technology firms to install so-called back doors in devices, potentially invading customer privacy.

Mr. al-Hussein said that American law enforcement agencies, in seeking trying to break the encryption protecting one phone, “risk unlocking a Pandora’s box,” and that there were “extremely damaging implications” for the rights of many millions of people, with possible effects on their physical and financial security. [Continue reading…]

Facebooktwittermail

U.S. defense secretary takes position against a data ‘back door’

The New York Times reports: Defense Secretary Ashton B. Carter assured an audience of computer security experts Wednesday that he was not in favor of a “back door” that would give the government access to data that is protected by encryption.

Speaking at the annual RSA Conference, Secretary Carter sought common ground with companies worried by Apple’s fight with the Federal Bureau of Investigation over access to an iPhone.

“Just to cut to the chase, I’m not a believer in back doors or a single technical approach,” Secretary Carter said to loud applause during a panel discussion at the conference. “I don’t think it’s realistic. I don’t think that’s technically accurate.” [Continue reading…]

Facebooktwittermail

Apple wins ruling in New York iPhone hacking order

The New York Times reports: A federal magistrate judge on Monday denied the United States government’s request that Apple extract data from an iPhone in a drug case in New York, giving the company’s pro-privacy stance a boost as it battles law enforcement officials over opening up the device in other cases.

The ruling, from Judge James Orenstein in New York’s Eastern District, is the first time that the government’s legal argument for opening up devices like the iPhone has been put to the test. The denial could influence other cases where law enforcement officials are trying to compel Apple to help unlock iPhones, including the standoff between Apple and the F.B.I. over the iPhone used by one of the attackers in a mass shooting in San Bernardino, Calif., last year.

Judge Orenstein, in his 50-page ruling on Monday, took particular aim at a 1789 statute called the All Writs Act that underlies many government requests for extracting data from tech companies. The All Writs Act broadly says that courts can require actions to comply with their orders when not covered by existing law. Judge Orenstein said the government was inflating its authority by using the All Writs Act to force Apple to extract data from an iPhone seized in connection with a drug case.

The government’s view of the All Writs Act is so expansive as to cast doubt on its constitutionality if adopted, Judge Orenstein wrote. [Continue reading…]

Facebooktwittermail

Living under business surveillance in America

One of the ironies of Libertarianism in America is its soft-spot for Capitalism — as though anything that brands itself free, like free-enterprise, actually promotes freedom. Libertarians never tire of warning about the threats posed by the NSA and other intrusive government agencies, while the coercive and covert power of commerce generates far less fury.

Yet anyone who is genuinely concerned about infringements on civil liberties through electronic systems of surveillance, probably needs to be more wary of business than they are of government.

Most of the data the government collects gets poured into digital black holes — the data being collected for business applications, however, is constantly being mined to extract all its value.

Government might be watching you, but business is telling you where to go.

The New York Times reports: Pass a billboard while driving in the next few months, and there is a good chance the company that owns it will know you were there and what you did afterward.

Clear Channel Outdoor Americas, which has tens of thousands of billboards across the United States, will announce on Monday that it has partnered with several companies, including AT&T, to track people’s travel patterns and behaviors through their mobile phones.

By aggregating the trove of data from these companies, Clear Channel Outdoor hopes to provide advertisers with detailed information about the people who pass its billboards to help them plan more effective, targeted campaigns. With the data and analytics, Clear Channel Outdoor could determine the average age and gender of the people who are seeing a particular billboard in, say, Boston at a certain time and whether they subsequently visit a store. [Continue reading…]

Facebooktwittermail

Apple vs. FBI: ‘Just this once’?

Julian Sanchez writes: Loudly as the Justice Department protests that this dispute is simply about one particular phone, that’s fairly clearly not the case. Forget other even more dangerous ways Apple could be compelled to use their private key and let’s stay focused on breaking iPhones for the moment. The Manhattan DA’s office alone has at least 175 iPhones that they’d like Apple to help them break into, and DOJ itself has 12 other ongoing lawsuits seeking access to iPhones. Realistically, if Apple loses here — and especially if they lose at the appellate level, which is where this is likely going given Apple’s decision to hire superstar lawyer Ted Olson for the case — they’re going to be fielding thousands of similar demands every year. As a practical matter, they’re going to need a dedicated team dedicated to developing, debugging, testing, customizing, and deploying the code used to brute force passcodes.

Now, when it comes to the Holy Grail of Apple’s security infrastructure — the private key — it’s almost certainly stored in secure vaults, on a Hardware Security Module that makes it difficult or impossible to copy the key itself off that dedicated hardware, and likely protected by elaborate procedures that have to be followed to authenticate major new software releases. If your adversaries realistically include, say, the Chinese and Russian intelligence services — and for Apple, you’d better believe it — it’s a serious enough security problem to guard against exfiltration or use of that Holy Grail private key. Doing the same for a continuously updated and deployed hacking tool is likely to be hugely more difficult. As the company explains:

Apple would do our best to protect that key, but in a world where all of our data is under constant threat, it would be relentlessly attacked by hackers and cybercriminals. As recent attacks on the IRS systems and countless other data breaches have shown, no one is immune to cyberattacks.

The Justice Department might not intend to “set a master key loose on the land” — but the predictable consequence of mandating compliance with requests of this type will be to significantly increase the chance of exactly that occurring. And that’s an increased risk that every individual or enterprise customer relying on iOS devices to secure critical data will need to take into account. [Continue reading…]

Facebooktwittermail

Ex-NSA chief opposes government effort to require ‘back doors’ in all devices

USA Today reports: Retired four-star general Michael Hayden, who as director of the NSA installed and still defends the controversial surveillance program to collect telephone metadata on millions of Americans, says he opposes proposals to force Apple and other tech companies to install “back doors” in digital devices to help law enforcement.

In an emerging court battle over access to information on the iPhone owned by one of the San Bernardino attackers, Hayden says “the burden of proof is on Apple” to show that limited cooperation with investigators would open the door to broader privacy invasions. Apple is being asked not to decrypt information on the smartphone but rather to override the operating system so investigators could try an endless series of passwords to unlock it.

“In this specific case, I’m trending toward the government, but I’ve got to tell you in general I oppose the government’s effort, personified by FBI Director Jim Comey,” Hayden told Capital Download in an interview about his memoir, Playing to the Edge: American Intelligence in the Age of Terror. “Jim would like a back door available to American law enforcement in all devices globally. And, frankly, I think on balance that actually harms American safety and security, even though it might make Jim’s job a bit easier in some specific circumstances.”[Continue reading…]

Facebooktwittermail

Russia steps up Syria cyber assault

Financial Times reports: Russia is mounting a far-reaching cyber espionage campaign against Syrian opposition groups and NGOs, as Moscow seeks to influence the flow of information on the country’s humanitarian crisis and obscure the full extent of its military operations there.

Targets include some of the most important human rights organisations and aid groups operating in the country, such as the Syrian Observatory of Human Rights, which reports on military incidents and is frequently cited in western media outlets, the Financial Times has learnt. The operation shares many of the hallmarks of Moscow’s sustained hacking campaign against the Ukrainian government in 2013 and 2014. [Continue reading…]

Facebooktwittermail

The archaic All Writs Act the government is using to try and unlock a terrorist’s iPhone

The Washington Post reports: The U.S. government and Apple are locked in a legal battle over unlocking an iPhone used by one of the San Bernardino shooters. But a new court order is throwing a law that dates to the days of the founding fathers into a high-tech debate over digital security.

On Tuesday, a U.S. magistrate judge in California ordered Apple to provide “reasonable technical assistance” to the government as it tries to bypass security features built into its products based on an interpretation of the “All Writs Act.”

The original form of that statute dates to the Judiciary Act of 1789, centuries before the iPhone was a twinkle in Steve Jobs’s eye. In its current form, the law gives federal courts the power to “issue all writs necessary or appropriate in aid of their respective jurisdictions and agreeable to the usages and principles of law.”

Basically, it’s “a very short, cryptic statute” that gives the courts “all sorts of incidental powers” to require things not specifically covered by other laws, according to Stephen Vladeck, a law professor at American University.

In the past, the act has been used to compel non-parties — like service providers of tech companies — to help in criminal investigations, Vladeck said. But that help has typically been limited to straightforward requests, like activating or turning off particular features and using systems that are already in place, he said.

The new order is different: It tells Apple to help the government by creating an entirely new software to help investigators bypasses security features. “That requires Apple to go much further than any company has ever been required to go in one of these cases,” said Vladeck. [Continue reading…]

Last October, Jennifer Granick and Riana Pfefferkorn wrote: Under the government’s interpretation of the All Writs Act, anyone who makes software could be dragooned into assisting the government in investigating users of the software. If the court adopts this view, it would give investigators immense power. The quotidian aspects of our lives increasingly involve software (from our cars to our TVs to our health to our home appliances), and most of that software is arguably licensed, not bought. Conscripting software makers to collect information on us would afford the government access to the most intimate information about us, on the strength of some words in some license agreements that people never read. (And no wonder: The iPhone’s EULA came to over 300 pages when the government filed it as an exhibit to its brief.)

The government’s brief does not acknowledge the sweeping implications of its arguments. It tries to portray its requested unlocking order as narrow and modest, because it “would not require Apple to make any changes to its software or hardware, … [or] to introduce any new ability to access data on its phones. It would simply require Apple to use its existing capability to bypass the passcode on a passcode-locked iOS 7 phone[.]” But that undersells the implications of the legal argument the government is making: that anything a company already can do, it could be compelled to do under the All Writs Act in order to assist law enforcement. [Continue reading…]

Facebooktwittermail

How a New York judge inspired Apple’s encryption fight

Reuters reports: Last October, prosecutors from the Justice Department asked a federal magistrate judge in Brooklyn to issue an order directing Apple to help the Drug Enforcement Administration bust security on an iPhone 5 seized from the home of Jun Feng, a suspected meth dealer.

The government had previously obtained many such orders against Apple and other companies under the All Writs Act, a 1789 statute that grants federal courts broad power to issue “necessary or appropriate” writs.

The act has been a powerful tool for prosecutors since 1977, when the U.S. Supreme Court ruled in U.S. v. New York Telephone that the All Writs Act extends, under certain conditions, to private companies in a position to assist “the proper administration of justice.”

Apple has a long history of compliance with All Writs Act orders. The company helped New York investigators extract data from a suspected child sex abuser’s iPhone in 2008; rushed a data extraction in 2013 from the phone of an alleged child pornographer in Washington; and in 2015 provided federal agents in Florida with data the company extracted from a drug suspect’s phone.

According to a Justice Department brief filed last fall, Apple never objected to All Writs Act orders in those cases – nor, for that matter, to any All Writs Act order directing the company to help federal investigators break into iPhones.

Apple’s policy of acquiescence abruptly changed in the Jun Feng case last year. And for all of the attention now focused on Apple’s announced opposition to a newly issued All Writs Act order directing the company to help Justice Department investigators break the passcode on an iPhone belonging to San Bernardino shooter Syed Farook, the Feng case is quite likely to produce a ruling before the Farook case.

The impending showdown over Farook’s phone is an irresistibly stark depiction of the competing interests of individual privacy and national security. But keep your eye on precedent from Feng. [Continue reading…]

Facebooktwittermail

FBI’s push to ‘fix a typo’ would really expand its surveillance authority

Robyn Greene writes: At last week’s Senate Intelligence Committee hearing on Worldwide Threats, FBI Director James Comey reiterated his call for a major expansion of the FBI’s surveillance authorities, but disingenuously downplayed it as fixing a “typo” in the law. In fact, Comey’s proposed fix, which he calls one of the FBI’s top legislative priorities, would be a major expansion of surveillance authority, and a major hit to Americans’ privacy and civil liberties. It would grant the FBI access to a range of revealing and personal details about Americans’ online communications — what are called Electronic Communications Transactional Records (ECTR), in legalese — without court approval.

Through Comey’s “ECTR fix,” the FBI would have the unilateral authority to obtain information from phone and Internet companies about your online communications such as logs of emails you send and receive, cell site data (including your location information), and lists of websites you visit. The FBI wants to get this information using National Security Letters (NSLs), which are demands for information issued directly by local FBI offices without any court approval or supervision.

Under current law, the FBI can only use NSLs to get information pertaining to a customer’s “name, address, length of service, and local and long distance toll billing records of a person or entity.” By contrast, if the FBI wants to compel a company to hand over the much more revealing private information that is included in ECTRs, they currently can’t use NSLs — instead, they have to get a court order after convincing a judge that they have a factual basis for demanding those records. Therefore, the FBI’s proposal that Congress add ECTRs to the NSL statute is far from a typo fix, and would instead be a major expansion of FBI’s authority to conduct surveillance with virtually no oversight and no accountability. [Continue reading…]

Facebooktwittermail

Apple’s stance highlights a more confrontational tech industry

Farhad Manjoo writes: The battle between Apple and law enforcement officials over unlocking a terrorist’s smartphone is the culmination of a slow turning of the tables between the technology industry and the United States government.

After revelations by the former National Security Agency contractor Edward J. Snowden in 2013 that the government both cozied up to certain tech companies and hacked into others to gain access to private data on an enormous scale, tech giants began to recognize the United States government as a hostile actor.

But if the confrontation has crystallized in this latest battle, it may already be heading toward a predictable conclusion: In the long run, the tech companies are destined to emerge victorious.

It may not seem that way at the moment. On the one side, you have the United States government’s mighty legal and security apparatus fighting for data of the most sympathetic sort: the secrets buried in a dead mass murderer’s phone. The action stems from a federal court order issued on Tuesday requiring Apple to help the F.B.I. unlock an iPhone used by one of the two attackers who killed 14 people in San Bernardino, Calif., in December.

In the other corner is the world’s most valuable company, whose chief executive, Timothy D. Cook, has said he will appeal the court’s order. Apple argues that it is fighting to preserve a principle that most of us who are addicted to our smartphones can defend: Weaken a single iPhone so that its contents can be viewed by the American government and you risk weakening all iPhones for any government intruder, anywhere.

There will probably be months of legal tussling, and it is not at all clear which side will prevail in court, nor in the battle for public opinion and legislative favor.

Yet underlying all of this is a simple dynamic: Apple, Google, Facebook and other companies hold most of the cards in this confrontation. They have our data, and their businesses depend on the global public’s collective belief that they will do everything they can to protect that data. [Continue reading…]

Facebooktwittermail

Apple encryption case risks influencing Russia and China, privacy experts say

The Guardian reports: Authoritarian governments including Russia and China will demand greater access to mobile data should Apple lose a watershed encryption case brought by the FBI, leading technology analysts, privacy experts and legislators have warned.

Apple’s decision to resist a court order to unlock a password-protected iPhone belonging to one of the San Bernardino killers has created a worldwide privacy shockwave, with campaigners around the world expecting the struggle to carry major implications for the future of mobile and internet security. They warned that Barack Obama’s criticism of a similar Chinese measure last year now risked ringing hollow.

Senator Ron Wyden of Oregon, a leading legislator on privacy and tech issues, warned the FBI to step back from the brink or risk setting a precedent for authoritarian countries.

“This move by the FBI could snowball around the world. Why in the world would our government want to give repressive regimes in Russia and China a blueprint for forcing American companies to create a backdoor?” Wyden told the Guardian.

“Companies should comply with warrants to the extent they are able to do so, but no company should be forced to deliberately weaken its products. In the long run, the real losers will be Americans’ online safety and security.” [Continue reading…]

Facebooktwittermail

The cyberattack on Ukraine’s power grid is a warning of what’s to come

By Nilufer Tuptuk, UCL and Stephen Hailes, UCL

When more than 100,000 people in and around the Ukrainian city of Ivano-Frankivsk were left without power for six hours, the Ukrainian energy ministry accused Russia of launching a cyberattack on the country’s national energy grid.

Now reports released by security researchers from the SANS Industrial Control Systems team and the Industrial Control Systems Cyber Emergency Response Team confirm their belief that a cyberattack was responsible for the power cut, making the incident one of the first significant, publicly reported cyberattacks on civil infrastructure.

This is a rare event, of which the most famous example is the Stuxnet malware used to destroy equipment in the Iranian nuclear programme. Many consider Stuxnet so sophisticated that national governments must have been involved. But as is frequently the case, attributing responsibility for Stuxnet has proved difficult, and it’s likely that, despite circumstantial evidence, it will be the same in this case. While the Ukrainian Security Service (SBU) and the international press were quick to blame Russian state-backed hackers, Moscow has remained silent.

Continue reading →

Facebooktwittermail

To what extent might Stephen Hawking and Elon Musk be right about the dangers of artificial intelligence?

structure3

Suzanne Sadedin, an evolutionary biologist, writes: I think they are right that AI is dangerous, and they are dangerously wrong about why. I see two fairly likely futures.

Future 1: AI destroys itself, humanity and most or all life on earth, probably a lot sooner than in 1000 years.

Future 2: Humanity radically restructures its institutions to empower individuals, probably via transhumanist modification that effectively merges us with AI. We go to the stars.

Right now, we are headed for Future 1, but we could change this. Much as I admire Elon Musk, his plan to democratise AI actually makes Future 1 more, not less, likely.

Here’s why:

There’s a sense in which humans are already building a specific kind of AI; indeed, we’ve been gradually building it for centuries. This kind of AI consists of systems that we construct and endow with legal, real-world power. These systems create their own internal structures of rules and traditions, while humans perform fuzzy brain-based tasks specified by the system. The system as a whole can act with an appearance of purpose, intelligence and values entirely distinct from anything exhibited by its human components.

All nations, corporations and organisations can be considered as this kind of AI. I realise at this point it may seem like I’m bending the definition of AI. To be clear, I’m not suggesting organisations are sentient, self-aware or conscious, but simply that they show emergent, purpose-driven behaviour equivalent to that of autonomous intelligent agents. For example, we talk very naturally about how “the US did X”, and that means something entirely different from “the people of the US did X” or “the president of the US did X”, or even “the US government did X”.

These systems can be entirely ruthless toward individuals (just check the answers to What are some horrifying examples of corporate evil/greed? and What are the best examples of actions that are moral, even uplifting, but illegal? if you don’t believe me). Such ruthlessness is often advantageous — even necessary, because these systems exist in a competitive environment. They compete for human effort, involvement and commitment. Money and power. That’s how they survive and grow. New organisations, and less successful ones, copy the features of dominant organisations in order to compete. This places them under Darwinian selection, as Milton Friedman noted long ago.

Until recently, however, organisations have always relied upon human consent and participation; human brains always ultimately made the decisions, whether it was a decision to manufacture 600 rubber duckies or drop a nuclear bomb. So their competitive success has been somewhat constrained by human values and morals; there are not enough Martin Shkrelis to go around.

With the advent of machine learning, this changes. We now have algorithms that can make complex decisions better and faster than any human, about practically any specific domain. They are being applied to big data problems far beyond human comprehension. Yet these algorithms are still stupid in some ways. They are designed to optimise specific parameters for specific datasets, but they’re oblivious to the complexity of the real-world, long-term ramifications of their choices. [Continue reading…]

Facebooktwittermail