Amazon patents AI-powered drones to provide ‘surveillance as a service’

At the first re:MARS event early this month Amazon proposed its plans to further digitize its delivery services by making the AI-powered drones deliver orders.

Amazon was recently granted a US patent on June 4 for these ‘unmanned aerial vehicles (UAV) or drones’ to provide “surveillance as a service.” The patent which was filed on June 12, 2015, mentions how Amazon’s UAVs could keep an eye on customers’ property between deliveries while supposedly maintaining their privacy.

“The property may be defined by a geo-fence, which may be a virtual perimeter or boundary around a real-world geographic area. The UAV may image the property to generate surveillance images, and the surveillance images may include image data of objects inside the geo-fence and image data of objects outside the geo-fence,” the patent states.



A diagram from the patent shows how delivery drones could be diverted to survey a location.

Source: USPTO

According to The Telegraph, “The drones would look for signs of break-ins, such as smashed windows, doors left open, and intruders lurking on people’s property. Anything unusual could then be photographed and passed on to the customer and the police”.

“Drones have long been used for surveillance, particularly by the military, but companies are now beginning to explore how they might be used for home security”, The Verge reports.

Amazon’s competitor, Alphabet Inc.’s Wing, became the first drone to win an FAA approval to operate as a small airline, in April. However, Amazon received an approval to start making drone deliveries only in remote parts of the United States. Amazon says it hopes to launch a commercial service “in a matter of months.”

The drones could be programmed to trigger automated text or phone alerts if the system’s computer-vision algorithms spot something that could be a concern. Those alerts might go to the subscriber, or directly to the authorities.

“For example, if the surveillance event is the determination that a garage door was left open, an alert may be a text message to a user, while if the surveillance event is a fire, an alert may be a text message or telephone call to a security provider or fire department,” the inventors write.

But this raises a lot of data privacy concerns as this may allow drones to peep into people’s houses and collect information they are not supposed to. However, Amazon’s patent stating that, “Geo-clipped surveillance images may be generated by physically constraining a sensor of the UAV, by performing pre-image capture processing, or post-image capture processing. Geo-clipped surveillance images may be limited to authorized property, so privacy is ensured for private persons and property.”

Amazon has been curating a lot of user data using various products including the smart doorbell made by Ring, which Amazon bought for more than $1 billion in February last year. This smart doorbell sends a video feed customers can check and answer from their smartphone. Amazon launched Neighbors, a crime-reporting social network that encourages users to upload videos straight from their Ring security cameras and tag posts with labels like “Crime,” “Safety,” and “Suspicious.” Over 50 local US police departments have partnered with Ring to gain access to its owners’ security footage.

Amazon’s Key allows Prime members to have packages delivered straight into their homes—if they install its smart lock on their door and Amazon security cameras inside their homes.

Last month, the US House Oversight and Reform Committee held its first hearing on examining the use of ‘Facial Recognition Technology’. The hearing included discussion on the use of facial recognition by government and commercial entities, flaws in the technology, lack of regulation and its impact on citizen’s civil rights and liberties. Joy Buolamwini, founder of Algorithmic Justice League highlighted one of the major pressing points for the failure of this technology as ‘misidentification’, that can lead to false arrests and accusations, a risk especially for marginalized communities.

Earlier this year in January, activist shareholders proposed a resolution to limit the sale of Amazon’s facial recognition tech called Rekognition to law enforcement and government agencies. Rekognition was found to be biased and inaccurate and is regarded as an enabler of racial discrimination of minorities. Rekognition, runs image and video analysis of faces, has been sold to two states; Amazon has also pitched it to Immigration and Customs Enforcement.

Last month, Amazon shareholders rejected the proposal on ban of selling its facial recognition tech to governments. Amazon pushed back the claims that the technology is inaccurate, and called on the U.S. Securities and Exchange Commission to block the shareholder proposal prior to its annual shareholder meeting. While ACLU blocked Amazon’s efforts to stop the vote, amid growing scrutiny of its product.

According to an Amazon spokeswoman, the resolutions failed by a wide margin. Amazon has defended its work and said all users must follow the law. It also added a web portal for people to report any abuse of the service. The votes were non-binding, thus, allowing the company to reject the outcome of the vote.

In April, Bloomberg reported that Amazon workers “listen to voice recordings captured in Echo owners’ homes and offices. The recordings are transcribed, annotated and then fed back into the software as part of an effort to eliminate gaps in Alexa’s understanding of human speech and help it better respond to commands”.

Also, this month, two lawsuits were filed in Seattle alleging that Amazon is recording voiceprints of children using its Alexa devices without their consent.

This shows Amazon may be secretly collecting user’s data and now, with the surveillance drones they can gain access to user’s home on the whole. What more can a company driven on user data ask for? We’ll have to see if Amazon stays true to what they have stated in their patent.

While drones hovering over for surveillance seems interesting, it is actually collecting large volumes of user data, and a lot of private information. Black hat hackers who use their skills to break into systems and access data and programs without the permission of the owners may gain access to this data, which is a risk. They can further sell the data to 3rd party buyers including advertisement companies who may further use it to forward advertisements on particular products they use.

Amazon employees managing the data from these drones may also have certain access to this data. As a network administrator or security professional, the rights and privileges allow them access most of the data on the systems of user’s network. Also, one can easily decrypt the data if they have access to the recovery agent account. This creates an alarming state whether this extra private is data safe or not? On what level can intruders misuse this?

According to The Verge, “Amazon has patented some pretty eccentric drone technologies over the years that have never made it to market; including a floating airship that could act as a warehouse for delivery drones, a parachute shipping label, and a system that lets a drone understand when you shout or wave at it”.

https://twitter.com/drewharwell/status/1141712282184867840

https://twitter.com/drewharwell/status/1141793761283989504

To know more about ‘surveillance as a service’ read the patent.

Amazon announces general availability of Amazon Personalize, an AI-based recommendation service

US regulators plan to probe Google on anti-trust issues; Facebook, Amazon & Apple also under legal scrutiny

Amazon shareholders reject proposals to ban sale of facial recognition tech to govt and to conduct independent review of its human and civil rights impact