By the time someone clicks "buy" on Amazon, Jenny Freshwater's team has probably expected it.
Freshwater is a software director in Amazon's Supply Chain Optimization Technologies group. Her team forecasts demand for everything sold by Amazon worldwide.
This task, into which NPR got exclusive insight, underlies the entire Amazon retail operation. And it's central to Amazon's wooing of some 100 million people who shell out up to $119 a year for a Prime subscription, which guarantees two-day shipping.
Inside Amazon, corporate executives like to evoke magic when they talk about fast delivery. For months, they used the code name Houdini before launching their fastest service, Prime Now, which delivers household basics within hours.
But a lot of it is thanks to artificial intelligence. With AI, computers analyze reams of data, making decisions and performing tasks that typically require human intelligence. AI is key to Amazon's retail forecasting on steroids and its push to shave off minutes and seconds in the rush to prepare, pack and deliver.
"It goes beyond just being able to forecast we need a hundred blouses," Freshwater says. "We need to be able to determine how many do we expect our customers to buy across the sizes, and the colors. And then ... where do we actually put the product so that our customers can get it when they click 'buy.' "
That's a key element to how Amazon speeds up deliveries: The team predicts exactly where those blouses should be stocked so that they are as close as possible to the people who will buy them.
(Note: Amazon is one of NPR's financial supporters.)
This process is even more essential now that the race is on for same-day and even same-hour delivery. Few other retailers have ventured into these speeds, because they're very expensive. And few rely quite so much on AI to control costs while expanding.
Both AI and forecasting are not unique to Amazon. All retail stores work hard to prepare, for example, for seasonal or weather-related demand. And all major retail companies have their own algorithms, automated warehouses and delivery tricks.
But it was Amazon Prime that got Americans hooked on two-day shipping, which shoppers now take for granted. Walmart, Target and many others now offer two-day deliveries even without membership fees.
While all large online retailers rely on AI, "Amazon definitely has the most powerful tools for all the little computational processes involved in moving the packages through many suppliers, routes of transit and all the steps that a package goes through," says Mike Liebhold, a senior tech researcher at the Institute for the Future.
Indeed, AI is woven through every part of an Amazon purchase, from the website to the warehouses to the actual delivery to your doorstep. In corporate lingo, Amazon calls that the "first mile," "middle mile" and "last mile."
Forecasting for the "first mile"
In 2013, Amazon got a patent for so-called "anticipatory shipping." The idea was to get your order as close as possible to your address before you actually click buy.
Since then, Amazon has built a massive warehousing footprint around the country. And it's been adding smaller warehouses closer to city centers where Prime Now promotes super-fast delivery options. It's also using Whole Foods locations for faster access to groceries and basic pickups.
Amazon regularly tests what new products people might want with extra-fast shipping, says Cem Sibay, who runs Amazon Prime in North America. "If you see customers consistently trying to select for same-day delivery, then the next step is obviously that some customers maybe actually even need this faster," he says.
Amazon decides what to stock by looking at every detail of each product. It takes a shirt and analyzes the color, length, silhouette, sleeve length and histories of purchases for similar clothing items. Freshwater, of Amazon's forecasting team, says her group does not have access to personally identifiable information on specific shoppers. But her team looks at past demand and online-shopping habits in aggregate to write algorithms and statistical models that teach the AI.
Some of Amazon's predictions are not surprising. "You don't sell very much Chicago Cubs gear in Hawaii," Freshwater says.
Others are less obvious. For example, AI has learned that not all new products mean the same type of forecast.
Take tax software. Everyone wants the latest version. But the release of a new DSLR camera? That actually triggers huge demand for the older versions, which are cheaper.
AI has also determined that online shoppers often abandon their online grocery cart entirely, if bananas are sold out — and that bananas are most in demand on Mondays.
Or consider sunscreen. It's obviously in demand in the summer, but Amazon's AI models have also detected mini-surges around winter holidays and spring breaks — when people go on trips to warm sunny places.
Whether you search on Amazon.com or shop using Alexa on the Echo device, AI determines what results or recommendations you see and tells you how fast you might get each item.
The "middle mile"
Inside Amazon's warehouses, AI powers the Kiva robots, which look like large Roombas carrying bright yellow shelves.
In traditional warehouses, it's the people who walk to the shelves. But in most Amazon warehouses, the robots bring the shelves to people.
The machines know what to bring and when to get orders packed so they are in time for delivery. AI systems keep track of all items in the warehouses, which can be as vast as 1 million square feet. AI constantly arranges and reshuffles the shelves so that things you're about to buy are ready to go.
Amazon is now rolling out new efficiency-boosting technology that eliminates the need for handheld scanners, says Brad Porter, head of robotics for Amazon Operations. The new system retrofits workers' stations with advanced cameras that can automatically scan items that workers hold in their hands. AI is learning to discern and interpret people's hand movements to recognize when a person places an item inside a shelf slot.
This kind of innovation is a controversial topic in retail, where layoffs are rampant, just as automation is reshaping the workforce. Economic research is divided on how much exactly AI will eliminate or create jobs, especially for lower-income Americans.
For example, Amazon has been hiring people by the hundreds of thousands for its highly automated warehouses. Its workforce has ballooned to 613,300 in a matter of a few years. Warehousing overall has been one of the fastest-growing industries, even though a lot of that work is part-time or temporary.
At the same time, Amazon has launched AI-powered stores that eliminate the need for a cashier, stoking fears about the future of jobs like these industry-wide. Plus, as Porter will point out, scientists have long been working on robots that can imitate human hands with their grip. For work like packing in warehouses, an agile robotic hand could erase one key edge that human workers now have over machines.
To Porter, modern automation is simply in the latest chapter of industrial progress.
"Industrial automation and robotics are here. They've been here for a long time," Porter says. "I think what's novel or hard in the logistics space and for Amazon is just the diversity and selection of inventory that we have and the scale at which we're doing it. And so yeah, I think we'll continue to see technology innovation in all these spaces."
The "last mile"
One area where AI has created a new type of job is in deliveries.
Amazon has many logistics contracts for deliveries, with the United States Postal Service, FedEx, UPS and other smaller companies. But it also took a page from Uber and began hiring drivers for side-gigs to make superfast deliveries in busy cities, paying between $18 and $25 an hour.
The app is officially called "Flex," but its original name is "Rabbit" — "along the magic theme," says Beryl Tomay, director of Amazon's Last Mile Delivery Technology.
The process relies on AI to constantly calculate how many drivers are needed at any given time. The app considers the weight and number of packages headed to the same neighborhoods and whether packages can fit in one of the available cars. The app even recommends the order of packing boxes into a car for the most efficient delivery.
One big challenge for Flex drivers is parking. Amazon's app does not guide where to leave cars while dropping off a package. However, Amazon does factor how long it took other drivers to deliver to an address in its estimates. AI's timing estimates also consider traffic and types of building.
"A single-family home is fairly straightforward, whereas if it's a commercial building or an apartment complex, that gets a little bit more difficult," Tomay says. "It's not always obvious where the unit is or are we taking it to a mailroom or receptionist?"
As Sibay describes it, when a delivery a one-hour race, every minute counts. So delays and missed deadlines get a thorough analysis.
"Were we not able to pick it in time from our warehouse shelves? Did we not have the right number of drivers?" says Sibay. Or maybe "the driver forgets his key at reception and has to walk a little longer. The driver's delivering a package and it's an elderly lady and they, you know, talk a little bit before he moves on to his next order."
It's hard for AI to predict all these scenarios, Sibay says. But next time, maybe the address with the Chatty Cathy will get a few more minutes baked into the algorithm.
RACHEL MARTIN, HOST:
Americans take for granted that most things they buy online will arrive on their doorstep two days after they clicked the order button. When Amazon introduced two-day shipping, it was a huge shift in retail thinking. To stay competitive, most major stores now offer similar speedy delivery. But getting you that package in just two days - or in some cases in just one hour - is extremely expensive. NPR's Alina Selyukh got a rare look at how Amazon makes it work.
ALINA SELYUKH, BYLINE: By the time someone clicks buy on Amazon, usually Jenny Freshwater had long predicted it.
JENNY FRESHWATER: In advance of them even knowing they want the product, in many cases.
SELYUKH: Her team's job is to anticipate demand for every single thing sold by Amazon in the world - not just how many blouses should Amazon sell but what color, size, sleeve length and most importantly...
FRESHWATER: Where do we actually put the product so that our customers can get it when they click buy?
SELYUKH: This is key to how Amazon cuts down delivery time, by stocking items as close as possible to the people who will buy them. That's why Amazon has been building smaller warehouses closer to city centers, where delivery might be within hours. And we should note, Amazon is one of NPR's sponsors.
FRESHWATER: We've started to build these AI algorithms, and the more we build, the better they get.
SELYUKH: AI algorithms, as in artificial intelligence, as in computers analyzing massive amounts of data. For example, AI knows that new doesn't always mean more sales. Like, with tax software, sure it does. Everyone wants the latest version. But a new DSLR camera actually creates more demand for the older version, which is cheaper. AI also knows that people often abandon their online grocery cart if bananas are sold out. And think about sunscreen. AI found many surges in the winter and spring.
FRESHWATER: Right around the holidays and then again when schools typically have spring breaks.
SELYUKH: Both AI and forecasting are not unique to Amazon. All retail stores work hard to plan for the future, and all major ones have their own algorithms and automated warehouses and delivery tricks. But it was Amazon Prime that got Americans hooked on two-day shipping. Now it's a race for a one-hour delivery with Amazon Prime Now. Few companies can afford that, and few rely quite so much on AI to control costs while growing.
CEM SIBAY: Really, AI is an underpinner technology for almost the entire Prime experience.
SELYUKH: Cem Sibay is an executive at Prime whose promise of free two-day shipping is the main reason millions of Americans shell out $119 a year for membership. Sibay and other Amazon executives push this illusion that fast delivery is magic. Like, the code name for Prime Now was Houdini. But the reality is forecasting on steroids and a meticulous shaving off of minutes and seconds on the journey of the package.
SIBAY: A lot of it is sort of end-to-end control of the experience as well.
SELYUKH: As in, Amazon has control of the entire process from the website to the warehouses to the actual delivery to your doorstep. In corporate lingo, that's first mile, middle mile and last mile. AI is woven through it all. In the first mile, when you order, AI analyzes your search and tells you upfront how fast your item could ship. In the warehouse - that's the middle mile - AI powers the Kiva robots. They look like large Roombas carrying yellow shelving units. In traditional warehouses, it's the people who walk to the shelves. Here, it's the robots that bring the shelves to people. The machines know what to bring and when to get each order packed in time for delivery. It's AI keeping track of all items in almost a million square feet of this warehouse. AI constantly arranges those shelves so that things you're about to buy are ready to go.
BRAD PORTER: There's a mix of industrial automation, manual processes and more sophisticated robotics.
SELYUKH: Brad Porter is the head of robotics for Amazon operations. This is controversial work in retail, where layoffs are rampant just as automation reshapes the workforce. Economists are divided on how much exactly AI will eliminate or create jobs, especially for lower-income Americans. In its defense, Amazon often points to how much it's actually been hiring. To Porter, we are in the latest chapter of industrialization.
PORTER: Industrial automation and robotics are here. They've been here for a long time.
SELYUKH: One area where AI has created a new type of job is in deliveries in the last mile. In busy cities, Amazon has to pull out all the stops. The company took a page from Uber and now hires drivers for side gigs, making superfast deliveries that pay as much as $25 an hour. AI fuels this. It matches package size to car size and even recommends what package to put in last. Sibay says when you're in a one-hour race, every minute counts. So AI's timing estimates consider traffic and building entry codes, and it learns from its mistakes.
SIBAY: The driver forgets his key at reception and has to walk a little bit longer. The driver is delivering a package, and it's an elderly lady. And they, you know, talk a little bit.
SELYUKH: It's hard for AI to predict all these scenarios, he says. But next time, maybe the address with a chatty Cathy will get a few more minutes baked into the algorithm.
Alina Selyukh, NPR News, Seattle.
(SOUNDBITE OF DECEPTIKON'S "INACCESSIBILITY") Transcript provided by NPR, Copyright NPR.