One of my favorite movies of all-time is The Zero Effect. If you haven’t seen it, you really should. Absolutely brilliant. One of the best scenes has Daryl Zero (played by Bill Pullman) explaining the trick to following someone without getting caught. It’s simple, he says. You just get where they’re going before they do. The obvious joke here is that it’s impossible to know where they’re going, which is why they need to be followed. But Pullman’s character in the film is a Sherlock-Holmes-sort-of-genius, and he expects that everyone ought to know where everyone else is going at all times. Just as a matter of course.

But we’re not all Daryl Zero. We can only guess at the future. We are resigned to follow.

A few years ago, I put forth a zany idea about where we in the book industry were heading: I thought there might come a day when agents and publishers would scour bestselling self-published books to wrangle in new talent. Doing so would allow them to mitigate risk; it would offload the arduous task of managing the slushpile by crowdsourcing it; and it would allow market research and experimentation to take place on someone else’s dime and time (the self-published authors’ and the readers’).

It wasn’t a Daryl Zero worthy prediction, even though it ended up coming true. The reason it wasn’t a great prediction was because it was too obvious. This discovery method was already taking place in other artistic forms. It was how music acts were discovered. Self-produced musicians would claw their way up, starting with gigs that lost or cost them money, all with the hopes of being seen and discovered by the right people. Musicians who wrote their own music without guidance, came up with band names, marketing ideas, logos, jacket art, even produced and recorded their own discs and handled direct-to-consumer transactions with fans.

Authors didn’t have a stage for their own gigs until e-books gained wide adoption. Once that happened, the rest was inevitable. Everything that musicians, photographers, comedians, fine artists and the like had been doing for generations was now open to writers. We could see where literature was heading because of the well-trodden path laid out before them.

By being at the forefront of this transition, Amazon has reaped most of the benefit. And not just monetarily, though that’s worth mentioning before we get to the meat of this rambling blog post. Amazon enjoys higher margins on self-published e-books. Before costs, they make 30% – 65% on every sale of a self-published work. For major publishers, their margin on e-books are likely less than 10%. They lose money on some of these transactions, just to hold the price down and ease adoption of e-readers. Their overall margin with traditionally published books might be under 5%. It certainly isn’t anywhere close to 30%.

But that’s just money. Where Amazon really wins is with the data they collect. When Amazon purchased Goodreads, it seemed to me that they were mostly purchasing the reviewing, buying, and reading habits of its users. Data like that is worth a lot of money. And its value is only going to go up in the future. This data makes it possible to recommend more items that customers will want to purchase. This is the most important trick in retail. It was my job as a bookseller. Keep this in mind, as I’ll get back to it when I posit how someone can win the next revolution in retail.But first, I’d like to digress and point out what might be an equally powerful advantage Amazon wins from their data: Not only are they seeing what customers buy, they’re seeing how authors sell. They see the next big thing before agents and publishers can. In fact, they can often see an author’s career trajectory take an upward turn before the author is even aware of it. Here is where I was wrong those years ago with my prediction. Agents and publishers aren’t going to be able to scour the bestseller lists for self-published up-and-comers, because soon they’ll find that Amazon has snagged them first. This is the power of Amazon’s vertical integration of its own publishing imprints. And it has only just begun.

I get a lot of emails from authors who are weighing publishing deals from various publishers. These emails invariably include sales numbers for reference. What I’ve noticed lately is that authors are getting offers from Amazon imprints much earlier in their career trajectories. The sales thresholds for a publisher’s interest aren’t getting more stringent. If anything, they seem to be getting softer. Amazon is simply getting the scoop. Which makes sense: they see the inflection in sales rates, incoming reviews, and read completion rates, before anyone else can.

This advantage of data is impossible to overstate. Employing hindsight (that clever genius), pundits have pointed out that Random House or a collection of publishers should have gotten into the e-reader device and direct e-book sales game early on. Rather than wait on a competitor to disrupt their business, they should have taken a page from Apple’s playbook and been the ones to disrupt. I’ll go one step further and use the power of hindsight to suggest something even more audacious: Publishers should have been the ones creating FREE self-publishing platforms. They should have created WattPad or their own version of Kindle Direct Publishing. Both offer ways to discover and profit from rising talent. Both offer ways to collect sales data from customers rather than relying on the infrequent and imprecise dribs and drabs of data that they get from retailers.

The investment costs for getting into the direct e-book sales game aren’t that enormous. It mostly takes web development and server space; publishers already owned the valuable content, all those manuscripts, which cost them a lot of money to acquire. Infinite numbers of copies of that content can be made and distributed for fractions of a penny per copy all across the word. But the chances of success in this market have dwindled to almost nothing, such is the dominance of established platforms. It is probably too late for direct sales from publishers. But there might still be time and demand to compete with a WattPad-type initiative. It would require major publishers to see self-publishing as a developmental program within their larger goal of discovering and acquiring talent.

I doubt this will happen, but it could. Aspiring writers would love to feel that they are in the Penguin Random House Development League. But the plan thus far from publishers has been to partner with vanity presses and attempt to wring money out of aspiring authors, the exact opposite direction they should have gone. The allure of quick profits got in the way of seeing what Amazon saw, which is the value of reader and author data. It’s an advantage that I don’t think can be overcome.

As publishers scramble to play catch-up, maybe they should consider Daryl Zero’s advice about following people. That’s what I like to do, however clumsily. I spend my time thinking about the next round, because retail is going to continue to change. Facebook will play a huge role in future retail, as a billion+ people spend much of their free time (and work time) connecting with friends and family, sharing suggestions, and placing themselves in front of ads. Google and Facebook are both sitting on mountains of data, and they are working on ways to leverage this advantage in retail. The book industry needs to think of ways to compete in this arena. Direct email campaigns are today the most powerful marketing tool self-publishers possess, but it requires getting email addresses from people who actually want to hear from us. How are publishers doing this?

They could start by offering backlist books at unbeatable prices, but only to subscribers or registered users. This would allow them to begin amassing their own big data, not just in reading habits but in customer contact info. Free e-book campaigns would be another great way of acquiring this information, especially if they could tie those free e-books to their own iPhone and Android apps in order to measure actual reading time, rate of completion, and gather reader reviews. Worrying about giving away content is shortsighted, when that content could be traded for this crucial data. Without that data, publishers don’t stand a chance competing in the arena that’s coming next.

What is coming next? I believe the future of retail is a shopping mall that caters to what we want, when we want it. Imagine walking through a virtual mall where the only other people we see are those we invite, and the only stores that exist are the ones we enjoy browsing. The future is an Amazon that knows not only that Mother’s Day is coming up, but what I got my mom last year, whether she returned it, how she rated the gift, and what my brother and sister are thinking about getting her. Future retail will suggest birthday gifts for our friends, serving as a reminder that those birthdays are coming up. We will never miss another anniversary. And we won’t be shown anything that our gift recipient has already bought for themselves. Instead, we’ll see items whose product pages they lingered on the longest. Even if they don’t add it to their wishlist, we’ll know what caught their eye.

Amazon, Facebook, and Google are all well poised to dominate in this future of retail, but booksellers and publishers could compete and carve out a niche if they threw their combined weight into the problem. There are some things physical bookstores still do better than online stores, and that’s the curation of things that we don’t know we’ll like until we see them. Online algorithms and interfaces are only so good. Until both get a whole lot better, it’ll still be more enjoyable for many people to browse a physical bookstore. The company or companies that win the next round will sort out the interface and algorithm issues first. It could be anybody. But I’m betting it won’t be the people who are hoping that change stops happening. Nor will it be the people who think most of the change has already taken place, and now they just need to hold their ground. It’ll be the people who see in advance and help steer all the change that’s coming.

The next revolution will mostly be a radical change in interface coupled with an incremental improvement in algorithms. The next interface is going to be virtual reality. Facebook’s acquisition of Oculus—a VR headset manufacturer originally funded by Kickstarter—was a brilliant move by a company well poised to become the future of the shopping mall. They have the data, our relationship trees, the ads we click on, the products we discuss, our birthdays and our anniversaries, everything needed to recommend the right product at the right time.

Facebook will know what foods we eat and enjoy, because we send them pictures of these items as well as our reviews; they just need image recognition engines and text parsers to make use of that data. They’ll know when our cars break down and that we need a mechanic or a spare part, because we complained about it on their site. Google will use maps data to know our route to and from work and remind us to get a coffee or a healthy breakfast. Amazon might know we’re going on vacation and have a product we ordered meet us there. These companies see the future, but there’s a lot of space to carve out. There are hundreds of millions of people who are going to always love a good book, and a good percentage are going to discover that all the convenience of online shopping can be married to the discovery process of a good bookstore when the two are merged by virtual reality.

Imagine a bookstore that knows your family, upcoming special dates and events, when you are shopping for yourself versus when you are shopping for your spouse. Imagine being able to “pick up” the books on display to read any page, check the back jacket, and then choose whether to buy the e-book, the paperback, the hardback, or even have some of the audiobook read aloud to you. Imagine a book trailer that pops up over the display of books, or a video window that shows an interview with or a message from the author. Or what if the books on the display change as you browse? You pick up one of Patricia Fitzgerald’s books, and the rest of her titles appear on the table, each on individual stands, to show you what else she’s written.

You could even summon a virtual bookseller to join you in your store to ask them questions. Amazon does this now with their Kindle Fire device. But imagine summoning a bookseller and being routed one who has read at least some of the books you are browsing. If you are standing in your own personal crime/thriller section, you would get a bookseller who enjoys those titles. The convenience of the machine will be married to the expertise of the human.

The company that creates this environment will begin collecting a new source of powerful data, and that’s our browsing patterns. How did we find what we ended up purchasing? How long did we linger over one book rather than another? How can we tailor our virtual bookstore for that customer’s next visit? How can we use their reading habit data so we know what they finished and what they gave up on?

Is it possible for brick and mortar stores and traditional publishers to compete in this space? I think they can. They just need to summon the willpower to make the early investments. It starts by gathering data on physical sales. Way back in 2007, Boekhandels Groep Nederland, the largest book distributor and retailer in the Netherlands, initiated an ambitious revamping of their physical stores. By working with Centraal Bookhuis, their printer, they began embedding RFID chips into every single book shipped. These chips could identify each book to remote sensors.

The advantages were enormous. It meant receiving books by passing the entire box through a scanner in the bookstore. It meant RFID antennas in the shelves could let customers and employees know where every book was in the store, even if the books was misshelved. Shrinkage (loss to theft) went down 60%, as the tags were more difficult to find and remove. Special order books were diverted at the receiving stage immediately and went to customers more quickly, increasing the chances of them using the store for future orders. Inventory at these stores is now done at the push of a button rather than shutting down for two days and scanning every single book. The labor cost savings have been enormous.

RFID has come way down in price, and it’s the only conceivable way for physical sales to be tracked the way online sales are. Every year that printers, publishers, and bookstores fail to adopt this technology and create industry standards for sharing the resulting data is a year of information and progress lost. Publishers could see what books are selling, in what quantities, where, and at what price (full or discounted). They could see demand for various formats. They could even do things that aren’t yet done online as effectively as they should be, like A/B testing of variations in cover art or back jacket material by supplying two different editions to two different regions. Simplifying the receiving process would result in less damaged books being comped and returned. Simplifying inventory and the ability to find an in-shop book would result in more sales.

(An aside: I was in London with Bella Andre a few weeks ago, and we went in search of one of her books for our display at the London Book Fair. The first shop we entered told us a nearby shop had six copies of the book. We went there and Bella watched as employees tore the store apart, only to come up empty. The books could not be found. Replacement copies weren’t going to be ordered, because of the six their computers said were in stock! Those six would never sell because they couldn’t be found. The problem would last until the yearly inventory came back around and the item was zeroed out. My store manager Bill and I saw this happen all the time at our shop, resulting in lost sales for us and the publisher.)

Disrupting how bookstores and publishers do business is a daunting and expensive task. Waiting for someone else to do it is dangerous and foolish. Future bookstores will be built around sales data and knowledge about customer reading habits. Right now, Amazon, Facebook, Google, and Apple are amassing such data. Publishers should be thinking about how they can get in this game. Creating writing communities to discover talent as early as possible would be a good idea. Giving away e-books using apps in order to glean reading habits, e-mail addresses, and selection algorithms would be a good idea. Integrating RFID into the supply chain as has been done in the Netherlands would be a good idea. There will still be a lot of catching up to do in other areas, but it might be time to start thinking about the future. And getting there first.