O'Reilly's Ben Lorica chats with Patrick Wendell, release manager of Apache Spark and co-founder of Databricks, about how he came to join the UC Berkeley AMPLab, the current state of Spark ecosystem components, Spark’s future roadmap, and interesting applications built on top of Spark.

Apache Spark: Powering Applications On-Premise and in the Cloud

O'Reilly Media spreads the knowledge of innovators. At O’Reilly, a big part of our business is paying attention to what’s new and interesting in the world of technology. We have a pretty good record at having anticipated some of the big technology developments in recent history. For instance, we launched the first commercial Web site, GNN, in 1993; we organized the meeting at which the term “open source” was first adopted; we were early investors in Blogger, which helped launch the blogging revolution; and more recently, our Web 2.0 conference launched a world-wide meme. We call this predictive sense the “O’Reilly Radar.” And while we’re certainly not always right, we are, at least, good at making interesting guesses.

Our methodology is simple: we draw from the wisdom of the alpha geeks in our midst, paying attention to what’s interesting to them, amplifying these weak signals, and seeing where they fit into the innovation ecology. Add to that the original research conducted by our Research team, and you start to get a good picture of what the technology world is thinking about.

Apache Spark: Powering Applications On-Premise and in the Cloud

O'Reilly Radar