Distributed computing frameworks like Hadoop and Spark have enabled processing of "big data" sets -- but that's not enough for modeling surprise/rare "black swan" or complex events. Just think of scenarios in disaster planning (earthquakes, terrorist attacks, financial system collapse); biology (including disease); urban planning (cities, transportation, energy power grids); military defense ... and other complex systems where unknown behaviors and properties can emerge. 

They can't be modeled based on (by definition impossible) limited data. And parallelization for this is hard. But what if companies and governments *could* answer these seemingly impossible questions -- through simulations? Especially ones where we can directly merge in knowledge and cues from the real world (sensors, sensors everywhere)? CEO of Improbable Herman Narula and Stanford University professor-in-residence at a16z Vijay Pande discuss this and more with Chris Dixon in this episode of the a16z Podcast. And as Herman says, "the cool stuff only happens at scale".

---

The views expressed here are those of the individual AH Capital Management, L.L.C. (“a16z”) personnel quoted and are not the views of a16z or its affiliates. Certain information contained in here has been obtained from third-party sources, including from portfolio companies of funds managed by a16z. While taken from sources believed to be reliable, a16z has not independently verified such information and makes no representations about the enduring accuracy of the information or its appropriateness for a given situation.
 
This content is provided for informational purposes only, and should not be relied upon as legal, business, investment, or tax advice. You should consult your own advisers as to those matters. References to any securities or digital assets are for illustrative purposes only, and do not constitute an investment recommendation or offer to provide investment advisory services. Furthermore, this content is not directed at nor intended for use by any investors or prospective investors, and may not under any circumstances be relied upon when making a decision to invest in any fund managed by a16z. (An offering to invest in an a16z fund will be made only by the private placement memorandum, subscription agreement, and other relevant documentation of any such fund and should be read in their entirety.) Any investments or portfolio companies mentioned, referred to, or described are not representative of all investments in vehicles managed by a16z, and there can be no assurance that the investments will be profitable or that other investments made in the future will have similar characteristics or results. A list of investments made by funds managed by Andreessen Horowitz (excluding investments and certain publicly traded cryptocurrencies/ digital assets for which the issuer has not provided permission for a16z to disclose publicly) is available at https://a16z.com/investments/.

Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Please see https://a16z.com/disclosures for additional important information.

The Cool Stuff Only Happens at Scale

The a16z Podcast discusses tech and culture trends, news, and the future -- especially as ‘software eats the world’. It features industry experts, business leaders, and other interesting thinkers and voices from around the world. This podcast is produced by Andreessen Horowitz (aka “a16z”), a Silicon Valley-based venture capital firm. Multiple episodes are released every week; visit a16z.com for more details and to sign up for our newsletters and other content as well!

Popular tracks tagged #distributed computing