Peter Hopkins: Hi, I’m Peter Hopkins, President and Co-Founder of Big Think. Joining me today is James Surowiecki. He’s a staff writer for The New Yorker, a very accomplished business writer and author of The Wisdom of Crowds, which explores how collective wisdom can actually transcend and amplify the knowledge of even the smartest individuals. James, it’s great to have you here today.

James Surowiecki: Thanks for having me.

Peter Hopkins: So, let’s get started. I want to begin with the book. It’s a really interesting concept. I think it defies a lot of expectation because groups are traditionally seen as somewhat dim-witted; you know, the stupid masses, if you will. What led to the insight that originally caused you to explore how crowds might actually have some wisdom?

James Surowiecki: Well, the story is actually kind of interesting in retrospect because what happened is when I first started writing about business in the mid-1990s, I really didn’t have much experience thinking about or writing about business and markets. And I was struck by the fact that a lot of times, although, as we’ve seen recently, by no stretch of the imagination always—markets actually tend to often be smarter than most of the people in them. So, we know that over time most investors, even very accomplished investors, underperform the stock market and the like. And the traditional explanation for why this is, if you looked at classic economics textbooks, they would talk about how investors are perfectly rational, they have perfect information, long-time horizons and the like. Well, that’s obviously not true at all. Investors are often very irrational; they act on very little pieces of information.

I was interested in how can you take all of these very, very imperfect, and very, very flawed individuals and, collectively, how can they arrive at reasonably good outcomes—not great outcomes, but reasonably good outcomes? And when I started looking at that, what I started coming across were a lot of examples that didn’t have anything to do with markets. Examples of groups again and again being able to collectively solve problems or come up with answers that were actually better in some cases than the smartest person in the group. And that, oftentimes, were very, very complex. So, over time, what I started to realize, or decide, depending on what you think of my book, is that it wasn’t that there was something about markets that was especially magical—and actually, in some ways I think markets are one of the more imperfect examples of the wisdom of crowds—but that actually they were just a specific example of this bigger phenomenon that I ended up calling “the wisdom of crowds.”

Peter Hopkins: Let’s outline a specific example so people can better understand. You use the example of a county fair in the book. Tease that out for us. Explain what that’s all about and how that would operate.

James Surowiecki: So, the story that opens the book is the story of the British scientist, Francis Galton, who is sort of famous in terms of—he was one of the first people to really use statistical analysis to try to think about populations and individual behavior and things like that. But this particular story is about a century ago. He was at this county fair in the west of England and, as he was walking through the fair, he came across this contest. And the contest was that an ox had been placed on display and people were basically lined up to guess the weight of the ox. Now, actually what they were being asked to do was to guess the weight of the ox after it had been slaughtered and dressed, which was sort of unfortunate for the ox, but if you think about it, it complicated the task. It was a little more complicated than just guessing the weight of an ox as it stood there. And it was actually a big crowd. It was actually 700 people, and there was a reward; if you did well, you’d win a prize. So there was some incentive for people to do a good job of trying to guess the weight of this ox.

So, after the contest was over, Galton went up to the guys who had organized the contest and he said, “Can I have the slips?” Sort of testimony to his brilliance as a scientist that he realized this might be useful information. And then what he did was, he just did a series of statistical tests on them. So, he graphed them to see if they formed a bell curve, to see if the distribution was normal. He calculated the standard deviation, and then he did a couple of things. He calculated the median guess and then he also eventually ended up calculating the mean guess, or the average guess.

Now, Galton, as a lot of people know, was this notorious elitist. He was—for all the virtues of his statistical work— he had this great vice, which was basically that he was the founder of eugenics. So, he basically thought that only a few people in the world really had the capabilities to make decisions or solve problems. So he, not surprisingly, thought that the group’s average guess was going to be way off the mark. He basically thought that you were taking the guesses of a few smart people, some mediocre people, and then a lot of morons. Because he thought these people were dumb, basically. And so he thought that the group’s estimate was going to be really flawed.

But as it turned out, the ox weighted—I always get the two numbers confused, but the ox weighed 1,197 pounds, I think, and the group’s average guess was that it would weigh 1,198 pounds. So it was basically, essentially perfect. And in this case it was actually better than any one person in the group. Now, that’s an extreme example. Most of the time the wisdom of crowds is not going to give you that perfect a guess and, in fact, if you do experiments like this one, say with a jar of jelly beans—that’s another way you can do it—the group’s average guess—there’s usually a few people in the group, a couple who do better than the group as a whole, but it’s actually, I think, a kind of great image and a great example of what the wisdom of crowds can actually look like.

And one other thing I’ll say about it that is important is that the group was not just made up of experts, so it wasn’t just made of butchers and farmers who probably were pretty good at doing this; it actually included a lot of other people. People like clerks or family members, people who were just there to kind of take a guess as a lark. And so the diversity of the group actually was something else that makes its intelligence interesting.

Peter Hopkins: Now, you write in the book that not all of these kinds of markets are created equal. There are certain characteristics that help make them better able to predict than others. Outline some of those. What makes certain crowds, I guess, smarter than others? What are the conditions that need to prevail to make them prescient in that way?

James Surowiecki: Yeah, I think that’s actually an important point. You hear the phrase “the wisdom of crowds—it’s bandied about a lot, which is nice. I mean, it’s nice that people have, to some extent, taken up the idea. But it isn’t as simple as just kind of casting about out widely. Ask as many people as possible, and you’re going to end up with good results. There are a lot of examples, and we just have lived through one of—well, we’ve lived through two if you think about the Internet bubble of the late ‘90s and now the housing bubble of this decade. There are a lot of examples of crowds being either really extreme in their behavior. So, really volatile or taking positions that are way out there. Or, alternatively, a lot of times crowds are often quite mediocre. They’re just sort of very ordinary in their thinking. And so I think it is really important to recognize that you need to do certain things to get people to be wise as a group.

I think the three things that are most important are— first of all you need, and this is kind of obvious, but it’s actually quite important—you need some way to aggregate lots of different individual judgments to produce a kind of collective judgment. So, for me, the wisdom of crowds isn’t just about a kind of suggestion box. A lot of people, for instance, sometimes talk about, kind of, the blogosphere as an example of the wisdom of crowds. And certainly it’s better to have lots of voices rather than a few, but if you don’t have a mechanism for taking a lot of different opinions and aggregating them in some way, then you just have a lot of different individual opinions, and you’re not necessarily coming up with a true kind of wisdom of crowds thing.

The second thing, which I actually think is probably the most important, is that you really need the group to be diverse. So, you really want people who are looking at a problem or trying to come up with an answer from lots of different perspectives. You want people who are using different kinds of tools to think about how to solve a problem. And that actually can be a big challenge. It can be a big challenge inside organizations because a lot of organizations push people to be homogeneous in their thinking. Sometimes organizations just aren’t good at recruiting for diversity. You hear a lot of lip service paid to diversity in corporate America, but a lot of times organizations tend to hire people who sort of “fit the mold.” And I’m not really talking here—well, to some extent that’s true just kind of sociologically. You tend to hire people with similar backgrounds to you or who look like you, literally, or who come from similar schools. I think the more important thing is people tend to hire people who have similar perspectives to them. When companies build teams, they oftentimes tend to hire people or recruit people who are going to fit in, and I think that’s a big problem. So diversity is very, very important.

And then the last thing which is connected to diversity but I think is slightly different from it, is that even once you have a diverse team or a diverse crowd, you also need to make sure that people are able to be independent in their thinking. You need to make it possible and easy for people to think for themselves rather than them having taking a lot of cues from each other. And although in principle it seems like, well, that should be easy. You know, most of us want to be independent thinkers. We don’t think about ourselves as being conformists. It can actually be pretty challenging. And I think actually, in some ways, the Internet magnifies the problem of what I kind of call “dependent thinking” because it’s very easy for small networks to get set up where people tend to only hear the voices of people they agree with. And it’s very easy for feedback loops to get set up that make it easy again for the same kind of ideas to get more and more reinforced.

So I would say the three things are aggregation, and then the two most important are diversity and independence.

Peter Hopkins: Now, there’s a lot to unpack there.

James Surowiecki: Yeah, sorry for going on like that.

Peter Hopkins: Oh, not, not at all, not at all. The diversity issue is a really interesting one.

James Surowiecki: Fascinating, yeah.

Peter Hopkins: Because for a long time the conversation about diversity has really been dismissed by its critics as tokenism and superficial inclusion, but this suggests there is way more to it. Are there examples that you have encountered of companies who are really explicitly thinking of diversity in terms of bringing these varied perspectives to their businesses, and how have they approached that?

James Surowiecki: Yeah, it’s a great question. And even as you ask that, nobody exactly comes to mind. I mean, I think that there are companies that are doing this. I think that the interesting thing about it is, if you look at corporate America, it definitely has gotten better in terms of recognizing the need, for instance, to break down silos. I mean, there are financial firms that have tried to do a better job of that, although it would be hard pressed to say the results have been extraordinary. But I think that’s one of the big things that the companies have done.

I think that if you look at a company like Google, which on the one hand, obviously, is not diverse in the sense that it tends to hire very, very smart people, but I think is actually trying to be diverse in terms of the kind of people it hires. And also, in terms of breaking down, not just silos between divisions, but also hierarchies inside the organization. So, I think that’s a company that has tried to do it.

IBM arguably is a company that also is trying to do this in terms of building cross-disciplinary teams. Working across the organization and the like. But I think, in general, corporate America is trying to do a better job of that. I think the big challenge is really that when the—to use a cliché— when the rubber hits the road, the problem is not necessarily the organizational-wide issue. The problem is oftentimes inside the organization when it comes to things, as I said earlier, like building teams or figuring out how you’re going to do projects. And one of the challenges there is that although we kind of like the idea of diversity maybe in theory, you know, we sort of like the idea of the independent thinker; we like the idea of getting lots of different opinions. In practice, it turns out that diversity is hard to implement on an organizational level. And it’s hard to implement because one of the things that diversity does, and one of the things it requires, is a willingness to have arguments. It’s a willingness to actually accept differences of opinion. It doesn’t sound like that radical of a concept. But, in fact, when you look at surveys, for instance, of people who have worked on diverse teams versus people who work on more homogeneous teams, what you find is that even when the diverse teams perform better, most of the people on them are less satisfied with their experience. And people say things like “I felt like I wasn’t listened to; I felt like I was taken advantage of” or whatever. A lot of which really means, my ideas weren’t necessarily implemented.

And so, I think that that’s one of the biggest challenges when it comes to turning diversity from a kind of a nice idea into something that is practically really powerful.

Peter Hopkins: That actually raises a very interesting question that we have hit on a number of times in our interviews at Big Think, which is, what are the expectations of the individual in this new economy? It seems—your work seems to suggest that there is a way of approaching information and an attitude that needs to prevail within the individual so that they are participating effectively. Can you distill any, almost, axioms or new lessons that a worker, particularly a displaced worker, who might be out of work thinking, how do I stay competitive? What should they be thinking about as they approach information? How should they be regulating their relationship with all this sea of stuff?

James Surowiecki: Well, that’s a really interesting question. Well, I think that there are a few questions inside that question. And so let me just say a couple of things about it.

The first is, I actually think it’s important when you think about the idea of the wisdom of crowds to recognize that it is not about replacing or making the individual irrelevant. In fact, if you think about the importance of diversity and independence in making crowds intelligent, it actually stresses in some ways the fundamental nature of individuals thinking for themselves. So, in fact, crowds work least well when people are just kind of going along with what everybody else is doing or kind of swimming with the crowd in a sense. There’s a paradox here. But the paradox is really that crowds are smartest when people in them are thinking as much like individuals as possible.

I would argue, in fact my argument for, if you want to try to understand things like bubbles and what happens during bubbles—stock market bubbles or other kinds of asset bubbles—is that, actually, independence really breaks down in some fundamental sense, and people start making their decisions primarily based on what everybody else is doing. There’s always an impulse to do that in markets, to try to ride the tide and then hope you can get out early, basically. But I think what happens during bubbles is that that’s really amplified.

So, one argument for what an individual should do is, actually there is a premium, or there should be a premium placed on trying to think for yourself. That sounds very ****, but I have to say, I think it’s true. There’s a premium placed, I think—there should be a premium placed really on diversity of thought and diversity of background. So, I actually think, interestingly, and maybe hopefully—maybe falsely hopefully— that going forward, organizations should place more of a premium on people who are able to approach problems from kind of diverse backgrounds. So, instead of necessarily saying we need to hire people only who have a very traditional model of training, very traditional backgrounds, that there really is a case for hiring people who have slightly more unconventional backgrounds. And so, in and of itself, I don’t think that’s something that people should try to avoid.

The other thing I would say in terms of the individual, in terms of diversity, I think is really interesting is, the Internet for me is actually one, if not the, one of the greatest tools for promoting diversity of thought that’s ever been invented. I mean, if you think about, as you said the “sea of information,” but if you think about the shear variety of information sources or perspectives that are out there, it’s extraordinary. Right? There’s no better tool for actually kind of tweaking your mind and looking at things from a different perspective and the rest of it. The problem is that I think most of us just almost naturally don’t really use the Internet in that way. That, actually, if you look at those kinds of network maps that trace the sources of where do people link to, where are links coming from and the like, you actually see much tighter connections than you would maybe hope for or would expect. And so one of the lessons I really think is important is the way I put it is, try to keep your connections on the Net loose, rather than tight. Have lots of loose connections rather than just a few tight connections. I think that applies to social networks, but I think it also applies and probably is more important when it comes to information networks.

Even if it infuriates you, read sites that don’t give you what you already kind of expect, what you already think. It’s very easy to do on the Net. It doesn’t take up a lot more time or whatever, but I think that that is really one of the keys on an individual level to kind of really reaping the best of what’s out there. And also to becoming a better thinker.

Peter Hopkins: Now I want to look at the example of the economic crisis because it is a perfect case study for understanding both the benefits and detriments of the wisdom of crowds. Today is the first meeting of the commission that is looking into the causes of the economic crisis, and you’ve got a lineup of all of the CEOs of the major banks from Lloyd Blankfein of Goldman Sachs, to Jamie Dimon of J.P. Morgan. Not all of these banks fared the same in this crisis, and perhaps they had a different approach to the wisdom of crowds. Have you been able to see any of the insights of your work in how these banks approached the crisis? Why Goldman and J.P. Morgan on the one hand fared so well, and why Merrill Lynch or now defunct Bear Stearns is just that. What distinguished them on the basis of your theories?

James Surowiecki: Well, it’s actually an interesting question, and I don’t know that there’s a simple answer to it. I mean, I think that no bank really fared that well during the crisis, but you’re right. There actually was a pretty significant gap between those that survived—not just those that survived, even of those that survived, those that did better and those that did worse. I think that there are a few things to keep in mind. One is that there were systemic problems on Wall Street that I think reach across all institutions. And that I think helped skew the collective judgments, so to speak, of the market in the last sort of decade or so. I think that one of the things is that the way compensation structures work on Wall Street, so the short-term versus long-term compensation issue makes a big difference. To the extent that you were rewarded primarily, or in some cases I think, maybe entirely on the basis of short-term results. Let’s say short-term being a year, or whatever it is. That, I think, actually inclines you to try to do more of the looking at what others are doing and trying to guess what they’re going to do next, rather than trying to actually value the asset at what it’s really, really worth.

And that tends to basically distort the entire market, especially when you have most of the people doing that. Because if you think about it, what you end up with is a situation where instead of trying to say to yourself, how much should this collateralized debt obligation that is made up of these home mortgages—how much is this actually worth? What’s the cash flow that it’s going to produce discounted by the— whatever or however you’re supposed to do it? Instead of doing it, you’re basically saying, everybody else says it’s worth “X,” right? And so, therefore, it must be worth “X,” or I’ll be able to at least gain the system in that way. That’s a huge issue. And if this thing isn’t going to blow up in a few years, even if I’m wrong, well then my incentive is to do it.

I think the same thing was true, obviously, in terms of the home mortgage market more generally. So, to the extent that, what’s called a securitization model, so people would make the mortgages and then package them together and sell them off, so they no longer had a stake in the individual mortgage that they made. That tended to, I think, diminish the “I’m looking at you trying to decide how reliable you are, how likely it is you’re going to pay me back,” etc. And as a result, again, you kind of get a distance between the real decision and the actual price of the asset. I think that had a big problem as well.

And I think, to be fair, one of the things that starts to happen, and interestingly, I don’t think it was the press’ fault in the housing bubble case. I mean, people disagree with me, I think, but I actually think if you go back and look, it’s surprising how many accounts there were, not necessarily in newspapers, but in magazines—not just business magazines but across the board—about the unreasonableness of the housing price run-up. But on Wall Street, there really was a kind of across-the-board conviction that housing prices would not fall across the board, basically. So, there really was a lack of diversity of thought, and people who were trying to offer up other opinions at a lot of institutions basically found themselves isolated. And so you really had that kind of breakdown of diversity that I talked about, and I think that that was one of the big problems.

And I think if you look at institutions that did better and those that did worse, those that did better did a better job of accepting diverse opinions, basically. And accepting the possibility that what might cost them in the short run would be better for them than in the long run.

You know, if you look at J.P. Morgan, for instance, and J.P. Morgan’s decision to get out of the CDO market and a lot of the subprime market. A lot of it really consisted of—they didn’t get out of it completely, but they were obviously able to avoid most of the excesses. A lot of it came simply from the fact that they looked at the numbers and they couldn’t make the numbers work. That is to say, they were quite literally just thinking for themselves in this way that I said that you needed to. And when they found that the numbers didn’t work—in other words, you couldn’t buy something at this price and have it justified by the return you were going to get—they basically were willing to say “no.” And I think that, in general on Wall Street, people were not willing to say “no.” They were basically willing to say “If people are willing to buy it, we are willing to sell it and etc.” So, I think that’s one of the biggest differences too.

Peter Hopkins: And so, there were some devil’s advocates out there.

James Surowiecki: Right.

Peter Hopkins: What was their role during all of this? How did they instigate the beginning of the end, if you will?

James Surowiecki: What do you mean, instigate the beginning of the end?

Peter Hopkins: I mean, there were some lone voices out there who were naysayers, so this can’t possibly be real. How did that go from a few lone voices to, ultimately, the whole thing crashing down on us?

James Surowiecki: So, that’s a fascinating question, like how does a crash finally happen? Right. And I think in this case, it’s actually pretty straight—well, I shouldn’t say straightforward, but I think the housing bubble in large part was a case of, to use an overused metaphor, a case of running on air and being okay until people looked down, basically. And that what happened was finally people looked down and realized that there was nothing below. And the reason that people looked down was they were forced to because at a certain point, incomes just basically would no longer support the housing prices that people were being expected to pay. As interest rates to some extent started to rise, and as the refinancing boom ended, the entire mortgage bubble had been built to some extent on people’s ability to refinance. When that started to fail, the whole edifice started to crumble, and I think at some point in bubbles—and no one really knows exactly why it happens—at some point in bubbles people are just no longer willing to keep paying just in the expectation that others will keep paying, and once that starts to happen, the whole thing starts to fall apart because there’s nothing really that—there are no fundamentals justifying the bubble. That was the case in 1999 when it came to technology stocks and it was the case in housing, I would argue, probably from about—depends on when, but I would say from 2004, 2005 on.

And so that was really how it started to accelerate. The other thing I would say, though, about the lone-voices thing that’s interesting is—I do think one of the other problems with this bubble, and it was also a problem in the technology bubble, maybe not to the same extent, is that even to the extent that there were lone voices or voices out there that were trying to challenge the dominant discourse on the housing bubble, it wasn’t actually until relatively late in the game. Hard for them to have their opinions registered in terms of market prices. So, it was possible to bet against the housing bubble using credit default swaps and the like. That’s how John Paulson, hedge fund manager, made $15 billion, I think. And what Gregory Zuckerman has called the greatest trade ever made. But it was actually hard to do that on any kind of systematic basis.

With the stock market at least, if you think a stock is overpriced, you can sell it short. It’s hard to do that—I mean, it’s not too hard to do that, but most people don’t do it during the Internet, but it’s hard to do it because it’s actually hard to borrow shares. But you can do it. But with the housing market, for the most part, if you thought the housing prices in Las Vegas were incredibly overpriced, if you were an ordinary person who was like “I want to bet against the housing market in Las Vegas.” Well, you couldn’t really do it. I mean, it was very difficult for an ordinary person or even an ordinary investor to actually do that. As a result, if you were against the housing market, you registered your opinion by just deciding not to participate in it. And so, again, you don’t really have that true diversity of information, diversity of opinion that sets prices because prices were being set primarily by people who thought the housing market was perfectly healthy. So, that was another problem.

Peter Hopkins: Is this a lesson in maintaining some fundamental principles as we go through these situations?

James Surowiecki: Yeah. I mean, I think the fact that we have now lived through two massive bubbles should make us skeptical of the ability of markets to simply self-correct in the absence of regulation and in the absence, I think, of better organizational models.

I think that a few lessons that need to be learned are, first of all, we need to think hard about how compensation affects people’s decision making. That’s sort of an obvious lesson. I thought it was one we had learned after the technology bubble, but we did not. So, I think the short-term/long-term problem which we had talked about for a very long time is not an illusion, and it is not just something that we can count on to go away. It is a big issue, and getting people to think long term is a huge, a huge problem. And when I say that, it isn’t just the banks, it’s also investors. Mutual fund investors.

So, let’s go back to ’99. One of the big problems, one of the things that continued to inflate the bubble long after it probably should have popped was the fact that, if you were a mutual fund manager who was not participating in the bubble—if you were someone who was not jumping on the NASDAQ bandwagon—after a certain point, it was very hard for you to get people to continue to give you money to invest because people looked around and said, “My neighbor”—I mean, the NASDAQ was in ’98, the NASDAQ returned maybe 46% I don’t remember what it was. In ’99, it returned like 80%. So, people looked around and said, “My neighbor is getting rich on these stocks and you as my mutual fund manager are still buying very traditional stocks or, even worse, you’ve got a bunch of money in cash. Give my money back; I’m going to go give it to the high flyer.” And so, as a result, there were a lot of mutual fund managers that said, “I have to buy these tech stocks, or else I’m basically not going to be able to continue to stay in business.” So, that’s a huge problem. It’s not just a problem of the banks. It’s a problem individual investors have to really think about, I think.

The other thing that I think is important is actually this question of shorting. That creating tools, mechanisms to make it easier for people to bet against assets as well as bet on them is important in terms of keeping markets honest and keeping markets accurate. It’s not that markets that don’t have a lot of shorting in them are necessarily going to be skewed, but when they are skewed, they are going to be really skewed. And short-sellers take a lot of bad-mouthing, and there were times in 2008 when they were definitely jumping on the bandwagon to kill companies I have no doubt about that, but in general, shorting makes markets healthier and not weaker.

And then the last thing I would say that’s kind of obvious, but is really important inside organizations is just to repeat myself, it’s really important to allow people to dissent from the prevailing wisdom and not be treated as if you’re not a team player, or you’re basically naive, or whatever it is. It’s really fundamental to try to aggregate the general wisdom of people inside the organization rather than just kind of saying, “If everyone else is doing it, I’m going to.” There’s a famous Chuck Prince quote about, he was speaking specifically about lending to private equity, but in some sense he was talking about it all, right?

Peter Hopkins: This is the CEO of Citigroup.

James Surowiecki: He’s the former CEO of Citigroup. Right. “As long as the music is playing, you have to get up and dance.” Basically. Well, that’s a recipe for disaster.

Peter Hopkins: Right, right. Well, this raises an important question. You’ve outlined a very complex dynamic, and the modern dichotomy, by all accounts—it’s a very complex thing that involves everybody from housewives in Topeka to gentlemen sitting in luxurious boardrooms just blocks from where we sit here in downtown Manhattan. Is this a system that can regulate itself, or do we need the government? I mean, what’s your sense of these things? Can these things be regulated with a good set of principles? People attaching themselves to real ideas of what does it mean to create value, and if I don’t get it, it isn’t value as Warren Buffet might say. Or is this something where the government has opportunity to interject itself in strategic and well-advised ways?

James Surowiecki: Well, I think there are definitely, I certainly think there is a tremendous—well, tremendous doesn’t sound like quite the right word. I think there’s a fundamental role for the government to play in regulation. And I thought that we were going to see some of it—well, we did see some of it in the technology bubble, frankly. And yet there have been things that the government has done that are good, although people disagree. I think Regulation FD, which basically got companies to disclose information to everyone rather than to a few people—I think that that’s a good thing. That tends to foster access to information, it tends to foster diversity of opinion and the like. I think that that tends to be a good thing. I think to the extent—just to get back on the shorting hobbyhorse—I think to the extent that the government makes shorting easier rather than harder, I think that that’s a good thing. And until the—sometime in the mid-‘90s—mutual funds for the most part could not short stocks. So, if you were in a mutual fund, you basically had to buy stocks if you wanted to be in the stock market. Well, what does that do? Well, that creates a kind of upward pressure. That changed, but even today, lots of mutual funds don’t do any shorting at all.

I think that’s a mistake. I think regulating the amount of leverage that institutions can use is actually important. For a reason that isn’t always—I think there’s a variety of reasons, but I think the most important reason is one that people maybe don’t think that much about. Which is one of the things you don’t want—at least in my mind—is you don’t want a small number of players in a market exerting excessive influence over that market.

Now that follows pretty straight out of my idea of what makes markets healthy. My view of what makes markets really healthy is not having a small number of really smart players. Well, supposedly smart, I guess. But a small number of really smart players playing in it. I actually think you want lots of players in the market. And that one of the things that access to leverage does—and by this we really mean allowing investment banks to leverage themselves up 20, 25, 30 times—is it magnifies, obviously, the effects of the mistakes these people make. And if you have a small number of players who dominate a market and they make mistakes, the consequences are just immense.

I mean, the classic example of this would be longterm capital in 1998. Long-term capital—you were using so much leverage and playing in small markets. In some cases, that in a lot of markets, they were basically almost the entire trade. So, when things started to go wrong, basically they were doomed. So, I think that’s another place where regulating leverage was something else that the government had a major role to play.

I do think also that, I think securities **** in principle is a very good idea still. I would still say that. I would think that to the extent that you can disperse risk—even though it creates these problems in terms of lending—I think to the extent that you can disperse risk, it’s probably better than concentrating it in the hands of a few people. But I think it actually makes sense to say, for instance, if you want to make a loan to a homeowner, you have to keep at least a percentage of that loan on your books. Not all of it, maybe only 10%, or whatever it is; but you have to keep some of it on your books because that keeps you focused on what’s the value of the loan, not what I can sell it for to somebody else; and I think that that is actually good.

The most complicated question that I have to say— I’m still very ambivalent about what the right answer to it is— the credit default swap question and derivative question, more generally. I tend to think that, in general, those things can be quite useful; that actually, it’s useful to allow people to lay off risk. It’s useful to allow people to take more risk to get it, and people who want to take less risk to sell it off. But I also think—and I actually have no problem as a lot of people with the credit default swap market, which allows people to bet on whether or not a company’s going to fail or default or whatever. In general, I think that’s not actually a bad thing. It gives you a market judgment on the health of a company. All those things, I think, are good. But, I think, to the extent that these things are all done behind closed doors, the informational value of them is less useful, and it also creates situations like the AIG one, where people are essentially writing hundreds of billions of dollars in insurance and do not have the ability to pay it off. That creates tremendous problems. So, that’s another place I think the government has a role to play. Although I don’t have a good sense of exactly what the role should be.

Peter Hopkins: Now you’ve also talked a lot about just the human tendency to bandwagon. Everybody just gets on, the bus going to apparently the Promised Land, and it ends off crashing or careening off the cliff. Are there some insights about how individuals should moderate their interactions with the market, particularly with the folks who are buying the mutual funds? The great masses, the real crowd, not just the Wall Street elite. In a world where the market is getting so complicated, all you can really tell at times is something is hot or it’s not. And it almost disposes us to a bandwagon mentality. How does the average person deal with that reality and not be a part of the problem?

James Surowiecki: Yeah, it’s a great question. So, the human tendency to bandwagon is very powerful, so one of the examples I talk about in the book, a simple example, is this great experiment they did in Times Square in the ‘60s. They put a guy on the street corner in Times Square, and they just had him gaze up at the sky, like this basically. And they just waited to see what happened. So, when they put one person on the street corner, maybe 5% or 10% of the pedestrians walking by would stop and be looking up at the sky. But when they put five people on the street corner, about half of the pedestrians stopped. And when they put eight people on the street corner, which I’ll grant you is a lot of people; but when they put eight people on the street corner, 80% of the pedestrians walking by stopped and just ended up gazing up at this empty sky.

And that’s a classic example. As the bandwagon starts to gather steam, it becomes more attractive. More and more people start to get on it. I mean, my colleague, Malcolm Gladwell’s book, The Tipping Point, is in some way an entire book about bandwagoning. Now, in Malcolm’s case, he’s trying to tell you how to use the bandwagon to your benefit if you’re a marketer or a company. But from the perspective of collective wisdom and the collective intelligence of groups, bandwagoning is a bad thing because basically the people in the group are not really—the vast majority of them are not thinking for themselves.

And the reason we do this—I think there are a few reasons. I mean one is, we learn to imitate very early on as people. It’s the way we learn, obviously, and the other reason— one other reason we imitate is imitation is actually often very useful. This is one of the things that we have to recognize. Bandwagoning isn’t just about basically being cheap, it’s also about realizing that a lot of times people know stuff that’s important, that’s valuable.

Peter Hopkins: It could be a plane crashing down into Times Square.

James Surowiecki: Yeah, whatever it is, right? And one example I use is one that is very relevant to people living in New York, which is—I live in Brooklyn, and in Brooklyn you have to move your car once a week for street cleaning. Each side of the streets are cleaned once a week. So you have to move it by 11:00 a.m. But the other thing about New York is we have a tremendous amount of religious holidays in New York City. So, on religious holidays you don’t have to move your car, which if you’re a driver in New York is heaven basically. So, on any given Tuesday or Thursday, which where I live happen to be the days when you have to move your car, you never really know what the rules are. So, what I do, I work at home and what I do—I did this yesterday—is you go outside at 10:45, and if all the cars are in the street, you basically say, “Ah, I guess street cleaning rules have been suspended, you don’t need to move your car.” And I don’t go online. I could do that. I could call 311 or whatever. I just assume that someone in that row of cars has figured out what the rules are. And the important thing is that it has never failed me. It has always worked. Now, I’m just bandwagoning. I’m just assuming that everybody else knows what’s going on. But it works a lot of the time.

And so there is this kind of tendency to do it. And it’s amplified, I think, by a couple of things. One is something I’ve already talked about, which is just this sense that a lot of time, if you don’t bandwagon, people start to assume that you don’t actually know what you’re doing. So in the NASDAQ case. If you weren’t buying NASDAQ stocks in 1999, you looked like a fool. You looked like you had no clue what you were doing as an investor. And so, people just do it because they want to look like they have an idea of what they’re talking about.

The other thing that happens is—I think this is especially important inside organizations—that if you don’t fall in line with what everybody else is saying, people start to think of you as not a team player, or they start to think of you as someone who doesn’t really “get” what’s going on. Doesn’t really get it, doesn’t really get the organization. And so there are all these pressures pushing people to do it.

I think for individuals the challenge is magnified—and this gets to what you were really asking about from an investment point of view. The challenge is actually magnified by the fact that I think there is actually too much information available to people. Now, it doesn’t really make any sense, you know, from any perspective: the more information people have, the better their judgment should be basically. But I think that, especially in the marketplace, when it comes to investing, one of the big problems is that there is a tremendous amount of information available, not just about the fundamental reality of the economy—whatever that is. There is also a tremendous amount of information available about what other people think about the fundamental reality of the economy, right? We are bombarded in a way investors have never been before with the opinions of others. So, the example I remember— I wrote a piece a long time ago when the bubble was bursting, the stock market bubble was bursting, about CNBC. Which, you know, I’ve **** anyway, but there’s no doubt about the coverage of the market that they provide. That is to say, not of the economy, not even of individual ****, but of the market, especially during times of crisis or times of euphoria, just amplifies what’s going on because—the example I always use is, if you go buy a television set, which is an example of a market transaction. When you go buy a television set, you do research, maybe read about what a good TV set is and blah, blah, blah. But when you go to buy the TV set, and the guy is trying to sell you the set, but you basically sit and you think to yourself, will this satisfy my needs, is it da, da, da, da. With investing, it’s like you’re trying to do that while people are yelling at you constantly. The television set is a buy—no, it’s a sell. It’s da, da, da, da, da. It’s very hard to think for yourself. It’s very, very hard.

Now, does that mean you should just ignore everything else? Well, on some level, I think we’d all be better off if individuals did that. It’s hard to do that as an individual, but I do think it is crucial to limit your exposure to the kind of day-to-day fluctuations of the market. It’s really crucial to limit your exposure to the endless flow of news because one of the things that the media tends to do, just by its nature, is exaggerate the importance of any one piece of news. And so, I think that’s a really huge thing as an individual investor. To the extent that you can remove yourself from the day-to-day flow, you’re probably going to be better off.

Peter Hopkins: Sounds like the moral of the story is that in certain cases in our new consumption, we have to consume more, and in other cases we have to really regulate it. So, it’s a lesson in kind of a whole new approach to information and thinking about it as information.

James Surowiecki: Yeah. I think that’s right. I think you—I guess I haven’t thought about it in exactly this way before, but I think that you’re right. I think the idea is that you need to be a better and more rigorous consumer of information. Both in terms of the diversity of the sources you access and then also the type—in the sense of, again, you want to try to look more at fundamentals rather than the day-to-day movement of the market.

Having said that, it’s not obvious to me that individual investors should be in the market on an individual stock basis or to some extent maybe even in an individual mutual fund basis at all. That actually, as difficult as it is to do psychologically, and difficult as it has been to do over the last year and a half, that indexing is probably one of the better strategies. Most individual investors should follow. I’m not saying all. I think there is an argument for the ability of certain mutual fund managers to out-perform, but it’s very hard if you’re an individual investor to not get caught up in the up-and-down movements of the market.

You know, actually, they do these great—they’ve done these studies—well, not actually great, they’re actually quite depressing. But these studies of the S&P500 have risen, I don’t know—whatever it is—9% a year—whatever it is—over the last 15 years. Whatever the numbers are. But the average investor’s return over that period is much, much smaller because the average investor is jumping into the market when it starts to get more expensive and jumping out of the market when it gets cheaper, and you know this psychologically. Just think about how hard it was in March of ’09 when the S&P was at whatever it was—680 or something like that. Nobody wanted to buy stocks then. But that’s the time to get in basically. And so that, I think, is a big challenge.

Peter Hopkins: Well, I want to switch gears a little bit. We’ve been talking a lot about the detriments of the crowds. But we should focus a little bit on their wisdom because there are a lot of people who are learning a lot from them. You drew a distinction for me when we were talking before the interview began between this sort of wisdom of the crowds and the markets that have emerged from them and the idea of crowdsourcing. Can you just quickly give us an overview of what you meant by that?

James Surowiecki: So, I think that—crowdsourcing, which is a term that Jeff Howe came up with a few years ago. I think that crowdsourcing describes a wide range of activities. Some of which include the wisdom of crowds sort of problem solving, but also include other things. And I think the way I would think about it is that some crowdsourcing actually involves not tapping the collective knowledge of the group; it actually involves sort of casting your net very widely in order to find the one or two people in the group who can solve the problem you’re looking for, or design the product you want, or whatever it is.

And so, I think that can actually be quite useful, and I think as a lesson in the fact that not all talent is located where we think to look, that traditional experts don’t have all the answers, that actually there are a lot of individuals out there who know quite a lot, and you can get their input and their information. That’s a very valuable lesson. But I think that, from my perspective, the best techniques are those that try to tap the collective knowledge of the group. And I think actually there are some that actually combine the two in a useful way. So, I’m not going to remember what it was. But I think Sony did this animated film, basically via a kind of crowdsourcing technique. And what it basically entailed was, obviously, individuals came up with the various components of it, but the crowd was constantly weighing in on the editing, basically; which should be included, which shouldn’t. And I think that is a really useful model where you actually have the crowd serve as the filter. The crowd serves as the judge. And that, I think, has enormous promise going forward.

You know, if you think about the wisdom of crowds in everyday life. I mean, one of the great examples for me that I talk about in the book and I think is obvious is Google search engine. I mean, that’s a search engine that’s built basically on the collective judgment of people on the Internet because the page-rank algorithms relies on hyperlinks from one Web site to another as a judgment of the value of that link and to the extent that you think—and now most search engines now use some form of this. To the extent that you think they do a good job of finding information, which I think they do an exceptionally good job. That’s a great example of the wisdom of crowds in action.

Peter Hopkins: Now, we’ve got a couple of questions from the audience via Twitter. This one is interesting I want to hit on. It asks, C. Winstonator asks, how dependent, if at all, is the wisdom of crowds on leadership figures within the crowd?

James Surowiecki: Yeah. So, that’s a really important question because one of the lessons of the wisdom of crowds obviously is that to some extent, we overrate leaders, at least to the degree that we think of leaders as synonymous with decision makers. I actually think one of the fundamental lessons of The Wisdom of Crowds, particularly for organizations, is that to the degree that you can push decision and at the very least kind of information input into the organization as a whole, the better the decisions you make are going to be. And that we need to move away from the kind of corporate savior model of the CEO.

Now, there are counterexamples. Whenever I talk about this people always say, what about Steve Jobs? And my answer is, okay, if you know your CEO is Steve Jobs, and that he has that kind of exceptional judgment, which he seems to have, okay, well then in that case you can really rely on him. But, generally you want to move away from that. Having said that, I think that it is very true that inside organizations, leaders are important, not so much again in terms of making the decisions. They are important in terms of implementing the idea of the wisdom of crowds, convincing people that this can actually work and then, just as important, taking the judgments or the idea that crowds generate, and putting them into practice. When I think about the role of leadership in the kind of new model organization, if I had my druthers, I really think that the role of the leader shifts to put less of an emphasis on straight decision making and more of an emphasis on aggregating knowledge, finding ways to get people to work together in useful ways and then taking the results of that process and really putting it into practice. Those are not easy things to do, but to the extent that you can do it, I think it’s incredibly valuable.

One CEO who comes to mind in terms of this is actually Eric Schmidt at Google, who is someone who has talked about the book The Wisdom of Crowds quite a bit, but he’s someone who I think is trying to instill that ethos inside the organization. And it’s a tough switch for most CEOs. Most CEOs didn’t get to where they were by saying “Okay, let’s hear what all of you think,” basically.

I remember talking to a management consultant a long time ago, who said—for some bizarre reason—these parents he knew asked him, “How will we know if our child is going to be a good…, is he CEO material or something.” One of the strangest questions ever asked, I think. And he said, “If his kindergarten teacher says does not play well with others, then he’s a good CEO,” basically. And that’s the old model CEO, and I think that’s not the model CEO we really need going forward.

Peter Hopkins: Now, this raises another question asked from user Behance Team: what are the risks to taking this kind of crowd-based, crowdsourced model for businesses? Is there a detriment, and do they risk in any way alienating or disaffecting the crowd itself in how they deal and how they approach this whole new model of information gathering?

James Surowiecki: I don’t think there’s much danger of disaffecting the crowd inside organizations for this reason. To the extent that you can get people to actually feel like they’re being listened to, that they actually have some input, maybe they’re not actually making actual decisions, but they’re having a real voice in what the company is doing. I think that’s just a plus. That actually makes a big difference in terms of employee motivation and in terms of getting people engaged with the organization.

Now, I don’t think that’s the real reason to do it. The reason to use the wisdom of crowds is that it’s going to make your decisions better. It’s going to make your organizations smarter. But I think that that is a useful by-product. I think the bigger challenge in terms of individuals and in terms of people is actually to some extent—it’s not quite the opposite, but it’s sort of the opposite, which is that the real challenge is getting people to participate. The real challenge is getting people—to convince people that this actually is going to be real. That it actually matters that it isn’t just kind of another one of these team-building exercises or suggestion boxes that people do, and then the CEO is like, “Well, none of these fit with what I want to do, so I’ll go ahead and do it.” So, I think that’s the other thing.

Now, one challenge people do talk about, whether this is kind of alienating for your top performers, and etc., etc., the people who are the experts. The Wisdom of Crowds is not in any way a war on experts. It’s not meant to say that you should get rid of experts, not meant to say that businesses should go out on Times Square and assemble a group of people and ask them a question. The more the people in the group know, the better its collective decision will be. It really just is an argument just of saying, instead of relying solely on the judgments of one or two people, that actually you really want to try to incorporate the views of more of your employees, and in some cases people outside your organization. I mean, companies have done a good job in some cases, not many but a few, of incorporating the opinions of their suppliers and of their customers, and I think that’s something else going forward that’s going to be very important. And I think that’s the big thing.

The last thing I would say about the risk question is, it’s a mistake to do this thinking that our judgment is suddenly going to become perfect. The argument that I would make is not that the wisdom of crowds is going to give you the best possible answer, it’s that it’s going to give you, most of the time, a better answer than any one person can give you. And I think that inside organizations in particular, there’s a tremendous amount of information, a tremendous amount of knowledge that just gets buried. The way organizations are set up just doesn’t do a good job of getting information from where it is to where it needs to be. And to the extent you can change that by using these kinds of wisdom of crowds prophesies, the results will be surprisingly good.

Just one last thing—I’m going on here, but Lou Platt used to be the CEO of Hewlett Packard, had the saying where he said, “If Hewlett Packard knew what Hewlett Packard knows, we would be a far stronger company.” And what he was saying was, the knowledge we want, the information we want, it’s here. It’s somewhere inside this organization. Let’s just find ways to get at it. And that, I think, is a real lesson.

Peter Hopkins: Great. Well, James, we have reached the end of the hour. This was really a fantastic session. I think we’ve looked at it from every perspective, and I think the audience is going to benefit tremendously from these insights. I want to thank the audience for joining us today, and thank James Surowiecki of The New Yorker as well for a really fascinating conversation about how crowds can really help us get smarter, and how we can get smarter about crowds. Again, my name is Peter Hopkins. I am the President and Co-Founder of Big Think. And I want to thank you for joining us.

James, thank you so much. I really appreciate it.

James Surowiecki: Thanks much.