The invisible hand made visible

My latest Bloomberg column just appeared (or will around 6pm EST today). It takes a look at a landmark economics paper from just over ten years ago, one that I am sure (or at least hope) every macroeconomist, indeed every economist, knows fairly well. Given it's fundamental importance, I'm guessing it is by now probably a staple of undergraduate economics education. I'm referring of course to this study by Robert Clower and Peter Howitt which, in comparison with traditional economic theories, took a major step in actually revealing the mechanisms by which Adam Smith's Invisible Hand might work.

To many economists, this probably sounds crazy. What about Arrow-Debreu and all the thousands of later papers in general equilibrium theory in the same tradition? Don't those all establish how a decentralized market can act to organize economic activity in a remarkably efficient way? We'll, actually, no -- because only a very small fraction of those studies have even tried to model the messy dynamic process by which a huge number of independent people might, through trial and error, through learning, come to lead the market to its final organized state. And NONE, so far as I am aware, established by investigation a plausible dynamical story consistent with realistic human behavior. Hence, the nice story and metaphor of the Invisible Hand really has no plausible counterpart in standard economic theory; it remains no more than a metaphor, even if economists seem loathe to admit that.

But this isn't to say that something like the Invisible Hand idea isn't true and really interesting. The insight inspiring (the late) Clower and Howitt is that human economies probably self-organize into functional states coordinating the disparate activities of many individuals in much the same way ant colonies organize themselves. It's not due to the far-sighted rationality of any one agent (ant or person), but due instead to organizing structures that emerge to help us relatively unintelligent individuals cope with a very complex world. In the words of Howitt from this nice paper from 2007:

The idea motivating the approach is that complex systems, like economies or anthills, can exhibit behavioral patterns beyond what any of the individual agents in the system can comprehend. So instead of modelling the system as if everyone’s actions and beliefs were coordinated in advance with everyone else’s, as in rational expectations theory, the approach assumes simple behavioral rules and allows a coordinated equilibrium to be a possibly emergent property of the system itself. The approach is used to explain system behavior by “growing” it in the computer. Once one has devised a computer program that mimics the desired characteristics of the system in question one can then use the program as a “culture dish” in which to perform experiments.

Now the first reaction of many economists upon first hearing about this methodology is that all economic models with an explicit micro-foundation, which is to say almost all models that one sees in mainstream macroeconomic theory, are “agent-based”. Some even have a multitude of heterogeneous agents (see Krusell and Smith, 1998 and Krebs, 2003, among others). So what’s the big deal?

The big deal, as Tesfatsion has emphasized on many occasions, has to do with autonomy. An agent in a rational-expectations-equilibrium model has a behavioral rule that is not independent of what everyone else is doing. In any given situation, her actions will depend on some key variables (prices, availability of job offers, etc.) or the rational expectation thereof, that are endogenous to the economic system. These variables will change when we change the agent’s environment, and hence her behavior cannot be specified independently of the others’. The household, for example, in a market-clearing model of supply and demand cannot choose what quantity to demand until told what price will clear the market. Likewise the agent on a Lucas island (a Phelps Island with rational expectors) cannot choose how much to sell until informed of the stochastic process determining aggregate and relative demand fluctuations.

The problem with assuming non-autonomous agents is that it leaves the model incomplete, and in a way that precludes a deep analysis of the coordination problem. For if the model does not allow people to act without knowing the equilibrium value of some variable, then someone must have computed that equilibrium value a priori. In such a model there is no way to describe out-of-equilibrium behavior, and the problem of reconciling peoples’ independently conceived plans is assumed to be solved by some unspecified mechanism that uses no scarce resources...

Now under certain assumptions about common information, someone endowed with enough information could figure out on her own what the market-clearing price is going to be, or what the rational expectation of the price level is, and in this sense could act autonomously even in a rational-expectations equilibrium framework. But an economy full of agents that were autonomous in this sense would not be decentralized in the Hayekian sense, because no market would be needed to aggregate the diverse information of heterogeneous people, each of whom can do the aggregation in her head. Each would be capable of acting as the economy’s central planner, although in this case the planner would not be needed. Moreover, such an economy would have no need for macroeconomists, because everyone would already know as much as could be known about the macroeconomy. The coordination problem would be trivial. So by “autonomous” agents I mean agents that are endowed with behavioral rules that can tell them what to do in any given situation, independently of each others’ rules, even when no one has access to a correct model of the economy.

What Clower and Howitt showed in their 2000 paper -- and Howitt has expanded upon since in work with other economists -- is that autonomous agents of limited intelligence working on their own in an economic setting can readily self organize their activities into a functioning system with an intelligence far beyond their own. In their model, the coordinating infrastructure is a network of firms that emerges to make it easier for people to find the goods they need, vastly simplifying the problem of matching producers and consumers. I won't spoil the story. See some of Howitt and his colleagues' most recent papers, such as this one, for a nice summary of the original model and developments since then.

One of the most important things emerging from this work, in my opinion, is a way to look at the mechanisms behind economic coordination in a much more specific way. When an economy gets hit by a crisis and goes into recession, things happen which cannot easily be reversed, and certainly not instantaneously. Firms go out of business and then do not exist. This leaves gaps in the coordinating network which puts additional stress on other firms or individuals. An economy, like a broken bone, has suffered real damage that requires both time and the consumption of resources to overcome. Of course, you cannot even begin to understand how the key coordinating infrastructure can be damaged, or how it might be repaired, if you have no theoretical apparatus to describe that coordinating structure in the first place -- this is the position of modern mainstream (neo-classical) economics.

Hence, it's important to remember when confronting the rhetorical arguments of the OpEd pages -- for or against "austerity" or "fiscal stimulus" or some other policy -- that the economist you are reading has either based his or her views on a theoretical model that doesn't even try to model the key mechanisms of self-organizing coordination, or has come to a belief for other reasons, perhaps (in the best cases) a deep reading of history. There are other possibilities, of course.

I must say, this whole thing seems amazing to me. I would have thought that Clower's and Howitt's lead would have immediately been picked up by any macroeconomist eager to make progress. It would have swept in a new approach to understanding macroeconomic dynamics, displacing the older approaches; after all, it actually describes how the Invisible Hand works, whereas they do not. Obviously, this hasn't been the case, which in itself says something disturbing about the state of today's economics. It's clearly not all about seeking a better understanding.

Read more

Questioning conventional wisdom

I recommend this fascinating interview with Harvard development economist Dani Rodrik. In his own words, he is someone who has worked from within the economic mainstream (especially where methods are concerned) but has not been afraid to accept logical conclusions that go against conventional wisdom, which is often not actually supported by any logic or theory. As he says,

...where I tend to part company with many of my colleagues is with the policy conclusions I reach. Many of my colleagues think of me as excessively dirigiste, or perhaps anti-market. A colleague at Harvard’s Economics Department would greet me by saying “how is the revolution going?” every time he saw me. A peculiar deformation of mainstream economics is the tendency to pooh-pooh the real-world relevance of all the theoretical reasons market fail and government intervention is desirable.

This sometimes reaches comical proportions. You get trade theorists who have built their entire careers on “anomalous” results who are at the same time the greatest defenders of free trade. You get growth and development economists whose stock in trade are models with externalities of all kinds who are stern advocates of the Washington Consensus. When you question these policy conclusions, you typically get a lot of hand-waving. Well, the government is corrupt and in the pockets of rent-seekers. It does not have enough information to undertake the right kinds of interventions anyhow. Somehow, the minds of these analytically sophisticated thinkers turn into mush when they are forced to take seriously the policy implications of their own models.

This is an interesting point. In essence, he is suggesting that some of the policy conclusions generally supported by the economic mainstream (deregulation, more markets, etc.) actually find no real foundation in theory. Yet many economists support these conclusions anyway for other reasons. He goes on to talk about the social forces within the profession:

There are powerful forces having to do with the sociology of the profession and the socialization process that tend to push economists to think alike. Most economists start graduate school not having spent much time thinking about social problems or having studied much else besides math and economics. The incentive and hierarchy systems tend to reward those with the technical skills rather than interesting questions or research agendas. An in-group versus out-group mentality develops rather early on that pits economists against other social scientists. All economists tend to imbue a set of values that tends to glorify the market and demonize public action.

What probably stands out with mainstream economists is their awe of the power of markets and their belief that the market logic will eventually vanquish whatever obstacle is placed on its path. As a result, economists tend to look down on other social scientists, as those distant, less competent cousins who may ask interesting questions sometimes but never get the answers right. Or, if their answers are right, they are so not for the methodologically correct reasons. Even economists who come from different intellectual traditions are typically treated as “not real economists” or “not serious economists.”

So the hurdles for the economists that want to depart from the conventional path are pretty high. Above all, they must play by the methodological rules of the profession. That means using the language of mathematics, the standard optimizing, general-equilibrium frameworks, and the established econometric tools. They must pay their dues and demonstrate they remain card-carrying members in good standing.

Now, Rodrik does suggest that he likes to work within this framework for various reasons. But what he then says is most interesting, touching back on the point of how widely held policy views link back to actual economic theory. Often, he suggests, they have no foundation at all in such theory, which is often employed more as a rhetorical tool than anything else:

In my own case, every piece of conventional wisdom I challenged had already become a caricature of what sounds economics teaches us. I wasn’t doing anything more than reminding my colleagues about standard economic theory and empirics. It was like pushing on an open door. I wasn’t challenging the economics, but the sociology of the profession. For example, when I first began to criticize the Washington Consensus, I thought I was doing the obvious. The simple rules-of-thumb around which the Consensus revolved had no counterpart in serious welfare economics. Neither were they empirically well supported, in view of East Asia’s experience with heterodox economic models. When you questioned supporters closely, you first got some very partial economic arguments as response, and then as a last resort some political hand-waving (e.g., “we need to get the government to stop doing such things, otherwise rent-seeking will be rife…”). My argument was that we should take economics (and political economy) more seriously than simply as rules of thumb. Economics teaches us to think in conditional terms: different remedies are required by different constraints. That way of thinking naturally leads us to a contextual type of policy-making, a diagnostic approach rather than a blueprint, kitchen-sink approach.

Similarly, when I questioned some of the excessive claims on the benefits of globalization I was simply reminding the profession what economics teaches. Take for example the relationship between the gains from trade and the distributive implications of trade. To this day, there is a tendency in the profession to overstate the first while minimizing the second. This makes globalization look a lot better: it’s all net gains and very little distributional costs. Yet look at the basic models of trade theory and comparative advantage we teach in the classroom and you can see that the net gains and the magnitudes of redistribution are directly linked in most of these models. The larger the net gains, the larger the redistribution. After all, the gains in productive efficiency derive from structural change, which is a process that inherently creates gainers (expanding sectors and the factors employed therein) and losers (contracting sectors and the factors employed therein). It is nonsensical to argue that the gains are large while the amount of redistribution is small – at least in the context of the standard models. Moreover, as trade becomes freer, the ratio of redistribution to net gains rises. Ultimately, trying to reap the last few dollars of efficiency gain comes at the “cost” of significant redistribution of income. Again, standard economics.

Saying all this doesn’t necessarily make you very popular right away. I remember well the reception I got when I presented my paper (with Francisco Rodriguez) on the empirics of trade policy and growth. The literature had filled up with extravagant claims about the effect of trade liberalization on economic growth. What we showed in our paper is that the research to date could not support those claims. Neither the theoretical nor empirical literature indicated there is a robust, predictable, and quantitatively large effect of trade liberalization on growth. We were simply stating what any well-trained economist should have known. Nevertheless, the paper was highly controversial. One of my Harvard colleagues asked me in the Q&A session: “why are you doing this?” It was a stunning question. It was as if knowledge of a certain kind was dangerous.

Years earlier, when I wrote my monograph Has Globalization Gone Too Far? I had been surprised at some of the reaction along similar lines. I expected of course that many policy advocates would be hostile. But my arguments were, or so I thought, based solidly on economic theory and reasoning. A distinguished economist wrote back saying “you are giving ammunition to the barbarians.” In other words, I had to exercise self-censorship lest my arguments were used by protectionists! The immediate qestion I had was why this economist thought barbarians were only on one side of the debate. Was he unaware of how, for example, multinational firms hijacked pro-free trade arguments to lobby for agreements – such as intellectual property – that had nothing to do with free trade? Why was it that the “barbarians” on one side of the issue were inherently more dangerous than the “barbarians” on the other side?

But ultimately, the reward of challenging conventional wisdom that has gone too far is that you are eventually proved right. The Washington Consensus is essentially dead, replaced by a much more humble approach that recognizes the importance of locally binding constraints. And many of the arguments I made about the contingent nature of the benefits from trade and financial globalization are much closer to the intellectual mainstream today than they were at the time.

Read more

Why game theory is often useless...

Economic theory relies very heavily on the notion of equilibrium. This is true in any model for competitive equilibrium -- exploring how exchange can in principle lead to an optimal allocation of resources -- or more generally in the context of game theory, which explores stable Nash equilibria in strategic games.

One thing physicists find wholly unsatisfying about equilibrium in either case is economists' near total neglect of the crucial problem of whether the agents in such models might ever plausibly find an equilibrium. You can assume perfectly rational agents and prove the existence of an equilibrium, but this may be an irrelevant mathematical exercise. Realistic agents with finite reasoning powers might never be able to learn their way to such a solution.

More likely, at least in many cases, is that less-than-perfectly rational agents, even if they're quite clever at learning, may never find their way to a neat Nash equilibrium solution, but instead go on changing and adapting and responding to one another in a way that leads to ongoing chaos. Naively, this would seem especially likely in any situation -- think financial markets, or any economy as a whole -- in which the number of possible strategies is enormous and it is simply impossible to "solve the problem" of what to do through perfect rational reflection (no one plays chess by working out the Nash equilibrium).

A brilliant illustration of this insight comes in a new paper by Tobias Galla and Doyne Farmer. This is the first study I've seen (though there may well be others) which addresses this matter of the relevance of equilibrium in complex, high-dimensional games in a  generic way. The conclusion is as important as it is intuitively reasonable:

Here we show that if the players use a standard approach to learning, for complicated games there is a large parameter regime in which one should expect complex dynamics. By this we mean that the players never converge to a fixed strategy. Instead their strategies continually vary as each player responds to past conditions and attempts to do better than the other players. The trajectories in the strategy space display high-dimensional chaos, suggesting that for most intents and purposes the behavior is essentially random, and the future evolution is inherently unpredictable.

In other words, in games of sufficient complexity, the insights coming from equilibrium analyses just don't tell you much. If the agents learn in a plausible way, they never find any equilibrium at all, and the evolution of strategic behaviours simply carries on indefinitely. The system remains out of equilibrium.

A little more detail. Their basic approach is to consider general two player games between, say, Alice and Bob. Let each of the two players have N possible strategies to choose from. The payoff matrices for any such game are NxN matrice (one for each player) giving the payoffs they get for each pair of strategies being played. The cute idea in this analysis is to choose the game randomly by selecting the elements of the payoff matrices for both Alice and Bob from a normal distribution centered on zero. The authors simply choose a game and simulate play as the two players learn through experience -- playing strategies from their repertoire of N possibilities more frequently if those strategies give good results.

With N = 50, the results show clearly that many games do not ever settle into any kind of stable behaviour. Rather, no equilibrium is ever found. The typical dynamics is reflected in the figure below, which shows the difference in payoffs to the two players (Alice's - Bob's) over time. Even though the two agents work hard to learn the optimal strategies, the complexity of the game prevents their success, and the game shows no signs whatsoever of settling down:

As the authors note, this kind of rich, complex, ongoing dynamics looks quite similar to what one sees in real systems such as financial markets (the time series above exhibits clustered volatility, as do market fluctuations). There are periods of relative calm punctured by bouts of extreme volatility. Yet there's nothing intervening here -- no "shocks" to the system -- which would create these changes. It all comes from perfectly natural internal dynamics. And this is in a game with N = 50 strategies. It seems likely things will only grow more chaotic and less likely to settle down if N is larger than 50, as in the real world, or if the number of players grows beyond two.

Hence, I see this as a rather profound demonstration of the likely irrelevance of equilibrium analyses coming from game theory to complex real world settings. Dynamics really matters and cannot be theorized out of existence, however hard economists may try. As the paper concludes:

Our results suggest that under many circumstances it is more useful to abandon the tools of classic game theory in favor of those of dynamical systems. It also suggests that many behaviors that have attracted considerable interest, such as clustered volatility in nancial markets, may simply be specific examples of a highly generic phenomenon, and should be expected to occur in a wide variety of different situations.

Read more

The limitations of markets

Economists aren't often as vocal as they should be about the limitations of markets -- especially the extreme assumptions required for them to deliver superior outcomes and some kind of "efficiency." I've documented here before some of the exuberant cheer-leading for the wonders of modern markets that was the norm before the financial crisis of 2008. No self doubt or balanced criticism about the dangers of markets there.

Now, only a few years after the crisis -- and with a global economic crisis just looming up before us -- the old hysteria is again getting underway with calls (especially from US politicians) for more privatization to get the damned inefficient government out of everything. For an intelligent, fact-based perspective, I'm simply going to quote the following extended discussion from economist Mark Thoma. He deserves a medal for saying what most other economists ought to be saying every day to everyone they meet:

To listen to some commentators is to believe that markets are the solution to all of our problems. Health care not working? Bring in the private sector. Need to rebuild a war-torn country? Send in the private contractors. Emergency relief after earthquakes, hurricanes, and tornadoes? Wal-Mart with a contract is the answer.

Whatever the problem, the private sector - markets and their magic - beats government every time. Or so we are told. But this is misplaced faith in markets. There is nothing special about markets per se - they can perform very badly in some circumstances. It is competitive markets that are magic, though even then we have to remember that markets have no concern whatsoever with equity, only efficiency, and sometimes equity can be an overriding concern.

In order to work their magical efficiency, markets need very special conditions to be present. There must be full information available to all participants. Product quality, locations and prices of alternative suppliers, every relevant piece of information must be known. Not quite sure if the wine is good or not? That's an information problem. Not sure if the used car has problems? Don't know where any gas stations are except the ones beside the freeway in a strange town? No way to monitor the quality of the building built in Iraq with U.S. aid? No way to be sure if consultants are worth the amount they are being paid? Information problems are common and they can cause substantial departures from the perfectly competitive, ideal outcome.

There also must be numerous buyers and sellers, enough so that no single buyer or seller's decisions can affect the market price. For example, if a firm can affect the market price by threatening to limit supply, the market does not satisfy this condition. If, as some claim, CEOs are in such short supply that they can individually negotiate their compensation, then the market is not producing an efficient outcome. Whenever there are a small number of participants on either side of the market - suppliers or demanders - this is potentially problematic.

In order for markets to work their magic, the product must be homogeneous. That is, the product or input to production sold by all firms in the market must be perfectly substitutable so that as far as the buyer is concerned, one is as good as the other. If some buyers favor one brand over another, if CEOs are perceived to have different and unique talents, this condition does not hold. In many cases the variety may be worth the inefficiency, not many of us would want just one style and color of shirt to be available in stores, but the inefficiency is there nonetheless.

In order for markets to work their magic there must be free entry and exit. Most people understand free entry, but free exit is sometimes less evident, so let me try to give an example. Starting a blog on Blogger or TypePad is easy. Entry is a snap and you can be up and running in no time at all. It's easy to join the competition and start supplying posts. But suppose that later you decide you want to switch to, say, TypePad from Blogger (or the other way around). That is not so easy. There is no way, at least no simple and convenient way, to export all of your old posts from Blogger and import them into TypePad, a significant barrier to exit if a large number of posts must be moved. Whenever barriers exist in markets that prevent free movement into and out of the marketplace or between firms within a market (on either side - there are sometimes barriers to purchasing as well), markets will underperform.

The list goes on and on. In order for markets to work their magic, there can be no externalities, no public goods, no false market signals, no moral hazard, no principle agent problems, and, importantly, property rights must be well-defined (and I probably missed a few). In general, the incentives that the market provides must be consistent with perfect competition, or nearly so in practical applications. When the incentives present in the marketplace are inconsistent with a competitive outcome, there is no reason to expect the private sector to be efficient.

Markets don't work just because we get out of the way. When government contracts are moved to the private sector without ensuring the proper incentives are in place, there will be problems - waste, inefficiency, higher prices than needed, etc. There is nothing special about markets that guarantees that managers or owners of companies will have an incentive to use public funds in a way that maximizes the public rather than their own personal interests. It is only when market incentives direct choices to coincide with the public interest that the two sets of interests are aligned.

If there is no competition, or insufficient competition in the provision of government services by private sector firms, there is no reason to expect the market to deliver an efficient outcome, an outcome free of waste and inefficiency. Why would we think that giving a private sector firm a monopoly in the provision of a public service would yield an efficient outcome? If the projects are of sufficient scale, or require specialized knowledge so that only one or a few private sector firms are large enough or specialized enough to do the job, why would we expect an ideal outcome just because the private sector is involved? If cronyism limits the participants in the marketplace, why would we expect an outcome that maximizes the public interest?

There is nothing inherent in markets that guarantees a desirable outcome. A market can be a monopoly, a market can be perfectly competitive, a market can be lots of things. Markets with bad incentives produce bad outcomes, markets with good incentives do better.

I believe in markets as much as anyone. But the expression free markets is often misinterpreted to mean that unregulated markets are all that is required for markets to work their wonders and achieve efficient outcomes. But unregulated is not enough, there are many, many other conditions that must be present. Deregulation or privatization may even move the outcome further from the ideal competitive benchmark rather than closer to it, it depends upon the characteristics of the market in question.

For government goods and services, when incentives consistent with a competitive outcome are present, we should get government out of the way and privatize, and there are lots of circumstances where this will be appropriate. There is no reason at all for the government to produce its own pencils and pens, buying them from the private sector is more efficient so long as the bids are competitive.

When competitive conditions are not met but can be regulated, the regulations should be put in place and the private sector left to do its thing (e.g.  mandating that sellers disclose problems with a house to prevent asymmetric information or mandating that government funded projects be subject to competitive bidding and monitoring to ensure contract terms are met). There's no reason for government to do anything except ensure that the incentives to motivate competitive behavior are in place and enforced.

But rampant privatization based upon some misguided notion that markets are always best, privatization that does not proceed by first ensuring that market incentives are consistent with the public interest, doesn't do us any good. There are lots of free market advocates out there and I am with them so long as we understand that free does not mean the absence of government intervention, regulation, or oversight, even libertarians agree that governments must intervene to ensure basics like private property rights. Free means that the conditions for perfect competition are approximated as much as possible and sometimes that means the presence - rather than the absence - of government is required.

Read more

Efficiency versus stability

UPDATED BELOW

I had an opinion piece published today in Bloomberg Views looking at the relationship between market efficiency and stability, a topic which hasn't received much attention in the economics literature until recently. The point of the essay was to explore two distinct recent studies which suggest that adding more derivative instruments to markets tends to make them less stable, even if they do push markets toward the ideal of market completeness and efficiency.

I wanted to make available here some further technical information on the two studies I mentioned, but as publication arrived very quickly and I've been pressed with other deadlines I haven't yet managed to write the post as I wanted. However, I can at least offer some information with the idea of updating it very shortly (later today, Thursday 25 August).

I've given some extensive discussion of the first study I mentioned, by economists William Brock, Cars Hommes and Florian Wagener, in an earlier post.

The second study by Matteo Marsili is quite technical and relies for parts of its analysis on ideas and techniques imported from physics. I will tomorrow try to give some simplified discussion of the gist of this argument. What makes this particularly fascinating is that it works fully within the confines of standard general equilibrium models, and examines how market stability should evolve as the market approaches the ideal of market completeness. Agents are assumed to be fully rational, there are no problems with asymmetric information, etc. Even here, however, Marsili finds that the equilibrium becomes more and more unstable as the ideal is approached. Efficient markets are also unstable markets.

UPDATE

Marsili's argument is one he has been developing in a series of papers (with various co-authors) over several years. This paper from last year offers what is perhaps the most concise argument. It looks at a market with informed (fundamentalist) traders and non-informed (noise) traders, and shows, first, that the market becomes efficient as the number of informed traders grows. They are assumed in the model to have different kinds of private information about market outcomes, and the market becomes efficient, roughly speaking, once there are enough traders to cover the space of outcomes so all private information gets aggregated into market prices. The paper then introduces a non-informed trader -- a chartist or trend follower -- and shows that this trader has a maximum impact on the market precisely at the point at which it becomes efficient. The conclusion is very much against standard economic thinking:

[The results suggest} that information efficiency might be a necessary condition for bubble phenomena - induced by the behavior of non-informed traders...

Another paper from two years ago approaches the problem from a slightly different angle. This study looks explicitly at how the proliferation of financial instruments (derivatives) provides more means for diversifying and sharing risks and takes the market to an efficient state. However, it finds that this state is what physicists refer to as a "critical state", which is a state characterized by extreme (essentially infinite) susceptibility to small disturbances. Any small noise stirs up huge fluctuations. Again, efficiency trails instability in its wake. As the paper asserts:

This suggests that the hypothesis of Arbitrage Pricing Theory (the notion that arbitrage works to keep market in an efficient state) may not be compatible with a stable market dynamics.

This paper also makes the important point that market stability really ought to be thought of as a public good because well functioning markets do help everyone. But like most public goods, private individuals acting in their own interests will not likely provide it.

Finally, the paper I discussed in the Bloomberg article is from last year and analyses a model set up specifically so as to include the finance sector. It is very much akin to standard general equilibrium models, and includes essentially two components:

1. There are investors who aim to take their current wealth and preserve it (or make it grow) into the future. They do this by investing in various instruments provided by a sector of financial firms. These investors are assumed to be rational and have full information and they invest their wealth optimally over the set of possible investments.

2. There are financial firms who create the investment instruments and take on risks in supplying them. They also act optimally, and they hedge their risks by trading between themselves. Again, the firms are rational and have full information.

Marsili then studies what happens to this world of investors and financial firms optimally making decisions as the number of different financial instruments grows. The first result confirms expectations -- the financial firms are ever more successful in hedging their risks and they can provide the financial instruments more cheaply. Investors can therefore invest more effectively. The market becomes efficient.

But there are also two unexpected consequences. As Marsili describes them,

As markets approach completeness, however, two "unintended consequences" also arise: equilibrium portfolios develop a marked susceptibility to idiosynchratic shocks and/or parameter uncertainty and hedging engenders divergent trading volumes in the interbank market. Combining these, suggests an inverse relation between financial stability and the size of the financial sector...

In other words, the character of the optimum portfolios for both the investors and the financial firms becomes hugely sensitive to tiny shocks to the economy. As the efficient state is approached, these agents have to work ever harder to adjust their holdings to remain in the optimal condition. The market only remains efficient through an ever faster and more vigorous churning of investment positions. This shows up in the hedging done by the financial firms, where the volume of trading required to remain optimally hedged actually becomes infinite as the market reaches efficiency.

All three of these papers show much the same thing -- efficiency bringing instability along with it. But this latter paper may be the most interesting as it shows directly how the size of the financial sector also naturally explodes as this efficient-unstable regime is approached. The effect sounds suspiciously like what has happened in the past 30 years or so with massive growth in the financial industries in most developed nations.

What I find really remarkable, however, is that all of this comes from the very models that economists have been using for a long time to make arguments about market efficiency. Why did it take a physicist to look at what happens to stability at the same point? This seems bizarre indeed.

Read more

Leverage Control -- A Subtle Story

I mentioned recently some work (in progress) by Stefan Thurner and colleagues exploring how leverage influences stability (price volatility) in a competitive, speculative market. Thurner spoke about this at a meeting on Tipping Points in Durham, UK. What I find most appealing about this work is that is explores this question with a model that is rich enough to exhibit many of the basic features we see in speculative markets -- competition between hedge funds and other investment firms to attract investors' funds, the use of leverage to amplify potential gains, the monitoring of leverage by banks who lend to the investment firms, occasional abrupt crashes and bankruptcies, etc.

Is it a perfect model? Of course not, there is no such thing; models are tools for thinking. But it is arguably better than anything else we currently have for running "policy experiments" to test what might happen in such a market if regulators take this or that step -- establishing tight limits to allowed leverage, for example. 

Stefan kindly sent me the slides from his talk, a few of which I'd like to mention here. As I said, this is work in progress, so these are preliminary results. They're interesting because they suggest that avoiding dangerous market instability through leverage limits comes with costs, and that our intuition isn't at all a reliable guide -- we need these kinds of models in which we can discover surprising outcomes (before we discover them in reality).

I won't give a detailed description of the model; it can be found in an early draft of the paper available here. Thurner and colleagues have been working to improve the model over several years, and it now reproduces a number of realistic market behaviors quite naturally. Thurner summarized these as follows:

 

In other words, the hedge funds act to eliminate mis-pricings (taking volatility out of the market), and profit by doing so. Funds have to be aggressive to survive in the face of stuff competition, but suffer if they get too large. Risks shorten the lifetime of a fund. Overall, the models also reproduces the right statistical fluctuations in the market.

As I discussed in my earlier post in this work, competition between hedge funds leads naturally to increasing leverage and drives the market to have a fat-tailed distribution of returns; it becomes subject (like real market) to large price fluctuations as a matter of course driven by its own internal dynamics (no external impacts required). In this condition, the market is highly prone to catastrophic crashes triggered by nothing by small price fluctuations linked to noise traders (unsophisticated investors buying and selling more or less at random). The figure below shows a typical example, plotting the wealth of various funds versus time, with a dramatic crash that affects all funds at once (different colors for different funds):

Now, a natural question is -- could these kinds of events be avoided with proper regulations? One idea would be to restrict the amount of leverage allowed with the aim of keeping the market returns in a mode Gaussian regime, i.e. eliminating fat tails. People could probably argue for decades about whether this would work or not without coming to an answer; this model makes it possible to do an experiment to find out, which is what Thurner and colleagues have done.

Two figures (below) show some of the results, and require some explanation. The different colors correspond to different possible regulatory regimes, and show how behavior changes with maximum allowed hedge fund leverage : BLUE (no other regulations), PALE GREEN (regulations akin to Basel I and II, in which banks loaning to hedge funds are restricted by capital requirements) and RED (a situation in which banks monitor hedge funds and reduce a hedge fund's allowed leverage below the maximum when the volatility in its assets grows; a kind of adaptive leverage control). 

First, consider a figure showing how how the action of hedge funds, and their use of volatility, actually benefits the market -- making it more efficient (in one sense). 

The figure shows the mean square price volatility versus allowed leverage. Increasing leverage lets the hedge funds pounce on opportunities more aggressively and wipe out mis-pricings more effectively. Die hard free market people should love this as it shows that the effect is strongest in the absence of any regulation. The regulated markets require higher leverage to get the same reduction in volatility.

But this isn't the whole story. Now consider another figure for the probability (per unit time) of a failure of one of the hedge funds:

Here the pure free market solution isn't so good, as this probability rises rapidly with increasing leverage. There is a relatively low value of leverage (around 5 in the model's units) where the market benefits of leverage have already been realized, and more leverage only leads to more failures (because it takes the market into the regime of fat-tailed returns; this can happen even if the mean square volatility remains small).

The regulated markets in this case perform marginally better -- the regulations reduce the number of failures, and the cost for this is marginally increased volatility.

A surprising outcome is that these same regulations, in the regime of very high leverage, actually do worse than no regulations at all -- they lead to higher market volatility AND more failures as well, a truly perverse regime.

All in all, then, this model offers a sobering perspective on how regulators might go about trying to avoid crashes linked to fat tails by limiting leverage. Some limitation clearly seems to be good. But too much can be bad, especially when coupled with other market regulations. You can't test out one idea in isolation, because they interact in surprising ways.

I'll probably have some further comments on this in the near future. It's a work in progress, as is my understanding of it -- and of what it means for the bigger picture.

Read more