Learning in macroeconomics...

I've posted before on macroeconomic models that try to go beyond the "rational expectations" framework by assuming that the agents in an economy are different (they have heterogeneous expectations) and are also not necessarily rational. This approach seems wholly more realistic and believable to me.

In a recent comment, however, ivansml pointed me to this very interesting paper from 2009, which I've enjoyed reading. What the paper does is explore what happens in some of the common rational expectations models if you suppose that agents' expectations aren't formed rationally but rather on the basis of some learning algorithm. The paper shows that learning algorithms of a certain kind lead to the same equilibrium outcome as the rational expectations viewpoint. This IS interesting and seems very impressive. However, I'm not sure it's as interesting as it seems at first.

The reason is that the learning algorithm is indeed of a rather special kind. Most of the models studied in the paper, if I understand correctly, suppose that agents in the market already know the right mathematical form they should use to form expectations about prices in the future. All they lack is knowledge of the values of some parameters in the equation. This is a little like assuming that people who start out trying to learn the equations for, say, electricity and magnetism, already know the right form of Maxwell's equations, with all the right space and time derivatives, though they are ignorant of the correct coefficients. The paper shows that, given this assumption in which the form of the expectations equation is already known, agents soon evolve to the correct rational expectations solution. In this sense, rational expectations emerges from adaptive behaviour.

I don't find this very convincing as it makes the problem far too easy. More plausible, it seems to me, would be to assume that people start out with not much knowledge at all of how future prices will most likely be linked by inflation to current prices, make guesses with all kinds of crazy ideas, and learn by trial and error. Given the difficulty of this problem, and the lack even among economists themselves of great predictive success, this would seem more reasonable. However, it is also likely to lead to far more complexity in the economy itself, because a broader class of expectations will lead to a broader class of dynamics for future prices. In this sense, the models in this paper assume away any kind of complexity from a diversity of views.

To be fair to the authors of the paper, they do spell out their assumptions clearly. They state in fact that they assume that people in their economy form views on likely future prices in the same way modern econometricians do (i.e. using the very same mathematical models). So the gist seems to be that in a world in which all people think like economists and use the equations of modern econometrics to form their expectations, then, even if they start out with some of the coefficients "mis-specified," their ability to learn to use the right coefficients can drive the economy to a rational expectations equilibrium. Does this tell us much?

I'd be very interested in others' reactions to this. I do not claim to know much of anything about macroeconomics. Indeed, one of the nice things about this paper is its clear introduction to some of the standard models. This in itself is quite illuminating. I hadn't realized that the standard models are not any more complex than linear first-order time difference equations (if I have this right) with some terms including expectations. I had seen these equations before and always thought they must be toy models just meant to illustrate the far more complex and detailed models used in real calculations and located in some deep economic book I haven't yet seen, but now I'm not so sure.

Read more

Difficulties with learning...

I just finished reading this wonderful short review of game theory (many thanks to ivansml for pointing this out to me) and its applications and limitations by Martin Shubik. It's a little old -- it appeared in the journal Complexity in 1998 -- but offers a very broad perspective which I think still holds today. Game theory in the pure sense generally views agents as coming to their strategies through rational calculation; this perspective has had huge influence in economics, especially in the context of relatively simple games with few players and not too many possible strategies. This part of game theory is well developed, although Shubik suggests there are probably many surprises left to learn.

Where the article really comes alive, however, is in considering the limitations to this strictly rational approach in games of greater complexity. In physics, the problem of two rigid bodies in gravitational interaction can be solved exactly (ignoring radiation, of course), but you get generic chaos as soon as you have three bodies or more. The same is true, Shubik argues, in game theory. Extend the number of players above three and as the number of possible permutations of strategies proliferates it is no longer plausible to assume that agents act rationally. The decision problems become too complex. One might still try to search for optimal N player solutions as a guide to what might be possible, but the rational agent approach isn't likely to be profitable as a guide to the likely behaviour and dynamics in such complex games. I highly recommend Shubik's short article to anyone interested in game theory, and especially its application to real world problems where people (or other agents) really can't hope to act on the basis of rational calculation, but instead have to use heuristics, follow hunches, and learn adaptively as they go.

Some of the points Shubik raises find perfect illustration in a recent study (I posted on it here) of typical dynamics in two-player games when the number of possible strategies gets large. Choose the structure of the games at random and the most likely outcome is a rich ongoing evolution of strategic behaviour which never settles down into any equilibrium. But these games do seem to show characteristic dynamical behaviour such as "punctuated equilibrium" -- long periods of relative quiescence which get broken apart sporadically by episodes of tumultuous change -- and clustered volatility -- the natural clustering together of periods of high variability. These qualitative aspects appear to be generic features of the non-equilibrium dynamics of complex games. Interesting that they show up generically in markets as well.

When problems are too complex -- which is typically the case -- we try to learn and adapt rather than "solving" the problem in any sense. Our learning itself may also never settle down into any stable form, but continually change as we find something that works well for a time, and then suddenly find it fails and we need to learn again.

Read more

The Wisdom (???) of Crowds

The notion that markets aggregate the opinions of many and thereby make superior estimations of value has a very long history. It's certainly at the root of the infamous Efficient Markets Hypothesis, which claims that markets gather and process information so efficiently that price movements have no predictable patterns and prices of financial instruments always reflect something very close to the true fundamental value of the assets in question. More recently, The Wisdom of Crowds has been the driving force behind prediction markets. One one way or another, this notion lurks behind the slippery and insidious idea that "markets know best" and that pretty much everything from water distribution to higher education should be organized as a market. 

But in his bestselling book on the topic, James Surowiecki was somewhat careful at the outset to acknowledge that the idea only works in some rather special situations (not that readers paid much attention). A crowd estimating the number of marbles in a jar or the correct price of a stock will only get superior results -- superior in accuracy to the guess of any one individual, and even of experts -- if the people are on average unbiased in their estimates; it won't work if they tend systematically to estimate too high or low. Moreover, the people have to make their estimates independently of one another. Any kind of social influence, one person copying or even being slightly swayed by the actions of another, also spoils the result. Wise crowds very quickly become dumb herds.

For an idea of such broad influence, it's surprising how few experiments have been done to probe in detail around the boundaries where wise crowds become unwise, how it happens and which are the key effects. This has been rectified by an impressive set of experiments carried out by Jan Lorenz and colleagues from ETH-Zurich, and published recently in PNAS. Their idea was to use a crowd of 144 student volunteers and have them perform estimation experiments in a range of conditions. They gave the participants monetary incentives to estimate accurately, and chose questions (on things like geography and crime statistics) for which the true answers are known. Then, in some trials, participants made their estimates on their own, without having any idea about the estimations of others, and in other trials, they were either informed in complete detail of what others had estimated or given at least average information on the others' estimates. The idea was to compare how well the crowd made estimates in the absence and presence of social influence.

What the results show is that social influence totally undermines the wisdom of crowds effect, and does so in three specific ways. It's interesting to consider these in some detail to see just how this whole "wise crowd" illusion falls apart in the face of a little social influence:

1. In what the researchers call the “social influence effect,” the mere act of listening to the judgements of others led to a marked decrease in the diversity of the participants estimates. That is, the estimates of the various people become more like one another -- people adjust their views to fit more closely with others -- but this does very little to improve the collective accuracy of the crowd. In effect, people think they are sharing information, but little information actually gets shared. The figure below illustrates what happens: in successive trials, a measure of the group's opinion diversity decreases dramatically if people hear either full or average information on the estimates of others, meanwhile the collective error decreases only marginally.

2. A second and even more interesting effect is what the researchers call the “range reduction effect.” Imagine that a government tries to use the wisdom of crowds, assembling a group and surveying their opinions, hoping to get a range of views and some idea of how much consensus there is on some topic. You would hope that, if the crowd's estimate was NOT accurate, this lack of accuracy would be reflected in a wide range of estimates from the individuals -- the wide range would signal a lack unanimity and confidence. A truly bad outcome would be a crowd that at once gives a very inaccurate estimate and does so with a narrow range of opinion differences, signalling apparent strong certainty in the result. But this is precisely what the research found -- in the social influence conditions, the individuals' estimates didn't "bracket" the true answer, with some being higher and others lower. Rather, the group narrowed the range of their views so strongly that the truth tended to reside outside of the group's range -- they were both inaccurate and apparently confident at the same time.

3. Finally, and worse still, is the “confidence effect”. The researchers interviewed the participants in the different conditions, asking them how confident they were in the accuracy of the group's final consensus estimate. Social influence, while it didn't make the crowd's estimate any more accurate, did fill the participants with strong confidence and belief in improved accuracy. Think 2005, housing bubble, mortgages with no income and no assets, etc. As hard as it is to imagine that people could have believed the market could not fail to go up further, most did. And they did in large part because they saw others apparently believing the same thing.

Altogether, this careful study points more toward the idiocy of crowds than their wisdom. Social influence is hard to eradicate. Even in markets, supposedly driven by anonymous individuals making their own estimates, lots of people are reading the newspapers and news feeds and listening to analysts, and, even when not, looking to price movements and using them to infer whether someone else may know something they don't. In these experiments, social influence makes everyone think and do much the same thing, makes it likely that the consensus view aims well wide of the actual truth, and, perversely, makes everyone involved increasingly confident that the group knows what it's doing. Some kind of Wisdom.

Read more

How derivatives make markets unstable: Part I

I posted a while back on some of the dirty secrets of the derivatives industry. I promised then to give a little more discussion at some point of two terrifically important pieces of research -- still not widely known, especially in mainstream finance -- which show how adding more derivatives to a market can make it less stable, not more stable. This goes directly against the received wisdom of economic (equilibrium) theory which claims that markets become more efficient as they become more complete, i.e. as it becomes possible to take essentially any kind of market position by virtue of a dense spectrum of financial instruments.

One of the papers I had in mind was this landmark study from several years ago in which William Brock, Cars Hommes and Florian Wagener considered the question of whether, in the run up to the recent crisis, "... highly leveraged positions using complex financial instruments may have amplified market volatility." The answer to which their analysis leads is -- yes, quite probably. More generally, they illustrate how more derivatives in general should make markets more unstable, increasing volatility.

Their paper is a little technical, but worth a read. I'll outline the gist of their argument, which starts with several straightforward observations and moves to a not-so-obvious conclusion:

Observation 1: They start by noting that people aren't the hyper-rational automatons of Milton Friedman's (or other neo-classical economists') favorite fantasies. Rather, people in the real world form their expectations and craft their behaviour in an adaptive way -- that is, they learn from experience.

Observation 2: They also note that people aren't identical. We not only learn, but our brains are different and we've all had different experiences in the past, so, at any moment, we've probably learned different things and have slightly different expectations (heterogeneous expectations, in economic lingo) about the future.

Observation 3: People are generally risk averse -- if they're willing to bet $100 on a gamble that could pay off, but involves risks, they'll be willing to bet more than $100 in the same gamble if you reduce the risks. In other words, people shy away from gambles more the riskier they are. This is basic empirical psychology.

Starting from these observations, Brock and colleagues then consider an "intertemporal" asset market (economist-speak meaning a market in which time exists) in which a lot of people look to past prices and try to predict future prices, buying and selling as they see fit. This market contains both risky and non-risky things to invest in -- stocks and risk-free bonds (which are guaranteed to increase in value by a factor R>1 over each interval of time). Stocks might rise more, but are less certain and hence riskier. In addition, the people can buy derivatives -- instruments which act like pure bets and give a pay off in certain circumstances.

What this all amounts to is that people in this market can 1) play it safe by buying bonds, 2) gamble more by buying stocks, and also 3) buy derivatives if they want which (in this model) have no effect except to offset some of the risks involved in buying stocks.

What Brock and colleagues then show is that the combination of the derivatives, the risk aversion of investors, and their tendency to learn by "reinforcement" -- to be more likely to follow strategies which have paid off in the past -- leads directly to trouble. I'll describe how in a moment, but one final thing before I do: the strength of reinforcement learning in the model (how quickly people shift to use better performing strategies) is controlled by one parameter β; bigger β means faster switching. In previous work, Brock and Hommes have shown that in an asset market in which people learn by the reinforcement process, there is a natural "tipping point" -- at a certain critical value of β -- where the market goes from being stable to being unstable. Intuitively, when people switch too quickly, taking even scanty short term evidence as proof of a strategy's superiority, fluctuations in the market become much stronger.

OK, so what happens in this market when you currently have, say, 15 possible derivatives covering lots of different possible outcomes, and now add a 16th derivative to cover other outcomes (i.e. we have derivatives on stocks and commodities, and suddenly invent some new ones to cover mortgage bonds)? Brock and colleagues show that the addition of this one new derivative makes the market go unstable more quickly, i.e. at a lower value of β. The mechanism involves a simple interplay of reduced risk and human confidence. This new derivative, by making it possible for investors to lower the risks associated with investments, leads them to invest more money. They take bigger bets. These bigger bets naturally amplify how quickly the bets that turn out to be correct amass profits. So, there are bigger differences in the payoffs to recent winning and losing strategies, which draws more followers to the winners more quickly (even if the fundamental switching rate of people haven't changed).

In brief: by the very act of reducing the risk of some strategies, the derivative invites more vigorous gambling on that strategy, leading to faster flows of people from one strategy to another. The extra derivative makes the market more volatile.

This model doesn't involve many questionable assumptions. It's a very basic model of the most central facts of any market, respecting some realities of human psychology. It suggests that derivatives hold inherent dangers. Yet as far as I can see, the ongoing discussion of regulating derivatives isn't taking this perspective into account. As Satyajit Das notes, the drive toward greater returns that is an essential part of the dynamics in the Brock, Hommes and Wagener model is a very real force in today's derivatives markets:

Investors searching for return drive speculation. Concerned about stagnant real incomes and inadequate retirement savings, individual investors seek out higher yielding investment structures, often based on derivatives. Pension funds and other institutional investors use derivatives to enhance returns to fully fund and meet their contracted liabilities. In an environment of diminishing returns and fierce competition for attractive investments, fund managers use derivative strategies to enhance returns through readily accessible leverage and capacity to create risk “cocktails”.

Facing increased pressure on earnings, corporations have increasingly “financialised”, resorting to speculative derivative trading to meet profit expectations. ... [Such] seculative activity amplifies rather than reduces volatility and systemic risks. Perversely, this may impede capital formation and also increase the cost of capital for companies.

What happens in the real world backs up the lesson of this simple model. Derivatives reduce risks only in a very narrow and restricted sense, while undermining the functioning of markets more generally. Of course, there's lots of money to be made by the people selling derivatives, so don't expect them to admit (or care about) any of this.

Read more