Assessing the summit's performance, as it has varied over the fourteen year history of the forum, is no easy task. There are a host of problems of evaluative criteria, inference, and evidence. Should summits be judged by their success in building a perfect world (optimizing world welfare)? When they fail, how can the analyst be confident that the lost opportunities were really there? Even when summits cope badly with clear and present dangers, can they confidently be charged with not preventing or even causing them (e.g. the 1980 allied disunity over pipeline sanctions, the 1982 recession, the 1987 stock market crash)? Whatever one's predispositions it is ultimately difficult to know what would have happened to a world without the summit.

Nor is it easy to specify the concrete entity - the "summit" or "summit system" - which constitutes the actor in this field. It is, at one end of the spectrum, merely a three day meeting in a three-hundred and sixty five day year, and, at the other, a continuous, year-long but largely invisible process of preparation and follow-up by sherpa networks, national bureaucracies, subordinate fora and international organizations. Can one confidently credit the summit with the successful consensus building and agreements that take place elsewhere (e.g. the OECD) before the summit, as leaders modify extreme national positions and look for agreements in anticipation of the forthcoming high profile event?

Moreover, there are severe problems of evidence. The secrecy of the forum is compounded by the competitive rush of each national delegation to put out the most favourable (and at times clearly false) version of events to the waiting media. This makes it impossible for even the most careful journalists to cross-check and multiple-source the stories they put out (even in those very rare instances when their language skills permit the interviewing of participants from several delegations). Because there are seven countries, with (by definition) different interests (otherwise harmony would prevail and co-ordination not be required), particular gatherings may be successful high performance summits for some countries and not for others, leaving the analyst with problems of aggregation and a need to account for national bias. Finally, there are problems of blending short-term successes with long-term failures and vica versa. This is a particularly severe problem for the evaluation of more recent summits, whose long term consequences have yet to be seen.

A further challenge arises from the multitude of functions a summit performs, disagreements about which purposes it was primarily created to fulfill (the Kissingerian Concert conception versus the Giscard-Schmidt Librarians versus the Carter Trilateralists), and varying views on what role it can and should play in the future. Indeed, over the years summit practitioners and observers have identified five major functions for seven-power summitry, which can be labelled, in order of their increasing political ambitiousness, "domestic political enhancement", "leadership education", "political direction", "policy co-ordination", and "socio-economic management".

"Domestic political enhancement" refers to the personal domestic political advantages which accrue to the heads by virtue of their summit participation. The relevance and claim to primacy of this function stem from the fact that the very origins of the summit as an institution established only with President Ford's decision to gather the leaders for a second year in Puerto Rico in 1976, can be attributed to his desire to seek a boost to his re-election prospects through the spotlight and endorsements for his policy that the summit would provide. Since that time the summiteers, especially in English-speaking countries, have been keenly sensitive to the political benefits of summit participation in general, and hosting in particular. Here the summit's advantages are personal and policy neutral, as leaders use the summit spotlight to secure maximum favourable publicity for themselves, regardless of whatever policy compromises or positions they might have to adopt to achieve this result. Such domestic political advantages begin with the imperative of any democratic leader to have him/her self, protegé or party re- elected (the re-election imperative). They extend to the desire to enhance one's popularity with strategic elites and the masses (public opinion), and, ultimately, to secure the personal and national prestige of "being there" as a member of one of the world's most exclusive clubs.

The second function, "leader education", deals with the individual-level advantages the preparatory and interpersonal experience of the summit brings to otherwise more domestically- oriented individuals. Here the summit increases leaders' international contacts by breaking down interpersonal barriers to international communication. By encouraging leaders to pick up the phone when the need arises, it promotes a process of personal internationalization. Moreover, the mere process of having to prepare for a summit, as well as meeting peers, expands leaders' awareness of international effects, causes, values, the common dilemmas confronted by national leaders, their constraints, and hence one's own. Most ambitiously, the summit provides a forum for sharing worldviews, developing common historical lessons and myths, and, in an international form of groupthink, taking shared leaps of faith into the dark (as at the superpower Reykjavik summit).

The third function, "political direction", is at the very essence of the summit as a private deliberative but public declaratory forum. Leadership education could most effectively be accomplished in secret, private sessions with not even personal representatives present to learn what the lesson was. This is the model of the hunting retreat summit, without the guides to gossip. But political direction demands publicity, even if only in the form of a sparse communiqué, or the private reports which attending officials transmit back to bureaucracies and their domestic and international clients afterward. And in contrast to furtive rendezvous at political funerals, consensus must be codified, and thus be precise and politically meaningful enough to be written down. This is not mere agenda setting (as important as that is) but the politics of legitimation, moral suasion and sending strong signals. The targets are national bureaucracies, international organizations (who seldom otherwise get attention from heads of state and government), markets and publics, whose behaviour is heavily influenced by, or directly dependent on, political direction. This may well be where the comparative advantage of the summit, as a summit, rests. For as Hunt notes "leaders are better at dealing with the broad thrust of policy than in settling the details of it" through particular package deals with precise targets. (Hunt 1984:658).

Beneath the rhetoric about providing political will, energizing political bureaucracies, and sending a clear signal to nervous exchange and stock markets, lie three specific components of "political direction". The first is legitimation, as leaders define what subjects are worth talking about at the summit and saying something about in the summit documents. The second is issue area definition, or descriptive linkage, as subjects are placed under certain categories (political or economic), dealt with in particular contexts, or linked to certain other areas (nuclear energy to nuclear proliferation or to north-south relations, debt to democracy). In this process the leaders go beyond the inherited and (national and international) organizationally-entrenched division of the world into fixed issue areas to create new, stable frames of reference, and isolate issues problems in their own right. Thirdly, in doing so, the leaders may also signal the trump values, suggest preference scales in a world of tradeoffs, or at least indicate a checklist of the range of relevant factors to be considered when a topic is addressed in the policymaking process. In doing so they signal the direction in which discussion, deliberation and decision-making should go. In short, they engage in governance in the essential sense.

The fourth function of the seven-power summit, and the focus of most assessments, is "policy co-ordination". For Putnam and Bayne, co-ordination takes place along a four point scale: mutual enlightenment (sharing information about national policy directions); mutual reinforcement (helping one another to pursue desirable policies in the face of domestic resistance); mutual adjustment (seeking to accommodate or ameliorate policy divergences): and mutual concession agreeing on a joint package of national policies designed to raise the collective welfare . (Putnam and Bayne 1987:260). In a similar four point scale, Ostry distinguishes among consultation, co-operation, harmonization, and coordination. (Ostry 1988:4). For both, the premium is on what Ostry describes as "significant modification of national policies in recognition of international economic interdependence", with a premium on package deals of convergent national elements traded together at the summit itself.

The fifth function of summitry, "socio-economic management", focuses on whether these deals or the looser policy directions make a difference, not vis-à-vis political subordinates the leaders control hierarchically but vis-à-vis recalcitrant actors in domestic and international society with the independence and resources to resist. Here one asks, does the dollar rise or fall, the stock market crash, literacy rise, Japanese consumption of imported goods rise, in response to summit declarations or decisions that these things should happen or should not. Securing such reliable implementation involves a three fold process. Firstly, once the summit is over, governments inside and out must at least acquiesce in the decision and therefore implement summit agreements, for as long as necessary, and as soon as necessary, to accomplish the agreed-upon task. Secondly, the inevitable stallers, dissidents, defectors, and backsliders must get caught and forced to comply, or at least punished enough that they will think twice before disobeying again. And thirdly, societal actors must behave in desired ways to meet the summit targets, with or without vigorous enforcement from the summiteers.

In assessing summit performance, Putnam and Bayne take as their focus the intensity, scope and duration of international policy co-operation, defined as "episodes in which the policies of one or more states are modified to reduce the costs (or increase the benefits) that those policies entail for the welfare of other states". (Putnam and Bayne 1987:2). They thus proceed to rank summits according to their promotion of policy change in an internationally-desired direction, along these three dimensions, with additional attention to lost opportunities and whether the summit made things worse. Such a focus, however, operates on only a restricted part of the range of summit functions identified above. Moreover, the criterion of "lost opportunities", which remains undefined, reduces the empirical referents for the rankings which result, while the criterion of "making things worse" assumes a firm knowledge of direct causal linkages in a multicausal world and a sound sense of counterfactuals.

To improve the ability to assess summit performance, it is first necessary to broaden the perspective by moving backward to include "political direction" and forward to include "socio- economic management" (at least in the latter's particular, concert-relevant sense of stabilizing the society, economy and polity from severe shocks). The two preliminary purposes of "domestic political enhancement" and "leadership education" can be excluded, for they are best treated as intervening variables whose role can be assessed in their contribution to the remaining three. For example, the summit is of little durable utility if it merely helps re-elect leaders who at the summit chose parochial over internationalist policies (the trade bilateralism and regionalism of the 1988 FTA and 1992 endorsements) for this just creates an incentive for parochialism rather than international co-operation in the future. Reciprocally, the best co-ordinated package deals in the summit record book are of little long-term use if leaders returning home (from Bonn 1978 or Tokyo 1979) to reliably implement their international commitments get defeated at the polls (as both of Canada's summiteers on these occasions did, in some degree due to their summit agreements).

This paper thus focuses on the three purposes of "political direction", "policy coordination", and "socioeconomic management" as the appropriate measures for assessing summit performance. But in recognition of the summit's particular status as an international concert, it re-orders these three, and somewhat redefines the last, in the process of measurement. It thus examines below, in turn, as the dependent variables of this study, three dimensions: "policy co-ordination", as measured by Putnam and Bayne's grading and as corrected by subsequent sherpa testimony; "political direction", as measured by the expanding and contracting summit agenda and declarations and its shifting focus on new and newly-defined areas; and "socio-economic management", in the particular concert manifestation of defending international order against major shocks generated by political, economic or social forces within or outside the summit countries.

This specification of what to measure carries implications for how to measure it. For many purposes there are good reasons to identify each summit as a separate experience and event, delivered by a new host, host sherpa, and sherpa meeting sequence. This conveniently provides fourteen data points for multivariate analysis. It is particularly appropriate if one wishes to evaluate summits against the high criterion of realizing through international policy co-ordination the goal of optimizing world welfare, which is an opportunity continuously available every day of every year. But for those who wish to include in their performance assessment the summit's functions of policy definition and defence of international order, in response to the threats that empirically occur (rather than in anticipation of the opportunities assumed to be there), this concept of a newborn summit each year is inappropriate. For as John Hunt reminds us, "occasions for major summit initiatives will not occur at convenient annual intervals, and we must accustom ourselves to summits which are routine as well as those which show results." (Hunt 1984:659) Indeed, through the dynamics of deterrence or anticipated reaction, the very absence of severe shocks and hence strong summit responses may be a sign of high summit success, just as the lack of lawbreaking and vigorous law enforcement is a sign of effective domestic-level governance. If one merely takes seriously the insurance policy view of summitry (as a guard against infrequent disasters), the analyst must be content with looking for medium- term patterns rather than yearly levels or fluctuations in the summit record, even at the expense of reducing the data points available for assessing competing causes and models.

With this caution, this paper looks first at the Putnam and Bayne grading of individual summits, including their preliminary ranking of Venice II in 1987 as no better than Puerto Rico 1976. Their assessments, listed in Table A, reveal, in their view, a two-stage pattern of success in the 1970's, followed by failure in the 1980's. Such an interpretation, however, produces a number of clear outlier summits - the failure of Puerto Rico 1976, the success of Williamsburg 1983, and the success of Tokyo II in 1986. It is also possible to identify from this data a three-phase ratcheting down at five year intervals (high performance 1975-1979, medium performance 1980-1984, and poor performance subsequently). But even here the problems of outliers, especially in the most recent and shortest period, remains. Another pattern in the data, and one which respects the interdependence between summits, arises when one notes that the first five years of summitry were devoid of failures (defined as C, D, or E summits) two years in a row, those of the second five year Period (1980-1984) had them, and those of the most proximate period have, by virtue of Toyko 1986, avoided them as well.

This very tentative and rather weak pattern of a three-Phase progression from high to low back to medium performing summits comes through much more clearly when one turns to a second measure of summit performance - the consensus judgments of the sherpa's who prepared them. This data, displayed in Table B, records the combined judgments of the direct references on each summit made by the leaders' personal representatives, in professional as opposed to political, contexts, through speeches or interviews. Perhaps not suprisingly, the sherpas are more generous in their overall judgments than Putnam and Bayne. But despite differences in nationality, the political affiliation of the leader they served, generation of service, and length of tenure, they are suprisingly united in their judgments of all individual summits, clusters or generations of summits, and the particular areas where success was achieved or not. They are insistent that the summit, from its inception, was focused on three core, continuing issues (money, trade, north-south) and that Putnam and Bayne in particular, and observers in general have significantly underestimated the performance of the summit in the trade field. Looking over the years, they see a natural grouping of summits into the highly productive first wave (1975- 1980), the far less productive second wave (1981-1984), and a third wave steadily improving from the nadir of Bonn II in 1985.

An examination of their summit-specific judgments, as recorded in Table B, sustains this view of a three stage, high-low-medium progression. (4) If one takes A and B summits as successful, and C and D summits as not, the period 1975 to 1980 had not a single failed summit, while the five years from 1981-1985 had not a single successful one. Yet Bonn II 1985 marked the start of an upward progression which continued and strengthened for the ensuing two years.

In making these judgments, and identifying the particular achievements on which they are based, the sherpas are generally united in awarding primacy of place to the task of policy co-ordination, especially as registered in package deals. But unlike Putnam and Bayne, they include, at a second level, the function of political direction. Moreover, perhaps betraying their status as institutional insiders, they award marks for a dimension not included independently in this analysis: those summits which strengthened the summit machinery or process itself.

If their judgments are to be accepted as accurate, in the light of Putnam and Bayne's evidence of a much weaker three-stage pattern on the top-ranked dimension of policy co-ordination, it follows that the sequence of a turn-of-the-decade decline and recent revival should come through very strongly on an independent measure of the second ranked dimension - "political direction". The data in Table C suggest that indeed it does. This chart takes as its unit of analysis each summit's concluding product, the core economic declaration and its immediate annexes if any. Given the focus of sherpa preparations during the previous year on producing this document, the fact that it is issued in the leaders' name, the thousands of media representatives who anxiously await it and analyze it, and the debates over particular words and phrases itself, this is a document with considerable political significance. It is through this document that the leaders communicate, at varying levels of generality, their consensus judgment on the way the world is and should be going (positions), and how it is divided and organized into discrete subjects or issue areas (position areas).

An examination of the number of position areas included in the communiqué at each summit shows a rather steady expansion of the summit's agenda and area of public consensus over its first seven years. Thus at the end of the first seven-year hosting cycle, the leaders dealt with over twice as many position areas at each summit as they had at the start. The next four summits however, saw a sharp decline, to the point where Bonn II in 1985 dealt with virtually the same number of position areas as Rambouillet a decade earlier. Tokyo II in 1986, however, began a vigorous expansion of the summit's field of attention and direction, which reached unparalleled heights at the Toronto summit in 1988.

This three-stage progression is also generally present if one focuses more precisely on the number of new position areas introduced at each summit, and the ratio of new to old subjects. Bonn I in 1978, generally regarded as a very successful summit, (at least outside of Germany) is unique on this measure in its forward-looking agenda. It is the only summit to have been dominated by new rather than old position areas. And while one can fault summits that identified emerging problems but failed to take effective action to prevent the later shocks these problems bred, Bonn's new agenda, largely generated by the energy field, helped pave the way for the success at Tokyo I the year after. More generally, the 1975 to 1981 period was characterized by forward looking (and admittedly in some respects trendy and episodic) summits, the period 1982 to 1986 by traditional topics, and the 1987 to 1988 summits by new position areas once again. Moreover, in contrast to the data on sheer numbers of position and position areas, this pattern is much less easily dismissed by reference to any alleged national or cultural preference by the French, Americans and British for short and familiar communiqués at summits they host, and the Japanese, Italians and Canadians for long, loquacious and inventive ones at theirs.

Has this pattern of early success, followed by failure and recent revival, also occurred with respect to the primary purpose of summitry - defending international order against major shocks to the system? Because shocks, by definition, constitute sharp, sudden, severe threats to common national values and international order, one can expect, at a minimum, that the leaders would deal with these shocks at the next summit, come to a consensus about how to deal with them, and issue a response commensurate with the magnitude of the threat. However to be successful in responding to a shock, a summit must also take action (or deliberate "non-action") that effectively addresses the problem within a year (when the next summit comes along to get its chance).

The data in Table D displays the record of the summit in responding to two sets of shocks: the "seminal" shocks that created the summit-as-concert in the first place, and the "successor" shocks that arose after it had been formed. The first pattern that is evident is the absence of "successor" shocks for the summit to respond to in its first four years (1975-1978). This phenomenon is due either to the absence of autonomously-generated shocks in the global system, or the very success of a well functioning summit-concert in deterring those who might be tempted to mount them. In contrast, the five years from 1979 to 1983 saw a succession of shocks as numerous and almost as threatening as those in the half decade immediately preceding the summit's birth. It may be that such an onslaught would have overwhelmed any summit and led to low performance on its part. Alternatively, the cascade of shocks can be attributed to the failure of the summit to deal adequately with the first few shocks in this successor sequence, thus giving rise to those which followed. In any event, the shock-response record of the summit was generally poor from 1979-1983. More recently, on the basis of the single completed case of the stock market crash of October 1987, its record has been good. And the very absence of shocks in the three-year period from 1984-1986, again suggests success for those who believe in the summit's deterrence effect.

---

Source: Prepared for the annual meeting of the International Studies Association and the British International Studies Association, London, March 29-April 1, 1989. Unpublished in print. Reproduced by permission of Professor John J. Kirton.