Review of the effects of protection in marine protected areas : current knowledge and gaps

The effectiveness of marine protected areas (MPAs) and the conservation of marine environments must be based on reliable information on the quality of the marine environment that can be obtained in a reasonable timeframe. We reviewed studies that evaluated all aspects related to the effectiveness of MPAs in order to describe how the studies were conducted and to detect fields in which research is lacking. Existing parameters used to evaluate the effectiveness of MPAs are summarised. Two-hundred and twenty-two publications were reviewed. We identified the most commonly used study subjects and methodological approaches. Most of the studies concentrated on biological parameters. Peer reviewed studies were based on control vs. impact design. BACI and mBACI designs were used in very few studies. Through this review, we have identified gaps in the objectives assigned to MPAs and the way in which they have been evaluated. We suggest some guidelines aimedat improving the assessment of the effects of protection in MPAs.


Introduction
Coastal marine environments host key habitats for many endangered marine populations, yet their accessibility and proximity to heavily inhabited areas makes them vulnerable to over exploitation through fishing, and to direct anthropogenic impacts.Traditionally, the demand of food in coastal areas makes fishing one of the most important activities impacting these areas.Fishing exerts direct pressure on the environment as well as on fish stocks, and there is unequivocal evidence that fishing has reduced the abundance and size of the most targeted and valuable species (Chapman & Kramer, 1999;Edgar & Barrett, 1999;McClanahan et al., 1999;Chiappone et al., 2000;Willis et al., 2003;Williamson et al., 2004).Poor planning and overpopulation of coastal areas has added to the problem due to resulting pollution and excessive recreational use (Bellan-Santini et al., 1994).In recent years, Marine Protected Areas (MPAs) have been increasingly seen as a way of reducing the intensity of these impacts (Ward et al., 1999).Since the creation of the first MPA in 1935 (Doumenge, 1993), MPAs have been established throughout the world as a management tool for compensating the effects of human impacts on the coastal marine environment (Agardy, 1994).Specifically, MPAs are implemented to reduce the effects of overfishing of coastal marine stocks, preserve marine biodiversity and protect key habitats (Francour et al., 2001;Halpern, 2003).They also provide a sustainable socioeconomic development for human communities in coastal areas (Sainsbury & Sumaila, 2003).
MPAs have been strongly advocated as a tool for the management of fisheries as they conserve fish stocks, increase the number and fecundity of the breeding population, increase the abundance of juveniles and act as nurseries and areas of biodiversity conservation (Bell, 1983;Russ & Alcala, 1998;Garcia-Charton et al., 2004).Recent empirical evidence suggests that establishing well-designed and managed marine reserves results in a rapid increase in the size and abundance of exploited species (Gell & Roberts, 2003;Lubchenco et al., 2003), thus reversing the detrimental effects of fishing (Dugan & Davis, 1993;Roberts & Hawkins, 2000).However, there is an increasing need to understand the long-term overall effectiveness of MPAs operating around the oceans (Pomeroy et al., 2005), as most studies assess different effects of MPAs over short time periods and at a local scale.Clearly, there is a need to assess the ability of MPAs to achieve their management objectives, taking into account the expectations of managers, monitoring needs and constraints (Pelletier et al., 2005).
This study therefore aims to: (a) provide a synthesis of studies that have been carried out to evaluate the effects of MPAs in terms of their objectives; (b) identify areas concerning the use of study subjects, descriptors and the most commonly used methods of investigation; (c) analyse the different kinds of results on the effects of protection; and (d) reveal areas where our understanding is poor and future research is necessary.

Material and methods
This study is based on a comprehensive search of papers available through published literature, together with a classical bibliographical search, from which a database of specific research on MPA evaluation parameters was constructed.We made keyword searches using 'MPA(s) ', 'indicators', 'ecological indicators' and 'social indicators'.References not published in journals were obtained through a classical search in several governmental institutions, research centres and universities.Great effort was put into obtaining technical reports, though there was considerable difficulty in obtaining these due to their restrictive circulation; for this reason, most came from the Southern European countries.
A database was made including these fields: year of publication, reference type, evaluation type (peer reviewed or technical reports), location of the study, geographical area, sampling design, study subject, considered taxa, variables selected, sampling method used, if confirmation applied, main results obtained, significant differences found and whether reserve effects were detected.Where studies covered several topics, we designated the primary topics as those given most attention by the author.We took into account every descriptive parameter that was used to measure the effects of protection.Not all studies gave information for each field.Where more than one reference by the same author clearly presented the same information, only one was included in our database, with a peer reviewed paper given preference over a technical report.Papers selected by the search that were revisions, or did not provide quantitative data, were not included in the database.Each study was ranked by an expert panel of scientists that scored manuscripts by summing values assigned to sampling design quality, statistical analysis performed and type of editorial evaluation (table 1).

Results
Our search of the literature led to the selection of 224 studies conducted between 1983 and 2006.A higher proportion (peer reviewed: 70.98%, n = 159; technical reports: 29.02%, n = 65) of these studies were peer-reviewed papers that increased during the nineties (fig.1).Most peer studies and technical reports reported significant differences between protected and unprotected areas (58.49% and 64.61%, respectively).

Study purposes and subjects
While the level of understanding in a particular subject area cannot be quantified in terms of absolute numbers of papers written, they do provide an indication of the extent of attention paid to the different subjects and highlight less studied areas that require further study.Both peer reviewed studies and technical reports mainly concentrated on the 'Effects on populations' and the 'Effects on assemblages', the latter being more frequent in peer reviewed studies (table 2).To a lesser extent, 'Effects on fishing yields' and 'Socioeconomic indirect effects' were also considered, while other topics such as spillover, direct socioeconomic effects and ecological indirect effects have rarely been addressed.In the review conducted, we could not find any 'larval exportation' and 'direct socioeconomic effects studies'.The study purposes used does not remain constant over time; peer reviewed publications assessed more different types of study purposes in the period from 1994 to 2006 (fig.2).Both peer reviewed studies and technical reports used most 'all fishes' to assess the effects of protection (table 3).'Commercial fishes' were most frequently used in peer-reviewed papers.There is a lack of studies for 'charismatic species' and 'Exploitative uses', within peer reviewed journals.Most studies considered the biological effects of protection (89.28%, n = 200), though there were a few studies that analysed socioeconomic effects using biological subjects (e.g.number of contacts with key species).Fishes were by far the most widely used taxa, though in some cases molluscs, crustaceans, echinoderms, gorgonian, seagrasses and algae, amongst others, were also considered.In socioeconomic studies, divers were the most frequently used subjects.In some cases, socioeconomic studies also considered travel costs, contingent behaviour and prices that tourists would pay.Protection levels (fully vs. buffer vs. general) replicated in space and time 3 Protection levels (fully vs. buffer vs.

general) replicated in space 2
Protection levels (fully vs. buffer vs.

general) replicated in time 2
Fixed transects (C vs. I) replicated in time 1 Others: no spatio-temporal replication 0

Statistically analysed 1
Statistically non-analysed 0 Publishing ranking Published (peer-reviewed) 2 Published (non-peer-reviewed) 1 Not-published 0 Variables selected Parameters on 'population structure' were the most studied variables, followed by variables on 'Assemblage structure', while behavioural studies must be highlighted for being scarce in technical reports (table 4).In general, most of the reviewed studies (84.37%) used parameters to assess changes to key species, populations and/or habitats potentially affected by the protection.Almost all the reviewed studies used parameters on the evaluation of protection on some very restricted biological subjects, specifically those that assess the condition of some species and/or habitats.

Sampling designs
Sampling designs were grouped into fourteen different types (table 5).Most peer-reviewed studies used a 'control vs. impact' sampling design, while technical reports mostly used 'only in protected replicated in time and space' sampling.The percentage of studies having what we would consider a 'good quality' design was higher in peer reviewed studies, though even here these more complex and logically suitable sampling designs were scarce.Sampling designs incorporating spatial and/or temporal hierarchical replication were more frequent from 1994 onwards.Interestingly, our review highlighted a great frequency of peer-reviewed studies that had 'no replication' and/or were carried out only within the protected area.Although they appear in the first studies reviewed, they are also frequent in recent years (fig.3).

Study approach and confirmation
Correlative studies were the most frequently applied to the assessment of the effects of protection, with only few peer-reviewed studies using an experimental approach.Most of the studies, both peer reviewed and technical reports, did not use any techniques to confirm whether the parameters assessed could be used in the future as indicators (Oreskes et al., 1994).Within those peer-reviewed studies and the technical reports that stated significant differences among the parameters evaluated, 70.96% and 78.57% respectively presented higher values of these parameters within the protected areas.Only 28.93% and 15.38% respectively exhibited non-significant differences.A considerable number of studies did not report back if any differences where found between protected and unprotected areas with the parameters studied.This can be due to an ineffective protection for several reasons (e.g.low effective protection, few protection years) but these studies where not very explicit on these reasons.

Study ranking
Peer-reviewed studies tended to have the highest ranking due to their higher quality sampling designs (table 6).Although most of the studies had some type of statistical analysis, peer-reviewed papers tended to use more rigorous methods and produce more quality results, a quality undoubtedly arising due to the review method applied.The results of the comparisons within peer-reviewed studies and the authors' origin revealed that the studies best ranked were carried out by Australian-New Zealand authors, followed by North American authors (fig.4).Papers from southern European researchers presented very heterogeneous values in the ranking.

Discussion
In this review, we highlight the differences among peer-reviewed studies and technical reports, primarily in terms of sampling designs and statistical analyses.Other fields considered only exhibited minimum quantitative differences between these two kinds of publications.Our results emphasize the very narrow range of methods and parameters used to assess the effects of protection in MPAs, as well as the low number of specific objectives proposed for MPAs tackled in the literature.Although our search for technical reports (the 'grey literature') was very thorough and particularly concentrated on MPAs reports, we consider that due to the difficulty of obtaining such information, there is a bias towards Southern European countries conducted in recent years (older reports are less likely to be listed online or available).For this reason, our results on grey literature should be restricted to this area since other areas are under-represented.
Regarding peer-review papers, we consider our sampling to be a good representation of global research trends as they are readily available online and inter-country availability is not an issue.If a review lacks a comprehensive search strategy, it is likely to suffer from a degree of publication bias.On the other hand, when the results of research are negative (Hull, 1999;Underwood, 1999), they are usually not published.As a consequence, reviews which fail to include these negative studies may overestimate the true effect of an intervention, resulting in false positive conclusions being drawn.It could be resolved if they had initially commissioned a comprehensive, systematic review of all the evidence (Scargle, 2000;Glasziou et al., 2001;Higgins & Green, 2005).Our review was undertaken with such a comprehensive search strategy to obtain every type of study, although if the studies were not published, they can't be obtained and some bias may exist.
MPAs have been considered a suitable management tool since the 1960s.However, in the early years, few studies were conducted and those that were did not suitably assess the benefits of MPAs or reserve effects or did not include all species that benefit from protection.Peer-reviewed publications increased mainly from the 1990s when MPAs became more popular, reaching 1306 declared MPAs around the world (Kelleher et al., 1995).The decline in papers after 2000 might simply reflect the delay between undertaking more complex research and getting it published or because this issue lost newness between the scientific community.

Driving forces of past trends in MPA study
Very few of the proposed objectives of MPAs have been examined (Jones, 1994;Rowley, 1994).Research effort has tended to concentrate on the conservation of biodiversity and fisheries resources, probably due to higher socioeconomic demand for such lines of study, but also possibly due to relative ease of study.This was already evidenced by IUCN, 2006 and has remained thus during subsequent years.In these areas uses are limited and within them the fishery.Therefore, one of the best descriptors is At best, the figure may rise to US$18/ha/year and is quite far from the US$1,000/ha/years needed to finance marine parks (IUCN, 2006).At the same time, there is also a tendency for certain study lines to be favoured by the background and personal preferences of researchers and decision makers and to the lack of considerations of certain subjects.It is likely that both of these factors have led to research not covering the prime objectives of MPAs (e.g. the effects on fishing yield).Fishes and decapods have been well studied, as they are relatively easy to sample and because they are the first organisms that show changes due to protection, but other more complicated and costly subjects to study, such as 'the cascade effect as an ecological indirect effect due to protection' have been less well covered.The ease with which fish and crustaceans can be studied is obviously the reason most population and/or assemblage descriptors, such as abundance, biomass, number of species and size, are the most commonly used, and explains the narrow range of generic methodologies used to record the data.Considering MPAs tend to be created using an ecosystem approach that takes into account the global links in the marine environment (Bohnsack, 1999), it seems that many studies fall into the error of excluding many potential study subjects and thus do not meet the study purposes.The complexity of the studies must increase searching for more complex and more difficult to prove interactions.This would explain the absence of studies evaluating the effects of protection at different stages and on components related with the design and functioning of MPAs: e.g.functional effects of enforcement on the management of MPAs, mitigate effects on the impacts, regulatory effects on main socioeconomic sectors affecting many coastal areas such as fishing and tourism.The opportunistic approach to the creation of many MPAs (McArdle, 1997;Roberts, 2005), in which neither size or the adequate scale needed for the MPA to accommodate the development of most of the species being protected are considered, has likely contributed to the inadequate state of research which is too general and vague.The politics of protection, which has centred mostly on species, is also deficient (Roberts, 2005).We therefore consider that MPAs research should contemplate a compromise in which study purposes are more evenly dispersed amongst the physical, bio-ecological and socioeconomic study lines.One  of the main stumbling blocks for the study of MPAs has been the lack of suitable methodologies to cover complicated topics such as spillover.This deficiency is, in our opinion, partly due to the lack of adequate investment needed to develop new techniques that go beyond purely observational methods.The necessary added investment has only really been available in some regions such as North America, Australia, New Zealand and Japan (OECD, 2007) and is reflected in the higher ranking of studies coming from these areas.Of course, these areas were also pioneers of MPAs and therefore have a longer track record of such research.
The differences in study ranking are also, in part, due to differences in experimental design training of researchers and the adoption of different study approaches, for example, the controversy between BACIPS (Stewart-Oaten et al., 1986) and BACI methodology (Green, 1979) or a more recent one, beyond-BACI (Underwood, 1991) and MBACI (Underwood, 1993).These methodological approaches have been embraced elsewhere over recent years.There is a gradual increase of a wide range of methodological approaches and the enlargement of those proved as more sturdy methodologies from a logical and statistical point of view.However, the results presented by technical studies are biased, as it was much easier to obtain studies of Southern Europe due to geographical proximity and Southeast Asia by being more available, obtaining results influenced by these events.However, the transfer of knowledge has not been as effective as the technical reports.Once again, the causes of this fact cannot be concluded directly from our review.Taking into account our experience, we can infer deficiencies in the training of the consult-ants that made the technical reports, making studies without a minimum replication.Consultants involved in technical reports lack any previous educational training in research, and they have to face the job market only with the knowledge acquired at university; these gaps in the study curriculum for professionals and/ or the small investment of institutions limit transfer of skills between consultants, managers and scientists.

MPAs research: the way forward
As the number of MPAs and associated capital and social investment increases, it becomes more and more important for managers to base decisions on sound scientific and social knowledge.There is therefore a growing need for reliable information on the patterns, processes and ecological consequences that protection has on communities.While descriptive studies have contributed greatly to our understanding of the structure of biological communities harboured by MPAs, we do not yet know the effects of protection on parameters such as adult biomass export, larval spillover, etc.It is clear that such lines of study need to be encouraged.Monitoring is also essential to all MPA management programmes in order to be able to effectively track changes over time with a review of management decisions accordingly.This is particularly relevant if MPA managers are to meet their objectives (Kelleher et al., 1995).Not only is better funding necessary to maintain long term monitoring programs, but these should strive to homogenize methodologies in order to allow posteriori comparisons at different temporal and spatial scales, overall when national and transnational programs promoting MPAs exist.office-based data relevant to management objectives and the health of the considered marine systems (Oakley, 2003).The use and selection of standardised protocols is not new and has previously been proposed at different forums (Goñi et al., 2000).It is essential that chosen indicators of the effects of protection are easily interpreted by managers and stakeholders so that they can contribute to efficient and transparent management (Mangi et al., 2007).At the same time, in view of the overlapping of different processes in space and time, it is essential that suitable scales for the evaluation of the effects of protection are chosen (Garcia-Charton et al., 2004;McClanahan et al., 2007).While it is true that selection of appropriate spatial and temporal scales used for detecting the effects of protection tends to be intuitive, it is essential that such sampling decisions are made with great care.An optimal strategy consists of studying the patterns of interest at multiple, simultaneous scales, identifying relevant scales of variability and then listing a series of hypotheses and testing them to account for the observed patterns (Underwood, 1997).This being the case, we are obliged to use increasingly complex sample designs requiring adequate spatial and temporal replication, with several control and impacted sites (Underwood & Chapman, 2003).Choice of these sites must be    made taking into account the dangers of pseudo replication (Hurlbert, 1984).The benefits lie in the power of resulting analyses and increased certainty of the results of protection (Underwood & Chapman, 2003).Wide heterogeneity has been detected in the papers studied, therefore the comparison and proper assessment of the politics used in MPAs as tools of management should be convenient to standardise methods of sampling when gathering field data.
Using the same patterns would allow to compare and analyse the long term series and will the comparison in a wide range the local studies using meta-analysis, using either spatial (Mosquera et al., 2000;Coté & Reynolds, 2000) or temporal comparison (Ojeda-Martínez et al., 2007).Meta-analysis data originated from several independent studies can be analysed quantitatively, providing major advantages over traditional synthesis and reviews (Hedges & Olkin, 1985;Gurevitch & Hedges, 1993).
It is noteworthy that confirmation, the process of verifying that any parameter used really responds to protection and not some other driving force, Oreskes et al., 1994, has been established in very few studies and is generally based on the comparison of results with historical long-term data or bibliographic refer-ences.This scarcity is likely due both to researchers' lack of awareness of its importance and the lack of reference information needed to confirm the assessed effects are due to the effects of protection.This highlights the need for long-term data series, preferably initiated before the time of protection (before data) and/or the need for good independent control sites.
We must also change the tendency to only publish positive results in peer-reviewed journals, as this is clearly not beneficial for the correct interpretation of the effects of protection.In particular, the current trend to conduct meta-analyses on published MPA work is clearly weighted in favour of studies that show positive results for protection.

Conclusions
Our review highlights the high heterogeneity among studies assessing the benefits of MPAs.A lot of emphasis is placed on the planning of MPAs and the evaluation of certain study purposes, study subjects and variables.Many parameters are studied but the study of the success of the protection itself is given less consideration.Insufficient attention is given to monitoring the extent to which MPAs achieve their objectives as a basis for taking action to improve management programs.Considering many of the studies are funded by the same institutions, there should be a concerted effort to require researchers to adopt standard methodological techniques that would allow widespread comparison and more cohesive management practices.Changes to the way we collect data and the questions asked by researchers are clearly required for effective, economically sound development of MPA policies.There is a pressing need for an integrated approach that treats MPAs as a whole instead of as a collection of separate biotic, social and economical entities.

Fig. 1 .
Fig. 1.Number of peer-reviewed publications and technical reports from 1983 to 2006.

Fig. 2 .
Fig. 2. Variation of the different types of study purposes for technical reports and peer-reviewed publications from 1983 to 2006: TR.Technical reports; P. Peer-reviewed publications.

Fig. 3 .
Fig. 3. Variation of the different types of sampling design category for technical reports (TR) and peerreviewed publications (P) from 1983 to 2006.

Table 1 .
Study rank according to their sampling design quality, statistical analysis performed and type of editorial evaluation.

Table 3 .
Study subjects used by technicl reports and peer-reviewed studies, in number (n) and frequency (%).

Table 4 .
Variables used by technical reports and peer-eviewed publications, in number (n) and frequency (%).Enough time has passed to now be able to assess the state of MPA related science and to recommend selected data collection methods that robustly capture data on the effects of protection due to MPAs in the context of all their proposed objectives.While the geographic scope of data collection methodologies is often initially designed for use at the local level, the use of consistent methodologies across larger regions is desirable.New protocols should outline steps necessary to obtaining environmental (including physical, biological conditions and ecological), fisheries and socioeconomic field, laboratory and No spatial and/or temporal replication Fixed transects (C vs. I) replicated in time Only in protected levels replicated in time and spaceProtected levels (fully vs. buffer vs. general) replicated in time Protected levels (fully vs. buffer vs. general)Protected levels (fully vs. buffer vs. general)