Skip to main content

Ambient ozone exposure and children’s acute asthma in New York City: a case-crossover analysis

Abstract

Background

Childhood asthma morbidity has been associated with ambient ozone in case-crossover studies. Varying effects of ozone by child age and sex, however, have been less explored.

Methods

This study evaluates associations between ozone exposure and asthma emergency department visits and hospitalizations among boys and girls aged 5-17 years in New York City for the 2005-2011 warm season period. Time-stratified case-crossover analysis was conducted and, for comparison, time-series analysis controlling for season, day-of-week, same-day and delayed effects of temperature and relative humidity were also performed.

Results

We found associations between ambient ozone levels and childhood asthma emergency department visits and hospitalizations in New York City, although the relationships varied among boys and girls and by age group. For an increase of interquartile range (0.013 ppm) in ozone, there was a 2.9-8.4% increased risk for boys and 5.4-6.5% for girls in asthma emergency department visits; and 8.2% increased risk for girls in hospitalizations. Among girls, we observed stronger associations among older children (10-13 and 14-17 year age groups). We did not observe significant modification by age for boys. Boys exhibited a more prompt response (lag day 1) to ozone than did girls (lag day 3), but significant associations for girls were retained longer, through lag day 6.

Conclusions

Our study indicates significant variance in associations between short-term ozone concentrations and asthma events by child sex and age. Differences in ozone response for boys and girls, before and after puberty, may point towards both social (gendered) and biological (sex-linked) sources of effect modification.

Peer Review reports

Background

As one of the six criteria air pollutants regulated by the U.S. Environmental Protection Agency (U.S. EPA), ozone still exceeds the National Ambient Air Quality Standard (NAAQS) in many U.S. urban areas, including New York City (NYC). Further, there is growing evidence of health effects occurring below the standard [1]. Ozone has been demonstrated in laboratory animals to cause increases in oxidative stress, epithelial injury and immunologic disease [2-5] and results from epidemiologic studies have found that short-term ozone exposure is associated with adverse health effects ranging from mild respiratory function impairment to increased mortality [6-10]. Effects of ozone on asthma outcomes have been examined widely, and significant associations between short-term ambient ozone exposures and asthma emergency department (ED) visits [11-15] and hospitalizations [16-18] have been reported, particularly during the warm season when ozone concentrations are higher [19,20].

According to the National Health Interview Survey Data from 2011, asthma is one of the most prevalent chronic conditions among U.S. children, with 10.5 million (14%) U.S. children having ever been diagnosed with asthma [21]. Children may be more susceptible to ambient air pollution than adults [22,23], and are disproportionately affected by asthma, as evidenced by higher asthma hospitalization rates for persons under age 18 [24]. Biological explanations for children’s heightened susceptibility to air pollutants may include higher ventilation rates, developing lung physiology, immature immune systems, and smaller peripheral airways, resulting in proportionally greater airway obstruction from inflammation [23,25-27]. Numerous epidemiologic studies have associated ozone with asthma exacerbations in children [28-31], but results have not always been consistent.

While children have been identified as a susceptible subpopulation, many of the prior studies of air pollution health effects have investigated associations across broad ranges of childhood ages, which may not reflect biologically meaningful periods for heightened sensitivity [32,33]. Differences in sex hormones, activity patterns, lung development, and immune systems between childhood and adolescence, which differ for boys and girls, would suggest that associations between air pollution and asthma could vary by both sex and age. Few epidemiologic studies, however, have been adequately powered to evaluate effect modification by age and sex groups simultaneously, potentially contributing to inconsistent findings among previous studies. In this study, we conducted case-crossover analysis to evaluate associations between ambient ozone concentration and asthma ED visits and hospitalizations among boys and girls aged 5-17 years in NYC, examining differential effects across sex and etiologically-relevant age groups. We compared case-crossover and time-series analytic methods, as a sensitivity test.

Prior ozone and asthma studies in NYC found that adjustment for other pollutants did not meaningfully change risk estimates. For example, Ito et al. [19] extensively examined the impact of model specifications on the risk estimates for asthma emergency department visits in NYC and found that inclusion of particulate matter less than 2.5 microns aerodynamic diameter (PM2.5), ozone and nitrogen dioxide (NO2) had little effect except that the risk estimates for sulfur dioxide (SO2) and CO were sensitive to the inclusion of NO2. In addition, in the evaluation of the role of age in modifying the effects of ozone and PM2.5 and asthma hospitalizations in NYC, Silverman and Ito [31] considered both single- and two-pollutant models, and found that adjustment for the other pollutant only slightly affected the risk estimates for these pollutants and that the age-specific effects pattern (i.e., the strongest association in the age 6-18 group) observed were not affected. Therefore, in this study we proceeded to focus on modification of ozone effects by sex and age in this same city.

Methods

Asthma case data

Ozone is a secondary pollutant formed through photochemical reaction of nitrogen oxides, volatile organic compounds, and sunlight, with notably higher concentrations during the warm season. Therefore, these analyses are limited to those months which correspond to the warm season in NYC. ED visits and hospitalizations have each been widely explored in response to short-term changes in air pollution exposures, but each captures a slightly different aspect of asthma exacerbation and use of medical services across sub-populations [34]. Hospital admissions are less frequent and thus may represent more severe cases, relative to outpatient events (wherein patients were treated then discharged) [35].

ED visit and hospitalization event data for asthma (ICD-9 code: 493) among children aged 5 to 17 years old in NYC from 2005-2011 (May 1-September 30) were obtained from the New York Statewide Planning and Research Cooperative System (SPARCS). We excluded case events for children younger than 5 years due to questionable reliability of asthma diagnosis among younger children [36]. Two categories of type of hospital admission - “emergent” (patient requires immediate medical intervention as a result of severe, life threatening, or potentially disabling conditions) and “urgent” (patient requires immediate attention for the care and treatment of a physical or mental disorder) - were selected to capture the acute effect of ozone. Patient data were grouped into these age groups: 5-9 years, 10-13 years, 14-17 years, and 5-17 years. Age groupings were selected, in part, based on the consideration of average age of onset of puberty [37,38]. The dataset was stratified by sex to compare associations for females and males.

Ozone and other pollutant data from regulatory monitors

To construct a city-wide time series of ozone concentrations, hourly monitoring data was retrieved from the seven EPA Air Quality System (AQS) regulatory monitoring stations in NYC, for the years 2005-2011 (May 1 - September 30). We considered the spatial distribution of, and data density within, these regulatory monitors to prevent biasing the city-wide time trend. As two monitors in the Bronx were deployed at the same location and covered complementary data years (years 2005-2006 for Site ID 360050083, and 2007-2011 for Site ID 360050133), these were combined and treated as one monitor. Likewise, data from one monitor in Queens (Site ID: 360810098) was removed, as only one year of data was available. A city-wide time series was then derived from the five remaining monitoring locations, including data from four of five NYC boroughs - two in the Bronx (hereafter referred to as “Bronx A” and “Bronx B”), one in Manhattan, one in Queens, one in Staten Island. Descriptive statistics showed no systematic missing values by hour, day of week, month, or monitor. Hourly concentrations across the five monitoring sites correlated well over the entire time period (r = 0.76 to 0.95), supporting a consistent citywide temporal trend.

To evaluate and interpolate missing values to create a robust city-wide time-series, we assessed the proportion of missing values for each monitor – Bronx A = 1.4%, Bronx B = 26.9%, Queens = 5.5%, Manhattan = 36.8%, and Staten Island = 6.7%. We then used non-missing monitor values as the “independent” variable in successive linear regression models to interpolate missing values at each of the other monitors (dependent variables), starting with Bronx A because it had the fewest missing values (n = 348).

Because ozone has a diurnal pattern – with troughs in early morning (4-6 am) due to limited photochemical activity and scavenging by fresh vehicular emissions, and peaks in early afternoon (12-2 pm) - interpolation was conducted by hour and day of week, to minimize error in the mean trend across days. Third, daily average ozone values were computed from hourly values for each monitor, and a city-wide daily average ozone trend was computed by averaging daily values across all monitors. Because the measures from the two Bronx locations were highly correlated across all hours, their daily values were averaged to avoid over-weighting ozone measurements at this location.

For sensitivity analyses, daily co-pollutant time series for PM2.5 and NO2 were provided by the NYC Department of Health and Mental Hygiene. As with the ozone estimates, these data were calculated using data obtained from EPA AQS regulatory monitoring stations located in NYC (Zev Ross, unpublished report). The city-wide metrics were computed following an approach used by Schwartz [39] that uses averages of scaled daily values to account for differences in variance and mean between sites. The resulting city-wide time daily series was merged with the ozone time series described above.

Weather data

Ambient daily temperature data from the four meteorological stations in the NYC area (JFK International Airport, LaGuardia Airport, Central Park, Newark International Airport) was retrieved from NOAA National Climatic Data Center (NCDC) [40]. Globally, minimum temperatures on land have increased more rapidly than maximum temperatures since 1950 (0.204°C/decade vs. 0.141°C/decade), resulting in a decline in the diurnal temperature range [41]. Physiologic recovery from daytime heat can be impaired if night time minimum temperatures are not sufficiently low [42]. In addition, a separate spatial pattern analysis of New York temperature data showed that minimum temperature has less spatial variation than maximum temperature and is generally representative of city-wide exposures (Zev Ross, unpublished data). Based on this reasoning, we chose to control for minimum temperature in the model. City-wide daily minimum temperature, mean temperature, and dew point temperature were averaged across the four stations. Relative humidity was calculated from mean temperature and dew point temperature using the standard NOAA equation [43].

Statistical analysis

Case-crossover model

We used conditional logistic regression with time-stratified referent sampling in the case-crossover design [44]. Estimated risks were computed for interquartile-range (IQR) increments of ozone, and expressed as percent excess risk. Previous studies have reported increased asthma exacerbation event risk with elevated air pollution levels on current day and up to four days lag [10,45-47]. We estimated risks for lag days 0 to 6 to capture possible delayed health effects of ozone beyond the lag period commonly investigated. We evaluated differential effects across age and sex subgroups. Because case-crossover design allows for examination of effect modification at the level of the individual, rather than subgroup, we also modeled potential interactive effects between ozone, sex, and age.

Referent selection

A key issue in the case-crossover design is selection of referent time period, for which multiple strategies have been proposed, such as: unidirectional selection (referent times only prior to case event) [48], symmetric bidirectional selection (referent times both before and after event evenly) [49], and time-stratified selection (referent times within a given period, such as same-month) [50]. A systematic review on case-crossover analysis showed that only 7.7% used unidirectional design and the remainder bidirectional, most commonly using symmetric or time-stratified bidirectional designs (91.1% combined) [51]. Bi-directional time-stratified methods commonly select referent days to include non-case days on the same day of week, month, and year as the event, which can control for seasonal trends and day-of-week [52,53]. Accordingly, we defined case times as the day of an asthma hospitalization or ED visit, and matched with referent (control) days from the same day of week, in the same calendar month and year [44,54], thus controlling for day of week and season. Weather factors were controlled for by including smooth functions of: current-day minimum temperature (3 degrees of freedom [df]), lagged 6 day average minimum temperature (3 df), and relative humidity (3 df) in the model.

Sensitivity analyses

First, we conducted time-series analysis using Poisson regression to compare the results with the case-crossover model. The time-series analysis controlled for the same weather factors as the case-crossover model with same df, and also included a smooth function of time (8 df) per calendar year to control for long-term trends, and day of the week. In two separate multivariate case-crossover models, we substituted dew point for relative humidity and included co-pollutants – PM2.5 and NO2. To further examine counter intuitive negative associations observed within the case-crossover model (explained in ‘Results’ below), we created a 0-3 lag day average ozone variable to compare these results to the single day exposures. Furthermore, to explore ozone’s functional form, we: examined scatterplots of ozone versus daily asthma counts, stratified models by ozone concentrations and, ultimately, subset by month groupings. To avoid missing any delayed effect further than lag day 6, we conducted a lag days 7-9 analysis separately in time-series model but not in case-crossover model, as in our time stratified case-crossover design, any day beyond lag day 6 would qualify as a referent day.

Data were processed and analyzed in SAS version 9.3 (SAS Institute, Inc., Cary, North Carolina) and R statistical package (version 2.14.0; R Development Core Team 2013). This study was reviewed and approved by the Institutional Review Boards of the University of Pittsburgh and Icahn School of Medicine at Mount Sinai.

Results

There were a total of 8,009 child asthma hospitalizations during the warm seasons (May-September) of 2005 to 2011, including 4,650 (58%) for males and 3,359 (42%) for females (Table 1). Among younger children, in age groups 5-9 years and 10-13 years, boys represented a higher proportion of asthma admissions (60 and 61%) than did girls (39 and 40%). Among older children (14-17 years), a higher proportion of admissions were for girls (54%) than for boys (46%). Over this same period, there were 35,907 asthma ED visits for 5-17 year olds, including 21,047 (58.6%) for males and 14,860 (41.4%) for females (Table 1). Similarly, the proportion was higher for boys than for girls among younger children (62 vs. 38% for both age subgroups), and higher for girls among older children (54 vs. 46%). Daily ozone values during our study period ranged from 0.005 ppm to 0.06 ppm (IQR 0.013 ppm).

Table 1 Asthma events for NYC children in the warm months from 2005 to 2011

ED visits

The percent excess risk of asthma ED visits with higher ozone was significant for both males and females on multiple lag days in the all-ages analysis (5-17 years) (Figure 1); excess risk per IQR increase in ozone ranged from 2.9% (95% CI: 0, 5.9%) to 8.4% (95% CI: 5.4%, 11.5%) for boys, and from 5.4% (95% CI: 2%, 9%) to 6.5% (95% CI: 3.1%, 10%) for girls. The lag times for the ozone effect appeared different for boys versus girls. For boys, the association showed a “U”-shaped lag distribution; no effect at lag day 0, peaking at lag day 3, followed by decreasing magnitude of association and no effect again at lag day 6. For girls, the association with ozone started at lag day 3, and persisted through lag day 6. We observed negative associations with same-day ozone (i.e., lag day 0) among girls, but not among boys.

Figure 1
figure 1

Percent excess asthma emergency department visit risk in NYC children per interquartile range of ozone concentration. Point denotes non-significant association; diamond denotes positive significant association; triangle denotes negative significant association.

Sex-and-age-stratified analysis revealed slight differences in significance and magnitude of associations across subgroups. The highest excess risk for asthma ED visits was found among boys aged 14-17 years on lag day 1 [13.4% (95% CI: 5.4%, 22%)]. For age group 5-9 years, associations appeared on lag day 1 and persisted through lag day 4 for boys; for girls no positive significant association was observed, and a negative association on lag day 0 was observed (but became non-significant when substituting the 0-3 lag day average ozone). For ages 10-13 years, associations started on lag day 1 and persisted through lag day 4 for boys; while for girls significant associations were observed from lag days 3 through 6. Associations for age group 14-17 years showed a similar pattern as age group 10-13 years; significant associations for girls started about 2 days later than for boys, but lasted through lag day 6, when associations for boys became non-significant.

Hospital admissions

The risk of asthma hospitalization was significantly associated with ozone only for girls (5-17 years) (Figure 2); for an IQR increase in ozone, we observed significant excess risk of 8.2% (95% CI: 1.1%, 15.8%) on lag day 2. In analyses stratified by both age and sex, negative associations were observed for boys in age group 5-9 years on lag days 1 and 2. Significant associations were observed among girls aged 10-17 years. The highest excess risk for asthma hospitalization was found among girls aged 10-13 years on lag day 2 [21.5% (95% CI: 6.6%, 38.6%)].

Figure 2
figure 2

Percent excess asthma hospitalization risk in NYC children per interquartile range of ozone concentration. Point denotes non-significant association; diamond denotes positive significant association; triangle denotes negative significant association.

Sensitivity analyses

Results from time-series analysis for childhood asthma ED visits and hospitalization are shown in Additional file 1: Figure S1 and Additional file 2: Figure S2, respectively. The associations and the lag structure patterns are generally consistent with the main case-crossover results. In a case-crossover model testing interaction terms for sex and age (results not shown), the Wald chi-square test indicated statistically significant modification of ozone effects by both age and sex, with associations very similar to those revealed in the main case-crossover analysis. Time-series analysis for lag days 7-9 showed that the associations dropped to non-significant levels on lag day 7 for all age groups (results not shown). The results of the sensitivity analyses substituting dew point for relative humidity and including PM2.5 and NO2 were essentially identical to the initial model. The counter-intuitive negative associations above became positive or non-significant when we substituted the 0-3 lag day average ozone for these ED visit groups: all 5-17 years [9.3% (95% CI: 5.9%, 12.9%)]; all 5-9 years [6.4% (CI 95% 1.7%, 11.4%)]; females 5-17 years [2.6% (-2.3%, 7.8%)]; and females 5-9 years [-0.5% (95% CI -7.5%, 7%)]. However, the negative associations persisted even when using the 0-3 lag day average ozone levels among the hospitalizations for those noted above. Stratifying by ozone level did not show a consistent pattern that differed by ozone level. However, subsetting by month groupings (May-July and August-September) revealed that the observed negative associations were driven primarily by these latter months. Possible explanations for this temporal phenomenon are discussed below.

Discussion

The boys had a higher proportion of asthma ED visits and hospital admissions than girls in the 5-13 year age group; the pattern reversed in the 14-17 year age group where girls had a higher proportion of asthma events, a pattern that is similar to those reported elsewhere [55-58]. Ozone levels were significantly associated with ED visits and hospitalizations for childhood asthma in NYC, although relationships varied between boys and girls. For girls, the strongest ozone and asthma ED visit association occurred in the 10-13 and 14-17 year old groups, whereas no significant modification by age was observed for boys. The boys exhibited more prompt response (lag day 1) to ozone, compared to girls (lag day 3), but the significant associations for girls remained longer, through lag day 6.

Differing patterns in sex-stratified results are of particular interest. A number of hypotheses have been proposed to explain this variation in asthma prevalence and response to pollution by age in boys and girls, including sex-linked biological differences in lung and airway size and growth rates [59], sex hormones [60], or to gender-linked variation in chemical co-exposures including cosmetic products, cigarette smoking, activity patterns, and obesity [61,62]. Our dataset lacked body mass index and personal behavior or product use data, thus we could not explore obesity or co-exposures as possible effect modifiers.

In the sex-stratified results of ED visit (Figure 1), we did not observe significant modification by age for boys. For girls, however, we found multiple significant associations in age groups of 10-13 and 14-17 years. This greater asthma exacerbation effect of ozone observed in older girls (10-17 years) compared with younger ones (5-9 years) likely concurs with active puberty for girls. This finding is consistent in general with a previous study that demonstrated age and sex-related differences in the risk of asthma hospital admission, with the risk among boys higher than girls <9 years of age, and reversed after puberty (10-19 years) [63].

No positive significant association was found on lag day 0 for either sex group, which may be expected, as asthma response may be progressive over subsequent days, or there may be significant delays in reaching medical care. The lagged response to ozone exposure was earlier for boys (lag day 1) than girls (lag day 3) in age groups of 10-13 and 14-17 years, suggesting more rapid ozone response for boys compared to girls of the same age groups. There are some possible explanations for this finding. Boys have smaller airways relative to lung volume than girls, and thus may be more susceptible to air pollutants [64], or differences in smooth muscle and vascular functions, and hormonal status may influence ozone response [65-67]. A Canadian study [33] found shorter lags between nitrogen dioxide exposures and asthma hospitalization among boys (2 to 3 days) than girls (6 to 7 days), but no associations for ozone. The authors proposed that boys have smaller airways relative to lung volume than girls, and thus may be more susceptible to specific stimuli. Lin et al.’s study only analyzed children aged 6-12 years, but nevertheless supports our general finding. An asthma cohort study in Southern California observed that, in that cohort, girls were less likely to play team sports and spent less time outdoors than boys [68], possibly contributing to differences in ozone exposure misclassification and observed associations with asthma outcomes. As outdoor ozone concentrations are higher than indoors, combined with higher ventilation rates during exercise, differences in activity patterns may partially account for differences in observed associations between ambient ozone and asthma.

Our finding of delayed ozone response among girls remaining significant through lag day 6 highlights that the health effects of ozone may be underestimated by only examining through 3 to 4 lag days, commonly done in previous studies. For example, one pediatric asthma study reported the strongest association between ozone and asthma ED visits on lag day 4 among 5-12 years age group [32]; another study conducted in Helsinki, Finland reported significant associations between ozone exposure and asthma ED visits for children (<15 years) from lag day 0 through lag day 3, and an excess risk estimate of 24.6% (95% CI: 21.2%, 67.8%) for lag days 0-4 average [10]; but neither study explored further lag day analysis. Silverman and Ito also examined the association of ozone with acute asthma in NYC and found significant associations for children of ages 6-18 years through lag day 3 but didn't stratify by sex [31].

We found fewer significant associations for asthma hospitalizations than ED visits (Figure 2). A significant association was observed only for girls on lag day 2 (5-17 years) for hospitalizations. When analyzed by age group, multiple positive significant associations were found for girls in age groups of 10-13 and 14-17 years, but not for boys. A study in Helsinki, Finland found stronger associations between ozone and childhood ED visits for asthma than did prior studies of asthma hospitalizations, and the authors suggested that ED visit data may be more sensitive than hospitalization data for capturing asthma exacerbations [10]. Another possible explanation is our time-stratified design which selects referent times before or after case time, or both, and thus could be considered “partially” bidirectional and assumes that cases are still at risk after an event [69]. Sampling post-event referent times for the subjects who stayed hospitalized longer than 7 days, thus, could bias results, as the referent exposures after the case day do not reflect actual exposures and risk of hospitalization for subjects not yet discharged. Although it has been shown that in case-crossover design, the bias resulting from sampling post-event referent times is small [54], we cannot rule out this possibility. This finding should also be interpreted with caution given the smaller statistical power with fewer hospitalization events. The age-stratified hospitalization analyses enabled a more specific consideration of ozone effects for key age ranges of interest but were somewhat limited in statistical power.

Interestingly, our results also initially suggested a “protective” effect of ozone. These largely disappeared when we considered the 0-3 lag day averages among ED visits, but not for hospitalizations. When interpreting this finding, it is important to note that the hospitalizations are largely a subset of the ED visits (i.e. a small percentage of the ED visits go on to be hospitalized), which could be altering the observed ozone association. However, negative associations have also been observed in previous studies; Paulu and Smith, for example, reported negative associations between asthma-related ED visits and ozone concentrations in one year (2003) of their study period (2000-2003). The authors found neither marked differences in source data between 2003 and the prior years, nor indication of confounding strong enough to change the direction of the association. An Israeli study also found negative correlations between emergency room visits and same-day ozone concentrations [70]. Buchdahl and colleagues observed a U-shaped relation between ozone and asthma incidence, indicating that ozone concentrations below a critical level were associated with higher incidence [71], and suggested confounding or possible protective effect of mid-level ozone concentrations. In our study, one plausible explanation is exposure misclassification due to intra-urban variability in ozone levels, not captured in AQS data, or strong inverse spatial correlations between ozone and combustion-related primary pollutants in NYC [72].

Furthermore, some researchers have proposed alternate modeling techniques to remove this physiologically counter-intuitive finding [73,74]. Our additional sensitivity tests revealed that the latter months (August-September versus May – July) show different ozone and asthma outcome relationships. We hypothesize that this finding may be due to declining ozone concentrations during these months with increasing asthma outcomes (as asthma ED visits and hospitalizations peak often in the fall due to rhinovirus infection seasonality and the start of the school year). The inability to control for this periodicity in asthma exacerbations – distinct from longer-term seasonal trends – is a limitation of our time-stratified case-crossover design referent sampling scheme (i.e., the same day of week of the same month in the same year). That is, for the same-day of the week in these latter months, asthma ED visits in the latter half of the month tend to be higher than those in the earlier half; and declining ozone concentrations over the month show the opposite pattern, yielding a negative ozone-asthma relationship. This finding led us to conclude that the residual confounding of the within-season temporal patterns in ozone and asthma exacerbations appears to act as a primary driver of our observed counterintuitive negative associations.

A key strength of our analysis is the confirmatory results between case-crossover and time-series models. Our main case-crossover model demonstrated significant associations between ambient ozone and childhood asthma events, and the time-series model and the case-crossover model with interaction terms provided comparable results, suggesting that associations were reasonably robust. This agreement between analytic techniques echoes previous comparison studies. A recent study of multiple air pollutants on cardiac arrests, using both time-series and case-crossover designs, found that risk estimates in case-crossover results were slightly less significant [75]; the authors argued that a possible explanation is that the case-crossover method applied 12 degrees of freedom per year, compared with 7 degrees of freedom per year for the time-series design. A cohort study conducted in five European cities compared time-series, case-crossover, and extended Cox regression designs, generating similar results, and the authors concluded that case-crossover analyses might underestimate the strength of associations [76]. We did observe such a pattern that the risk estimates were slightly smaller for case-crossover results than time-series results from lag days 1 to 3, but slightly larger for case-crossover results from lag days 4 to 6.

There are several important limitations to our study. First, the use of central site monitoring locations to estimate ozone concentrations likely produces measurement error; assuming this error is non-differential, it may bias effect estimates towards the null. Further, it has been noted that ozone has considerable small-scale spatial variations within urban areas [77], and thus better study designs would capture this spatial variation as well. Carracedo-Martínez pointed out in a systematic review that the potential advantages of incorporating individual-level information in case-crossover design has not been fully explored [51]. Further, our study period was only 35 months, offering a relatively small count of asthma hospitalizations, relative to longer-term studies. Pollen could be a contributor of asthma burden, however, the daily counts were not available for this study. The peak pollen times in this geographical area are spring (trees – a time period not included in our study), June (grasses) and September (weeds). Given the highly time specific and relatively short duration of pollen peaks, we would posit that omission of pollen might bias toward the null. Despite these limitations, our study reasonably corroborates prior results, and contributes towards better understanding differential effects of ozone on childhood asthma exacerbations by sex and age.

Conclusions

Our study showed that short-term ozone exposures in an urban U.S. setting were significantly associated with asthma-related ED visits and hospitalizations in children aged 5-17 years. Our results are generally consistent with a growing body of evidence demonstrating an association between short-term ozone exposures and asthma ED visits and hospitalizations, but revealed substantial variation in effects by sex and age. There was a pattern of a strong ozone-ED visits association for females aged 10 to 17, whereas males exhibited significant associations among all age groups, suggesting that pubertal onset may play a differential role in asthma exacerbation among males and females. The lag time for induction of asthma appeared to be shorter among boys than girls, which may be related to a combination of sex-linked biological differences and gender differences in behaviors and exposure patterns.

Abbreviations

AQS:

Air quality system

CDC:

Centers for Disease Control and Prevention

CI:

Confidence interval

ED:

Emergency department

ICD:

International Classification of Diseases

IQR:

Inter-quartile range

NAAQS:

National Ambient Air Quality Standard

NCDC:

National Climatic Data Center

NOAA:

National Oceanic and Atmospheric Administration

NYC:

New York City

RH:

Relative humidity

SPARCS:

New York Statewide Planning and Research Cooperative System

U.S. EPA:

United States Environmental Protection Agency

References

  1. US EPA. Health Effects of Ozone in the General Population. 2014. http://www.epa.gov/apti/ozonehealth/population.html. Accessed October 1 2014.

  2. Groves AM, Gow AJ, Massa CB, Laskin JD, Laskin DL. Prolonged injury and altered lung function after ozone inhalation in mice with chronic lung inflammation. Am J Respir Cell Mol Biol. 2012;47(6):776–83. doi:10.1165/rcmb.2011-0433OC; 10.1165/rcmb.2011-0433OC.

    Article  CAS  Google Scholar 

  3. Hollingsworth JW, Maruoka S, Li Z, Potts EN, Brass DM, Garantziotis S, et al. Ambient ozone primes pulmonary innate immunity in mice. J Immunol. 2007;179(7):4367–75.

    Article  CAS  Google Scholar 

  4. Triantaphyllopoulos K, Hussain F, Pinart M, Zhang M, Li F, Adcock I, et al. A model of chronic inflammation and pulmonary emphysema after multiple ozone exposures in mice. Am J Physiol Lung Cell Mol Physiol. 2011;300(5):L691–700. doi:10.1152/ajplung.00252.2010.

    Article  CAS  Google Scholar 

  5. Wiegman CH, Li F, Clarke CJ, Jazrawi E, Kirkham P, Barnes PJ, et al. A comprehensive analysis of oxidative stress in the ozone-induced lung inflammation mouse model. Clin Sci (Lond). 2014;126(6):425–40. doi:10.1042/CS20130039; 10.1042/CS20130039.

    Article  CAS  Google Scholar 

  6. Bell ML, McDermott A, Zeger SL, Samet JM, Dominici F. Ozone and short-term mortality in 95 US urban communities, 1987-2000. JAMA. 2004;292(19):2372–8. doi:10.1001/jama.292.19.2372.

    Article  CAS  Google Scholar 

  7. Choi M, Curriero FC, Johantgen M, Mills ME, Sattler B, Lipscomb J. Association between ozone and emergency department visits: an ecological study. Int J Environ Health Res. 2011;21(3):201–21. doi:10.1080/09603123.2010.533366; 10.1080/09603123.2010.533366.

    Article  CAS  Google Scholar 

  8. Filleul L, Cassadou S, Medina S, Fabres P, Lefranc A, Eilstein D, et al. The relation between temperature, ozone, and mortality in nine French cities during the heat wave of 2003. Environ Health Perspect. 2006;114(9):1344–7.

    Article  Google Scholar 

  9. Gold DR, Damokosh AI, Pope CA, Dockery DW, McDonnell WF, Serrano P, et al. Particulate and ozone pollutant effects on the respiratory function of children in southwest Mexico City. Epidemiology. 1999;10(1):8–16.

    Article  CAS  Google Scholar 

  10. Halonen JI, Lanki T, Tiittanen P, Niemi JV, Loh M, Pekkanen J. Ozone and cause-specific cardiorespiratory morbidity and mortality. J Epidemiol Community Health. 2010;64(9):814–20. doi:10.1136/jech.2009.087106.

    Article  Google Scholar 

  11. Stieb DM, Szyszkowicz M, Rowe BH, Leech JA. Air pollution and emergency department visits for cardiac and respiratory conditions: a multi-city time-series analysis. Environ Health. 2009;8(25). doi:10.1186/1476-069X-8-25.

  12. Cassino C, Ito K, Bader I, Ciotoli C, Thurston G, Reibman J. Cigarette smoking and ozone-associated emergency department use for asthma by adults in New York City. Am J Respir Crit Care Med. 1999;159(6):1773–9. doi:10.1164/ajrccm.159.6.9809042.

    Article  CAS  Google Scholar 

  13. Jaffe DH, Singer ME, Rimm AA. Air pollution and emergency department visits for asthma among Ohio Medicaid recipients, 1991-1996. Environ Res. 2003;91(1):21–8.

    Article  CAS  Google Scholar 

  14. Peel JL, Tolbert PE, Klein M, Metzger KB, Flanders WD, Todd K, et al. Ambient air pollution and respiratory emergency department visits. Epidemiology. 2005;16(2):164–74.

    Article  Google Scholar 

  15. Wilson AM, Wake CP, Kelly T, Salloway JC. Air pollution, weather, and respiratory emergency room visits in two northern New England cities: an ecological time-series study. Environ Res. 2005;97(3):312–21. doi:10.1016/j.envres.2004.07.010.

    Article  CAS  Google Scholar 

  16. Thurston GD, Ito K, Hayes CG, Bates DV, Lippmann M. Respiratory hospital admissions and summertime haze air pollution in Toronto, Ontario: consideration of the role of acid aerosols. Environ Res. 1994;65(2):271–90. doi:10.1006/enrs.1994.1037.

    Article  CAS  Google Scholar 

  17. Anderson W, Prescott GJ, Packham S, Mullins J, Brookes M, Seaton A. Asthma admissions and thunderstorms: a study of pollen, fungal spores, rainfall, and ozone. QJM. 2001;94(8):429–33.

    Article  CAS  Google Scholar 

  18. Lee JT, Cho YS, Son JY. Relationship between ambient ozone concentrations and daily hospital admissions for childhood asthma/atopic dermatitis in two cities of Korea during 2004-2005. Int J Environ Health Res. 2010;20(1):1–11. doi:10.1080/09603120903254033.

    Article  CAS  Google Scholar 

  19. Ito K, Thurston GD, Silverman RA. Characterization of PM2.5, gaseous pollutants, and meteorological interactions in the context of time-series health effects models. J Expo Sci Environ Epidemiol. 2007;17 Suppl 2:S45–60. doi:10.1038/sj.jes.7500627.

    Article  Google Scholar 

  20. Strickland MJ, Darrow LA, Klein M, Flanders WD, Sarnat JA, Waller LA, et al. Short-term associations between ambient air pollutants and pediatric asthma emergency department visits. Am J Respir Crit Care Med. 2010;182(3):307–16. doi:10.1164/rccm.200908-1201OC.

    Article  Google Scholar 

  21. CDC. National Health Interview Survey Data 2011. 2014. Available at: http://www.cdc.gov/nchs/nhis.htm. Accessed October 20, 2014.

  22. American Academy of Pediatrics Committee on Environmental Health. Ambient Air Pollution: Health Hazards to Children. Pediatrics. 2004;114(6):1699–707. doi:10.1542/peds. 2004-2166.

    Article  Google Scholar 

  23. Trasande L, Thurston GD. The role of air pollution in asthma and other pediatric morbidities. J Allergy Clin Immunol. 2005;115(4):689–99. doi:10.1016/j.jaci.2005.01.056.

    Article  CAS  Google Scholar 

  24. Moorman JE, Rudd RA, Johnson CA, King M, Minor P, Bailey C, et al. National surveillance for asthma--United States, 1980-2004. MMWR Surveill Summ. 2007;56(8):1–54.

    Google Scholar 

  25. Bateson TF, Schwartz J. Children’s response to air pollutants. J Toxicol Environ Health A. 2008;71(3):238–43. doi:10.1080/15287390701598234.

    Article  CAS  Google Scholar 

  26. Selgrade MK, Plopper CG, Gilmour MI, Conolly RB, Foos BS. Assessing the health effects and risks associated with children's inhalation exposures–asthma and allergy. J Toxicol Environ Health A. 2008;71(3):196–207. doi:10.1080/15287390701597897.

    Article  CAS  Google Scholar 

  27. Gilliland FD. Outdoor air pollution, genetic susceptibility, and asthma management: opportunities for intervention to reduce the burden of asthma. Pediatrics. 2009;123 Suppl 3:S168–73. doi:10.1542/peds. 2008-2233G.

    Article  Google Scholar 

  28. Akinbami LJ, Lynch CD, Parker JD, Woodruff TJ. The association between childhood asthma prevalence and monitored air pollutants in metropolitan areas, United States, 2001-2004. Environ Res. 2010;110(3):294–301. doi:10.1016/j.envres.2010.01.001.

    Article  CAS  Google Scholar 

  29. Dong GH, Chen T, Liu MM, Wang D, Ma YN, Ren WH, et al. Gender differences and effect of air pollution on asthma in children with and without allergic predisposition: northeast Chinese children health study. PLoS One. 2011;6(7):e22470. doi:10.1371/journal.pone.0022470.

    Article  CAS  Google Scholar 

  30. Samoli E, Nastos PT, Paliatsos AG, Katsouyanni K, Priftis KN. Acute effects of air pollution on pediatric asthma exacerbation: evidence of association and effect modification. Environ Res. 2011;111(3):418–24. doi:10.1016/j.envres.2011.01.014.

    Article  CAS  Google Scholar 

  31. Silverman RA, Ito K. Age-related association of fine particles and ozone with severe acute asthma in New York City. J Allergy Clin Immunol. 2010;125(2):367–73.e5. doi:10.1016/j.jaci.2009.10.061.

    Article  CAS  Google Scholar 

  32. Babin SM, Burkom HS, Holtry RS, Tabernero NR, Stokes LD, Davies-Cole JO, et al. Pediatric patient asthma-related emergency department visits and admissions in Washington, DC, from 2001-2004, and associations with air quality, socio-economic status and age group. Environ Health. 2007;6:9. doi:10.1186/1476-069X-6-9.

    Article  Google Scholar 

  33. Lin M, Chen Y, Burnett RT, Villeneuve PJ, Krewski D. Effect of short-term exposure to gaseous pollution on asthma hospitalisation in children: a bi-directional case-crossover analysis. J Epidemiol Community Health. 2003;57(1):50–5.

    Article  CAS  Google Scholar 

  34. Winquist A, Klein M, Tolbert P, Flanders WD, Hess J, Sarnat SE. Comparison of emergency department and hospital admissions data for air pollution time-series studies. Environ Heal. 2012;11:70. doi:10.1186/1476-069X-11-70.

    Article  CAS  Google Scholar 

  35. Lipfert FW. A critical review of studies of the association between demands for hospital services and air pollution. Environ Health Perspect. 1993;101 Suppl 2:229–68.

    Article  Google Scholar 

  36. Potter PC. Current guidelines for the management of asthma in young children. Allergy Asthma Immunol Res. 2010;2(1):1–13. doi:10.4168/aair.2010.2.1.1.

    Article  Google Scholar 

  37. Biro FM, Greenspan LC, Galvez MP. Puberty in girls of the 21st century. J Pediatr Adolesc Gynecol. 2012;25(5):289–94. doi:10.1016/j.jpag.2012.05.009.

    Article  Google Scholar 

  38. Herman-Giddens ME, Steffes J, Harris D, Slora E, Hussey M, Dowshen SA, et al. Secondary sexual characteristics in boys: data from the Pediatric Research in Office Settings Network. Pediatrics. 2012;130(5):e1058–68. doi:10.1542/peds. 2011-3291.

    Article  Google Scholar 

  39. Schwartz J. The distributed lag between air pollution and daily deaths. Epidemiology. 2000;11(3):320–6.

    Article  CAS  Google Scholar 

  40. NOAA. NNDC Climate Data Online. 2014. http://www7.ncdc.noaa.gov/CDO/cdoselect.cmd?datasetabbv=GSOD&countryabbv=&georegionabbv=. Accessed January 8 2014.

  41. Vose RS, Easterling DR, Gleason B. Maximum and minimum temperature trends for the globe: An update through 2004. Geophys Res Lett. 2005;32(23):L23822-L.

    Article  Google Scholar 

  42. US EPA. Excessive heat events guidebook. 2006. www.epa.gov/heatisland/about/heatguidebook.html. Accessed July 20 2014.

  43. NOAA. NOAA's National Weather Service - Relative Humidity. 2014. http://www.erh.noaa.gov/bgm/tables/rh.shtml. Accessed August 2 2014.

  44. Janes H, Sheppard L, Lumley T. Case-crossover analyses of air pollution exposure data: referent selection strategies and their implications for bias. Epidemiology. 2005;16(6):717–26.

    Article  Google Scholar 

  45. Norris G, YoungPong SN, Koenig JQ, Larson TV, Sheppard L, Stout JW. An association between fine particles and asthma emergency department visits for children in Seattle. Environ Health Perspect. 1999;107(6):489–93.

    Article  CAS  Google Scholar 

  46. Burnett RT, Smith-Doiron M, Stieb D, Cakmak S, Brook JR. Effects of particulate and gaseous air pollution on cardiorespiratory hospitalizations. Arch Environ Health. 1999;54(2):130–9. doi:10.1080/00039899909602248.

    Article  CAS  Google Scholar 

  47. Jalaludin B, Khalaj B, Sheppeard V, Morgan G. Air pollution and ED visits for asthma in Australian children: a case-crossover analysis. Int Arch Occup Environ Health. 2008;81(8):967–74. doi:10.1007/s00420-007-0290-0.

    Article  CAS  Google Scholar 

  48. Boutin-Forzano S, Adel N, Gratecos L, Jullian H, Garnier JM, Ramadour M, et al. Visits to the emergency room for asthma attacks and short-term variations in air pollution. A case-crossover study. Respiration. 2004;71(2):134–7. doi:10.1159/000076673.

    Article  CAS  Google Scholar 

  49. Lin M, Chen Y, Burnett RT, Villeneuve PJ, Krewski D. The influence of ambient coarse particulate matter on asthma hospitalization in children: case-crossover and time-series analyses. Environ Health Perspect. 2002;110(6):575–81.

    Article  Google Scholar 

  50. Villeneuve PJ, Chen L, Rowe BH, Coates F. Outdoor air pollution and emergency department visits for asthma among children and adults: a case-crossover study in northern Alberta, Canada. Environ Health. 2007;6:40. doi:1476-069X-6-40.

    Article  Google Scholar 

  51. Carracedo-Martinez E, Taracido M, Tobias A, Saez M, Figueiras A. Case-crossover analysis of air pollution health effects: a systematic review of methodology and application. Environ Health Perspect. 2010;118(8):1173–82. doi:10.1289/ehp.0901485.

    Article  Google Scholar 

  52. Bateson TF, Schwartz J. Control for seasonal variation and time trend in case-crossover studies of acute effects of environmental exposures. Epidemiology. 1999;10(5):539–44.

    Article  CAS  Google Scholar 

  53. Bateson TF, Schwartz J. Selection bias and confounding in case-crossover analyses of environmental time-series data. Epidemiology. 2001;12(6):654–61.

    Article  CAS  Google Scholar 

  54. Lumley T, Levy D. Bias in the case-crossover design: implications for studies of air pollution. Environmetrics. 2000;11(6):689–704. doi:10.1002/1099-095X(200011/12)11:6<689::AID-ENV439>3.0.CO;2-N.

    Article  CAS  Google Scholar 

  55. Choi IS. Gender-specific asthma treatment. Allergy Asthma Immunol Res. 2011;3(2):74–80. doi:10.4168/aair.2011.3.2.74.

    Article  CAS  Google Scholar 

  56. Millar WJ, Hill GB. Childhood asthma. Health Rep. 1998;10(3):9-21 (ENG)–9-2 (FRE).

    Google Scholar 

  57. Schatz M, Clark S, Camargo Jr CA. Sex differences in the presentation and course of asthma hospitalizations. Chest. 2006;129(1):50–5. doi:129/1/50.

    Article  Google Scholar 

  58. Schatz M, Clark S, Emond JA, Schreiber D, Camargo Jr CA. Sex differences among children 2-13 years of age presenting at the emergency department with acute asthma. Pediatr Pulmonol. 2004;37(6):523–9. doi:10.1002/ppul.20018.

    Article  Google Scholar 

  59. Vink NM, Postma DS, Schouten JP, Rosmalen JG, Boezen HM. Gender differences in asthma development and remission during transition through puberty: the TRacking Adolescents’ Individual Lives Survey (TRAILS) study. J Allergy Clin Immunol. 2010;126(3):498–504.e1-6. doi:10.1016/j.jaci.2010.06.018.

    Article  Google Scholar 

  60. Osman M. Therapeutic implications of sex differences in asthma and atopy. Arch Dis Child. 2003;88(7):587–90.

    Article  CAS  Google Scholar 

  61. Clougherty JE. A growing role for gender analysis in air pollution epidemiology. Environ Health Perspect. 2010;118(2):167–76. doi:10.1289/ehp.0900994.

    Article  CAS  Google Scholar 

  62. Castro-Rodriguez JA, Holberg CJ, Morgan WJ, Wright AL, Martinez FD. Increased incidence of asthmalike symptoms in girls who become overweight or obese during the school years. Am J Respir Crit Care Med. 2001;163(6):1344–9. doi:10.1164/ajrccm.163.6.2006140.

    Article  CAS  Google Scholar 

  63. Chen Y, Dales R, Stewart P, Johansen H, Scott G, Taylor G. Hospital readmissions for asthma in children and young adults in Canada. Pediatr Pulmonol. 2003;36(1):22–6. doi:10.1002/ppul.10307.

    Article  Google Scholar 

  64. Bjornson CL, Mitchell I. Gender differences in asthma in childhood and adolescence. J Gend Specif Med. 2000;3(8):57–61.

    CAS  Google Scholar 

  65. Redline S, Gold D. Challenges in interpreting gender differences in asthma. Am J Respir Crit Care Med. 1994;150(5 Pt 1):1219–21. doi:10.1164/ajrccm.150.5.7952543.

    Article  CAS  Google Scholar 

  66. Almqvist C, Worm M, Leynaert B, working group of GALENWPG. Impact of gender on asthma in childhood and adolescence: a GA2LEN review. Allergy. 2008;63(1):47–57. doi:ALL1524.

    CAS  Google Scholar 

  67. von Mutius E. Progression of allergy and asthma through childhood to adolescence. Thorax. 1996;51 Suppl 1:S3–6.

    Article  Google Scholar 

  68. McConnell R, Berhane K, Gilliland F, London SJ, Islam T, Gauderman WJ, et al. Asthma in exercising children exposed to ozone: a cohort study. Lancet. 2002;359(9304):386–91. doi:S0140-6736(02)07597-9.

    Article  CAS  Google Scholar 

  69. Navidi W. Bidirectional case-crossover designs for exposures with time trends. Biometrics. 1998;54(2):596–605.

    Article  CAS  Google Scholar 

  70. Garty BZ, Kosman E, Ganor E, Alon N, Kibilis N, Garty J, et al. Effects of Atmospheric Pollutants on Children Asthma Outbreaks. Open Allergy J. 2009;2(1):9–15. doi:10.2174/1874838400902010009.

    Article  CAS  Google Scholar 

  71. Buchdahl R, Parker A, Stebbings T, Babiker A. Association between air pollution and acute childhood wheezy episodes: prospective observational study. BMJ. 1996;312(7032):661–5.

    Article  CAS  Google Scholar 

  72. Clougherty JE, Kheirbek I, Eisl HM, Ross Z, Pezeshki G, Gorczynski JE, et al. Intra-urban spatial variability in wintertime street-level concentrations of multiple combustion-related air pollutants: the New York City Community Air Survey (NYCCAS). J Expo Sci Environ Epidemiol. 2013;23(3):232–40. doi:10.1038/jes.2012.125.

    Article  CAS  Google Scholar 

  73. Diaz J, Garcia R, Ribera P, Alberdi JC, Hernandez E, Pajares MS, et al. Modeling of air pollution and its relationship with mortality and morbidity in Madrid, Spain. Int Arch Occup Environ Health. 1999;72(6):366–76. doi:90720366.420.

    Article  CAS  Google Scholar 

  74. Jimenez E, Linares C, Martinez D, Diaz J. Role of Saharan dust in the relationship between particulate matter and short-term daily mortality among the elderly in Madrid (Spain). Sci Total Environ. 2010;408(23):5729–36. doi:10.1016/j.scitotenv.2010.08.049.

    Article  CAS  Google Scholar 

  75. Silverman RA, Ito K, Freese J, Kaufman BJ, De Claro D, Braun J, et al. Association of ambient fine particles with out-of-hospital cardiac arrests in New York City. Am J Epidemiol. 2010;172(8):917–23. doi:10.1093/aje/kwq217.

    Article  Google Scholar 

  76. Peters A, von Klot S, Berglind N, Hormann A, Lowel H, Nyberg F, et al. Comparison of different methods in analyzing short-term air pollution effects in a cohort study of susceptible individuals. Epidemiol Perspect Innov. 2006;3:10. doi:1742-5573-3-10.

    Article  Google Scholar 

  77. Monn C. Exposure assessment of air pollutants: a review on spatial heterogeneity and indoor/outdoor/personal exposure to suspended particulate matter, nitrogen dioxide and ozone. Atmos Environ. 2001;35(1):1–32. doi:10.1016/S1352-2310(00) 00330-7.

    Article  CAS  Google Scholar 

Download references

Acknowledgements

The authors are grateful to Dr. Kazuhiko Ito and Zev Ross for their valuable consultations, assistance with the co-pollutant data acquisition and processing, and suggestions on study design and data analysis. This research was supported by NIH R21ES021429 and K23ES024127 (PS).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Perry Elizabeth Sheffield.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

PS participated in study conception and design, acquisition of data, data analysis and interpretation, and revision of the manuscript. JZ participated in study design, performed data analysis and interpretation, drafting and revision of the manuscript. JS participated in study conception and design, acquisition of data, data analysis interpretation, and revision of the manuscript. JC participated in study conception and design, acquisition of data, data analysis interpretation, and revision of the manuscript. All authors read and approved the final manuscript.

Additional files

Additional file 1: Figure S1.

Results from time-series analysis for child asthma emergency department visits as percent excess asthma risk in NYC children per interquartile range of ozone concentration. Point denotes non−significant association; diamond denotes positive significant association; triangle denotes negative significant association.

Additional file 2: Figure S2.

Results from time-series analysis for child asthma hospitalizations as percent excess asthma risk in NYC children per interquartile range of ozone concentration. Point denotes non−significant association; diamond denotes positive significant association; triangle denotes negative significant association.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sheffield, P.E., Zhou, J., Shmool, J.L.C. et al. Ambient ozone exposure and children’s acute asthma in New York City: a case-crossover analysis. Environ Health 14, 25 (2015). https://0-doi-org.brum.beds.ac.uk/10.1186/s12940-015-0010-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12940-015-0010-2

Keywords