Figures
Abstract
Background
No global positioning system (GPS) technology study has been conducted among a sample of young gay, bisexual, and other men who have sex with men (YMSM). As such, the purpose of this study was to evaluate the feasibility and acceptability of using GPS methods to understand the spatial context of substance use and sexual risk behaviors among a sample of YMSM in New York City, a high-risk population.
Methods
Data came from a subsample of the ongoing P18 Cohort Study (n = 75). GPS feasibility and acceptability among participants was measured with: 1) a pre- and post-survey and 2) adherence to the GPS protocol which included returning the GPS device, self-report of charging and carrying the GPS device as well as objective data analyzed from the GPS devices. Analyses of the feasibility surveys were treated as repeated measures as each participant had a pre- and post-feasibility survey. When comparing the similar GPS survey items asked at baseline and at follow-up, we present percentages and associated p-values based on chi-square statistics.
Results
Participants reported high ratings of pre-GPS acceptability, ease of use, and low levels of wear-related concerns in addition to few concerns related to safety, loss, or appearance, which were maintained after baseline GPS feasibility data collection. The GPS return rate was 100%. Most participants charged and carried the GPS device on most days. Of the total of 75 participants with GPS data, 75 (100%) have at least one hour of GPS data for one day and 63 (84%) had at least one hour on all 7 days.
Citation: Duncan DT, Kapadia F, Regan SD, Goedel WC, Levy MD, Barton SC, et al. (2016) Feasibility and Acceptability of Global Positioning System (GPS) Methods to Study the Spatial Contexts of Substance Use and Sexual Risk Behaviors among Young Men Who Have Sex with Men in New York City: A P18 Cohort Sub-Study. PLoS ONE 11(2): e0147520. https://doi.org/10.1371/journal.pone.0147520
Editor: Javier R. Lama, Asociacion Civil Impacta Salud y Educacion, PERU
Received: May 22, 2015; Accepted: January 5, 2016; Published: February 26, 2016
Copyright: © 2016 Duncan et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data cannot be made publicly available due to ethical restrictions imposed by the New York University Institutional Review Board in order to protect sensitive information. A minimized and anonymized dataset is available upon request to Dr. Perry Halkitis (perry.halkitis@nyu.edu).
Funding: This work was supported by the NYU Center for Drug Use and HIV Research Pilot Project Awards Program (Drs. Dustin Duncan and Farzana Kapadia, co-Principal Investigators). The NYU Center for Drug Use and HIV Research is funded by the National Institute on Drug Abuse (Grant# 5P30DA011041-17). The parent study, Project 18, was also funded by the National Institute on Drug Abuse (Grant # 1R01DA025537; 2R01DA025537-06A1, Dr. Perry Halkitis, Principal Investigator).
Competing interests: The authors have declared that no competing interests exist.
Introduction
Spatial epidemiology has made important contributions in understanding connections between physical and social neighborhood environments and health and health behavior [1]. The existing research among men who have sex with men (MSM) shows neighborhood factors can influence their drug use and high-risk sexual behaviors [2–9]. For example, one study in 385 young MSM (YMSM) in New York City found that neighborhood-level gay presence was significantly and positively associated with consistent condom use during anal intercourse [3]. Consistent with several other studies on neighborhoods and MSM health [2–9], a limitation of this study was that the neighborhood unit of analysis used was the residential ZIP code. This is problematic as the field of spatial epidemiology is fraught with several issues including spatial misclassification, where a neighborhood-level exposure is characterized incorrectly [10], and the “residential trap,” where static geographic information systems (GIS) approaches limit the study of exposure to the residential area [11, 12]. These methodological issues are important and inhibit our understanding of neighborhood effects on health, e.g., simulation research suggests that defining neighborhoods with too large a spatial unit (e.g. an administrative unit) will underestimate the neighborhood effect on a health outcome [13, 14]. GIS methods usually employ administratively defined neighborhood boundaries (i.e. census tracts or ZIP codes) or spatial buffers around a geocoded location known as static egocentric buffers without recognizing that the contexts in which individuals spend time are not limited to these defined areas [10]. Furthermore, standard GIS techniques do not capture individual travel patterns and behaviors, and GIS methods make isotropic assumptions regarding one’s exposure to specific neighborhood environments, i.e. assumes equal movement in all directions. This issue is known as the uncertain geographic context problem, which arises because of the spatial uncertainty in the actual neighborhoods that exert contextual influences on the individuals being studied and the temporal uncertainty in the timing and duration in which individuals experienced these contextual influences [15]. The neighborhood research in MSM populations is similar to the vast majority of general neighborhoods and health research, whereby most neighborhood health research among MSM suffers from the aforementioned limitations of GIS-based research and focus exclusively on their residential neighborhoods. The concept of “spatial polygamy”, however, argues that people experience and interact with multiple neighborhoods, which can influence their health [16]. Like other populations, MSM are spatially polygamous, including in urban areas such as New York City [17, 18]. For example, one recent study assessing concordance between neighborhoods (defined as the five boroughs of New York City) in which YMSM reside, socialize, and have sex in found that 75% of these men reported discordance between all three of these borough types [17].
Studies are increasingly using mobility data to assess the impact of multiple contextual exposures. There are different approaches to assessing spatial mobility and spatial polygamy including the use of travel diary, web-based assessment of regular destinations as well as use of Global Positioning System (GPS) data. For example, VERITAS (Visualization and Evaluation of Route Itineraries, Travel Destinations, and Activity Spaces) is a web-based application nested within a computer-assisted personal interview that allows researchers to assess daily mobility, travel to regular destinations, and perceived neighborhood boundaries using interactive mapping technology [19] and has been used in previous spatial epidemiology research [20, 21]. Each method to examine spatial mobility has strengths and limitations; for example, a limitation of surveying a participant’s regular locations can be recall bias and this method can be implicated in spatial misclassification. A recent web-based mapping study in MSM found locational errors when participants identified locations, including when they mapped their homes (the median home location error across participants was 0.65 miles) [22]. A well-suited alternative method to address these issues is the use of GPS technology [23, 24], which can record mobility patterns in real-time to assess lengths of exposure to various spatial contexts. While we are not aware of any GPS studies done in MSM populations, there has been some GPS feasibility research conducted among certain marginalized groups such as drug-using populations [25–28]. For example, Mirzazadeh and colleagues [27] conducted focus group discussions among injecting drug users and found that participants identified several concerns about carrying GPS devices, including what authorities might do with the data, that other people who inject drug and dealers may suspect them as informants, and adherence to carrying and use. However, most participants felt concerns were surmountable with detailed informed consent on the purpose of the study and practical ways to carry, charge, and hide devices [27]. Results from a study that conducted focus group discussions among adults in Iquitos, Peru found concerns regarding whether the GPS unit would audio/videotape them and concerns regarding the confidentiality of information [29]. This suggests the need for research assessing the feasibility and acceptability of GPS methods to studying neighborhood contexts among MSM, who may mistrust medical research and research more broadly [30, 31] and because past feasibility of GPS research might not be applicable across population groups.
We focused on YMSM in this study because they are a high-risk group, especially in urban areas such as New York City. As an example, MSM comprise the largest group of individuals in the United States (U.S.) infected with HIV [32]. Stark disparities in HIV prevalence and incidence continue to persist among MSM, especially YMSM [33]. In 2010, YMSM (aged 13–24) accounted for 72% of all new HIV infections among all persons aged 13–24 years, and nearly one-third of new HIV infections among all MSM, with a 22% increase in the number of new HIV infections in this group from 2008–2010 [34]. Rates of new HIV infections are greatest in metropolitan areas with populations of 500,000 people compared to smaller metropolitan and non-metropolitan areas [34]; thus, HIV disparities are greatest in urban areas such as New York City [35], so New York City is an ideal location to conduct this research. Additionally, substance use is highly prevalent among YMSM. A recent nationally representative study found YMSM, who were between the ages of 15 and 24, were more likely to have used marijuana or powder cocaine in the past year and more likely to have engaged in binge drinking in the past year than their heterosexual counterparts [36]. Our prior research with the Project 18 (P18) cohort study demonstrates that past month substance use (e.g. alcohol use and marijuana use) as well as past month condomless sexual behavior (e.g. unprotected receptive oral intercourse and unprotected receptive anal intercourse), as measured using a Timeline Follow Back method [37], increased between baseline and 18-month follow-up [38]. The purpose of this novel study is to evaluate the feasibility and acceptability of utilizing GPS technology in a sample of YMSM in New York City in the P18 cohort study (regardless of their drug use, sexual risk behaviors and HIV status)—employing a state-of-the-science GPS technology.
Materials and Methods
The P18 Cohort Study
Participants in this study were enrolled in Project 18 (P18), a prospective cohort study of sexual behavior, substance use and mental health burdens including 600 racially/ethnically and socioeconomically diverse YMSM recruited between June 2009 and May 2011. The P18 cohort study has been described in detail elsewhere [39, 40]. Briefly, participants were eligible for this study if they were 18 or 19 years old, biologically male, lived in the New York City metropolitan area, reported having had sex (any physical contact that could lead to orgasm) with another male in the 6 months preceding screening, and self-reported a HIV-negative or unknown serostatus. P18 participants predominantly lived in one of the New York City boroughs (69%) [17]. For this GPS sub-study, our goal was to recruit 25 Black men, 25 Hispanic men and 25 White men to participate in this study. To achieve this sample, we emailed n = 200 active study participants of which n = 82 individuals called in to be screened, n = 81 were screened, and n = 75 were scheduled to come in to obtain their GPS devices and take a brief GPS feasibility survey and a survey on perceived neighborhood characteristics (e.g., exposure to neighborhood crime, neighborhood social norms about drug use and sex/HIV and gay friendliness of the neighborhood). Participants were compensated $25 for their first 30 minute visit and $75 at their second visit after returning a week later with the GPS device and charging materials. Assessments for the GPS sub-study were conducted between October and November 2014 at New York University’s Center for Health, Identity, Behavior and Prevention Studies (CHIBPS). Other survey data including substance use, sexual behavior, HIV status and socio-demographics were assessed during a prior office visit and were conducted beginning in July 2014. The Institutional Review Board at New York University approved the research protocol and written informed consent was obtained prior to participation in this study.
GPS Protocol and Data Processing
In this study, we used QStarz’s BT-Q1000XT GPS Travel Recorder (Qstarz International Co., Ltd., Taipei, Taiwan), which has been used in multiple prior research studies [41–50]. Also consistent with other research studies [42, 44, 47, 50–56], GPS tracking of the sample was conducted for a week, i.e. seven (7) days and participants in this study generally wore the GPS device 5 week days and 2 weekend days. We followed a GPS protocol used in past work where the GPS device was set to record locations in 30-second intervals [23]. Prior to implementing the GPS study protocol, key study staff (e.g. Project Directors) participated in the GPS Overview and Training Workshop. The purpose of the workshop was to provide an overview of GPS technology and to review the GPS protocol so that the protocol was implemented with integrity. In the workshop, staff members were trained to use the GPS units and were taught important points to discuss with participants (e.g. how to wear/carry the GPS unit). Participants were instructed to place the small GPS device in their pockets or bags, if their clothing did not have any pockets, and to keep the device with them during the study orientation and baseline assessment. Participants also were instructed to charge the GPS devices nightly and to wear the GPS devices at all times (except when sleeping, swimming or showering). We additionally asked participants to complete a travel diary, which consisted of questions that asks: “Did you charge the GPS monitor today?” (Response options: “Yes” and “No”) and “Did you carry the GPS monitor with you today?” (Response options: “Yes—for all journeys”, “Yes—for some journeys”, “No—but did not make journeys” and “Did not travel today”)–which was meant to help the participant remember to charge the unit and carry it with him throughout the week. The GPS device (battery fully charged and with a unique study GPS serial number) was given to participants in a large zip lock bag, which also contained a mini USB charging cord for the GPS device, a USB wall adapter for charging, a pamphlet containing background information on GPS technology, and a travel diary. Participants were reminded daily (except Sunday) via text to charge their GPS devices. Upon completion of our weeklong GPS protocol (i.e. carrying the device for all journeys, charging the device daily, and completing the travel diary), participants came back to our research office to return the GPS devices at which time participants completed the GPS feasibility and acceptability survey and were given the larger incentive. Whenever a participant forgot to fill out their travel diary (which was relatively rare), project staff members filled the diary out jointly with participants during this follow-up visit.
Several security measures were put in place to protect the GPS mobility data of our participants. First, as previously described, all GPS devices had a unique ID number that was separate from the participant ID number. Second, the GPS data was downloaded by project staff on a single computer in our office that was password protected and in a locked office. The geospatial analyst was sent the GPS data via a secure password protected web-based system and was not given attribute data of any participant. Strict data confidentiality was maintained in this study. GPS data was downloaded using the Qstarz proprietary software and stored as.gpx files. The.gpx files, were transformed into shapefiles, and stored in an ESRI geodatabase for further analysis and map creation. GPS data were processed with several scripts, and with models built using ArcGIS version 10.2 (ESRI, Redlands, CA) and the Python programming language (Python Software Foundation. Python Language Reference, version 2.7. Available at http://www.python.org).
GPS Feasibility and Acceptability
GPS feasibility and acceptability was measured in multiple ways. First, we measured GPS feasibility and acceptability using survey-based methods, consistent with previous research [23, 51, 57, 58]. In particular, we implemented a pre- and post-survey including 7 items that were similar across both surveys to facilitate comparisons [23, 57]. For example, at baseline participants were asked “GPS makes it more interesting to participate in the study” and “I am worried about someone trying to steal the GPS,” while at follow up, participants were asked “GPS made it more interesting to participate in the study” and “I was worried about someone trying to steal the GPS” [23, 57]. Participants were asked to use a Likert scale from 1 (strongly agree) to 5 (strongly disagree) for these items [23, 57]. Similar to past research [23], the survey also included 21 post-only GPS questions, including “Overall, was it easy to use the GPS?”; “Did you feel comfortable wearing the GPS?”; “I would participate in another GPS research study”; and “Would you participate in a GPS study that tracked you for two weeks?” Response options for the majority of these questions were “Yes” and “No”.
Second, we evaluated adherence to the GPS protocol, which included returning the GPS device, self-report of charging and carrying the GPS device (as assessed via the travel diary), and objective data analyzed from the GPS devices (i.e., the total number of GPS data fixes per participant, the number of days the devices were worn per participant, and the amount of GPS data obtained from the GPS device per day) [23, 46, 58–60]. GPS data may be analyzed in various ways depending upon the desired unit of analysis; here we are interested in activity spaces. We did not require that these points necessarily be consecutive in nature because we were assessing overall wear time. In this study, we examined the number of days of wear time of the GPS unit per participant and used various cutoff values to determine GPS wear time across the study participants. Valid and useable GPS data for a day was defined if a participant had at least 120 GPS data points in a given day as this would indicate an hour of wear time with our GPS device attempting to log in 30 second intervals, which is a threshold used in previous GPS research [23]. As additional measures applied in previous GPS research, we used a cutoff of 5 hours, 8 hours and 12 hours of wear time to ascertain GPS data quality [23, 51]. Finally, we present a map of a participants’ GPS data as an additional measure of feasibility. There are different ways to map GPS data [61]. We recognize that caution should be taken when mapping GPS data, including when mapping YMSM, due to concerns regarding confidentiality. However, in this case, New York City is the most populous city and most densely population metropolitan area in the U.S., including a large YMSM population, so we believe that there is little risk of identification given that all other demographic information is presented in aggregate. We have also chosen to obscure the participants GPS data occurring while at their home neighborhood (ZIP code). That is, GPS points in the participant’s ZIP code were hidden to protect confidentiality.
Socio-Demographic Characteristics
We collected data on sociodemographic characteristics such as: age (years), gender (male, transfemale, genderqueer), ethnicity (Hispanic or non-Hispanic) and race (Black, Asian-Pacific Islander, Native American, White, multiracial and other). Participants reported whether or not they were currently enrolled in school (yes, no) as well as their highest level of education completed (high school or less, some college/ technical school, college degree or more). Participants were also asked where they currently lived: in a family apartment/house, their own apartment/house, with friends/roommates or in temporary housing/shelter. They reported their total individual annual income which is categorized here as <$15,000, $15,000-$35,000, >$35,000 which approximates national poverty level (<$15,000) and above (>$15,000). Data were also collected on sources of income including: regular job, public assistance, odd jobs, own business, parent, selling drugs and sex for money/drugs. We assessed sexual identity using the 6-point Kinsey scale, which ranges from 0-exclusively heterosexual to 6-exclusively homosexual. For the purpose of this analysis, sexual identity was dichotomized as exclusively homosexual versus not exclusively homosexual. To assess for relationship status, participants reported whether or not they currently had a main/stead partner (yes, no). In addition, participants reported if they were foreign born (yes, no). Participants were also asked to report their HIV status. If they reported being negative, unknown or positive without proof of status, their status was confirmed with the INSTI™ HIV-1/HIV-2 Rapid Antibody Test from bioLytical™ Laboratories. All positive test results were further verified with a confirmatory HIV test.
Statistical Analysis
First, we computed descriptive statistics on the socio-demographic data. After this, we examined the feasibility of GPS methods among the sample by computing descriptive statistics on the GPS feasibility and acceptability measures. IBM SPSS statistics software (version 21.0; IBM Corp, Armonk, NY) was used to conduct the statistical analyses. The analytic sample for the survey data included participants who answered the pre (n = 75) and post feasibility surveys (n = 74). Analyses of the feasibility surveys were treated as repeated measures as each participant had a pre- and post-feasibility survey. When comparing the similar GPS survey items asked at baseline and at follow-up, we present percentages and associated p-values based on chi-square statistics computed from McNemar's test for related samples [62]. Statistical significance was determined at p < 0.05. The GPS map of a participant was created in ArcGIS (ESRI, Redlands, CA).
Results
Socio-Demographics
Table 1 shows the sample socio-demographics of the YMSM. Most participants were 22 or 23 years of age (92%) and identified as male (93%). By design, most participants were Black, Hispanic, or White (96%). Most were currently not enrolled in school (68.0%) and lived with their family (41.3%). The annual individual-level income was <$15,000 for approximately 50% of the participants, with the main source of income being a regular job. The vast majority (87%) of participants were born in the U.S. Almost half (46%) of the sample completed high school or less education. Almost two-thirds (60%) reported being exclusively homosexual and about 47% reported currently having a partner. A very small percent of the sample was HIV-positive (4%).
GPS Feasibility and Acceptability
Overall, we found that it was feasible and acceptable to use GPS devices among our sample of YMSM. Seventy-five YMSM agreed to participate in the study where one of the criteria was wearing a GPS device for a week and all GPS devices were returned (100%).
Table 2 compares GPS feasibility pre- and post-survey measures. For both pre- and post-survey measures, participants reported high ratings of GPS acceptability and ease of use and low levels of wear-related concerns in addition to few concerns related to safety, loss, or appearance. Therefore, GPS feasibility was maintained after baseline GPS feasibility data collection. These results remained stable from pre to post assessment with the exception of two survey measures. At baseline and follow-up participants were asked "The GPS seems uncomfortable to wear" and "The GPS irritated my skin or was uncomfortable to wear", respectively, where 6.7% and 0% (p = .063) reported they "strongly agree or agree". This marginally significant result suggests that participants became more comfortable with the GPS device once they had used it. Furthermore, at baseline and follow-up participants were asked, “I am concerned that I will lose the GPS" and "I was concerned that I would lose the GPS", respectively, where 8% and 17.6% (p = .039) reported that they "strongly agree or agree". This result suggests that participants became more concerned about losing their GPS device after participating in the GPS protocol. However, it is worth noting that though there was a significant increase in concern over losing the GPS, the overall concern with losing the GPS at post-GPS protocol assessment was still relatively low (17.6%) and no GPS devices were lost over the course of the weeklong protocol. Finally, at baseline and follow-up participants were asked, “I am concerned about how I will look wearing the GPS” and “I was concerned about how I looked wearing the GPS”, respectively, where 10.7% and 2.7% (p = 0.267) reported that they "strongly agree or agree".
The GPS feasibility post survey items overall also demonstrate that use of GPS was viewed as feasible by participants (Table 3). For example, when asked “Overall, was it easy to use the GPS?” 97.3% reported “yes.” Approximately 93% of participants reported “yes” to “Did you feel comfortable wearing the GPS?” To the question, “Did the GPS device get in the way of your everyday activities?” 97% reported “no.” In addition, 98.6% of the sample answered, “yes” to “I would participate in another GPS research study” and 100% answered “yes” to “Would you participate in a GPS study that tracked you for two weeks?”
Table 4 shows an aspect of the GPS protocol compliance, self-reported carrying and charging of the GPS unit, based on the travel diary. We found that, for the most part, participants charged and carried the GPS device on most days. For example, the percentage of participants that charged their GPS for the day ranged from 97.3% to 88.0% for all 7 days of the study. The percentage of participants that carried their GPS on at least some of their daily travels ranged from 96.0% to 82.7% for all 7 days of the study.
Objective GPS data from all participants over all days resulted in a total N of 958,716 GPS data records written into.gpx files. There were likely data errors due to multipath reflectance and timing issues; in addition there were some duplicate data. Multipath reflectance is the process by which signals between GPS satellites and GPS receiver “bounce” or reflect off tall buildings; when the signal connects with the GPS receiver this reflectance causes errors in the location calculation. Here we were looking at activity spaces and areas visited, so we were not concerned with small-scale (i.e. 10–20 meter) variation. Regardless, in an attempt to minimize the effect of data errors on the analysis we applied several techniques to minimize these effects. First, duplicate.gpx data were removed from analysis. Second, data that had either erroneous timestamps (i.e. year 1980) or data with erroneous Latitude or Longitude values (0, 0) were removed. Finally, a nearest neighbor analysis with a distance threshold of 100m was run on data files from each participant in QGIS to remove GPS data points with sudden large jumps in position. These techniques remove a minimal amount of data (N = 5,024 <0.005%), leaving a total sample size of 953,692. The total number of GPS points per participant for the duration of the study varied from a low of 1,739 points (representing approximately 14.5 hours of data) to a high of 22,135 points (representing approximately 184.5 hours of data) and mean count of 12,715 per participant (representing approximately 106.0 hours of data). Of the total of 75 participants with GPS data, all 75 (100%) had at least one hour of GPS data for one day as indicated by 120 GPS fixes on a day given our 30-second sampling rate and 63 (84.4%) had at least one hour (120 GPS points) on 7 or more days (Table 5). When participant data (n = 75) was disaggregated by the number of hours participants wore the GPS, we found that participants had a range of viable data. Using a cutoff of 12 hours of wear time indicating a full day of GPS use (indicated by 1440 GPS data points on any given day), 28 of 75 participants or 37.3% had at least 12 hours of wear time on 7 or more days.
Fig 1 is a map of the raw GPS data, from one participant, overlaid onto their residential ZIP code. This map shows that this participant spent significant time outside of his residential neighborhood. This map also illustrates his movement not being isotropic. During this week of observation, this participant traveled mostly in and between north Brooklyn and lower Manhattan.
Note: “High” means a high density of GPS points, whereas “low” means a low density of GPS points. GPS points located in participants’ home ZIP code were hidden to protect participant privacy.
As an exploratory analysis, we examined GPS feasibility and acceptability by race/ethnicity (i.e. Black, Hispanic, and White) for the different GPS feasibility and acceptability metrics and did not find any appreciable differences (results not shown), suggesting that GPS research is feasible and acceptable for YMSM of different racial/ethnic groups.
Discussion
This is the first GPS study to be conducted among any sample of MSM. In this study, we evaluated the feasibility and acceptability of GPS methods among a sample of YMSM in the New York City-based P18 Cohort Study, as part of a larger sub-study to understand the spatial context of substance use and sexual risk behaviors. We successfully recruited 75 YMSM, where one of the criteria was being willing to wear a GPS device for a week. Overall, we found that it was feasible and acceptable to use GPS devices among our sample of YMSM. For example, the GPS return rate was 100%. Of the participants with any GPS data, 100% had at least one hour of GPS data for one day and 84.4% had at least one hour on 7 or more days. While this met our goal for feasibility, it should be noted that one hour of wear time may not be sufficient to fully quantify activity spaces and a greater amount of GPS data spanning a longer time duration may be required. In addition, while our success rate for one hour of wear time of 7 days was 100%, this fell off when the wear time threshold was increased. Indeed, when participant data was disaggregated by the number of hours participants wore the GPS, we found that participants had a range of viable data.
It is difficult to compare our results with other GPS studies because no other GPS research has been conducted to date in any MSM population. GPS research (including GPS feasibility research) has been conducted in a variety of populations and generally demonstrates that using GPS devices is feasible and acceptable for different population groups, including marginalized populations. In recent a GPS feasibility study among a sample of 120 low-income housing residents in New York City where they wore a GPS device for a week (the study also used QStarz’s BT-Q1000XT GPS Travel Recorder), participants (mean age of 39 years and 55% female) reported high ratings of pre-GPS acceptability and ease of use and low levels of wear-related concerns in addition to few concerns related to safety, loss or appearance, which were maintained after the baseline GPS feasibility data collection. Our results replicate this finding in the YMSM population. The overall GPS return rate in the New York City study of low-income housing residents was 95.6%. Out of the total of 114 participants with GPS data, 112 (98.2%) delivered at least one hour of GPS data for one day and 84 (73.7%) delivered at least one hour on 7 or more days. The qualitative interviews indicated that overall, participants enjoyed wearing the GPS devices, that they were easy to use and charge and that they generally forgot about the GPS device when wearing it daily [23]. Our results are therefore highly comparable to this study.
We recognize that feasibility and acceptability are likely determined by myriad factors. Feasibility and acceptability were likely influenced by our financial incentive, our use of the travel diary, our sending daily reminder texts to charge their GPS device as well as the weeklong time frame of our GPS protocol. In addition, participants’ were allowed to wear the GPS device in their pocket, which may have increased compliance and might explain our finding a lower proportion of people who reported being concerned with how they looked with the GPS at follow-up. We also note that YMSM may be image-focused (as previous research suggests [63]) and participants were worried about it being bulky and noticeable, but after wearing it for a week they realized it was small and out of view. This study provided a cash incentive, and participants seemed to appreciate this method of compensation. The majority of the participant compensation was given after the GPS was returned and the cash incentive may have influenced the 100% return rate in this study. Furthermore, on the consent form we stated “The purpose of this study is to learn more about how global positioning systems (GPS) technology can be used to help us understand what types of neighborhood conditions influence health and health-related behavior of young men”. As such, here and elsewhere the altruistic motives of study participation were emphasized (i.e., producing useful information to reduce HIV and improve neighborhood conditions for MSM through effective policies), which may have also increased participation rate. Over the course of the cohort study (i.e., past 6 years), we have built a strong relationship with our participants. This relationship has allowed a rapport to be developed, which may have increased trust and therefore feasibility and acceptability of these geospatial methods. The participants’ extensive experience with research, combined with appointment reminders, probably increased compliance as well, but we are not able to quantify by what degree.
Future Research
Future research utilizing GPS-based methods are warranted and needed among YMSM, including larger samples of YMSM given the health disparities they experience. There should be other GPS feasibility research among MSM samples in different populations (e.g. homeless MSM), different ages (e.g. older MSM populations) and across geographies including rural populations, for example, in the southern United States (known as the Deep South)–where the prevalence of HIV is particularly high [64, 65]. Because our findings highlight the importance of improving GPS protocols, future studies should find ways to increase adherence to GPS charging and wearing protocols. Different and more advanced GPS protocols could be used. For example, future GPS protocols can have longer assessment periods and assessments over time among the same participants, which can help researchers to understand the extent of spatial overlap in people’s spatial patterns over time [66]. This has rarely been done to date and may vary, in part, due to seasonal variation. Indeed, people’s spatial mobility might vary by meteorological seasons (spring, summer, fall, and winter) [67]. Future research could also test the feasibility and acceptability of using a dedicated GPS device (which was used in this study) as compared to other GPS devices (e.g., a GPS-enabled smartphone). Among MSM, there has been some smartphone-based research on geosocial networking smartphone applications like Grindr [68–72], which could be integrated into GPS research on spatial mobility and spatial polygamy. It is also important for future research to examine how characteristics of activity space neighborhoods (measured by GPS devices) can influence health among MSM including substance use and sexual risk behaviors, while accounting for selective daily mobility bias [19, 24]. For example, activity space neighborhood-level lesbian, gay, bisexual and transgender (LGBT) hate crimes may be associated with poor health among MSM [73–75]. To our best knowledge, no research has been conducted examining relationships between activity space neighborhoods and health behavior (e.g. drug use and sexual risk behaviors) in MSM. We encourage researchers to examine a variety of other neighborhood characteristics including neighborhood-level HIV prevalence in one’s activity space: we recently found that incident infections were marginally higher among those residing in neighborhoods with higher rates of HIV prevalence [76]. Because we [76] and others [17] have found stark racial disparities in HIV (e.g. Black YMSM have higher prevalence and incidence of HIV), it could be useful to examine these activity space neighborhood effects by race/ethnicity and importantly results from this study show that GPS feasibility and acceptability did not vary by race/ethnicity (e.g. GPS devices may be used in different MSM populations including Black YMSM). In this future research, it is important to understand mobility patterns and examine “deliberate” neighborhoods (e.g. those that individuals intentionally travel to regularly) versus “incidental” neighborhoods (e.g. those that individuals may go to for a singular occasion or irregularly)—which in part could be done using qualitative methodologies or by extending the assessment period of the protocol, as suggested earlier. In addition to advancing the literature on health and place, this future research will minimize spatial misclassification [10] as well as overcome the residential trap [11, 12]. Furthermore, the proposed future research will inform specific neighborhood-level policy interventions. For example, increasing community efforts to combat LGBT hate crime neighborhood rates through increased local police attention in high-crime locations may be an HIV prevention intervention (if this association exists). Second, given that we will know the travel patterns of MSM, we will be able to identify optimal geographic locations for drug use and HIV testing/prevention interventions, which currently have limitations. Finally, this research will facilitate the identification of geographic locations suitable for recruiting MSM in research studies (an improved method of venue-based sampling).
Limitations
In spite of this study’s substantial strengths, it also has limitations. In part, we used survey methods to examine the success of GPS feasibility and acceptability. Consequently, participants may have succumbed to social desirability reporting, i.e. the tendency of survey respondents to answer questions in a manner that will be viewed favorably by others. In addition, it is possible that participants could have changed their spatial patterns due to them being GPS tracked, which would lead to potential reactivity bias. However, these issues may be minimal in this study. To illustrate, we asked the question, “Did using the GPS device cause you to alter your behavior?” in the post GPS survey and almost 99% of the sample said “no”. We, therefore, do not believe that the GPS devices altered most participants’ spatial behavior. This study was based on a convenience sample in an existing cohort, which may have consisted of individuals who were more motivated to be in the study and to complete the GPS protocol. By using an existing cohort indeed, there is also selection bias. This could have increased the likelihood of success of the GPS protocol. In addition, while 75 participants is a relatively small sample size for general population health research, given that many recent GPS studies have fewer than 100 participants, our sample size is similar to the sample sizes of most GPS-based research to date. Furthermore, this study was conducted among a relatively small sample of YMSM in New York City and included very few Asian and Multiracial/Other MSM. Additionally, our study was conducted among a sample of predominantly racial/ethnic minority HIV-negative YMSM in New York City. Although relative homogeneity of participants was desired for this feasibility pilot study, these findings, therefore, might only be generalizable to similar MSM populations in urban areas similar to New York City, such as racial/ethnic minority HIV-negative YMSM in urban settings. Because this study was conducted in New York City, a large metropolitan location, issues with large buildings and potential GPS errors due to multipath reflectance are possible [77]. In addition, individuals in New York City often travel via the subway system and while they are underground GPS receivers are unable to obtain signals from GPS satellites, which may lead to additional GPS data loss. Although GPS signals may be lost periodically in the context of a large metropolitan areas, these data are still valuable in determining general activity space and in examining how participants travel throughout different neighborhoods.
Acknowledgments
We thank the participants of the study and the CHIBPS research staff that contributed to the project. We additionally thank William Brady for conducting the statistical analyses and John Twarog for entering the GPS travel diary data. Finally, we thank Dr. Vincent Guilamo-Ramos and Forrest Brooks for commenting on an early version of this manuscript. This work was supported by the NYU Center for Drug Use and HIV Research Pilot Project Awards Program (Drs. Dustin Duncan and Farzana Kapadia, co-Principal Investigators). The NYU Center for Drug Use and HIV Research is funded by the National Institute on Drug Abuse (Grant# 5P30DA011041-17, Dr. Sherry Deren, Principal Investigator). The parent study, Project 18, was also funded by the National Institute on Drug Abuse (Grant # 1R01DA025537; 2R01DA025537-06A1, Dr. Perry Halkitis, Principal Investigator).
Author Contributions
Conceived and designed the experiments: DTD FK. Analyzed the data: DTD SDR. Wrote the paper: DTD WCG. Obtained the funding: DTD FK. Made the figure: SDR. Led the GPS training of research staff: WCG. Verified the GPS travel diary data: WCG. Formatted the references: WCG. Recruited participants: MDL SCB. Collaborated on the funded proposal: SRF PNH. Principal investigator of the parent study: PNH. Read and approved the final version of the manuscript: DTD FK SDR WCG MDL SCB SRF PNH.
References
- 1.
Kawachi I, Berkman LF. Neighborhoods and Health. New York: Oxford University Press; 2003.
- 2. Carpiano RM, Kelly BC, Easterbrook A, Parsons JT. Community and drug use among gay men the role of neighborhoods and networks. J Health Soc Behav. 2011;52(1):74–90. pmid:21362613
- 3. Frye V, Koblin B, Chin J, Beard J, Blaney S, Halkitis P, et al. Neighborhood-level correlates of consistent condom use among men who have sex with men: a multi-level analysis. AIDS Behav. 2010;14(4):974–85. pmid:18712593
- 4. Kelly BC, Carpiano RM, Easterbrook A, Parsons JT. Sex and the community: the implications of neighbourhoods and social networks for sexual risk behaviours among urban gay men. Sociol Health Illn. 2012;34(7):1085–102. pmid:22279969
- 5. Phillips G II, Birkett M, Kuhns L, Hatchel T, Garofalo R, Mustanski B. Neighborhood-Level Associations with HIV Infection Among Young Men Who Have Sex with Men in Chicago. Arch Sex Behav. 2015;44(7):1773–86. pmid:26168977
- 6. Mustanski B, Birkett M, Kuhns LM, Latkin CA, Muth SQ. The role of geographic and network factors in racial disparities in HIV among young men who have sex with men: an egocentric network study. AIDS Behav. 2014;19(6):1037–47.
- 7. Buttram ME, Kurtz SP. Risk and protective factors associated with gay neighborhood residence. Am J Mens Health. 2013;7(2):110–8. pmid:22948299
- 8. Raymond HF, Chen Y-H, Syme S, Catalano R, Hutson M, McFarland W. The role of individual and neighborhood factors: HIV acquisition risk among high-risk populations in San Francisco. AIDS Behav. 2014;18(2):346–56. pmid:23760633
- 9. Vaughan AS, Rosenberg ES, Sullivan PS. Spatial Relationships Between Gay Stigma, Poverty, and HIV Infection Among Black and White Men Who Have Sex with Men in Atlanta. AIDS Res Hum Retroviruses. 2014;30(8):740–1. pmid:25013969
- 10. Duncan DT, Kawachi I, Subramanian S, Aldstadt J, Melly SJ, Williams DR. Examination of how neighborhood definition influences measurements of youths' access to tobacco retailers: a methodological note on spatial misclassification. Am J Epidemiol. 2014;179(3):373–81. pmid:24148710
- 11. Chum A, Patricia O. Contextual determinants of cardiovascular diseases: Overcoming the residential trap by accounting for non-residential context and duration of exposure. Health Place. 2013;24:73–9. pmid:24056364
- 12. Cummins S. Commentary: investigating neighbourhood effects on health—avoiding the ‘local trap’. Int J Epidemiol. 2007;36(2):355–7. pmid:17376797
- 13. Spielman SE, Yoo E-h. The spatial dimensions of neighborhood effects. Soc Sci Med. 2009;68(6):1098–105. pmid:19167802
- 14. Spielman SE, Yoo E-H, Linkletter C. Neighborhood contexts, health, and behavior: understanding the role of scale and residential sorting. Environ Plann B Plann Des. 2013;40(3):489–506.
- 15. Kwan M-P. The uncertain geographic context problem. Ann Assoc Am Geogr. 2012;102(5):958–68.
- 16.
Matthews SA. Spatial polygamy and the heterogeneity of place: studying people and place via egocentric methods. Communities, Neighborhoods, and Health. New York, New York: Springer-Verlag; 2011. p. 35–55.
- 17. Duncan DT, Kapadia F, Halkitis PN. Examination of spatial polygamy among young gay, bisexual, and other men who have sex with men in New York City: the p18 cohort study. Int J Environ Res Public Health. 2014;11(9):8962–83. pmid:25170685
- 18. Koblin BA, Egan JE, Rundle A, Quinn J, Tieu H-V, Cerdá M, et al. Methods to measure the impact of home, social, and sexual neighborhoods of urban gay, bisexual, and other men who have sex with men. PloS One. 2013;8(10):e75878. pmid:24146785
- 19. Chaix B, Kestens Y, Perchoux C, Karusisi N, Merlo J, Labadi K. An interactive mapping tool to assess individual mobility patterns in neighborhood studies. Am J Prev Med. 2012;43(4):440–50. pmid:22992364
- 20. Perchoux C, Kestens Y, Brondeel R, Chaix B. Accounting for the daily locations visited in the study of the built environment correlates of recreational walking (the RECORD Cohort Study). Prev Med. 2015;81:142–9. pmid:26303373
- 21. Stewart T, Duncan S, Chaix B, Kestens Y, Schipperijn J, Schofield G. A novel assessment of adolescent mobility: a pilot study. Int J Behav Nutr Phys Act. 2015;12(1):18.
- 22. Dasgupta S, Vaughan AS, Kramer MR, Sanchez TH, Sullivan PS. Use of a Google Map Tool Embedded in an Internet Survey Instrument: Is it a Valid and Reliable Alternative to Geocoded Address Data? JMIR Res Protoc. 2014;3(2).
- 23. Duncan DT, Regan SD, Shelley D, Day K, Ruff RR, Al-Bayan M, et al. Application of global positioning system methods for the study of obesity and hypertension risk among low-income housing residents in New York City: a spatial feasibility study. Geospat Health. 2014;9(1):57–70. pmid:25545926
- 24. Chaix B, Meline J, Duncan S, Merrien C, Karusisi N, Perchoux C, et al. GPS tracking in neighborhood and health studies: a step forward for environmental exposure assessment, a step backward for causal inference? Health Place. 2013;21:46–51. pmid:23425661
- 25. Martinez AN, Lorvick J, Kral AH. Activity spaces among injection drug users in San Francisco. Int J Drug Policy. 2014;25(3):516–24. pmid:24374172
- 26. Byrnes HF, Miller BA, Wiebe DJ, Morrison CN, Remer LG, Wiehe SE. Tracking adolescents with global positioning system-enabled cell phones to study contextual exposures and alcohol and marijuana use: a pilot study. J Adolesc Health. 2015;57(2):245–7. pmid:26206448
- 27. Mirzazadeh A, Grasso M, Johnson K, Briceno A, Navadeh S, McFarland W, et al. Acceptability of Global Positioning System technology to survey injecting drug users' movements and social interactions: A pilot study from San Francisco, USA. Technol Health Care. 2014;22(5):689–700. pmid:24990173
- 28.
Brouwer KC, Weeks JR, Lozada R, Strathdee SA. Integrating GIS into the Study of Contextual Factors Affecting Injection Drug Use Along the Mexico/US Border. Geography and Drug Addiction. New York, New York: Springer; 2008. p. 27–42.
- 29. Paz-Soldan VA, Stoddard ST, Vazquez-Prokopec G, Morrison AC, Elder JP, Kitron U, et al. Assessing and maximizing the acceptability of global positioning system device use for studying the role of human movement in dengue virus transmission in Iquitos, Peru. Am J Trop Med Hyg. 2010;82(4):723–30. pmid:20348526
- 30. Eaton LA, Driffin DD, Kegler C, Smith H, Conway-Washington C, White D, et al. The role of stigma and medical mistrust in the routine health care engagement of black men who have sex with men. Am J Prev Med. 2015;105(2):e75–e82.
- 31. Hoyt MA, Rubin LR, Nemeroff CJ, Lee J, Huebner DM, Proeschold-Bell RJ. HIV/AIDS-related institutional mistrust among multiethnic men who have sex with men: Effects on HIV testing and risk behaviors. Health Psychol. 2012;31(3):269. pmid:22059617
- 32. Centers for Disease Control and Prevention. Diagnoses of HIV Infection in the United States and Dependent Areas, 2013. HIV Surveillance Report. 2015;25:1–25.
- 33. Koblin BA, Mayer KH, Eshleman SH, Wang L, Mannheimer S, Del Rio C, et al. Correlates of HIV acquisition in a cohort of Black men who have sex with men in the United States: HIV prevention trials network (HPTN) 061. PloS One. 2013;8(7):e70413. pmid:23922989
- 34. Centers for Disease Control and Prevention. Estimated HIV Incidence among adults and adolescents in the United States, 2007–2010. HIV Surveillance Supplemental Report. 2012;17(4).
- 35. Neaigus A, Jenness SM, Hagan H, Murrill CS, Torian LV, Wendel T, et al. Estimating HIV incidence and the correlates of recent infection in venue-sampled men who have sex with men in New York City. AIDS Behav. 2012;16(3):516–24. pmid:21983693
- 36. Brewster KL, Tillman KH. Sexual orientation and substance use among adolescents and young adults. Am J Public Health. 2012;102(6):1168–76. pmid:22021322
- 37.
Sobell LC, Sobell MB. Timeline follow back: A calendar method for assessing alcohol and drug use (Users Guide). Toronto: Addiction Research Foundation; 1996.
- 38. Halkitis PN, Kapadia F, Bub KL, Barton S, Moreira AD, Stults CB. A longitudinal investigation of syndemic conditions among young gay, bisexual, and other MSM: the P18 Cohort Study. AIDS Behav. 2014;19(6):970–80. Epub 2014 Sep 6.
- 39. Halkitis PN, Figueroa RP. Sociodemographic characteristics explain differences in unprotected sexual behavior among young HIV-negative gay, bisexual, and other YMSM in New York City. AIDS Patient Care STDs. 2013;27(3):181–90. pmid:23442029
- 40. Halkitis PN, Kapadia F, Siconolfi DE, Moeller RW, Figueroa RP, Barton SC, et al. Individual, psychosocial, and social correlates of unprotected anal intercourse in a new generation of young men who have sex with men in New York City. Am J Public Health. 2013;103(5):889–95. pmid:23488487
- 41. Christian WJ. Using geospatial technologies to explore activity-based retail food environments. Spat Spatiotemporal Epidemiol. 2012;3(4):287–95. pmid:23149325
- 42. Dessing D, de Vries SI, Graham JM, Pierik FH. Active transport between home and school assessed with GPS: a cross-sectional study among Dutch elementary school children. BMC Public Health. 2014;14(1):227.
- 43. Epstein DH, Tyburski M, Craig IM, Phillips KA, Jobes ML, Vahabzadeh M, et al. Real-time tracking of neighborhood surroundings and mood in urban drug misusers: Application of a new method to study behavior in its geographical context. Drug Alcohol Depend. 2014;134:22–9. pmid:24332365
- 44. Harrison F, Burgoine T, Corder K, van Sluijs EM, Jones A. How well do modelled routes to school record the environments children are exposed to?: a cross-sectional comparison of GIS-modelled and GPS-measured routes to school. Int J Health Geogr. 2014;13(1):5.
- 45. Hurvitz PM, Moudon AV, Kang B, Saelens BE, Duncan GE. Emerging technologies for assessing physical activity behaviors in space and time. Front Public Health. 2014;2:2. pmid:24479113
- 46. O'Connor TM, Cerin E, Robles J, Lee RE, Kerr J, Butte N, et al. Feasibility study to objectively assess activity and location of Hispanic preschoolers: a short communication. Geospat Health. 2013;7(2):375–80. pmid:23733298
- 47. Yen IH, Leung CW, Lan M, Sarrafzadeh M, Kayekjian KC, Duru OK. A Pilot Study Using Global Positioning Systems (GPS) Devices and Surveys to Ascertain Older Adults’ Travel Patterns. J Appl Gerontol. 2015;34(3):190–201.
- 48. Robinson AI, Oreskovic NM. Comparing self-identified and census-defined neighborhoods among adolescents using GPS and accelerometer. Int J Health Geogr. 2013;12(1):57.
- 49. Evenson KR, Wen F, Hillier A, Cohen DA. Assessing the contribution of parks to physical activity using global positioning system and accelerometry. Med Sci Sports Exerc. 2013;45(10):1981–7. pmid:23531716
- 50. Klinker CD, Schipperijn J, Christian H, Kerr J, Ersbøll AK, Troelsen J. Using accelerometers and global positioning system devices to assess gender and age differences in children’s school, transport, leisure and home based physical activity. Int J Behav Nutr Phys Act. 2014;11(1):8.
- 51. McCluskey A, Ada L, Dean CM, Vargas J. Feasibility and validity of a wearable GPS device for measuring outings after stroke. ISRN Rehab. 2012;2012.
- 52. Clark RA, Weragoda N, Paterson K, Telianidis S, Williams G. A pilot investigation using global positioning systems into the outdoor activity of people with severe traumatic brain injury. J Neuroeng Rehabil. 2014;11(1):37–7.
- 53. Wiehe SE, Kwan M- P, Wilson J, Fortenberry JD. Adolescent Health-Risk Behavior and Community Disorder. PloS One. 2013;8(11):e77667. pmid:24278107
- 54. Yan K, Tracie B, Marie-Ève M, Mélanie H, Jean-Luc B, Benoit T, et al. Innovation through wearable sensors to collect real-life data among pediatric patients with cardiometabolic risk factors. Int J Pediatr. 2014;2014.
- 55. Hurvitz PM, Moudon AV. Home versus nonhome neighborhood: quantifying differences in exposure to the built environment. Am J Prev Med. 2012;42(4):411–7. pmid:22424255
- 56. Zenk SN, Schulz AJ, Matthews SA, Odoms-Young A, Wilbur J, Wegrzyn L, et al. Activity space environment and dietary and physical activity behaviors: a pilot study. Health Place. 2011;17(5):1150–61. pmid:21696995
- 57. Zenk SN, Schulz AJ, Odoms-Young A, Wilbur J, Matthews SA, Gamboa C, et al. Feasibility of using global positioning systems (GPS) with diverse urban adults: Before and after data on perceived acceptability, barriers, and ease of use. J Phys Act Health. 2012;9(7):924–34. pmid:21952361
- 58. Webber SC, Porter MM. Monitoring mobility in older adults using global positioning system (GPS) watches and accelerometers: a feasibility study. J Aging Phys Act. 2009;17(4):455–67. pmid:19940324
- 59. Oliver M, Badland HM, Mavoa S, Duncan MJ, Duncan J. Combining GPS, GIS, and accelerometry: methodological issues in the assessment of location and intensity of travel behaviors. J Phys Act Health. 2010;7(1):102–8. pmid:20231761
- 60. Duncan JS, Badland HM, Schofield G. Combining GPS with heart rate monitoring to measure physical activity in children: A feasibility study. J Sci Med Sport. 2009;12(5):583–5. pmid:19036637
- 61.
Duncan DT, Regan SD. Mapping multi-day GPS data: a cartographic study in NYC. J Maps. 2015:In Press.
- 62. McNemar Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika. 1947;12(2):153–7. pmid:20254758
- 63. Tiggemann M, Martins Y, Kirkbride A. Oh to be lean and muscular: Body image ideals in gay and heterosexual men. Psychol Men Masc. 2007;8(1):15.
- 64. Reif S, Geonnotti KL, Whetten K. HIV infection and AIDS in the Deep South. Am J Public Health. 2006;96(6):970–3. pmid:16670228
- 65. Reif SS, Whetten K, Wilson ER, McAllaster C, Pence BW, Legrand S, et al. HIV/AIDS in the Southern USA: A disproportionate epidemic. AIDS Care. 2014;26(3):351–9. pmid:23944833
- 66. Gonzalez MC, Hidalgo CA, Barabasi A-L. Understanding individual human mobility patterns. Nature. 2008;453(7196):779–82. pmid:18528393
- 67.
Kirchner TR, Gao H, Anesetti-Rothermel A, Carlos H, House B, editors. Longitudinal human mobility and real-time access to a national density surface of retail outlets. 3rd International Workshop on Urban Computing; 2014 August 24; New York, New York.
- 68. Goedel WC, Duncan DT. Geosocial-Networking App Usage Patterns of Gay, Bisexual, and Other Men Who Have Sex With Men: Survey Among Users of Grindr, A Mobile Dating App. JMIR Public Health and Surveillance. 2015;1(1):e4.
- 69. Landovitz RJ, Tseng C-H, Weissman M, Haymer M, Mendenhall B, Rogers K, et al. Epidemiology, sexual risk behavior, and HIV prevention practices of men who have sex with men using GRINDR in Los Angeles, California. J Urban Health. 2013;90(4):729–39. pmid:22983721
- 70. Rendina HJ, Jimenez RH, Grov C, Ventuneac A, Parsons JT. Patterns of lifetime and recent HIV testing among men who have sex with men in New York City who use Grindr. AIDS Behav. 2014;18(1):41–9. pmid:23925515
- 71. Rice E, Holloway I, Winetrobe H, Rhoades H, Barman-Adhikari A, Gibbs J, et al. Sex risk among young men who have sex with men who use Grindr, a smartphone geosocial networking application. J AIDS Clinic Res. 2012;(S4):5.
- 72. Winetrobe H, Rice E, Bauermeister J, Petering R, Holloway IW. Associations of unprotected anal intercourse with Grindr-met partners among Grindr-using young men who have sex with men in Los Angeles. AIDS Care. 2014;26(10):1303–8. pmid:24754563
- 73. Duncan DT, Hatzenbuehler ML. Lesbian, gay, bisexual, and transgender hate crimes and suicidality among a population-based sample of sexual-minority adolescents in Boston. Am J Public Health. 2014;104(2):272–8. pmid:24328619
- 74. Duncan DT, Hatzenbuehler ML, Johnson RM. Neighborhood-level LGBT hate crimes and current illicit drug use among sexual minority youth. Drug Alcohol Depend. 2014;135:65–70. pmid:24326203
- 75. Hatzenbuehler ML, Duncan D, Johnson R. Neighborhood-level LGBT Hate Crimes and Bullying Among Sexual Minority Youths: A Geospatial Analysis. Violence Vict. 2015: 30(4):663–75. pmid:26160063
- 76. Halkitis P, Kapadia F, Ompad D. Incidence of HIV infection in Young Gay, Bisexual, and other YMSM: The P18 Cohort Study. J Acquir Immune Defic Syndr. 2015.
- 77. Georgiadou Y, Kleusberg A. On carrier signal multipath effects in relative GPS positioning. Manuscripta Geodaetica. 1988;13(3):172–9.