- Open Access
Validity of environmental audits using GigaPan® and Google Earth Technology
© The Author(s) 2018
- Received: 7 February 2018
- Accepted: 2 July 2018
- Published: 6 July 2018
Health behaviors are shaped by the context in which people live. However, documenting environmental context has remained a challenge. More specifically, direct observation techniques require large investments in time and resources and auditing the environment through web-based platforms has limited stability in spatio-temporal imagery. This study examined the validity of a new methodology, using GigaPan® imagery, where we took photos locally and, stitched them together using GigaPan® technology, and quantified environmental attributes from the resulting panoramic photo. For comparison, we examined validity using Google Earth imagery.
A total of 464 street segments were assessed using three methods: GigaPan® audits, Google Earth audits, and direct observation audits. Thirty-seven different attributes were captured representing three broad constructs: land use, traffic and safety, and amenities. Sensitivity (i.e. the proportion of true positives) and specificity (i.e. the proportion of true negatives) were used to estimate the validity of GigaPan® and Google Earth audits using direct observation audits as the gold standard.
Using GigaPan®, sensitivity was 80% or higher for 6 of 37 items and specificity was 80% or higher for 31 of 37 items. Using Google Earth, sensitivity was 80% or higher for 8 of 37 items and specificity was 80% or higher for 30 of 37 items. The validity of GigaPan® and Google Earth was similar, with significant differences in sensitivity and specificity for 7 items and 2 items, respectively.
GigaPan® performed well, especially when identifying features absent from the environment. A major strength of the GigaPan® technology is its ability to be implemented quickly in the field relative to direct observation. GigaPan® is a method to consider as an alternative to direct observation when temporality is prioritized or Google Earth imagery is unavailable.
- Environmental audit
- Built environment
Built environment characteristics can encourage or constrain an individual’s ability to engage in healthy behaviors such as physical activity and dietary behaviors and have been identified as potential causal factors contributing to obesity and physical inactivity [1–6]. Built environment features include structures supportive of physical activity such as sidewalks, lighting, or buffers to separate pedestrians from motorized traffic. The built environment also includes features that may be barriers, such as the absence of curb cuts, litter, and highly trafficked roads. These environmental features can have a significant impact on individuals’ decisions to engage in physical activity behavior, making them a critical component when evaluating population health [7, 8]. Characterizing fine details of the built environment is a necessary component for: identifying specific features associated with health behaviors, examining current community environments for advocacy work, and documenting where resources are most needed within the local, state, or national level.
While research has documented the importance of place on health and health behaviors, reviews of current methodology used to document features within the built environment have concerning limitations that have been well established [9, 10]. The current gold standard for documenting features within the built environment is direct observation. Direct observation methods require people to walk or drive through an environment to document features around them . However, this method requires significant time and financial resources, thus other methods with reduced cost or effort to collect these data are often used [11, 12]. Examples of these methods include self-report measures, archival data, and web-based audits . Yet, each of these methods has notable limitations. Self-report measures are subject to same-source bias, where the outcome of interest may impact how people report or perceive surrounding environmental attributes . Additionally, the administration of self-report questionnaires can be challenging due to low response rates from participants, and self-reported information has been shown to have low correlation with objective built environment measures [10, 14]. Archival data, such as from Geographic Information Systems (GIS), often lack information on specific exposures of interest, such as the quality of features within the built environment . Moreover, considerable time and expense is required to create exposures from these data . Furthermore, GIS data are restricted by the spatial scale and priorities of the organizations collecting the data, which may have different goals and purposes for the use of their archival data [15, 16]. Web-based audits, such as Google Earth, may have insufficient resolution for documenting important fine-grained attributes (e.g. presence of graffiti, advertising) and web-based imagery is not always available in remote areas of the world [17, 18]. In addition, the timing of web-based imagery is not controlled by the researcher and temporal change to environmental attributes are difficult to capture [19, 20]. Because we rely on these methods to provide evidence for community-based policies that are relevant to health, development in this area is critical.
To examine the validity of GigaPan® methodology a total of 614 street segments were sampled from two communities in Pittsburgh, Pennsylvania. The study sample was comprised of street segments from 13 census tracts. Within this study, a street segment was defined as a segment extending from one street intersection to the next street intersection, and encompassing both sides of the street.
Street segment observation form
To code built environment features, an audit tool developed by Bridging the Gap (BTG) called the ‘BTG Street Segment Observation Form’ was used . The audit instrument was designed to assess key street-level features of the neighborhood environment that are thought to be related to physical activity behavior and active transport. The street segment observation form included 50 items to characterize the built and social environment, focusing on the following environmental constructs: land use, traffic and safety, and public amenities.
Four trained neighborhood residents (i.e. community-based fieldworkers), trained as part of the staff, walked the length of each segment to complete an audit of the built environment for all street segments through direct observation. Fieldworkers recruited to complete direct observation of the built environment were formally trained in the use of the BTG Street Segment Observation Form through written and visual instruction. This included training both in the classroom and in the field on how to recognize the presence of features and how to rate the quality of the features, if applicable. Field Coordinators supplied staff with maps to indicate the street segments they were to observe and conducted field visits to ensure data collection quality. In pairs, field staff walked each street segment and completed the BTG Street Segment Observation Form. A community member hired as part of the research team oversaw data collection and reviewed 10% of the sample to identify and resolve any inconsistencies. All direct observation audits were completed from August 2015 to October 2015.
The GigaPan® robotic camera mount (GigaPan®, Epic; $339.95) was used within the field along with an everyday camera (Canon®, PowerShot S120; $100.00). Four neighborhood residents were trained as staff (i.e. community-based fieldworkers), to collect GigaPan® photos for this project. All community-based field workers who were taking GigaPan® photos read the GigaPan® manual, watched an instructional video, and completed practice photos in the field using the equipment. Each GigaPan® photo was stitched together using the GigaPan® Epic Stich software and labeled with a unique identifier corresponding to the street segment where it was taken. All GigaPan® photos were taken from September 2015 to December 2015, stitched together on site, and then transported to a remote location for coding.
Remote coders were trained within a classroom setting where all coders read the manual and audit tool items were discussed in a group setting using written, visual, and verbal instructions. All coders were oriented to the street segment observation form and instructed on how to navigate GigaPan® photos. Practice GigaPan® photos were independently audited and then discussed within a group setting until there was consensus among all coders. Remote coders used a digital version of the direct observation street segment observation form for all practice audits. All coders were certified as reliable by achieving 80% reliability on a group of sample segments.
Street segment audits were completed in Google Earth by replicating the process used in previous research . A shapefile containing all the street segments was exported to a KMZ file to load into Google Earth. Each line segment was labeled with a unique identifier for the street. Once in Google Earth, coders were able to see the unique identifier provided by the KMZ file, and could easily navigate to the individual street segment and traverse along the segment of interest via Google Street View to identify features in the environment. The training for audits of Google Earth imagery was similar to the training for GigaPan®. Remote coders were trained within a classroom setting where all coders read through the manual and audit tool items were discussed in a group setting using written, visual, and verbal instructions. All coders were oriented to the digital street segment observation form and instructed on how to navigate street view within Google Earth. Practice Google Earth segments were identified for independent coding. All coders met with a reliability coder to discuss any disagreement in a group setting until there was consensus among all coders. Remote coders used a digital version of the direct observation street segment observation form for all practice audits and were provided five attempts to achieve 80% reliability before data collection. Google Earth imagery dates were collected throughout the coding process. Google collected and uploaded new imagery throughout the study period. Google Earth imagery dates ranged from July 2007 to September 2016.
The GigaPan® or Google Earth imagery from 575 street segments were assigned to eight research staff for coding. Research staff were restricted to one audit for a unique street segment, however all research staff audited both GigaPan® and Google Earth imagery. Research staff were restricted to auditing a unique street segment once to eliminate any environmental information the auditor could carryover from one method to another.
To characterize the validity of each method against direct observation, we estimated sensitivity and specificity. A method’s sensitivity is defined as the proportion of “true positives” it detects (i.e. the proportion of present environmental features, as defined by direct observation, and also defined to be present by the alternative measurement method). A method’s specificity is defined as the proportion of “true negatives” it detects (i.e., the proportion of absent environmental features, as defined by direct observation, and also defined to be absent by the alternative measurement method). To evaluate the relative performance of the alternative measurement methods, we compared confidence intervals. Non-overlapping confidence intervals provide a conservative estimate of a statistically significant difference between measures [23, 24].
To calculate sensitivity and specificity all variables were recoded as dichotomous, with 1 indicating the presence of a feature and 0 indicating absence. For measures including ‘neither’, ‘one’, or ‘both sides’ of the street as response categories, (e.g. sidewalk, detached housing) data were recoded to represent ‘neither’ in comparison to ‘one or more sides of the street’. For one ordinal item, the number of traffic lanes, the recoded indicator represents ‘zero and one lane’ in comparison to ‘two or more lanes’ of traffic. Variables previously found to have low reliability (Kappa < 0.20) were excluded from validity analysis. Sensitivity and specificity was calculated using the full analytic sample. In addition, in an effort to minimize bias, sensitivity and specificity were calculated on a second, more robust subsample, which excluded segments with any documented problems. All analyses were performed using STATA, version 14.2 (StataCorp LP, College Station, Texas).
A total of 37 variables were characterized into three broad constructs: land use, traffic and safety, and public amenities.
Validity of GigaPan® and Google Earth technology in comparison to direct observation for all land use categories for street segments in Pittsburgh, PA
GigaPan® versus direct observation
Google Earth versus direct observation
Car repair shop
Bars on windows
Traffic and safety
Validity of GigaPan® and Google Earth technology in comparison to direct observation for all traffic and safety categories for street segments in Pittsburgh, PA
Gigapan® versus direct observation
Google Earth versus direct observation
Number of lanes of trafficb
Special speed limit
Street or sidewalk lighting
Street and sidewalk buffer
Continuous sidewalk both ends
Ped. signal at traffic light
Children/special pop sign
Marked bike lane
Median with traffic island
Validity of GigaPan® and Google Earth technology in comparison to direct observation for all public amenity categories for street segments in Pittsburgh, PA
GigaPan® versus direct observation
Google Earth versus direct observation
Garden, flowers, or planter
Public trash can
Bench/shelter at transit
Benches or other seating
Amount street trees
Trees shading sidewalk
Within the robust subsample, 77 street segments had documented issues that were excluded (Fig. 2). After computing sensitivity and specificity within this subsample, parameter estimates were similar to estimates obtained in the analytic sample. Therefore, only results from the analytic sample are presented.
This study evaluated the validity of measures obtained using GigaPan® imagery by comparing them to the same measures obtained via direct observation (gold standard) and web-based Google Earth audits (less time and cost). Among items measured using GigaPan®, 16.2% exhibited high sensitivity (sensitivity > 80%) and 83.8% exhibited high specificity (specificity > 80%). Google Earth performed similarly to GigaPan® with 21.6% exhibiting high sensitivity and 81.1% exhibiting high specificity. Given the results of this study, GigaPan® was shown to be a reasonable method for characterizing the built environment. A major difference in validity between the GigaPan® and Google Earth methods is that GigaPan® performed better overall for specificity (i.e. when an attribute was coded as absent via direct observation, GigaPan® audits identified the absence of the attribute correctly). On the other hand, the Google Earth method provided better overall sensitivity of environmental features (i.e. when an attribute was coded as present via direct observation, Google Earth audits identified the attribute correctly).
The GigaPan® methodology had a number of strengths. First, a major strength of the GigaPan® technology is its ability to be implemented quickly in the field. Temporal change to environmental attributes may be difficult to capture using Google Earth because researchers cannot control the timing of imagery . GigaPan® serves as a potential solution to capture temporal changes in the environment, allowing researchers to study behavior changes in response to an environmental change. Second, GigaPan® photographs provided fine-grained detail of entire street segments for remote coders to evaluate. The panoramic photos were able to be digitally stored in one central location. In our study, one panoramic photo and all associated images used to create that file was around 30–50 mega-bytes when the files are utilized within the GigaPan® format, however when exported to a tif file as a single panoramic the size increases substantially. Third, measuring characteristics of the built environment has remained a challenge in both national and international settings. GigaPan® may serve as a potential solution to unique challenges faced using virtual audits (e.g. Google Earth), such as limited imagery in international or rural environments [19, 26]. Previous research has demonstrated the feasibility of training international citizens to use innovated technology that captures information about the neighborhood environment [27, 28]. Utilizing citizen scientists and GigaPan®, photos could be captured and audited for features of the environment. Fourth, with GigaPan®, data collectors, who take the photos, can be trained remotely and not incur travel expenses. Using laypersons to obtain the photos could result in cost savings in multi-regional, national, or international studies. The cost of materials needed to obtain GigaPan® panoramic photos amounted to less than $500 ($339.95 for GigaPan® Epic, $100.00 for a Canon camera, $23.00 64 GB SD card, $4.00 for AA batteries, $30.00 for a tripod), assuming availability of a computer. Additionally, there has been increased availability of open source technologies that mimic the capabilities of GigaPan®. Technologies such as Hugin, which provide users the capability of combining overlapping images to allow for fine grain detail within a panorama . Therefore, this technology may have external generalizability beyond the use of a GigaPan® device. However, given the comparability of the results, between GigaPan® and GSV, if imagery is available and time appropriate, GSV is free and requires no time to collect.
Using GigaPan® to document built environment attributes also had notable limitations. GigaPan® is limited by its single vantage point to document the environment. Since the panorama was taken from a single vantage point, coders had difficulty viewing features around obstructions (e.g. cars, signs). Furthermore, because the length of the segment is highly variable the GigaPan® photo was clearer for a short segment and provided additional challenges for a long segment. Multiple photos along the segment can provide details for longer segments, however this introduces additional administrative challenges for documenting and completing an audit. There were also limitations of this study as a whole. Direct observation audits were not obtained at the same time that GigaPan® or Google Earth imagery was taken, which could have limited the validity of both methods of environmental audits. The imagery obtained by community members using GigaPan® was captured from 0 to 145 days after direct observation whereas Google Earth imagery was taken anywhere from 0 to 3003 days before or after direct observation. Additionally, while we had an original sample of 575 street segments, our analysis sample was reduced to 464 street segments due to issues with GigaPan® and Google Earth imagery (Fig. 2). The most commonly reported problems included GigaPan® capturing a fraction of the street segment, missing imagery, and capturing a segment that was not a street.
Using the GigaPan® methodology provides objectively measured micro-scale environmental features, which is a limitation of previous studies using GIS to characterize the built environment . GigaPan® audits obtained comparable results as Google Earth audits when capturing the built environment, and serves as a potential alternative to direct observation methods. Although the results were comparable the time involved in getting images to audit was not comparable to Google Earth. Google Earth is readily available while GigaPan® photos need to be taken. Within this project we were able to implement GigaPan® technology quickly in the field. GigaPan® could be used in the future to capture temporal changes to environmental attributes. GigaPan® provides an accurate representation of the built environment when other measures of characterizing the built environment are unavailable, too expensive, or timing of imagery is prioritized. Future research should examine the association between built environment attributes characterized using GigaPan® and health behaviors (e.g. physical activity) or health outcomes (e.g. obesity) to determine its predictive validity. This method could be of great value to the field, particularly in remote areas where web-based imagery is not currently available.
ET analyzed, interpreted, and was a major contributor in writing the manuscript. CA was a major contributor throughout data analysis and writing stages. RB was instrumental in acquisition of data and interpretation of data. PC made substantial contributions to conception and study design. TD was instrumental in conception of the study and direction of GigaPan® data collection. NC made substantial contributions to study conception, design, data acquisition, and interpretation of data. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Availability of data and materials
The datasets generated and/or analyzed during the current study are not publicly available due to data use agreement, but are available from the corresponding author on reasonable request.
Consent for publication
Ethics approval and consent to participate
This study was reviewed by the University of Michigan institutional review board and deemed exempt from further review.
This research project was supported by an investigator-initiated grants (R21 CA188481, Natalie Colabianchi, PI and R01 CA164137, Tamara Dubowitz, PI) from the National Cancer Institute (NCI), National Institutes of Health, Department of Health and Human Services. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Cancer Institute or the National Institutes of Health.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Oppert JM, Charreire H. The importance of the food and physical activity environments. Nestle Nutr Inst Workshop Ser. 2012;73:113–21.View ArticlePubMedGoogle Scholar
- Harrison F, Jones AP. A framework for understanding school based physical environmental influences on childhood obesity. Health Place. 2012;18(3):639–48.View ArticlePubMedGoogle Scholar
- Ferdinand AO, Sen B, Rahurkar S, Engler S, Menachemi N. The relationship between built environments and physical activity: a systematic review. Am J Public Health. 2012;102(10):e7–13.View ArticlePubMedGoogle Scholar
- Ding D, Gebel K. Built environment, physical activity, and obesity: what have we learned from reviewing the literature? Health Place. 2012;18(1):100–5.View ArticlePubMedGoogle Scholar
- Ding D, Sallis JF, Kerr J, Lee S, Rosenberg DE. Neighborhood environment and physical activity among youth a review. Am J Prev Med. 2011;41(4):442–55.View ArticlePubMedGoogle Scholar
- Frank LD, Engelke P. Multiple impacts of the built environment on public health: walkable places and the exposure to air pollution. Int Reg Sci Rev. 2005;28(2):193–216.View ArticleGoogle Scholar
- Bauman AE, Reis RS, Sallis JF, Wells JC, Loos RJF, Martin BW. Correlates of physical activity: why are some people physically active and others not? Lancet. 2012;380(9838):258–71.View ArticlePubMedGoogle Scholar
- Cain KL, Millstein RA, Sallis JF, Conway TL, Gavand KA, Frank LD, Saelens BE, Geremia CM, Chapman J, Adams MA, et al. Contribution of streetscape audits to explanation of physical activity in four age groups based on the Microscale Audit of Pedestrian Streetscapes (MAPS). Soc Sci Med. 1982;2014(116):82–92.Google Scholar
- Booth KM, Pinkston MM, Poston WS. Obesity and the built environment. J Am Diet Assoc. 2005;105(5 Suppl 1):S110–7.View ArticlePubMedGoogle Scholar
- Brownson RC, Hoehner CM, Day K, Forsyth A, Sallis JF. Measuring the built environment for physical activity: state of the science. Am J Prev Med. 2009;36(4 Suppl):S99–123.View ArticlePubMedPubMed CentralGoogle Scholar
- Brownson RC, Kelly CM, Eyler AA, Carnoske C, Grost L, Handy SL, Maddock JE, Pluto D, Ritacco BA, Sallis JF, et al. Environmental and policy approaches for promoting physical activity in the United States: a research agenda. J Phys Act Health. 2008;5(4):488–503.View ArticlePubMedGoogle Scholar
- Heath GW, Brownson RC, Kruger J, Miles R, Powell KE, Ramsey LT. The effectiveness of urban design and land use and transport policies and practices to increase physical activity: a systematic review. J Phys Act Health. 2006;3(s1):S55–76.View ArticlePubMedGoogle Scholar
- Diez-Roux AV. Neighborhoods and health: where are we and were do we go from here?: Environnement résidentiel et santé: état de la question et perspectives pour le futur. Rev Epidemiol Sante Publique. 2007;55(1):13–21.View ArticlePubMedPubMed CentralGoogle Scholar
- Hajna S, Dasgupta K, Halparin M, Ross NA. Neighborhood walkability: field validation of geographic information system measures. Am J Prev Med. 2013;44(6):e51–5.View ArticlePubMedGoogle Scholar
- Cummins S, Macintyre S, Davidson S, Ellaway A. Measuring neighbourhood social and material context: generation and interpretation of ecological data from routine and non-routine sources. Health Place. 2005;11(3):249–60.View ArticlePubMedGoogle Scholar
- Lucan SC. Concerning limitations of food-environment research: a narrative review and commentary framed around obesity and diet-related diseases in youth. J Acad Nut Diet. 2015;115(2):205–12.View ArticleGoogle Scholar
- Midekisa A, Holl F, Savory DJ, Andrade-Pacheco R, Gething PW, Bennett A, Sturrock HJW. Mapping land cover change over continental Africa using Landsat and Google Earth Engine cloud computing. PLoS ONE. 2017;12(9):e0184926.View ArticlePubMedPubMed CentralGoogle Scholar
- Clews C, Brajkovich-Payne R, Dwight E, Ahmad Fauzul A, Burton M, Carleton O, Cook J, Deroles C, Faulkner R, Furniss M, et al. Alcohol in urban streetscapes: a comparison of the use of Google Street View and on-street observation. BMC Public Health. 2016;16(1):442.View ArticlePubMedPubMed CentralGoogle Scholar
- Schootman M, Nelson E, Werner K, Shacham E, Elliott M, Ratnapradipa K, Lian M, McVay A. Emerging technologies to measure neighborhood conditions in public health: implications for interventions and next steps. Int J Health Geogr. 2016;15(1):20.View ArticlePubMedPubMed CentralGoogle Scholar
- Vanwolleghem G, Ghekiere A, Cardon G, De Bourdeaudhuij I, D’Haese S, Geremia CM, Lenoir M, Sallis JF, Verhoeven H, Van Dyck D. Using an audit tool (MAPS Global) to assess the characteristics of the physical environment related to walking for transport in youth: reliability of Belgian data. Int J Health Geogr. 2016;15(1):41.View ArticlePubMedPubMed CentralGoogle Scholar
- Zenk SN, Slater S, Rashid S. Collecting contextual health survey data using systematic observation. In: Johnson TP, editor. Handbook of health survey methods, vol First. Hoboken: Wiley; 2015. p. 421–45.Google Scholar
- Clarke P, Ailshire J, Melendez R, Bader M, Morenoff J. Using Google Earth to conduct a neighborhood audit: reliability of a virtual audit instrument. Health Place. 2010;16(6):1224–9.View ArticlePubMedPubMed CentralGoogle Scholar
- Liese AD, Colabianchi N, Lamichhane AP, Barnes TL, Hibbert JD, Porter DE, Nichols MD, Lawson AB. Validation of 3 food outlet databases: completeness and geospatial accuracy in rural and urban food environments. Am J Epidemiol. 2010;172(11):1324–33.View ArticlePubMedPubMed CentralGoogle Scholar
- Afshartous D, Preston RA. Confidence intervals for dependent data: equating non-overlap with statistical significance. Comput Stat Data Anal. 2010;54(10):2296–305.View ArticleGoogle Scholar
- Curtis JW, Curtis A, Mapes J, Szell AB, Cinderich A. Using google street view for systematic observation of the built environment: analysis of spatio-temporal instability of imagery dates. Int J Health Geogr. 2013;12(1):53.View ArticlePubMedPubMed CentralGoogle Scholar
- Wu Y-T, Nash P, Barnes LE, Minett T, Matthews FE, Jones A, Brayne C. Assessing environmental features related to mental health: a reliability study of visual streetscape images. BMC Public Health. 2014;14(1):1094.View ArticlePubMedPubMed CentralGoogle Scholar
- Winter SJ, Goldman Rosas L, Padilla Romero P, Sheats JL, Buman MP, Baker C, King AC. Using citizen scientists to gather, analyze, and disseminate information about neighborhood features that affect active living. J Immigr Minor Health. 2016;18(5):1126–38.View ArticlePubMedPubMed CentralGoogle Scholar
- Rosas LG, Salvo D, Winter SJ, Cortes D, Rivera J, Rodriguez NM, King AC. Harnessing technology and citizen science to support neighborhoods that promote active living in Mexico. J Urban Health Bull N Y Acad Med. 2016;93(6):953–73.View ArticleGoogle Scholar
- Hugin—Panorama photo stitcher. http://hugin.sourceforge.net/. Accessed 1 Apr 2018.
- Christiansen LB, Cerin E, Badland H, Kerr J, Davey R, Troelsen J, van Dyck D, Mitáš J, Schofield G, Sugiyama T, et al. International comparisons of the associations between objective measures of the built environment and transport-related walking and cycling: IPEN adult study. J Transp Health. 2016;3(4):467–78.View ArticlePubMedPubMed CentralGoogle Scholar