WHAT YOU NEED TO KNOW ABOUT CANOPY CHANGE ANALYSIS
POSTER #15325
LIAR, LIAR! YOUR MAP’S ON FIRE HOW IT WORKS
THE PROCESS
STRATAGIES TO ACCESS ACCURACY
A wide variety of available methods, data sources, terms, protocols, and considerations exist for analyzing canopy change over time. Using case studies comparing a variety of imagery, mapping, and estimation techniques, the following common canopy change analysis concerns can be addressed: 1. How to gauge the accuracy of a prior tree canopy study and develop specifications for a new analysis that will produce confidence in results 2. The advantages and limitations of using sampling-based techniques (i-Tree Eco or Canopy) vs. remote sensing-based (aerial, satellite, and LiDAR) 3. The availability of remotely sensed data, software requirements, and how bias can be introduced 4. Technical terms such as accuracy assessment, standard error matrix, users vs. producer’s accuracy, and standard error (SE) percent 5. How to determine what repeat frequency makes sense in your city
QUANTIFY CHANGE
OVERALL ACCURACY
IDENTIFY CANOPY
Vegetation
Impervious
Soil/Dry Veg
Water
Ref. Pixels
Tree Canopy
329
3
2
0
1
335
Vegetation
8
144
16
0
0
168
Impervious
4
7
389
0
0
400
Soil/Dry Veg
0
0
0
3
0
3
Water
0
0
0
0
94
94
Total
341
154
407
3
95
1,000
96%
ERROR
METHODOLOGIES AND TOOLS USED IN CANOPY ANALYSES
TOOLS
A Comparison of Methods
I-TREE TOOLS The i-Tree tools are a suite of free, peer-reviewed urban forest analysis software tools developed by USDA Forest Service researchers and partners. The suite includes over ten different desktop and web-based apps that allow users to analyze and quantify their urban forest’s structure, benefits, and threats based on public- domain science using point-based or plot-based statistical sampling. The i-Tree tools allow users to assess land cover and quantify the economic value of trees based on factors like energy savings, carbon sequestration, and air or water quality, while also prioritizing future plantings. Specifically, the i-Tree Canopy tool can be used to quickly obtain a percentage of tree canopy cover within a given boundary by assigning values of “Tree” (1) or “Non-Tree” (0) to randomly-placed points overlaid on aerial imagery.
Sources: 1. i-Tree Canopy Technical Notes. USDA Forest Service, 2011. 2. Popkin, Gabriel. “DC says its tree canopy is growing. Federal researchers disagree.” Washington, DC: The Washington Post, 2018. 3. Nowak, David J and Eric J Greenfield. “Declining urban and community tree cover in the United States.” Syracuse, NY: USDA Forest Service, 2018. 4. [PG Mississauga study] 5. [PG DC study]
Top-down approach Bottom-up approach Requires field surveying Uses Google imagery and public-domain, peer-reviewed data inputs (included in tool)
i-Tree tools X X X X
Requires additional data inputs from user (aerial or satellite imagery, LiDAR, geographic boundaries of interest) Tree canopy % is modeled with statistical point- or plotbased sampling. Results are uniform throughout AOI. Tree canopy areas are classified pixel-by-pixel throughout AOI. Results are spatially-explicit and % can be calculated at any scale. Faster to complete Free to use Results will vary with inputs (imagery resolution, previous project data, image interpreter, etc.) Introduction of bias is possible Accuracy can be assessed
REMOTE SENSING TOOLS
TOOLS WORKING TOGETHER
Remote sensing refers to processes that utilize imagery collected via plane or satellite to derive on-the-ground data. A wide variety of input data sources and specific methodologies are possible and will yield varying results. The resolution of base imagery can range from 30 meters or greater (USGS Landsast) to 1 meter (USDA National Agriculture Imagery Program) to mere centimeters, and each of these imagery sources can create a slightly different tree canopy dataset. Additional data inputs can also be included, such as LiDAR (elevation data). There are also various technical methodologies available to use to derive tree canopy polygons. A common method is Object Based Image Analysis (OBIA), in which a software program is used to classify features based on their spectral signatures, textures, pattern relationships, and height in an iterative approach.
Both methods can be used to verify, calibrate, and compare canopy results. A canopy change analysis of Mississauga, Ontario performed by Plan-It GEO in 2015 utilized both i-Tree Canopy and remote sensing to estimate canopy coverage at two time periods (1992 and 2014), and both methods produced similar results. The i-Tree Canopy results indicated that Mississauga experienced a 3.5% increase in canopy from 15% to 18.6% from 1992-2014 with a 1.1% SE. The remote sensing results confirmed a 19% canopy cover in 2014 with 94% overall accuracy.
Remote Sensing X
X X X X X X
X
X X
X X
There are a variety of ways to introduce error into a canopy change analysis. Some of these include: •Inconsistent methodology and/or data sources •Object lean produced by image collection •Overestimation in one year vs. underestimation •Showing gaps in tree canopy vs. broad-brushing large stands of trees •Re-evaluating canopy too quickly •User error
There are also ways to check the accuracy of canopy analysis results •To test the accuracy of a statistic, use the Standard Error •To test the accuracy of a map, use a Confusion Matrix
Plan-It Geo recently completed a tree canopy change analysis with of Washington, DC that is useful for evaluating the accuracy of both a point-based and full geospatial canopy analysis. Both methods were performed– remote sensing by Plan-It Geo and pointbased sampling by USFS researchers-- and the two methods (somewhat contentiously) produced very different results.
Tree Canopy
96%
Tree Canopy
98%
Vegetation
94%
Vegetation
86%
Impervious Bare Ground/Soil Water
96% 100% 96%
USERS ACCURACY
Any baseline analysis will almost inevitably need to re-survey canopy cover from time to time to understand changes and adjust policies, plans, and outreach accordingly. The ability to control for various factors that influence accuracy in paramount. This presentation provides urban planners with the primary options, terminology, technologies, and accuracy assessment protocols associated with urban tree canopy change analysis.
Tree Canopy
PRODUCERS ACCURACY
CLASSIFICATION DATA
One of the primary indicators used in urban forestry and sustainability is the metric of percent tree canopy cover in a city or region. While there are many other important criteria such as urban forest composition (species and age diversity), resource management (staff capacity, training, and policy), and community framework (awareness of urban tree benefits, engagement, etc.), tree cover is a relatively easy metric to obtain and understand. Management plans, community rating systems, and even climate action plans now reference tree canopy and canopy cover goals.
BEGIN WITH IMAGERY
SOURCES OF ERROR
REFERENCE DATA
Impervious Bare Ground/Soil Water
CONFUSION MATRIX
The accuracy of a tree canopy map derived using remote sensing can be calculated using a Confusion Matrix. To create a Confusion Matrix, an accuracy assessment must be performed. This process requires creating a set of random sample points throughout the area of interest. Then, much like the i-Tree Canopy analysis method, a user will classify each accuracy point based on their interpretation of the imagery. The same set of points will also be assigned a value based on what type of landcover that point falls on within the imagery. The user’s values will then be compared with the land cover data’s values to get an estimation of accuracy. Overall accuracy is computed by dividing the number of correctly-identified points by the number of total points. There are also two other types of possible error with land cover data. User’s Accuracy The User’s Accuracy, also known as Error of Commission, represents the probability that a point identified as a tree in the map is truly a tree on the ground. User’s accuracy is lessened by Type II errors, or false-positives (e.g. trees included in the land cover data that are not actually trees in the aerial image and real world). Producer’s Accuracy The Producer’s Accuracy, also known as Error of Omission, represents the probability that a point that is truly a tree on the ground is represented in the land cover data. Producer’s accuracy is lessened by Type I errors, or false-negatives (e.g. trees that exist in the imagery and real world that were not correctly classified in the land cover data).
97% 100% 100%
STANDARD ERROR
The accuracy of an i-Tree Canopy or similar sample-based method can be calculated using the Standard Error (SE). SE is a statistical estimate of the uncertainty of the statistical estimate being calculated (i.e. tree canopy coverage). The more points that are classified in the analysis, the more precise the estimation will be because standard error decreases as the number of points increases. SE is calculated with the following equation. A USFS study performed in 2018 used a pointbased statistical sampling method to quantify tree canopy change in all US states and the District of Columbia. In this analysis, 1000 random points were distributed throughout urban and community areas within each region and classified as Trees or Non-Trees based on 2010 and 2015 imagery. This study found that tree canopy in Washington, DC decreased by 2.2%, or approximately 170 acres, over the 5-year period from 36.1% to 33.9% with a 1.5% SE.
No matter methods used, there is always a margin of error in canopy
BOLD STATEMENT, FACT, QUOTE OR OUTCOME
change analysis; consider these stratagies to avoid being a liar, liar with your canopy map on fire
AVOID BEING A LIAR, LIAR WITH YOUR MAP ON FIRE
COMPARITIVE OVER OR UNDER ESTIMATIONOF REMOTE SENING DATA
SENSITIVITY ANALYSIS
EXPLORE To explore the impacts of over- or underestimation in remotely sensed canopy change analysis, a sensitivity analysis was performed. Derived tree canopy polygons were buffered by 1, 2, and -1 meters and then tree canopy coverage was calculated. RESULTS Results indicated that the impact of over- or under-estimation was related to the initial amount of canopy. Beginning with .04 acres, overestimating tree canopy by 2 meters led to an increase of .03 acres, while beginning with 9.25 acres, overestimating by 2 meters led to an increase of 2.20 acres. IMPACT Results also show that although over- or under-estimation of canopy may appear slight in the map, it can have a significant impact when compounded throughout the entire project area. Additionally, underestimation of canopy had a greater impact on overall canopy losses than overestimation had on canopy gains, since overestimating 9.25 acres of canopy by 1 meter led to an increase of 1.24 acres while underestimating it by the same amount led to a decrease of 1.31 acres.
Original 0.04 acres
0.28 acres
9.25 acres
+1 Meter Buffer 0.05 acres/+0.01
0.34 acres/+0.06
10.49 acres
+2 Meter Buffer 0.07 acres/+0.03
0.40 acres/+0.12
11.45 acres
-1 Meter Buffer 0.03 acres/-0.01
0.22 acres/-0.06
7.95 acres
01 02 03 04 05
RECOMMENDATIONS
HOW YOUR RESULTS WILL BE PUT INTO PRACTICE
Determine where your data will be used. This can help inform what type of analysis you perform as well as what level of accuracy you strive to obtain. Determine whether time and affordability or precision are more of a priority when developing your analysis
CAREFULLY REVIEW PREVIOUS RESULTS
Make sure the data you are using to draw conclusions about canopy change are actually comparable. If your previous data was underor overestimated compared to your current data, the change results will be as well, as illustrated in the sensitivity analysis above
ENSURE YOU ARE COMPARING THE SAME AREA
Geographic boundaries like land use or zoning codes, neighborhoods, and even city boundaries can change over the time period of a canopy change analysis. It is important to use the same geographic boundaries in all time periods assessed in order to get meaningful results. When calculating percentages based on area, be sure to be consistent with determining whether water will be included in your total study area or not
TIME YOUR ANALYSIS INTENTIONALLY
Re-assessing canopy too frequently can create additional bias. A number of factors affect how often you should perform a canopy change analysis, such as initial canopy cover, rate of canopy change, and when new base imagery and/or LiDAR will be available
PREPARE FOR UNEXPECTED RESULTS
Remain objective when designing, performing, and assessing your study to get the most accurate results. Human error can also create bias when analysts expect a certain result and then try to prove it through their analysis
Plan-It Geo specializes in trees and technology. We are innovators in web and mobile software, GIS and LiDAR for urban tree canopy assessments, tree inventories, and management plans. Our ISA certified arborists and technical staff build and use Tree Plotter software, the most comprehensive, user-friendly, supported, and customizable inventory and data management software in the world 1-833-TREEMAP (873-3627) | 7878 Wadsworth Blvd, Suite 340 Arvada, Colorado 80003