Utilizing Machine Learning Algorithms for Reconstructing Grid-Edge Topologies for Renewable Integration Presented by Elizabeth Cook Created by Elizabeth Cook, Dr. Yang Weng, and Bilal Saleem October 28, 2019
Agenda • • • • • •
Traditional Grid vs. Future Grid Impact of DER in Regards to Existing Power System The New Complexity of the Grid Key Technology Areas Required for the Future Grid Current Status of Utility System Data Summary Integrate AMI Data to Develop Processes to Ensure Accuracy of System Topology • Methodology to Integrating Voltage Interval AMI Data in Topology Recovery
2
Traditional Grid vs. Future Grid
Power Plants
Electric Grid Power Flow
Customers
Traditional Grid • One-way power flow • Utility to have control over electricity flow • Match energy generation with consumption
3
Traditional Grid vs. Future Grid
Power Plants
Electric Grid Power Flow
Two Way Power Flow
Customers
Traditional Grid • One-way power flow • Utility to have control over electricity flow • Match energy generation with consumption
Future Grid • Two-way power flow • Intermittent resources (solar and wind) • More difficult to match generation and consumption 4
Impact of DER in Regards to Existing Power System Operational Challenges
Operational grid challenges on the grid associated with large scale Distributed Energy Resources (DERs) and loads: • Lack of visibility of distribution system • Uncontrollable nature of DER output • Photovoltaic (PV) inverters in large amounts can affect the frequency response and voltage profile of the system • Forecast assumptions of “net load” seen by operators • Demand response variability and forecast uncertainty • Uncertainty/assumptions associated with commercial, industrial and residential storage
5
Impact of DER in Regards to Existing Power System Bulk System Reliability Associated with DERs
North American Electric Reliability Corporation (NERC) has identified the following factors on the Bulk Electric System associated with DERs: •
Need for establishing requirements to regulate the aggregated voltage at the point of interconnection to the transmission
•
Potential over-generation during minimum load periods due to DER plus grid connected base load and non-dispatchable generation
•
Need for developing standards for DERs wishing to participate in ancillary service markets
•
Coordination and reconciliation of IEEE 1547 interconnection standards with proposed DER grid codes on fault current, low-voltage ride-through, frequency ride-through etc.
•
Potential system protection coordination due to current flow reversal
•
System restoration coordination between transmission/distribution resources
6
New Complexity of the Grid
Climate Change/Grid Resiliency
DC Fast Charger Impacts (EV)
DER Two-Way / Participating in Unpredictable Wholesale Power Flows Markets
Frequency of Distribution Outages and Use Of Switching Higher Amount of Masked Loads
Volatility in Frequency and Voltage Integration of Smart Inverter Functions & Capabilities
Public Safety Power Shut Off Capability Integration of Non-Wires Alternatives for Grid Services
Communications Network Reaching Capacity Limits Maintaining Cybersecurity
7
Key Technology Areas Required for the Future Grid
Distribution Planning & Interconnection Tools
Leverage increased amounts of data to analyze past, current and future models to make accurate decisions about future infrastructure needs, minimizing costs and maximizing benefits of DERs.
Distribution Operations Tools
Enhance operational capabilities to improve safety and reliability. Assess, monitor, analyze, and manage grid resources to enable quick response to outages and integrate DERs for customer and grid benefit.
Customer Service and Operational Efficiency
Utilize improvements in the metering infrastructure to improve customer service and drive operational efficiency while also improving security and expanding connectivity to 3rd party IoT devices.
8
Current Status of Utility System Data New Current Situation • • •
Data Overload Constraints Applied Engineers spend time reconciling data
Data Sources • • • •
The Old Way • Lack of data • Inadequate data • Slow and Costly
AMI meters GIS Data System Models AMI Data Warehouse
Streamline Data and Provide Transformational Information 9
Integrate AMI Data to Develop Distribution Feeders Current State “Top Down” Allocation • Use limited data to estimate loading for forecasting and simulation Future State “Bottom Up” Aggregation • Integrate granular AMI data to: • Build more statistically precise forecast shapes • Perform more accurate load allocations
Integrating AMI Data enables better planning with more granular data and enhances capabilities to inform operational tools as well 10
Methodology to Integrating Voltage Interval AMI Data in Topology Recovery
11
What is Principal Component Analysis?
The voltage data included 30days worth of 5-min voltage interval data for 10,000 meters. (i.e., 10,000 x 8,640). It is difficult to present the highdimensional data (3+) therefore to best understand the data in a lower dimension, we use principal component analysis to reveal the internal structure of the data in a way that best explains the variance in the data. This is done by using only the first few principal components so that the dimensionality of the transformed data is reduced.
The scatter plot is the original 2-dimensional data, the principal components are the two orthogonal vectors. The first principal can be used to approximate the original data.
12
What are Cluster Algorithms? When you know nothing about the data, utilizing cluster algorithms allows you to identify common denominators or certain attributes to explore different features within a dataset. For instance, the notion of a “ cluster� cannot be specifically defined therefore research has developed multiple algorithms to distinguish different clusters.
13
Integrating Voltage Interval AMI Data in Topology Recovery Voltage Interval AMI Data (provided by Feeder level)
All voltage array s sorted by Street Name (i.e., Chestnut St.)
House Numbers split by Intersections
...(100-190)
All voltage array s sorted by Street Name (i.e., Walnut St.)
(200-290)
House Numbers split by Intersections
(100-190)
(200-290)
All voltage array s sorted by Street Name (i.e., Evergreen St.)
House Numbers split by Intersections
[ Meter 1 ID, House #, Street Name, V11, V12, V13, V14,
V18640 ]
[ Meter N ID, House #, Street Name, VN1, VN2, VN3, VN4,
VN8640 ]
...(100-190)
(200-290)
PCA
[ Meter 1 ID, House #, Street Name, P1, P2, P3, Mean V1, Std. Dev. V1] [ Meter 2 ID, House #, Street Name, P1, P2, P3, Mean V2, Std. Dev. V2] [ Meter 3 ID, House #, Street Name, P1, P2, P3, Mean V3, Std. Dev. V3]
Organized and Cleaned Utility Voltage Data
[ Meter N ID, House #, Street Name, P1, P2, P3, Mean VN, Std. Dev. VN]
+ GPS Coordinates of All Utility Poles
+
GPS Coordinates of a Portion of Pole-top Transformers
+
Voltage Data Array Normalized and Weighting Criteria Applied
Driving Distances Calculated from each Meter Point API BING Maps
X2
Weighting Criteria
14
Overview of Clustering Approaches Clustering Approaches Approaches
Methods
Applications
DBSCAN
Filter Outlier AMI meters
Density based
K-means
Voltage Interval AMI Data
Determine connectivity AMI meters Transformers
Integrate Utility GIS Data
Integrate Public Map Data
Optimize within cluster sum of distances
BIRCH
Hierarchical Clustering Fixed max cluster Diameter
15
What is DBSCAN? Density-based Spatial Clustering of Applications with Noise (DBSCAN) – Data points • đ?œ€ (Eps) : radius of neighborhood w.r.t. some point • minPts : minimum number of points required to form a dense region – Core point: more than minPts points in đ?œ€ -neighborhood – Border point: fewer than minPts within đ?œ€ but is in neighborhood of a core point – Noise (Outlier): not a core or border point; robust to noise
đ?œ€
đ?œ€=1 minPt =3
16
Using DBSCAN to Remove Outlier Meters • Sometimes there are 1 or 2 AMI meters farther away from the others which act as an outlier for the algorithm and will disturb the results • Using DBSCAN helps to remove the AMI meters with incorrect location data • One important step is to remove missing values (and/or Bad data)
17
What is K-means? … You have some data that you needed to put into “3” clusters. Step 1: Select the number of clusters you want to identify in your data. In this case k=3. That is to say we want to identify 3 clusters. Step 2: Randomly select three random data points these are the original first cluster. Step 3: Measure the distance between the 1st point and the three initial clusters.
D3 = distance of random third data point D2 = distance of random second data point D1 = distance of random first data point
Step 4: Assign the 1st point to the nearest cluster. In this case the nearest cluster is the blue cluster Step 5: Calculate the mean of each cluster until we are done. And then restart over, calculating the mean of each cluster than re-cluster based on the new means. It repeats until the clusters do not change Step 6: Now that the data is clustered, we sum the variations within each cluster and then do it all again. Then assess the quality of the clustering by adding up the variation within each cluster.
Calculated mean of cluster set
Variation of blue cluster
Variation of green cluster
Variation of yellow cluster
18
Integrate Utility GIS Data Integrate Utility GIS Data
GPS Coordinates of All Utility Poles
Voltage Interval AMI Data
Integrate Public Map Data
Clustering Approaches Separate StreetWise Chunks
StreetWise Clustering
19
Integrate Public Map Data Integrate Public Map Data
Public Map Data
Voltage Interval AMI Data
Clustering Approaches
Integrate Utility GIS Data Improve Generality
Improve Accuracy
20
Summary of Topology Recovery Approach Clustering Approaches Approaches
Methods
Integrate Utility GIS Data
Integrate Public Map Data Public Map Data
Applications GPS Coordinates of All Utility Poles
DBSCAN
Filter Outlier AMI meters
Density based
Voltage Interval AMI Data
K-means
Determine connectivity AMI meters
Separate StreetWise Chunks
StreetWise Clustering
Improve Generality
Improve Accuracy
Transformers
Optimize within cluster sum of distances
BIRCH Hierarchical Clustering Fixed max cluster Diameter
21
Visual Summary of Topology Recovery Voltage Data Array Normalized and Weighting Criteria Applied Clustering Algorithms
Clustering of Meters to Transformer Using Clustering Algorithms (MtoT)
Filter Outlier Smart Meters DBSCAN
Determine connectivity of Smart Meters to Transformer
K-means
K- Nearest Neighbor (KNN) Clustering Algorithm Connecting Meters to Poles
Chow-Liu Algorithm Used to Identify Connection of Pole to Pole Using the Tree Span CF to Connect the Secondary Network
Chow-Liu Algorithm Used to Identify Connection of Pole to Pole Using the Tree Span CF to Connect the Primary Network
22
Observations • In topology recovery, researchers typically assumes that we know all the measurements at the exact boundary of each network and at the same voltage level but this is not true. • If a primary distribution feeder covers a thousand streets, processing all the data together is computationally expensive and could be erroneous. Therefore, it is more efficient to process a sub-network on a street belonging to the same circuit at a time. • To measure the data points in the sub-network, we can look into both the voltage signal space and geographical space.
• For voltage data, we can check the similarity. For geographical space, we can use Bing Map APIs to decode the latitude and longitude based on the house address and pole addresses.
23
Street-wise Meter Segmentation Approach • •
•
KML file generation for viewing the connections on Google Earth, etc. – Used FastKML library from GitHub For horizontal/vertical streets – longitude only horizontal streets (latitude does not change horizontal direction); latitude only vertical streets (longitude does not change vertical direction)
Street-wise smart meter segmentation – In order to prevent any far away meters with same voltage profile, we segmented the houses on the basis of street names
24
System Topology along Straight or Curved Streets •
Used street distance as a metric instead of latitude or longitude for clustering – Street distance can be used for any straight or curved street – Obtained through Bing maps APIs – Tested on a real street and results the same as longitude – 100% result on the curved Street
25
Example of Topology Identification Completed
Example of Topology Identification: Red: Primary Green: Secondary pole to pole Yellow: Service wires to AMI meters
26
Summary of Integrating AMI Data to Develop Processes to Ensure Accuracy of System Topology
Integrating AMI Data enables better planning with more granular data and enhances capabilities to inform operational tools as well
27
Conclusion • Current State: A current industry practice is to dispatch individual personnel to the field to log and record all assets by visually observing changes and entering the modification into a database. • Problem Statement: With the increased need to observe and manage distribution system, there is great opportunity to develop methodologies for utilities to identify their topology without doing a field verification as field verification is expensive. • Viable Solution: This body of work has utilized machine learning, mutual information, and external data extraction to incorporate utility data to identify an EDC’s grid that is not a simple grid block design but has mountainous terrain with sections of rural and urban areas which include overhead and underground facilities and is highly networked.
28