Environmental Monitoring Data – Current Challenges and the Future Analytics, Big Data & The Cloud Conference April 24, 2012 Ray Keller Data Management Monitoring and Science Division Alberta Environment and Water Ray.Keller@gov.ab.ca
Environmental Monitoring Data Introduction
The Story So Far – Current Challenges • • • • •
Background Monitoring Program – Business Drivers Data management challenges Data vs. information vs. knowledge, etc. Where is the data? What is the issue?
Implementing the Data Management Framework - Future • • •
Addressing the data and data management challenges Steps towards an integrated data management system Development of an Enterprise wide Data Management Strategy
Environmental Monitoring Data Background
Monitoring – Business Drivers • Persistent rapid pace of development • Getting closer to reaching environmental limits • Alberta is shifting to an outcomes-focused, cumulative effects management (CEMS) system • Increased public scrutiny and criticism • “Old” monitoring, evaluation and reporting system was built to meet different needs in a different time • Potential to use new technologies and approaches
Environmental Monitoring Data Background
Requirements for a new Monitoring, Evaluation and Reporting System •
Numerous monitoring reports released from Federal, Provincial and independent organizations highlighted the need for a holistic, more comprehensive and integrated environmental monitoring, evaluation and reporting system
•
All reports and recommendations indicated similar elements for the MER system: •Greater scientific rigor and oversight •Monitoring data and information must be more transparent and accessible to the broader stakeholder community in a coordinated, publicly accessible manner •The system must be broader and more comprehensive to support CEMS
Environmental Monitoring Data The Story So Far
Current Data Management Challenges • Data & information fragmented, stored in different formats & applications • Capacity to monitor, evaluate & report are at different development stages • Data quality & ownership is an issue due to: • A lack of common standards & policies • Unclear roles & accountabilities • There is a need to integrate environmental with social & economic data • There is a need to clarify accountability & roles regarding data
Environmental Monitoring Data The Story So Far
Current Data Management Challenges
• • • •
Discovery – cannot find the right data / information Integration – cannot manipulate & combine data / information Insight – cannot extract value & knowledge from data / information Dissemination – cannot provide data / information forKnowledge consumption & distribution • Management – cannot manage & control data / information volumes & growth • Quality – cannot be trusted / used decision-making negatively affected Data Info Data
Data Data
Application Application Application info
Info
Currently the data processes are application centric/information driven – oriented to operation & operations management
Environmental Monitoring Data Enterprise Data Management at AEW The Story So Far
Current Data Management Challenges
Lack of knowledge about or quality in data creates undesirable cycle The result is a self-perpetuating cycle of data production at an ever increasing rate because people: • Do not know what data exists • Do not understand the data so they won’t use it • Do not trust the data
May not be a problem if: • Decision-making scope is narrow / shallow • Issues resolution is within a small group or specific to a particular business unit (internal issues) or single site issue
Environmental Monitoring Data The Story So Far
Data Management Required
• Need to provide data to support decision-making at all levels • Need to expose all available data to all levels
Data
Traceability
Environmental Monitoring Data Paradigm Shifts related to Data Multiple shifts in paradigm have resulted in the need for greater integration of data and with public demand for more open and transparent data, it requires a different approach to how we manage data Change in business paradigm - Movement to Cumulative Effects Management • • • •
Single site to multi-dimension Consideration of environment with social and economic Integration across environmental media (effects) Integration across the policy landscape (policy/planning/management/operations)
Paradigm of asset management • Data is viewed as a strategic asset in a knowledge economy
Paradigm of Open Data • • • •
Credible and trusted source Social media Risk to credibility and reputation Move from “internal” to “public/open”
Environmental Monitoring Data The Solution
Adapt to a Data Stewardship Model
A New Way of Working DM BM / DM
IT
Binary model: No Independent Data Mgmt IT & Business frictions
DM deployment
BM
IT
Ternary model: Data Mgmt No IT & Business frictions
• Embracing Data Management means fundamentally changing the way that work is completed in the organization • Data Management function is owned by the data stewards • Data stewards work collaboratively & cooperatively with business & IT stewards to carry out data management activities
Environmental Monitoring Data The Solution
Modernize the DM Function
Environmental Monitoring Data The Solution
Requirements of an EDM Framework
1. Make it easy to discover / find data 2. Make it easy to integrate data from a variety of sources 3. Make it easy to extract value & knowledge from data 4. Make it easy to consume data 5. Manage & control data volumes & growth 6. Satisfy open data requirements (transparency & accountability) 7. Adapt to AEW organization with minimal impact 8. Improve the quality of the enterprise data 9. Manage data as a strategic asset 10. Make it easy to develop new data products in a timely fashion
Environmental Monitoring Data The Solution
Adopt EDM Model
Environmental Monitoring Data The Solution
EDM Mission & Goal
EDM Mission To support Alberta Environment & Water in becoming a trusted source for environmental data by managing & using data as a strategic asset & exploiting that asset in business decisions, processes & relations
EDM Goal To make AEW data easily accessible, trusted & secure and support informed planning, decision making & communications.
Environmental Monitoring Data Sharing Data Assets - Current
Band 3 - Unique
Assets implemented or acquired once for the GoA and shared by all • Corporate • Standardized • Centralized (planned and managed centrally)
Band 2 - Consistent Band 1
Shared Assets are used for distinct business purpose and not for the intent of sharing or integration Assets that are not necessarily widely shared across the GoA, but must still be integrated strategically across GoA in order to deliver strategic value
Environmental Monitoring Data Sharing Data Assets - Future
Band 3 - Unique
Band 2 - Consistent
Assets implemented or acquired once for the GoA and shared by all • Corporate • Standardized • Centralized (planned and managed centrally)
Band 1
Shared Assets are used for distinct business purpose and not for the intent of sharing or integration Assets that are not necessarily widely shared across the GoA, but must still be integrated strategically across GoA in order to deliver strategic value
Environmental Monitoring Data The Solution
Need to move to a data-centric view
• Develop & implement standardized data management functions • Determine, develop & deliver trusted datasets for stakeholder use .
Source: Data Management Association Body of Knowledge (DAMA-DMBOK)
Environmental Monitoring Data Data Analytics What does all this mean to data and data analytics for the environment? • Common data framework is required to: • Move data from the unique to the shared – concept of core data • Must be collaborative • Must deal with complexity of security and traceability – science • Data quality issues • Data needs to be available in the open domain • Large change in culture will be required • Scientific focus • Tension will be between being “open” and “exclusive”
Questions