2%
6%
11% 8% 14% 18%
6%
12%
25%
23% 22% 14% 10%
16%
30%
26%
27%
Research and insights on current industry challenges, standards, strategies, and best practices in the water monitoring and data management industry.
Published by
Table of Contents
Executive Summary Section 1: ProďŹ le Type of Organization Geographic Location Business Roles
4
4 6 6 7 7
SECTION 2: Network Design Needs Served Areas of Focus Station Density Size of Network Number of Variables Partnering & Co-operating
8 8 9 10 10 11 11
SECTION 3: Technology Primary Monitoring Technologies Communications Technologies Monitoring Techniques
12 12 13 13
SECTION 4: Quality Management Quality Management System Standard Operating Procedures
14 14 15
SECTION 5: Training & HR Number of Hydrographers Hydrographer Activities Unscheduled Field Trips Demographic Composition Gender Balance Average Salaries Salary Change Education Training
15 15 16 16 17 17 18 18 19 19
SECTION 6: Data Consumer & Stakeholder Expectations Data Dissemination Methods Real-Time Products & Services Data Production Process Version Control Granularity & Timeliness Analysis Metadata
20 20 21 21 22 22 23 23
SECTION 7: Data Management Data Management Challenges Data Management Systems Eectiveness Satisfaction Features Conclusions & Remarks Published by Aquatic Informatics
23 24 25 26 27 28 29 30
Global Report
The hydrological monitoring industry is undergoing rapid evolution worldwide. The last decade has seen a paradigm shift in environmental monitoring technologies, standards, and best practices. The industry transformation will continue over the next decade.
The changes are real, significant, and substantial.
2%
6%
The growing global population is placing unprecedented burden on water resources, needed not only for safe drinking water, but also to support our agriculture, energy, transportation, manufacturing, and civil infrastructure. Increasing climate uncertainty invalidates many historic assumptions creating a need for new information for water management and prediction. Regulations are growing in complexity. Water resource managers, hydrologists, and scientists are facing enormous pressure to meet the mounting expectations of stakeholders for high quality, real-time information.
11% 8% 14% 18%
In the fall of 2012, Aquatic Informatics conducted the “Global Hydrological Monitoring Industry Survey” to provide a better understanding of the current state of the industry and to offer further insights on how to address emerging challenges. This report quantifies the current trends and identifies the strategies being adopted in water monitoring and data management. It provides water resource professionals with the information that they need to benchmark their hydrological monitoring program against that of other organizations.
6% 44% 12%
25%
23% 22% 14% 10%
16%
30%
26%
27%
Industry Report
Executive Summary Aquatic Informatics collected responses online for the “Global Hydrological Monitoring Industry Survey” in September and October of 2012. Over 700 water program managers, hydrologists, engineers, scientists, and hydrographers participated in this international survey, representing hundreds of academic institutions, government agencies, national monitoring organizations, and engineering consulting firms. The results show increased demand for hydrological monitoring networks to serve multiple needs (including reference for climate change and inventory for water availability) and to serve multiple purposes (including water quantity and continuous water quality monitoring). Yet 72% of water professionals reported that they would need more monitoring stations to adequately meet all of their program goals. There is good news or perhaps optimism in the industry: respondents forecast their network size to grow by 53% over the next decade. The projected increase in the volume of available data will provide new information for expanding priorities. Data consumers today want real-time information. Meeting the growing information needs of stakeholders is also being addressed by the industry through the mass adoption of real-time monitoring and communications technologies. Digital multichannel data loggers and solid state electronic sensors are currently used by 71% and 67% of respondents, respectively. By 2022, automated sample collection and multi-parameter water quality sensors are expected to be used by 43% and 66% of respondents, respectively. With more continuous data being collected, water resource managers are turning to real-time communications technologies for data transmission. Respondents reported using digital data retrieval (e.g. logger files) for 63% of their stations in 2012. By 2022, approximately 40% of stations are forecasted to use web enabled sensors, satellite, and/or telephone to transmit water monitoring data.
66% use, or plan to start using, the U.S. Geological Survey (USGS) accepted standard operating procedure reference documents.
The rapid adoption of continuous monitoring and real-time communication technologies has resulted in large volumes of complex data. Naturally, water resource professionals today are concerned with ensuring data quality, defensibility, and interoperability. They are implementing clearly documented Quality Management Systems and adopting internationally accepted standards. More specifically, 62% of question respondents have adopted “clearly communicated objective(s) for data quality.” 66% of respondents use, or plan to start using, the U.S. Geological Survey (USGS) accepted standard operating procedure reference documents. Water managers want to produce data that they can trust. Consumers of hydrological data today also have high expectations – they want immediate access to continuous information that is quality controlled. Tolerance for data faults in dynamic data has declined. Demand for metadata and higher level analysis has increased. Quality controlled data is expected to be published sooner. Results show that by 2022, the most popular data publishing methods are forecasted to be Web 2.0 (e.g. dynamic content), Web services (e.g. SOAP, REST, WaterML, XML), and mobile device dissemination (e.g. iPhone), adopted by 59%, 52%, and 58% of water professionals, respectively. How are water resource professionals managing the growing volumes of hydrological data to meet stakeholder demands for real-time information? They depend on their data management systems to capture, store, correct, analyze, visualize, and report on vast volumes of environmental data from disparate sources. But the performance of data management systems varies. Office Software like Microsoft® Excel is still the most commonly used tool, used by 35% of respondents as their primary data management system. Other data management systems in use include custom solutions and integrated environmental systems. Additionally, 23% of
4
Industry Report
Up to 32% more respondents with actively licensed commercial software reported satisfaction with their data management system.
respondents reported using actively licensed commercial software. These respondents reported being better able to meet evolving stakeholder expectations for real-time data products and services, metadata availability, higher level analysis, and timely reporting and publishing. Up to 32% more respondents with actively licensed commercial software reported satisfaction with their data management system in areas such as system responsiveness to emerging technologies, performance, reliability, data security, and breadth of features. Competing program goals; limited network resources; the rapid adoption of real-time technologies; massive and growing volumes of data; increased data complexity; mounting data consumer expectations … there is no shortage of challenges. Water resource managers are being proactive in meeting these challenges. The “Global Hydrological Monitoring Industry Survey” has brought to light trends that promise a bright future. New technologies and data management solutions are being implemented so that real-time, high-quality data is more readily available to make important and timely decisions about precious water resources. Quality Management Systems and internationally accepted standards are being adopted to improve data quality and interoperability. There is more good news. Survey results show that water resource managers forecast the size of their teams to grow by 23% more hydrographers. Salaries have increased in the past decade, which will help attract more hydrologists and hydrographers to the profession. More women are joining the field – they are forecast to make up 34% of the workforce by 2022. And education levels are on the rise – 74% of the workforce is reported to currently hold a bachelor’s degree or higher level education. A common theme across environmental monitoring agencies is that hydrometric networks are being updated in response to both rapidly evolving end-user expectations and emergent technological opportunity. This finding is not unexpected. What is surprising is how diverse these adaptations have been. There is a lot of experimentation as each agency endeavors to solve many and various challenges with whatever opportunities are most readily available. Widespread convergence on the most successful solutions to common problems is forecast for the coming decade. The detailed findings - described in the following sections - can help you benchmark your organization’s progress and guide planning for future investments in a successful, modern network.
2
5
Industry Report
Section 1. Profile The “Global Hydrological Monitoring Industry Survey” was distributed to public and private organizations globally. It was promoted by Aquatic Informatics, the International Association of Hydrological Sciences (IAHS), the World Meteorological Organization (WMO), and the American Water Resources Association (AWRA). As a result of the broad distribution of the survey, respondents represent a variety of organizations, global geographic regions, and business roles.
The over 700 respondents represent a variety of organizations, global geographic regions, and business roles.
Type of Organization Question participants represented various types of environmental monitoring organizations, including: academic institutions (23%), national government agencies (19%), state and provincial governments (19%), consulting engineering firms (15%), and regional/municipal governments (14%).
23%
Academic Institution
3% 1% 5%
Environmental NGO Government – International Government – Municipal
19%
Government – National (e.g. USGS)
9%
Government – Regional
19%
Government – State/Provincial
1%
Government – Tribal (First Nation) Private Sector – Consulting Engineering
2% 1% 4%
Private Sector – Hydropower Private Sector – Mining / Oil / Gas Other 0
Which of the following best describes your organization?
6
50
15%
100
150
200
Industry Report
Geographic Location The survey results represent environmental monitoring organizations with networks located in 90 countries around the world, including a high representation from the United States (42%) and Canada (15%). Six continents are represented: North America (60%), Asia (12%), Europe (11%), Africa (8%), South America (5%), and Australia and the Oceania Region (5%). South America
Survey results represent environmental monitoring organizations with networks located in 90 countries.
5%
Africa
8%
Asia 12%
5% North America
60%
11%
Australia & Oceania Region
Europe
In what country is your environmental monitoring network located?
Business Roles The majority of participants ďŹ t the role of scientist / researcher (34%), engineer / hydrologists (31%), program manager (13%), ďŹ eld operations (9%), and consulting engineer / contractor (6%).
6% 2%
Consulting engineer / contractor Data consumer
31%
Engineer/hydrologist
9% 13%
Field operations Program manager
34%
Scientist/researcher
5%
Other 0
Which of the following best describes your role?
7
50
100
150
200
250
Industry Report
SECTION 2: Network Design Reference (climate/land-use change)
Scientific investigation
Water treaty obligations
Real-time risk management (e.g. flood forecasting)
2%
59%
18% 5%
13%
3%
50%
23% 8%
15% 51%
25% 10% 10%
57%
25%
4% 6% 5%
Stormwater or urban runoff
And yet, water professionals report insufficient network density to meet all of their program purposes. This is perhaps why a growing majority of organizations are cooperating or co-managing at least some component of their monitoring network with other agencies. The good news is that survey respondents remain optimistic about the growth of their monitoring network. The average number of stations per network is forecasted to grow by 53% over the next decade, and even more drastic growth is expected in the number of combined geophysical and station health variables measured per station. While added resources will improve the ability of water resource managers to address the challenges of managing their network design, it will also contribute to the growing complexity and volume of data used to manage important water resources.
45%
10% 14%
2%
Planning / policy
27% 27%
2%
Watershed stewardship
Other
17%
1%
Real-time operations (e.g. reservoir mgmt, fish flow releases)
Needs Served 50%
21% 9%
5% 1%
15%
11% 12%
27% 600
500
450
400
300
350
200
250
150
100
50
0
Increased Stable Decreased Don't Know Not Applicable
25% 25%
2%
Water license & regulatory compliance
57%
24%
4% 5% 6%
The business of monitoring water is complex and growing in complexity. Over the past decade there has been a significant increase in the demand for hydrological monitoring networks to serve multiple needs, including scientific investigation, reference for climate/land-use change, planning and policy, and inventory for water availability. Hydrological monitoring programs are also serving multiple purposes, including water quantity monitoring, continuous water quality management, meteorological monitoring, and groundwater management. Allocating resources to these areas of focus is a challenge with increasing pressures to provide better data to meet all program priorities.
67%
24%
3% 3% 2%
Inventory of water availability
68%
20%
1% 3% 5%
Have the needs served by your hydrological monitoring network (or client networks) changed? Please select all the applicable needs served by your organization below by indicating any change in their demand over the past decade (since 2002)?
8
Hydrological monitoring networks are designed to serve a multitude of needs – 99% of respondents indicated that their network serves more than one need and over 115 participants selected “other” for this question. Some of the most common needs served include: scientific investigation (by 94% of question respondents), reference for climate/land-use change (90%), planning / policy (86%), and inventory of water availability (85%). It is also clear from the responses that the needs served by hydrological monitoring networks have increased in demand over the past decade.
Industry Report
Areas of Focus
92% of question participants rated “water quantity” as either a primary (70%) or secondary (22%) area of focus.
Hydrological monitoring is a multipurpose activity – water resource professionals identified an average of 4.2 primary and/or secondary areas of focus for their monitoring network. It is clear that “water quantity” rated highest in popularity, applicable to 92% of question participants as either a primary (70%) or secondary (22%) area of focus. Nearly 90 participants selected “other” in this question, noting areas of focus such as sediment transport, stable isotopes, and stormwater management. Note: Many people identified multiple areas of focus as primary. Water quantity
5
Continuous water quality monitoring
44% 41%
12% 30%
Discrete water quality
43%
30%
23%
Meteorological monitoring Other
41%
26%
Groundwater
40% 43%
14% 8%
4%
27% 550
500
450
400
300
350
200
250
150
100
50
0
Primary Secondary Not Applicable
70%
22%
6%
Have the areas of focus of your monitoring network (or client networks) changed? Please indicate your primary area of focus and all secondary areas of interest.
Since 2002, there has generally been an “increase” in the resources dedicated to the following program purposes: water quantity (50%), continuous water quality (47%), meteorological monitoring (44%), groundwater (39%), and discrete water quality (33%). Water quantity 5% Continuous water quality monitoring
7%
Groundwater
7%
Meteorological monitoring 1%
6%
29%
33%
14% 39%
25% 13%
44%
30% 8% 8% 16%
Please indicate how the proportion of total resources dedicated to that program has changed in the past decade (since 2002).
9
400
300
350
200
250
150
100
50
0
Increased Other Stable Decreased Don't Know
5%
47%
28%
7% 8%
Discrete water quality
50%
30%
9%
Industry Report
Station Density
72% of question
respondents indicated that they would need to increase the density of their network to fully meet their program goals.
Water professionals who responded to this question feel that their current monitoring programs have an insufficient number of monitoring stations to adequately meet all of their program goals. Indeed, most (72%) indicated that they would need to increase the density of their network to fully meet their program goals: 22% by more than 100%, 21% by 50 to 100%, and 29% by up to 50%.
22% 21%
Increase by more than 100% (i.e. more than double the network density) Increase by 50% to 100%
29%
Increase by up to 50% No change – the monitoring network is sufficiently dense
16% 3%
Could be decreased without compromising end user requirements
10%
Don't Know
0
50
100
150
200
250
In your opinion, how would the density of your network’s monitoring stations need to change in order to fully meet your program purposes?
Size of Network There is good news: the size of networks is on the rise. With an average of 269 monitoring stations per network today, respondents reported an 18% increase in network size since 2002. Participants are optimistic about the growth of their network, forecasting a further 53% increase in their network over the next decade.
228 269
2002 (PAST): # of stations 2012 (PRESENT): # of stations
412
2022 (FORECAST): # of stations 0
90
180
270
360
450
Is the size of your network (or client networks) changing over time? Please indicate the number (#) of monitoring stations that your organization managed in 2002, 2012, and 2022, including stations that you cooperated on or co-managed.
10
Industry Report
Number of Variables The average number of variables monitored per station has also grown since 2002 by approximately 29%, and the volume of data is expected to grow at a more rapid rate over the next decade. (Note: specific averages are not provided in this report because a number of participants appeared to provide a “total” rather than “average” number of stations, skewing the data.) 2002 (PAST): # of geophysical variables 2002 (PAST): # of station health variables 2012 (PRESENT): # of geophysical variables 2012 (PRESENT): # of station health variables 2022 (FORECAST): # of geophysical variables 2022 (FORECAST): # of station health variables
19
9 12
25 70
37 0
20
40
60
80
For an average station, how many geophysical variables and station health variables are you monitoring? Please indicate the number of variables monitored in 2002, 2012, and 2022.
Partnering & Co-operating
66% of organizations represented in the survey co-operate or co-manage at least a component of their monitoring network with other agencies.
The majority of organizations represented in the survey co-operate or co-manage at least a component of their monitoring network with other agencies, perhaps as a cost saving measure and to increase the volume of data they need to meet their program goals. The percentage of respondents who co-operate with other organizations has grown from 54% in 2002 to 66% today, and this is expected to remain fairly stable into 2022. Co-operating organizations specified, in order of popularity, included the USGS, state/provincial governments, Water Survey Canada, the US Bureau of Reclamation, and many others.
54%
Co-operated or co-managed in 2002
34% 10%
66%
Co-operating or co-managing in 2012
27% 6%
62%
Will co-operate or co-manage in 2022
23% 450
360
270
180
90
0
Yes No Don’t Know
13%
Does your organization co-operate or co-manage any component of its monitoring network with other agencies / organizations? Is this changing over time?
11
Industry Report
SECTION 3: Technology Staff Gauges
48%
Analog paper charts
35%
19%
8%
37%
Digital multi channel data loggers
66%
Mechanical sensors
26%
6%
Non-contact sensors
26%
Mechanical current meter
67% 64%
41% 42%
15%
Primary Monitoring Technologies 47%
The decline in the use of analog paper charts comes as no surprise – by 2022 only 8% of respondents expect analog paper charts to be significantly used in their network. The exciting trend is the rapid rate at which the industry is mass adopting technologies for continuous monitoring.
52%
11% 15% 19%
Dilution gauging Paper field forms
51% 49%
30% 23%
Handheld field computers
59% 49%
Automated sample collection
19%
Multiparameter water quality
37%
Other
65%
43%
26%
4%
On the water quantity side, there has been major growth in the use of digital multichannel data loggers, from 37% of respondents in 2002 to 71% today. The use of solid state electronic sensors has also grown significantly from 34% of respondents ten years ago to 67% today. There is a similar trend for the use of acoustic measurement devices from 15% a decade ago to 47% today, and usage is forecasted to grow to 52% by 2022.
60% 59%
Manual bottle samples
61%
66%
8% 9% 450
360
270
180
90
0
Are the primary technologies used in your network changing? Please select all the technologies significantly used (i.e. making up 10% or more of your network) in 2002, 2012, and 2022.
12
As previously noted, results from the “Global Hydrological Monitoring Industry Survey” show that water resource professionals expect to manage 53% more monitoring stations by 2022 and that the average number of geophysical and station health variables monitored per station is on the rise. But these are not the only factors contributing to the drastic growth in the quantity and complexity of water data being managed. The trends are revealing mass adoption of continuous monitoring and real-time communication technologies, resulting in incredible volumes of data. Today’s water resource managers are collecting, storing, managing, analyzing, and publishing more continuous hydrological data than ever before.
37%
28%
Acoustic measurement devices
71%
39% 37% 34%
Solid state electronic sensors
2002 2012 2022
59% 60%
On the water quality side, the use of manual bottle sampling is reaching its peak – while 65% of respondents report using manual bottle sampling today, only 49% expect to use it significantly by 2022. The trends are towards the adoption of automated sample collection and multi-parameter water quality sensors used respectively by 37% and 61% of respondents today, with growth expected to continue to 43% and 66% by 2022.
Industry Report
Communications Technologies By 2022, approximately 40% of stations are expected to use web enabled sensors, satellite, and telephone to transmit water monitoring data.
The use of analog data retrieval is on the decline – while used for 38% of stations in 2002, this number is expected to drop to 22% by 2022. The most popular communications technology used today is digital data retrieval (e.g. logger files) used for 63% of stations, an increase of 18% since 2002. The most drastic trend is the adoption of web enabled sensors (e.g. Internet Protocol Communications) – while only used for 4% of stations in 2002, they are used for 20% of stations today, and are expected to double in usage by 2022. By 2022, approximately 40% of stations will each use web enabled sensors, satellite, and telephone to transmit water monitoring data. An interesting note: the results show that redundant technologies are being used as a best practice to guarantee timely availability of critical water data. Analog data retrieval 22%
38%
26%
46%
Digital data retrieval 16% 20%
Radio
16%
Satellite 2002 2012 2022
4%
Web enabled sensors
33%
27% 20%
0%
68%
28% 24%
Telephone
63%
40% 41% 40%
20%
40%
60%
80%
Are your data communications technologies changing? Please select the approximate percentage (%) of stations using the following technologies in 2002, 2012, and 2022. Sums can exceed 100% due to redundant communications technologies used at a single station.
Monitoring Techniques The stage discharge rating curve is the most common water monitoring technique, currently used by 89% of professionals who responded to this question. This technique may be reaching its height in popularity and applicability – while still the most common technique, only 75% of participants expect to be using it by 2022. This can perhaps be explained by the rise in use of the index velocity model. While only 15% of respondents used index velocity modeling in 2002, 31% use it today, and 36% forecast using it by 2022. The second most popular technique is control structures, currently used by 56% of survey participants. 70%
Stage discharge rating curves 41%
Control structures 15%
Index velocity 2002 2012 2022
13
89%
75%
3% 6% 7%
Other 0
100
56% 54%
31% 36%
200
300
400
500
Are your hydrometric monitoring techniques changing? Please select all techniques significantly used (at least 5% of the time) in 2002, 2012, and 2022 (forecast).
Industry Report
SECTION 4: Quality Management With larger volumes of data to manage than ever before, today’s water resource professionals are concerned with ensuring data quality and interoperability. They want to use the right and most current hydrological science and principles. They want to produce credible and defensible data. They want to compare their data to those of their peers. Most importantly, they want to produce data that they can trust. This is why today’s water managers are implementing clearly documented Quality Management Systems and adopting internationally accepted standard operating procedures.
Quality Management System
62% of question
respondents have implemented “clearly communicated objective(s) for data quality.”
The most highly adopted component of a Quality Management System today is “clearly communicated objective(s) for data quality” – currently implemented by 62% of respondents to this question, with an additional 19% reporting plans for implementation by 2022. “Clearly communicated objective(s) for service delivery” and “data security” are also currently implemented by 49% and 42% of respondents, respectively. These numbers are forecasted to jump by over 20% in the next decade. Clearly communicated objective(s) for data quality
8%
Clearly communicated objective(s) for service delivery
11%
12%
Clearly communicated objective(s) for data security
16%
14%
Audit of SOP compliance
17%
End-user feedback for measurement of QMS objectives
49%
21%
16% 17%
Verification of data quality objective(s) using SOP
42%
22%
38%
23% 22% 23% 21%
18% 15%
34% 31%
19% 17%
28% 39% 23% 350
260
210
140
70
0
Continuous improvement Currently Implemented based on feedback Plan to Implement No Plans to Implement Don't Know
62%
19%
Please indicate your organization's plans for the implementation of the following components of a Quality Management System (QMS).
14
Industry Report
Standard Operating Procedures
66% use, or plan to start using, the U.S. Geological Survey (USGS) accepted standard operating procedure reference documents.
Most water resource professionals surveyed also use or plan to start using one or more sets of internationally accepted standard operating procedure reference documents. The most popular source is the U.S. Geological Survey (USGS), being adopted by 66% of respondents. Water professionals are also turning to the World Meteorological Organization (WMO) and International Standards Organization (ISO), selected by 43% and 39% of participants, respectively. This is good news for the industry – water professionals are working towards a better future with greater data quality, consistency, and interoperability.
U.S. Geological Survey (USGS)
66%
39% 43%
International Standards Organization (ISO) World Meteorological Organization (WMO)
13%
Other 0
50
100
150
250
200
300
350
If you use, or plan to start using, any internationally accepted standard operating procedure reference documents please specify which ones.
SECTION 5: Training & HR While the average number of hydrographers per organization has not changed since 2002, the demographic composition of the workforce has. Women are making up a larger portion of the workforce – they are expected to account for one third of hydrologists and hydrographers by 2022. Survey results also show that the workforce is more educated, with nearly three quarters holding a bachelor’s degree or higher level education. Perhaps the most interesting trend for hydrologists and hydrographers, is the growth in salaries: 49% of respondents reported an increase or significant increase over the past decade. Results for the United States show that hydrographers in the middle of their career currently make an average annual salary of $63,725 USD.
Number of Hydrographers While the complexity of hydrological program management has grown, the teams responsible for monitoring water resources have not. Respondents reported having approximately 24 hydrographers in their organization in 2002 as well as today. It is feasible that the mounting pressure on today’s hydrographers to do more with limited resources is leading to optimism in the growth of teams, which are expected to grow by an average of 6 hydrographers over the next decade. 2002: # of hydrographers
24
2012: # of hydrographers
24
2022: # of hydrographers
30 0
5
10
15
20
25
30
Is the number of stream hydrographers in your organization changing? Please indicate the number (#) of stream hydrographers in your organization in 2002, 2012, and 2022 (forecast).
15
Industry Report
Hydrographer Activities “Field work” and “data processing & management” have and will continue to take up the majority of the stream hydrographer’s schedule. What is interesting is that over the past decade slightly less time is being dedicated to field work and more time is being dedicated to data processing, a trend that will continue over the next decade. This trend makes sense as new technologies require fewer scheduled field visits, while the growth in the volume of data they produce means more time is needed for data management.
39% 38%
Field work 37% Data processing & management
30%
13%
Hydrology training & professional development
11%
General training (e.g. health & safety)
17%
Administration, communication, & other
10%
0%
39%
20%
15% 18% 16%
2002 2012 2022
35%
21% 23% 30%
20%
50%
40%
Are the activities of the stream hydrographer(s) in your organization changing? Please select the approximate percentage (%) of time dedicated to various responsibilities in 2002, 2012, and 2022 (forecast). The totals should sum to 100%.
Unscheduled Field Trips From 2002 to 2022, the percentage of field trips that are unscheduled is expected to increase moderately from 38% to 42%.
Almost 60% of field visits today are scheduled. From 2002 to 2022, the percentage of field trips that are unscheduled is expected to increase moderately from 38% to 42%, with the growth coming from unscheduled survey field visits in response to hydrological events. Unscheduled maintenance field visits are expected to remain stable.
Unscheduled survey field visits (e.g. event response)
2002 2012 2022
19%
Unscheduled maintenance field visits (e.g. emergency repairs)
22% 23%
19% 19% 0%
5%
10%
15%
20%
20%
25%
Is the percentage of field trips that are unscheduled changing over time? Please indicate the percentage (%) of total field trips unscheduled in 2002, 2012, and 2022 (forecast).
16
Industry Report
Demographic Composition Recent hires are a growing demographic. While they made up 30% of hydrographer teams in 2002, they are expected to make up 38% by 2022. This means hydrology training and mentorship will continue to be important components of building successful teams.
30%
Recent hires (0 to 10 years)
35% 38%
35%
By 2022, female hydrologists are forecast to make up 34% of the workforce.
Mid-career
36% 40%
26%
Eligible for retirement within 10 years
29% 24%
2002 2012 2022
10%
0%
30%
20%
40%
Is the demographic composition of your hydrologist / hydrographer workforce changing? Please select the approximate percentage (%) of your hydrologists / hydrographers that are recent hires, mid-career, and retiring within the next 10 years. The totals should sum to 100% for each 2002, 2012, and 2022 (forecast).
Gender Balance While historically a more male dominated profession, hydrology is attracting a growing number of females. Women accounted for only 16% of hydrographers in 2002, as reported by survey respondents. This number is forecasted to more than double to 34% by 2022.
16% Percentage of Women 2002 2012 2022
24% 34% 0%
5%
10%
15%
20%
25%
Please indicate the gender balance of your hydrologist / hydrographer workforce?
17
30%
35%
40%
Industry Report
Average Salaries In the United States, the average salary for recent hires (with 0 to 10 years of experience) is US$47,767 according to question respondents.
Salaries are a sensitive topic – only 200 survey participants completed this question. Salaries reported varied widely internationally, making it challenging to provide representative data outside of Canada and the Unites States. The average annual salaries for these countries are reported here in U.S. dollars (the exchange rate used was provided by the Bank of Canada on September 28, 2012). For the United States, respondents reported that recent hires (0 to 10 years) earn an average salary of $47,767 while those eligible for retirement within 10 years earn $77,373 annually.
$54,106 $47,767 $70,631 $63,725 $84,134 $77,373
Recent hires (0 to 10 years)
Mid-career Eligible for retirement within 10 years 10K
20K
30K
40K
50K
60K
70K
80K
90K
100K
110K
Recent Hires (0 to 10 yrs)
Mid-Career
Retirement within 10 yrs
Aswered Question
Canada
$54,105.71
$70,631.33
$84,133.82
28
United States
$47,766.80
$63,724.64
$77,372.97
88
What are the average salaries for your hydrologist / hydrographer workforce? Please indicate the average salary of your recent hires, mid-career, and retiring hydrologists / hydrographers, including the correct currency. Please skip this question if you cannot make a reliable estimate of the average salary for at least one category. Please indicate the currency for the salary information provided above.
Salary Change Organizations today must pay more to compete for qualified hydrographers. 49% of survey respondents indicated that hydrologist salaries have increased or increased significantly over the past decade, while only 13% reported a decrease or significant decrease. Increased Significantly
Don’t Know 6%
8%
Decreased Significantly 3%
Decreased
10%
Increased
43% 30% Stable
How have your organization’s salaries for hydrologists / hydrographers changed relative to 10 years ago (accounting for inflation)?
18
Industry Report
Education Results show that 74% of hydrologists and hydrographers in the workforce today hold a bachelor’s degree or higher level education.
The workforce for hydrologists and hydrographers is becoming more educated. Today, 74% hold a bachelor’s degree or higher level education, as reported by survey respondents. This number can be compared to 62% in 2002 and is forecasted to grow to 82% by 2022. This higher education is likely a contributing factor in the increase of salaries for hydrologists and hydrographers.
62% Percent with bachelor’s degree or higher
74%
82% 2002 2012 2022
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Are the education levels of your hydrologist / hydrographer workforce changing? Please indicate the approximate percentage (%) with a bachelor’s degree or higher in 2002, 2012, and 2022 (forecast).
Training In-house training is the most popular source of hydrology training used by 41% of organizations surveyed, followed by academic courses (24%), and self-taught options like Internet research (24%). Survey participants reported their general satisfaction the training options. Depending on the training source, only between 18% and 21% said that they were not satisfied, indicating some room for improvement. Vendor training
16% In-house training
24%
41% 24%
Academic courses
14% Training from other monitoring agencies
Self-taught What are all the sources of hydrology training used by your organization? Please indicate the approximate percentage (%) of training delivered via the following methods / sources. The total should add to 100%. Are you satisfied with the training resources?
19
Industry Report
SECTION 6: Data Consumer & Stakeholder Expectations Data consumer and stakeholder expectations are mounting. Today, end-users expect immediate access to continuous environmental information that is accurate. They want access to dynamic hydrological data via the web and mobile devices, and more, they want access to web services. Stakeholders have lower tolerance for data faults in dynamic data. They are also less accepting of missing, estimated, and degraded quality data. They expect published data to be auditable, supported by version control and traceability to source. Quality controlled data must be available sooner. In fact, today’s data consumers expect more than high quality real-time data. Their demand for metadata and supplementary data has also increased significantly over the past decade. And they expect higher level analysis and interpretation. Survey respondents report an increase in the importance of modeling, geospatial analysis, advanced statistics, trending, frequency analysis, specialized plotting, synthesis reports, and retrospective analysis. The evolving expectations of data end-users create no shortage of challenges for today’s time- and resource-limited water resource professionals.
Data Dissemination Methods
2012, Web 2.0, Web
services, and mobile publishing are used by 46%, 34%, and 28% of question respondents.
Once amongst the most popular data dissemination methods, expensive print and telephone communications are on the decline – only 35% and 24% of respondents expect to be using these methods by 2022, respectively. The trend shows the rapid adoption of communication methods that support real-time data publication. Web 2.0, Web services, and mobile publishing are used by 46%, 34%, and 28% of survey respondents today, representing growth rates ranging between 500 to 900% since 2002. These three real-time communications methods are forecasted to be the most popular by 2022.
Hard copy paper publications Telephone / storefront human-mediated exchange
35%
24%
40% 50%
Physical digital media 27%
53%
38% 7%
Web 2.0 (e.g. dynamic content)
46%
7%
Web services (e.g. SOAP, REST, WaterML, XML, etc.)
34%
3%
Data dissemination via mobile devices 2002 Other 2012 2022
66%
47%
Web1.0 (e.g. static HTML)
1% 0
59% 52%
28%
5%
69%
57%
35%
58%
10%
50
100
150
200
250
300
350
Have your organization’s primary data dissemination methods and technologies changed? Please select all the communication methods and technologies significantly used in 2002, 2012, and 2022 (forecast).
20
Industry Report
Real-Time Products & Services Today, stakeholders and end-users are generally less tolerant of data faults related to real-time products and services. Their acceptance of spurious real-time data errors is more likely to have decreased (25%) than increased (16%). The same trend is true for the acceptance of systematic real-time data errors, which is more likely to have decreased (29%) than increased (13%). The expectations of data end-users have evolved over the past decade. They want real-time data they can trust, requiring today’s water resource managers to automate the QA/QC of their real-time data.
24%
Real-time data communications fault (i.e. missing data)
14%
10%
16%
Spurious real-time data errors (e.g. spikes, at lines)
29%
25%
17%
10% 13%
27% 19%
11%
29%
120
90
60
30
0
Systematic Increased real-time data Stable errors (e.g. sensor Decreased mis-calibration) Don't Know Not Applicable
26% 27%
Has end-user dependence on real-time products and services changed? Please indicate how end-user acceptance for data faults in real time data has changed over the past decade (since 2002).
Data Production Process
Participants reported an increase in the number of complaints for missing data (33%), estimated periods (22%) , and degraded data (23%) since 2002 .
Data consumers are generally less tolerant today of missing, estimated, and degraded quality data than they were 10 year ago. More survey participants reported an increase in the number of queries or complaints for missing data (33%), estimated periods (22%), and degraded data (23%), than a decrease (19%, 12%, and 19% respectively).
Missing data
22% 12%
23% 19% 7%
33%
21%
9%
Degraded quality data (e.g. poor rating curve)
27%
22%
150
120
90
60
30
0
Increased Stable Decreased Don't Know Not Applicable
16%
6%
Estimated periods e.g. reconstructed from surrogates)
33%
26%
19%
Have end-users become more sophisticated in their understanding of the data production process? Please indicate if the number of queries or complaints about missing / estimated / degraded quality in the ďŹ nal data product has changed in the past decade (since 2002).
21
Industry Report
Version Control A more drastic change in data end-user expectations pertains to version control and source traceability. Most (or 56% of) survey respondents reported that the importance of “communicating the data version to the public” has increased over the past decade, with only 3% reporting a decrease. 49% indicated that the importance of “authenticating exactly who made changes to the data, when, how, and why” has increased, while only 2% reported a decrease. Users want published data they can trust, and data auditability builds that trust. Ability to say exactly what data was publicly accessible
3%
Ability to communicate the data version to the public
56%
25%
3% 5%
Ability to authenticate data changes
11% 49%
27%
2%
15%
5%
44%
25%
3%
18%
8%
120
90
60
30
0
Increased Ability to Stable differentiate & Decreased verify data path Don't Know Not Applicable
12%
8%
49%
27%
Have expectations for version control and traceability to source for published data changed? Please indicate how the importance of the following aspects of data auditability has changed over the past decade.
Granularity & Timeliness
By 2022 , 46% of respondents expect to report daily means and unit values dynamically (in near real-time).
Quality controlled data is available sooner today than it was ten years ago, and this trend will continue into 2022. Ten years ago, 39% of respondents published their daily means annually and 36% published unit values annually. By 2022, 46% of respondents expect to report daily means dynamically and 46% forecast publishing unit values dynamically.
2002 Daily means
6%
2002 Unit values
6%
9%
17% 36%
15% 11%
17%
13% 14%
2022 Daily means
10% 9% 10% 11% 8% 10%
46%
46% 120
90
60
30
0
Annual 2022 Unit values Quarterly Weekly Dynamic Not Applicable
39%
18%
Are expectations changing with respect to the granularity and timeliness of quality controlled data? Please indicate the duration of the complete data production cycle (e.g. for final published, archival quality, products or reports) past, present, and forecast. The ‘dynamic’ choice represents continuous data production using web services.
22
Industry Report
Analysis Specialized plotting (e.g. double mass curves, piper plots) Advanced statistics, trend / frequency analyses Geospatial analysis (e.g. collective analyses by bio-geo-physical classification)
4%
6% 6%
3% 4%
9%
350
280
210
140
SECTION 7: Data Management
43%
9% 6% 35% 35%
4% 9%
Standards reference documents
39% 42%
35%
2% 2%
13% 37%
50%
6% 300
240
180
120
60
0
Have expectations for higher level analysis and interpretation of the data changed? Have expectations for metadata and supplementary data access changed? Please indicate how the importance of the following services has changed over the past decade (since 2002).
23
Over the past decade, the demand for metadata and supplementary data has certainly increased. Most water professionals who responded to this question reported an increase in demand for site location and description (63%), station operation information and gauge history (53%), station equipment information (53%), and standards reference documents and training materials (50%). So in addition to managing growing volumes of data, water resource professionals are expected to manage increasing volumes of metadata. The use of effective modern data management systems is more critical than ever before.
53%
36%
3%
Unpublished observational data Increased Stable Decreased Don't Know Not Applicable
6%
6% 7% 3%
Rating curves
53%
35%
3% 5% 1%
Field visit notes
63%
32%
1% 3% 1% 2%
Metadata
83%
9% 1% 4% 2%
2% Station operation information/ gauge history
52%
30%
70
Station equipment information
76%
11%
1%
The demand for higher level analysis and interpretations has clearly increased since 2002. The importance of “modeling” has increased according to 83% of respondents. The importance of “geospatial analysis” and “advanced statistics, trend, and frequency analysis” has increased as reported by 76% and 72% of participants, respectively. Today’s data consumers want more than just high quality data, they also expect higher level analysis, including specialized plotting, synthesis reports, and retrospective analysis.
72%
18%
0 Site location & description
42%
13% 10%
2% 5% 3%
Synthesis reports / retrospective analysis / case studies Modeling (e.g. hydrologic / hydraulic / water quality)
29%
In today’s economy, water resource managers are expected to do more with less. The collections of hydrological data sets have become immensely large and complex, resulting from the rapid adoption of real-time technologies. To add to these challenges, data consumers want dynamic access to quality controlled data and higher level analysis. Water resource managers must rely on their data management systems to capture, store, correct, analyze, visualize, and report on vast volumes of disparate environmental data sources. They are dependent on their data management systems to publish hydrological data efficiently, accurately, and defensibly.
Industry Report
Specialized hydrologic data management systems are commercially available to meet the evolving needs of hydrologists and to support current industry standards for water information management. Results from this survey show that actively licenced commercial software is used by 23% of water professionals. These respondents reported being able to better meet the evolving expectations of data consumers for faster reporting and publishing, the availability of metadata, and the delivery of real-time products and services. These water resource professionals also reported higher satisfaction rates with their commercial data management system, in all areas including system responsiveness to emerging technologies, performance, reliability, data security, and breadth of features.
Data Management Challenges
Many organizations
do not have enough time,
qualified staff, nor the right data management tools to publish massive volumes of continuous data as quickly as expected.
Survey respondents were asked to describe their water data management challenges in an open-ended question. Nearly 250 water resource professionals shared their challenges in their own words. The most common responses can be categorized as follows. Data Volumes. Many organizations are collecting more hydrological data than is being processed – they are data rich, but information poor. These organizations simply do not have enough time, qualified staff, nor the right data management tools to centralize, quality control, analyze, and publish massive volumes of continuous data to support the real-time information demands of stakeholders and end-users. Lack of Funding/Staff/Training. Funding is limited. The number of qualified staff available to manage complex datasets is limited: “there is too much data and not enough staff.” Organizations are finding it challenging to attract, train, and retain qualified staff to consistently process hydrometric data, even with standard operating procedures in place. Data Quality Control/Standards/Validation. Many water resource managers are finding it impossible to turn continuous data into real-time quality controlled information. They are finding it hard to implement standard operating procedures that support changing international standards. Staff turnover and the geographic dispersion of teams are making change management difficult. They are also finding it challenging to automate quality assurance and quality control (QA/QC) without the right tools in place. Data Consolidation/Data Management System. Many water resource managers are ill-equipped to centrally manage of their water data. They are expected to store data in different locations, using disconnected software tools. Their legacy systems cannot be updated to meet current needs and data volumes. Historical data is difficult to access. Data is stored in various measures/units. These water resource managers need a customizable modern data management platform to support their real-time water data processing needs. Data Dissemination. Respondents shared their challenges in publishing timely data to meet internal organization and external public demands. It is simply taking too long to process continuous data, and the reporting and publishing tools are either inflexible or not available to them.
24
Industry Report
Data Management Systems
28% of respondents
use a commercial hydrological data management system, either actively licensed (23%) or unsupported (5%).
Office Software (e.g. Microsoft Excel) is still the most commonly used tool to manage hydrological data – used by 35% of respondents as their primary data management system. 34% reported the use of a custom solution (24% build in-house and 10% build by a contractor) and 28% use a commercial hydrological data management system, either actively licensed (23%) or unsupported (5%). Note: the USGS recently announced the retirement of its custom ADAPS system, in favor of the commercially available AQUARIUS Software. It is likely that large and small organizations around the world (which look to the USGS for best practices) will now follow their lead and adopt commercial data management systems.
Office software (E.g. Microsoft Excel)
Actively licenced commercial hydrological data management system
23%
35%
5% Integrated environmental system software (e.g. YSI, FTS, etc.)
3%
24%
10%
Unsupported (or highly customized) commercial hydrological data management system
Custom solution – built by contractor Custom solution – built & supported in-house Which data management system does your organization primarily use? How many years have you been using it?
25
Industry Report
Effectiveness
Respondents using commercial hydrological software were more likely to rate their system as very effective or effective (by 6 to 22% depending on the type of consumer expectation).
Today’s data management systems could be more effective in enabling water professionals to meet the expectations of data consumers. This is particularly true for organizations using custom solutions, integrated environmental systems, and Microsoft Office. 36% of these respondents (not using commercial software) rated their system as somewhat ineffective or ineffective in enabling them to meet consumer expectations for faster reporting and publishing. 34% of them reported system ineffectiveness in making metadata available – while only 10% of respondents with commercial software rated their system as somewhat ineffective or ineffective. Of respondents not using commercial software, 28% rated their system as ineffective in the timely delivery of real-time products and services. Organizations using commercial hydrological software were more likely to rate their system as very effective or effective (by 6 to 22% depending on the type of consumer expectation).
22%
Timeliness - real-time products & services 2%
47%
12% 12% 4% 33%
Accuracy - archival data quality & completeness
2% 1%
48%
11%
4%
25% Defensibility - version control & traceability to source
9% 7%
1%
20% Reporting - faster publishing of archival quality data 1%
24%
Analytics - higher level analysis & interpretation 2%
6%
44%
13% 10%
20%
4%
8%
11%
39%
16%
200
160
120
80
40
0
Very Effective Somewhat Metadata Effective availability of Somewhat metadata Ineffective Ineffective Don't Know Not Applicable
39%
22%
11%
5%
42%
13%
What is the effectiveness of your current data management system in enabling you to meet the expectations of today's data consumers?
26
Industry Report
Satisfaction
Up to 32% more respondents who are using commercial hydrological software reported being very or somewhat satisfied with their system when compared to respondents using custom solutions, integrated environmental systems, and Microsoft Office.
Water resource managers can and should be more satisfied with their data management system. Again, more (up to 32% more) respondents who are using commercial hydrological software reported being very or somewhat satisfied with their system when compared to respondents using custom solutions, integrated environmental systems, and Microsoft Office. While 47% of respondents not using commercial software reported dissatisfaction with their system’s responsiveness to emerging technologies and consumer expectations, only 16% with commercial software reported dissatisfaction. Respondents not using commercial software also showed high dissatisfaction with their system’s performance (37%), reliability (25%), data security (24%), and breadth of features (44%). The dissatisfaction rate of respondents using commercial software in these areas ranged from only 11% to 15%.
29%
Total Cost of Ownership
11%
5%
39%
16% 22%
Ease of use / training 10%
3%
49%
15%
29% Data security
11% 10% 11%
Responsiveness (e.g. to emerging technologies & evolving end-user expectations)
15% 17%
7%
38%
37%
22%
23%
Performance (e.g. speed of data queries) 4%
41%
19%
11%
28% Reliability 3%
7%
Breadth of features/ functionality
15%
39%
22%
15%
6%
200
250
150
100
50
0
Very Satisfied Somewhat Satisfied Somewhat Unsatisfied Unsatisfied Don’t Know
43%
16%
What is your level of satisfaction with your existing system (inclusive of support) relative to the following attributes?
27
Industry Report
Features Respondents rated the importance of 16 features of a data management system. This question confirmed that water resource professionals want a system with a wide breadth of features: 64% to 96% of respondents rated features as very or somewhat important, depending on the feature. The most important features as reported by water resource managers include: ease of importing data (96%), time series data editing and correction (92%), data history and auditability (92%), data visualization (90%), and database security, backup, failover, and general system enterprise awareness (89%).
Features of a Data Management System
Very Somewhat Somewhat Not Don’t Important Important Unimportant Important Know
Count Response
User web access to data management system
234
100
27
24
7
392
Ease of importing data (e.g. time series, gauging
316
62
5
3
3
389
Level of integration with telemetry systems
204
109
31
27
14
385
Development & maintenance of rating curves & shifts
217
95
26
21
24
383
Support for other river discharge models (e.g. index
113
154
40
43
32
382
Discrete sample analysis (e.g. cluster analysis)
92
163
51
43
34
383
Integration with enterprise GIS
214
121
17
21
17
390
Database security, backup, failover, & general system
252
98
15
4
19
388
Data history / auditability (who, what, when, why)
230
133
10
4
7
384
Data approval, locking, & workflow control
162
157
30
13
19
381
Data visualization options & interactivity
238
117
13
6
8
382
Time series data editing & correction
266
98
14
2
6
386
Automated corrections
115
174
49
25
19
382
Reporting & report configurability
202
138
25
11
6
382
Real time alerts & notifications
183
118
50
23
8
382
Data dissemination & publication to data consumers
210
112
36
12
11
381
Question Totals answered question
Please rate the importance of each of the following features in a data management system.
28
394
Industry Report
Conclusions & Remarks Hydrometric network managers continue to be very responsive to the many and various industry challenges and opportunities. Monitoring operations today are very different than they were a decade ago. While the average number of hydrographers has not changed, more needs are being served with more timely and higher quality data. It is clear that even this vastly improved effectiveness is not considered complete. Much more change is anticipated over the coming decade. How are you managing the increase in needs served by your network? Are you responding by increasing the number of parameters monitored per station, by increasing the station density or by co-operating with other agencies?
2%
6%
How effectively are you using enabling technologies? The use of multi-channel data loggers, multi-probe sensors and acoustic doppler measurement devices are replacing more traditional technologies.
11%
Are you meeting widely held expectations for data timeliness? What proportion of your network is connected to telemetry systems and what methods are you using for data publication?
8% 14% 18% 6%
The use of advanced technologies requires a better educated, highly skilled workforce. Are you paying enough and are your training methods sufficient to meet your needs over the coming decade? Are you able to clearly state what your objectives are for data quality, service, and security? Are you able to demonstrate compliance with internationally recognized standard operating procedures? Do you operate a data management system with sufficient features, performance, reliability, and security to support both (1) the ever increasing volume and complexity of water data and (2) the stakeholder expectations for timely delivery of information products and analyses?
12%
The answers to these questions and the trends highlighted in the report can help you benchmark your network, helping you plan for future investments to ensure the continued success of your hydrological monitoring program.
18%
23% 22% 14% 10%
16%
30%
26%
27%
34%
Industry Report
Published by Aquatic Informatics Aquatic Informatics™ Inc. provides leading software solutions that address critical water data management and analysis challenges for the rapidly growing environmental monitoring industry. It understands the challenges of environmental data management. Its flagship product AQUARIUS is carefully engineered to ensure a smooth transition to modern best practices for hydrometric monitoring. AQUARIUS is the time series data management software used by the largest and most advanced hydrological and environmental monitoring agencies, including the USGS and Water Survey Canada, and it is scalable to fit the needs of any size of monitoring network. AQUARIUS has a simple design that combines an intuitive and efficient user interface with the latest hydrological science and techniques. AQUARIUS allows hydrologists and technicians to manage the data they collect more quickly and to a higher level of quality, so they can deliver more effectively on the evolving demands of stakeholders. AQUARIUS is fully configurable to adapt to and support any Quality Management System. AQUARIUS excels in retrospective data analysis as well as real-time continuous work flow management. It has a unique portfolio of features for real-time sanity checking, error detection, data cleaning, data flagging, automatic bias corrections, and rating shift management to streamline quality controls. AQUARIUS automatically builds an auditable data processing, correction, and editing log, ensuring data defensibility. The AQUARIUS rating development tool is engineered to support the latest global standards set by the USGS, ISO, WMO, and OGC, to ensure the highest confidence in calculations of flow. Its flexible reporting and publishing tools include an extensive list of industry standard report templates and an integrated report template builder for customized reporting. 37.72
37.22
36.72 0
Aquatic Informatics Inc. 1100 – 570 Granville St. Vancouver, B.C. V6C 3P1 tf: 1.877.870.AQUA (2782) p: +1.604.873.AQUA (2782) info@aquaticinformatics.com www.aquaticinformatics.com © 2012 Aquatic Informatics Inc.
watch AQUARIUS see how it works
2
The AQUARIUS architecture is designed to manage and synchronize data from multiple networks for fast reliable solutions to difficult water management issues. Advanced support for data migration ensures continuity with legacy systems and its state-of-the-art architecture ensures your system is secure, scalable, and integrated. The intuitive AQUARIUS toolboxes combined with comprehensive training and support resources ensure rapid deployment and implementation in any operational environment. To watch a VIDEO demonstration of AQUARIUS or to learn more, please visit www.aquaticinformatics.com.