Table 10: User assessment of the DHIS2 platform at endline (October 2021

Table 10: User assessment of the DHIS2 platform at endline (October 2021) Website: DHIS2 Website URL: https://dhis2.health.gov.mw/dhis-web-dashboard/

Heuristic being assessed

Visibility of system status

The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.

Relevant task or task step

Searching for relevant information.

Match between system and the real world

The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

User control and

freedom Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Supports undo and redo. Is the language used on the website familiar to its intended audience?

Navigation and

control.

Consistency and standards

Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions. Exploring the web site.

Usability description

When figures are loading, it displays a moving circle to indicate loading status.

It is not obvious when the user has been logged off. If the user has been logged off, things simply stop loading rather than a logged off message. Must refresh the page to log in again. It is not clear what some of the metrics mean. There is no explanation for the metrics visible on the Dashboards or in the Indicator dictionary.

The acronyms do not have explanations.

Evaluator’s comments on usability

The loading icon is universally recognisable (the moving circle).

It would be helpful to automatically display a message when the user has been logged out, asking them to sign back in.

It would be very helpful to have descriptions of the metrics and acronyms.

Changing the filters on dashboards is fairly easy, there is no complicated undo/redo necessary.

Changing between dashboards and apps is easy. Some apps lose the app icon in the upper right corner of the screen but for these it is easy to go back or home.

On the dashboard “HIV (By District)” when filtering by a fixed period of months or quarters, the figures subtitles still show “Last 6 months” or “Last 4 Quarters” on the graph view (see Figure 18). In the table view it says the correct period.

The dashboard “03.2 – HIV (By District)” shows national data and “03.3 – HIV (National)” shows data by district (they appear to be switched). Same with the Maternal & Child Health district and national dashboards. See Figure 19 below.

There are 2 data visualizer apps –‘Classic Data Visualizer’ and ‘Data Visualizer’. The only filters which are easy to use are period and organisation unit. The others are not obvious what they mean.

Other than not displaying which information is on which dashboard, the dashboards are easy to navigate.

The pivot table app has so many options it would be difficult to navigate unless the user knew exactly what they were looking for, but it seems very useful. On dashboards, filter information is not displayed in the figures, and sometimes the subtitles contradict the filters, so it is not always obvious if the data shown is for the period selected. It is easier on the table view than the figure view. It should ideally be straightforward both ways.

There appears to be a moderate level of consistency in the way data is presented between different dashboards (comparing HIV site, district, and national, and Maternal & Child Health site, district, and national).

It seems unnecessary to have 2 data visualizer apps. Perhaps it is for personal preference for more experienced users, but if most users prefer one, the other is redundant.

Error recognition, diagnosis, and recovery from errors

Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

Recognition rather than recall

Make objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate. Advanced search Instructions for use of the system are not obvious.

Flexibility and efficiency of use

Accelerators -- unseen by the novice user – may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

Aesthetic and minimalist design

Dialogues should not contain information which is irrelevant or rarely needed.

Help and documentation

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large. There are 2 apps for indicator definitions – ‘Indicator Dictionary’ and ‘Indicator Search’.

Simple and

advanced search. Can you get the same results? The only common error message on the Dashboards is “No data”.

On the Maps app, for any ‘Earth Engine’ layers, (precipitation, temperature, population density, etc), an error at the bottom of the screen reads “No value present: To show this layer you must first sign up for the Earth Engine service at Google. Please check the DHIS 2 documentation.”

The ‘Indicator dictionary’ app should contain definitions of all indicators, but many of them are not descriptive enough or do not specify the denominator.

Does the website have any shortcuts for proficient users? It seems that the pivot table app can be used very flexibly to display data in a variety of ways that are most convenient for the user. It contains a lot of additional data not shown on dashboards. It seems unnecessary to have 2 indicator definition apps, particularly if neither is up to date with definitions for all indicators.

The “no data” error message could be more precise. For example, the ART registration data is available by quarters but not by months, which is not obvious without trial-and-error. If you filter by months, it says “no data”, and if you filter by quarters, data is displayed. Perhaps a regular user would know this information already.

It is taking me a lot of trial-and-error to find the information I need, since I cannot find descriptions for some of the data. For example, the time frame for “cumulative” is not specified – is it cumulative since the beginning of the year? Since the beginning of reporting on the system? The last 12 months? Without that information it is difficult to know the difference between, for example, Cumulative ART Registrations and New ART Registrations for the same time period. I checked the Indicator Dictionary app and it did not provide a useful answer.

The pivot table app seems very useful for advanced users, even better than the dashboards for finding the desired data. With an instruction tutorial it could be used by novice users too.

General website

appeal and initial impressions

Easy to read font, good contrast Good initial impressions of website aesthetic. Data is presented in easyto-read figures on the dashboards.

How do I use the advanced search? There are no instructions or help buttons visible on Dashboards. The information button says “No description” when clicked. It is unclear where I can find instructions on website use. It would be helpful to have some basic instructions on how to perform common actions, including use of filters, and information on what each dashboard shows. The Indicator Dictionary and Indicator Search apps are not as useful as they should be.