eKarton IS EUROREC Seal Level II

Test log Report - Eurorec Seal - 2

Assessment Type: System assessed: - SW provider: (name, address)

-

Application/Module: Version:

Assessment performed: - Organization: - Location: - Date and time: Final findings:

Attendees: - Evaluator:

-

SW provider:

-

Others:

Attachment: -

Detailed Test log Report

Self-assessment

ZipSoft d.o.o.

RakovaÄ?ka 32, 21000 Novi Sad e-Karton 4

External assessment

Detailed Test log Report

GS001512.01 The system enables to link a role to a user.

Scenario: 1. Add a new user 2. Assign a role to the user Observations:

Criteria: Check that the role has been appropriately assigned to the user.

Compliance:

We have created a new user account and assigned two roles to the user (General practitioner and Emergency unit). Each user must have at least one role assigned. Users can have multiple roles assigned (Picture GS001512.01_1). Each role defines access permissions and capabilities of each user that accesses the system. Users have access only to those system features and functions to which they have been granted access.

Attachments:

Available user roles

Roles assigned to user

Picture GS001512.01_2: Assigning roles to the user

GS001519.04 The system shall include the information necessary to identify each patient, including the first name, surname, gender and date of birth.

Scenario: Criteria: 1. Add new a patient with the following Check if the patient data in the system includes attributes: first name, surname, gender and date at least: name, surname, date of birth, gender of birth Observations: Compliance: On the dialogue window for creating a new patient record (Picture GS001519.04_1), the system enables entering patient demographic data: JMBG (National identification number), first name, last name, date of birth and gender.

Attachments:

Picture GS001519.04_1: Creating a new electronic patient’s medical record

GS001523.03 The system enables the capture of all patient demographic data necessary to meet legislative and regulatory requirements.

Scenario: Criteria: 1. Add a new patient with all relevant Check if all relevant (requested) demographic demographic attributes data of the patient are displayed Observations: Compliance: According to “The Regulation about Technological and Functional Requirements for Integrated Health Information System” of the Ministry of Health of Serbia, as a regulatory requirement, the system enables entering all required patient demographic data.

Attachments:

Picture GS001523.01_1: Overview of demographic data in the patient’s medical record

Picture GS001523.01_2: Overview of demographic data in the patient’s medical record

GS001531.02 The system displays all current health problems associated with a patient.

Scenario: Criteria: 1. Search for a patient Check if only current health problems are 2. Add 2 health problems (health items) displayed.

3. Assign the status of inactivity to one health problem Observations: Compliance: We have added a new diagnosis (I10) as a new health problem (Picture GS001531.02_1). On the patient record window, the system shows a list of all current health problems (Picture GS001531.02_1).

Attachments:

Picture GS001531.02_1: Entering a diagnosis as a new health problem

Picture GS001531.02_2: Overview of all current health problems

GS001537.03 Each version of a health item has a date and time of data entry.

Scenario: Criteria: 1. Search for a patient Check if the date and time for the diagnosis, the 2. Add a health item, e.g. a diagnosis intervention and the consultation note are the 3. Add a health item, e.g. an intervention same as the date and time of the registration.

4. Add a health item, e.g. a consultation note The registration is the date and time when the data was entered in the system (time-stamp).

Observations: Compliance: We created a new contact by adding a new diagnosis (Picture GS001537.03_1), a service (Picture GS001537.03_2) and a medical document (Picture GS001537.03_3).

After saving the contact, the patient record window GS001537.03_4) shows the date and time of registration.

(Picture

Attachments:

Picture GS001537.03_1: Adding a new diagnosis

Picture GS001537.03_2: Adding a new service

Picture GS001537.03_3: Adding a new medical document

Picture GS001537.03_4: Patient’s record window; an overview of entered information (date and time of entry)

GS001538.02 Each version of a health item identifies the actor who has actually entered the data.

Scenario: Criteria: 1. Log in user #1 Check that the user #1 is the actor who entered 2. Search for a patient the diagnosis and the user #2 is the actor who 3. Add a health item, e.g. a diagnosis entered the intervention.

4. Log in user #2 5. Search for the patient 6. Add a health item, e.g. an intervention Observations: Compliance: Each version of a health item has the date and time of creation and information about the user responsible for creation.

We added two health items as two different users.

On the patient’s record window (picture GS001538.02_1) the system shows contacts created by two different users of the system.

Attachments:

Picture GS001538.02_1: Overview of previous contacts in the patient's medical record created by two different users of the system

GS001539.02 Each update of a health item results in a new version of that health item.

Scenario: Criteria: 1. Search for a patient Check that the diagnosis and the consultation 2.Search for a health item, e.g. a diagnosis note both have two versions.

3. Update the diagnosis to a more specific diagnosis 4. Search for a health item, e.g. a consultation note 5. Update the consultation note Observations: Compliance: We have changed the status of the diagnosis to ‘inactive’. For each status change, the user is required to enter the reason for the status change (picture GS001539.02_1). On the history window (Picture GS001539.02_2) the system shows who made the changes, version number, the status of the item, etc.

Attachments:

Picture GS001539.02_1: Entering the reason for the status change

Picture GS001539.02_2: History window displaying the complete history of item versions

GS001544.04 The system supports the use of clinical coding systems, where appropriate, for data entry of health items.

Scenario: Criteria: 1. Enter a data e.g. for diagnose using clinical Check if clinical coding is used where appropriate coding system (eg. Diagnose) Observations: Compliance: The system supports the use of the following clinical coding systems: -

ICPC-2 (International Classification of Primary Care) (Picture GS001544.04_1),

-

ICD-10 (International Classification of Diseases) (Picture GS001544.04_2).

Attachments:

Picture GS001544.04_1: Entering the reason for patient's contact using ICPC-2 code list

Picture GS001544.04_2: Entering the diagnosis using ICD-10 code list

GS001550.06 The system presents a current medication list associated with a patient.

Scenario: Criteria: 1. Search for patient Check if only current medication list is displayed.

2. Add medication #1 3. Add medication #2 4. Assign status of inactivity to the medication #2 Observations: Compliance: Each therapy associated with a patient requires one diagnose, dose, dosing instructions and duration to be defined.

We have added new medication - Bromazepam (Picture GS001550.06_1). The system displays current medication list according to the duration of each defined therapy (Picture GS001550.06_2).

The user can stop a therapy by changing the status of medication (Pictures GS001550.06_3 and GS001550.06_4).

Attachments:

Picture GS001550.06_1: Defining a 7-day therapy

Picture GS001550.06_2: Patient’s current medication list

Picture GS001550.06_3: Confirmation for discontinuation of the therapy

Picture GS001550.06_4: Entering the reason for discontinuation of the therapy

Picture GS001550.06_4: Updated patient’s current medication list

GS001559.02 The system presents a medication history associated with a patient.

Scenario: Criteria: 1. Search for patient Check if a medication history associated with a 2. Add medication #1 patient is displayed.

3. Add medication #2 4. Assign status of inactivity to the medication #2 Observations: Compliance: On the history window (Pictures GS001559.02_1 and GS001559.02_2), the system displays a complete and detailed medication history.

As stated above, the user can stop the therapy by changing the status of medication.

Attachments:

Picture GS001559.02_1: Patient’s medication history

Picture GS001559.02_2: A detailed patient’s medication history

Picture GS001559.02_2: The patient’s history window; medication history list

GS001573.02 The current medication list can be printed.

Scenario: 1. Search for patient 2. Add medication #1 3. Add medication #2 4. Assign status of inactivity to the medication #2 Observations:

Criteria: Check if only current medication list can be printed.

Compliance:

The system allows the user to print the current medication list (Picture GS001573.02_01).

The current medication list can also be exported to an Excel document (Picture GS001573.02_02).

Attachments:

Picture GS001573.02_1: The patient’s current medication list; Print preview

Picture GS001573.02_1: The patient’s current medication list displayed in an Excel document.

GS001577.03 The system provides a catalogue of medicinal products.

Scenario: Criteria: 1. Search for patient Check if you can enter the medication, using 2. Add a medication using catalogue of catalogue. medicinal products Observations:

Compliance:

The system uses the official catalogue of registered medicinal products.

The catalogue includes both generic and commercial names of medicinal products.

The picture GS001577.03_1 displays the search results for the required medication (acetisal) and its chemical parallels (acetylsalicylic acid).

Chemical drug parallels appear in the search results only if entered search criteria include at least 5 characters. For search criteria shorter than 5 characters, only drugs whose commercial name contains the given search criteria are presented.

Attachments:

Picture GS001577.03_1: Search results for the criteria "Aceti" with chemical parallels

GS001579.02 Each version of a health item has a status of activity, e.g. active or current, inactive, history or past, completed, discontinued, archived.

Scenario: Criteria: 1. Search for patient Check that the status of activity for a health item 2. Search for a health item, e.g. diagnose, (eq. diagnose, intervention, consultation note) intervention or consultation note can have different status of activity.

3. Display the current status of activity 4. Update the status of activity 5. Repeat step 2-4 minimum 3 times and update status of activity Observations: Each health item has a status of activity: active/inactive.

We have changed the status of activity for the diagnosis R94.5 to inactive (Picture GS001579.02_1).

The system requires entering the reason for changing the status of a health item (Picture GS001579.02_2).

The picture GS001579.02_3 shows that the diagnosis of R94.5 is now inactive.

The picture GS001579.02_4 shows the versions history of the health item (the status of the chosen health item was changed several times).

Health items that are inactive are always displayed with strike through font style.

Attachments:

Compliance:

Picture GS001579.02_1: Changing the status of the diagnosis R94.5

Picture GS001579.02_2: Input of the reason for the change of the diagnosis status

Picture GS001579.02_3: Inactive diagnosis R49.5

Picture GS001579.02_04: An overview of the complete history of the versions of a health item

GS001590.02 The system presents a list of the allergens with an active status.

Scenario: Criteria: 1. Search for patient Check if only current allergen list is displayed.

2. Add allergen #1 3. Add allergen #2 4. Assign status of inactivity to the allergen #2 Observations: Compliance: We have added a Penicillin allergy (Picture GS001590.02_1). Then we added a dust mite allergy.

The status of the dust mite allergy was changed to ‘inactive’ (Picture GS001590.02_2).

The Picture GS001590.02_3 displays the list of the patient’s allergies. The Penicillin allergy is active and the dust mite allergy is inactive.

Attachments:

Picture GS001590.02_1: Adding the Penicillin allergy

Picture GS001590.02_2: Changing the status of dust mites allergy to inactive

Picture GS001590.02_3: Penicillin allergy is active and the dust mite allergy is inactive

Picture GS001590.02_3: The patient’s current allergen list

GS001593.02 Deletion of a health item results in a new version of that health item with a status "deleted".

Scenario: Criteria: 1. Search for patient Check that the health item has the status 2. Search for a health item, e.g. diagnose, “deleted”. intervention or consultation note 3. Delete the health item 4. Repeat step 2-3 minimum 3 times (different health items) Observations: Compliance: We submitted a request to delete a patient contact (Picture GS001593.02_1).

We chose a contact from the list (Picture GS001593.02_2). The user who is currently logged in can only delete the contacts they created.

After choosing a contact, we entered the reason for deletion of the contact (Picture GS001593.02_3).

The picture GS001593.02_4 shows the patient’s history window, with the deleted contact marked in red.

Attachments:

Picture GS001593.02_1: Submitting the request for deletion of a patient’s contact

Picture GS001593.02_2: The list of contacts which can be deleted

Picture GS001593.02_3: Entering the reason for deleting the contact

Image GS001593.02_4: The review of the contact with a status “deleted�

GS001594.02 Each version of a health item has a person responsible for the content of that version. The person responsible for the content can be a user or a third party.

Scenario: Criteria: 1. Search for patient Check that the health items has the correct 2. Add a health item, e.g. a diagnose person responsible for the content.

3. Add a health item, e.g. an intervention 4. Add a health item, e.g. a consultation note Observations: Compliance: We logged in as a nurse. We added a new diagnosis and a service (Picture GS001594.02_1). The user must choose the physician who is responsible for that service. Picture GS001594.02_1 shows a patient’s history window with information about the person responsible for the contact and a different person responsible for the service item.

Attachments:

Picture GS001594.02_1: Dialog box for adding a service

Picture GS001594.02_2: The patient’s history window

GS001595.01 Each change of status of a health issue results in a new version of that health issue.

Scenario: Criteria: 1. Search for patient Check if all versions of health items are in the 2. Add 2 health items, e.g. An allergen and system. medication 3. Assign them status active/current 4.Change the status Observations: Compliance: As mentioned in GS001539.02, every change in the status of health items results in an automatic creation of a new item version. Picture GS001595.01_1 shows the history of the versions of the patient’s allergies.

Attachments:

Status of a health item Picture GS001595.01_1: The history of the versions of the patient’s allergies

GS001598.02 A complete history of the versions of a health item can be presented.

Scenario: Criteria: 1. Search for patient Check if all versions of health items are 2. Add 2 health items, e.g. allergen and displayed. medication 3. Assign them status active/current 4.Change the status Observations: Compliance: As stated before, the system displays all versions of a health item. (see GS001539.02 and GS001595.01)

Attachments:

GS001610.03 The system enables to document a patient contact.

Scenario: Criteria: 1. Search for patient Check that the system enables to document a 2. Add all health items to document one patient patient contact. contact Observations: Compliance: We created a new contact with the following health items: -

Diagnosis (Picture Image GS001610.03_1),

-

Service (Picture GS001610.03_2),

-

Therapy (Picture GS001610.03_3)

-

Medical document (Picture GS001610.03_4),

-

Referral (GS001610.03_5),

-

Vital signs and reason for contact (Picture GS001610.03_6)

-

Sick leave (GS001610.03_7).

The picture GS001610.03_7 displays an overview of the patient’s contacts, including the last contact.

Attachments:

Picture GS001610.03_1: A dialog box for adding diagnosis

Picture GS001610.03_2: A dialog box for adding a service

Picture GS001610.03_3: A dialog box for adding therapy

Picture GS001610.03_4: Creating a medical document

Picture GS001610.03_5: A dialog box for adding a referral

Picture GS001610.03_6: Entering the vital signs and the reason for contact

Picture GS001610.03_7: Entering sick leave information

Picture GS001610.03_8: The patient’s history window; An overview of the patient’s contacts

GS001611.01 The system is able to present all the documentation associated to a contact for that patient.

Scenario: Criteria: 1. Search for patient Check that system is able to present for one 2. Add all health items to document one patient patient contact all the documentation associated contact with that patient.

Observations: Compliance: The picture GS001610.03_7 displays a patient’s history window which contains an overview of all patient’s medical items.

Attachments:

GS001638.01 The system is able to present the history of the individual results for discrete lab tests.

Scenario: Criteria: 1. Search for patient Check if the history of lab tests is displayed 2. Add 3 lab tests Observations: Compliance:

We added 3 lab test results (Picture GS001638.01_1). Each lab result can be individually displayed. The picture GS001638.01_2 shows each lab test result individually. Laboratory results are marked in red color when the value is outside of the reference values. Otherwise, values are presented in blue color.

The picture GS001638.01_3 shows a diagram for each individual lab test result.

Attachments:

Picture GS001638.01_1: Adding lab test results

Picture GS001638.01_2: A dialog box for displaying individual lab results

Picture GS001638.01_3: A dialog box for displaying a diagram of individual lab results

GS001901.02 Each version of a health item has a date of validity.

Scenario: Criteria: 1. Search for patient Check if all health items have date of validity.

2, Add at least 3 health items and their date of validity e.g.: diagnose with date of validity 4 days ago, lab test with date of validity 2 days ago, medication with date of validity per today Observations: Compliance: We added a new contact with the date of validity 2 days ago (Picture GS001901.02_1), a lab test result with the date of validity 2 days ago (Picture GS001901.02_3) and medication with the date of validity per today (Picture GS001901.02_2).

The picture GS001901.02_4 shows the patient’s history window with validity date and creation date marked for each item. The picture GS001901.02_5 shows the patient’s medication list with validity date marked.

Attachments:

Picture GS001901.02_1: The date of validity for the contact

Picture GS001901.02_2: Dialog for adding a therapy; Date of validity

Picture GS001901.02_3: Adding laboratory results; Date of validity

Picture GS001901.02_4: A patient’s history window; Date of validity

Picture GS001901.02_5: A patient’s medication list; Date of validity

GS001932.01 The system supports concurrent use.

Scenario: 1. User #1 login 2. Search patient A 3. Display/add health items 4. User #2 login (concurrent) 5. Search patient A 6. Display/add health items Observations:

Criteria: Check if it is possible to login at the same time and to work with health items of the same patient concurrently.

Compliance:

We logged in to the system as two different users (Picture GS001932.01_1).

In both instances, we opened the same patient file.

In the first instance of application, we entered diagnoses, therapies and services (Picture Picture GS001932.01_2). Then we entered different dignoses, therpies and services in the second instance of application (Picture GS001932.01_3).

After saving the contact in both instances of application, the patient’s history window displays all the health items added in both instances of application (Picture GS001932.01_4).

Attachments:

Picture GS001932.01_1: Patient’s file, two instances of applications with different logged users

Picture GS001932.01_2: Entering diagnoses, therapies and services in the first instance of application

Picture GS001932.01_3: Entering diagnoses, therapies and services in the second instance of application

Picture GS001932.01_4: The patient’s history window in both instances of application

GS001947.02 The system makes confidential information only accessible by appropriately authorised users.

Scenario: Criteria: 1. User #1 login.

Check if user #1 and user #2 can access only the 2. Display/add health items to the patient (user data in step 2 (not in 3).

1 has access right to see/enter this data) 3. Try to display/add health items for which he has no access rights.

4. Repeat steps 1 to 3 for user #2 with different access rights.

Observations: Compliance: We logged in to the system as a user with high access level settings. We created a new contact and chose to save the contact as accessible only with a high access level (’Only users in my group can access’) (Picture GS001947.02_1).

After logging in as a user with lower access level settings, the contact is not accessible.

Attachments:

Picture GS001947.02_1: Choosing the access levels for saving a contact

Picture GS001947.02_5: Access denied to a user with lower access level settings

GS002175.02 The system enables the implementation of a privilege and access management policy.

Scenario: Criteria: 1. Search user #1 Check if the privilege/access rights can be 2. Change privilege/access rights changed.

3. Add user #2 4. Assign privilege/access rights to the user #2 5. Repeat test GS001947 Observations: Compliance: The privilege and access rights can be changed by changing the role settings of the user. The picture GS002175.02_1 shows a dialog box for role configuration. We configured the role to have low access level (’All users can access’).

We created a new user (Picture GS002175.02_2) and assigned the role to the user (Picture GS002175.02_3) Attachments:

Picture GS002175.02_1: A dialog box for role configuration

Picture GS002175.02_2: A dialog box for creating a new user

Picture GS002175.02_3: A dialog box for assigning roles to the user

GS002182.01 The audit trail contains the registration of users logging in or out.

Scenario: Criteria: 1. User #1 login.

Check if the system keeps track of all logins and 2. Add health items to the patient logouts.

3. User #2 login.

4. User #1 logout.

5. User #3 login.

6. User #3 logout.

Observations: Compliance: The picture GS002182.01_1 shows the system audit trail.

Attachments:

Picture GS002182.01_1: A dialog box for displaying the system audit trail

GS002184.01 The audit trail contains the registration of security administration events.

Scenario: Criteria: 1. Login as administrator Check if the change of access rights has been 2. Change the access rights to one user. registered in the audit trail.

Observations: Compliance: All changes applied to the settings of user accounts and roles are registered in the audit trail (Picture GS002814.01_1).

Attachments:

Picture GS002184.01_1: A dialog box for displaying the system audit trail;

GS002198.02 Audit trails cannot be changed after recording.

Scenario: 1. Display the audit trail 2. Try to change an item in the audit trail Observations:

Criteria: Check that the audit trail could not be changed.

Compliance:

The audit trail is automatically generated by the system. No changes to the audit log are allowed. The logs are read only.

Attachments:

GS002211.01 The system enables a user to change his password.

Scenario: Criteria: 1. User login Check if the user can login with a new password.

2. Change the password Observations:

Compliance:

A logged user can change their password.

The picture GS002211.01_1 displays the dialog box for changing password. The user is required to enter the new password twice.

Attachments:

Picture GS002211.01_1: A dialog box for changing the password for current user

GS002243.01 Security service issues and operation of the system are well documented.

Scenario: Criteria: 1. Check the documentation Check if security service issues all well covered in the SW documentation.

Observations: Compliance: Please see the documentation attached.

Attachments:

GS002265.01 Each health item is uniquely and persistently associated with an identified patient.

Scenario: Criteria: 1. Insert three types of health items.

Check that each health item is uniquely and persistently associated with an identified patient.

Observations: Compliance: We added a new diagnosis (I11), medication (Cephalexin 500mg.) and a service (Picture GS002265.01_1).

When we open the patient’s file again, we can see the previously saved data with all of the health items (Picture GS002265.01_2).

At the database level in each table, there is a mandatory field "uid_pacijenti" - unique patient identification. The unique patient id is automatically generated when a patient’s file is created.

Attachments

Picture GS002265.01_1: Entering a diagnosis, therapy, service

Picture GS002265.01_2: The patient’s history window; The diagnosis, therapies and services added during the contact.

GS002266.01 Each version of a health item is uniquely and persistently identified.

Scenario: Criteria: 1. Search for patient Check that the diagnose an the consultation note 2. Search for a health item, e.g. a diagnose both have two versions.

3. Update the diagnose to a more specific diagnose 4. Search for a health item, e.g. a consultation note 5. Update the consultation note Observations: Compliance: In the personal history of a selected patient, we found a previously specified diagnosis (J01) (Picture GS002266.01_1) and set the status to inactive. We added a new, more specific diagnosis (J01.1) to the patient's personal history (Image GS002266.01_2). As shown in the picture GS002266.01_3, the patient’s personal history includes both diagnoses - J01 and J01.1.

We have also updated the information about an existing vaccine for tuberculosis and changed the date of vaccination (Picture GS002266.01_4).

The picture GS002266.01_5 shows that all versions of a health item are displayed (in this case, both versions of vaccination).

Attachments:

Picture GS002266.01_1: A dialog box for displaying a patient’s personal history of diagnoses

Picture GS002266.01_2: Adding a diagnosis

Picture GS002266.01_3: All versions of the diagnosis are displayed

Pictures GS002266.01_3: Updating the information about vaccination

Picture GS002266.01_4: All versions of a health item (vaccination)

GS002268.01 Each user is uniquely and persistently identified.

Scenario: 1. Search for an existing user 2. Create a new user with the same attributes.

Try inserting the same user-ID 3. Create a new user 4. Delete the new user

Criteria: Check if the system allows to create a new user with the already existing user-ID. Check if it is possible to persistently delete an existing user.

Observations:

Compliance:

The system automatically generates a unique user identification for each user. The system prevents the deletion of any user accounts. It is only allowed to activate and deactivate user accounts. The picture GS002268.01_1 displays a dialog box for creating and modifying user accounts.

Attachments:

Picture GS002268.01_1: A dialog box for modifying a user account; Activating/deactivating a user account

GS002269.01 The system enables to assign different access rights to a health item (read, write,...) considering the degree of confidentiality.

Scenario: Criteria: 1. Search for a health item and assign it an Check that the system appropriately appropriate access rights controls/limits access to the health items, considering the degree of confidentiality.

Observations: Compliance: When saving a new contact, a user can assign one of the five access levels to that contact (Picture GS002269.01).

Attachments:

Picture GS002269.01_1: Selecting an access level

GS002281.01 All patient data can be accessed directly from the patient record.

Scenario: Criteria: 1.Login user #1 Check that all the health items entered by user 2. Search for patient A #1 can be displayed when logging in as user #2 3.Add 2-4 health items, e.g. diagnose, intervention 4.Login user #2 5.Search for patient A 6.Display all the health items Observations:

Compliance:

All patient data can be accessed from the patient’s history window (Picture GS002281.01_1).

Attachments:

Picture GS002281.01_2: A patient’s history window

GS002287.02 The system distinguishes administrators, privileged users and common users. Administrators assign privileges and/or access rights to privileged and common users. Privileged users assign privileges and/or access rights to common users.

Scenario: Criteria: 1. Login as an Administrator.

Check that the system distinguishes 2. Change/grant privileges and access rights to a administrators, privileged users and common less privileged user. users. Administrators assign privileges and/or 3. Try to assign priviliges to a higher or equally access rights to privileged and common users. privileged user.

Privileged users assign privileges and/or access 4. Repeat steps 1-3 for a "privileged user". rights to common users.

Observations: An administrator assigns privileges and access rights to all users. However, for crucial and critical system settings, the system requires a ‘super administrator password’.

The picture GS002287.02_1shows an administrator creating a user account.

The picture GS002287.02_2 shows a dialog box for entering a ‘super administrator’ password.

Attachments:

Picture GS002287.01_1: A dialog box for creating a new user account

Compliance:

Picture GS002287.01_2: Entering a super administrator password to access the highest level of administration

GS002300.02 The system is available in the languages required by the regulatory authorities.

Scenario: Criteria: 1. Display the cases/outputs in all languages The system and its components is able to required by the legal authorities. perform in the languages reuired by the legal authorities.

Observations: Compliance: Since the system is used on the territory of the Republic of Serbia, all system applications are in Serbian language (Picture GS002300.02_1).

Multilingual support is also implemented in the system.

Attachments:

Picture GS002300.02_1: e-Karton WS application in Serbian language

GS002307.02 Each patient and his EHR is uniquely and persistently identified within the system.

Scenario: Criteria: 1. Insert new patient with a new ID and some Check that the system does not allow changing attributes the patient's ID 2. Try to change ID Observations: Compliance: When inserting a new patient, the system automatically generates a unique patient ID. The system does not allow changing a patient’s ID. Administrators can only see a patient’s ID when performing certain administrative tasks (Picture GS002307.02_1).

Attachments:

Picture GS002307.02_1: A dialog box for creating a new EHR; A patient’s ID;

GS002312.01 The system is able to make a distinction between patients with same name, first name, gender and date of birth.

Scenario: Criteria: 1. Insert three patients with the same name, Check that the inserted patients have the correct first name, gender and date of birth name, firstname, gender, date of birth.

2. Search all patients by the previously inserted name.

3. Change a name of the patient #1.

4. Change a piece data other than name firstname - gender - date of birth of Observations: Compliance: We added three patients with the same first name „Petar“, last name „Petrović”, gender „Male“ and date of birth „01.02.2000“. The picture GS002312.01_1 shows search results for the criteria “Petar”.

Since all the patients have a unique patient ID, the system always makes a distinction between patients.

Attachments:

Picture GS002312.01_1: A dialog box for displaying search results for patients; Search criteria "Petar Petrovic"

GS002415.04 The system takes the access rights into account when granting access to health items, considering the role of the care provider towards the patient.

Scenario: Criteria: 1. Grant read-only access rights (role) to user#1 Check the access righs: User #1, read only and rgd. health item-A user#2, write.

2. Grant write access rights (role) to user#2 rgd. health item-B 3. Login as User#1 4. Search patient 5. Try to update item-A 6. Repeat step 3-5 for User#2 Observations: Compliance: For the role ‘General Practice’ we granted read-only access rights for the services provided for patients (picture GS002415.04_1).

For the role “Pediatrics” we granted read and write access rights for the services provided for patients (picture GS002415.04_2).

We logged in as a user with the user role “General practice” and created a new contact. Although we could see the previously entered services, there was no option for adding a new service (Picture GS002415.04_3).

Then, we logged in as a user with the user role “Pediatrics” and created a new contact. This time, there was an option for adding a new service (Picture GS001531.02_4).

Attachments:

Picture GS002415.04_1: A dialog box for configuring a role; The role ”General Practice”

Picture GS002415.04_2: A dialog box for configuring a role; Role “Pediatrics”

Picture GS002415.04_3: For the user logged in as ‘General Practice’, there is no option for adding a service item

Picture GS002415.04_4: The user registered in the group ‘Pediatrics’ can add a service item

GS002437.04 The system offers to all the users nationally approved coding lists to assist the structured and coded registration of health items.

Scenario: Criteria: 1. Select two cases of the nationally - Check that the nationally approved coding-lists are approved coding lists. displayed/offered by the system when user registers 2. Insert health item using the relevant structured and coded health items. coding list.

- Also a relevant mapping of the entered data to the national coding-lists would be considered as appropriate.

Observations: Compliance: For adding a new diagnosis the system uses the ICD-10 codelist for item selection (Picture GS002437.04_1). For adding a new service provided to the patient, the system uses a codebook provided by the National Health Insurance Fund (Picture GS002437.04_2).

Attachments:

Picture GS002437.04_1: Selecting a diagnosis from the ICD-10 code list

Picture GS002437.04_2: Adding a new service; The Code list of services provided by the National Health Insurance Fund

GS002489.02 Data entry is only done once. Entered health items are available everywhere required.

Scenario: Criteria: 1. Insert/update health item that is in use in Check that entered health items are available different parts of the system everywhere in the system where required.

2. Display the same health item in another part(s) of the system, where required.

Observations: Compliance: We added three new diagnoses: J02, I11 and S03 (picture GS002489.02_1).

Then we opened a sick leave for the patient. In the input field for the reason for sick leave (Dg.) a drop-down menu offers a choice of three diagnoses that we added in the first step (Picture GS002489.02_2).

Attachments:

Picture GS002489.02_1: Adding a new diagnosis

Picture GS002489.02_2: Entering data about the sick leave; choosing a diagnosis

GS002497.03 The system displays patient identification data (name, first name, age and sex) on each data entry interface.

Scenario: Criteria: 1. Select a patient Check that patient's identification data (name, first 2. Display data entry interfaces where diagnosis name, age and sex) are displayed on each data and other patient relevant data are entered. entry interface.

Observations: Compliance: Patient’s identification data (first and last name, age and sex) is always displayed on top of the patient’s history window (Picture GS002497.03_1).

Attachments:

Picture GS002497.03_1: A patient’s history window and identification data

GS002582.02 The system displays, when prescribing a medicinal product, known allergies of the patient, if it does not alert the user for a specific allergen.

Scenario: Criteria: 1. Select an existing patient with already Check if the system displays, when prescribing a registered allergies, or register/insert allergies medicinal product, known allergies of the with a selected patient. patient, if it does not alert the user for a specific 2. Insert medicinal product to this patient. allergen.

Observations: Compliance: Every time a patient’s record is opened, the system will display a notification window about known allergies, risk factors and the expiration date of health insurance at the bottom right corner of the screen (Picture GS002582.02_1).

Attachments:

Picture GS002582.02_1: A patient’s history window; Notification about the patient’s known allergies

Picture GS002582.02_2: Notification about the patient’s known allergies

GS002625.01 The system enables the user to modify patient's administrative data.

Scenario: Criteria: 1. Select a patient Check that the system enables modifying the 2. Modify the patient's administrative data patient's administrative data (address). (address) 3. Display the modified data Observations: Compliance: The picture GS002625.01_1 displays a patient’s administrative data before any changes. We changed the address, country and citizenship of the patient. The picture GS002625.01_2 displays the patient’s administrative data after the changes.

Attachments:

Picture GS002625.01_1: Patient’s administrative data before the changes

Picture GS002625.01_2: Patient’s administrative data after the changes

GS002638.01 The system distinguishes actual or active medication items from past medication items when including and displaying medication items in lists or in a journal.

Scenario: Criteria: 1. Select a patient Check that system distinguishes actual or active 2. Insert two medicinal products medication items from past medication items 3. Assign a non-active status to one of the two when including and displaying medication items in prescribed medicinal products. lists or in a journal.

Observations: Compliance: We added two medication items (Palitrex and Brufen). Both active medication items are marked red in the picture GS002638.01_1. We discontinued the Brufen therapy. The picture GS002638.01_2 shows that the currently logged user discontinued the therapy. Therefore, the discontinued therapy is no longer visible on the list of active medications.

Attachments:

Picture GS002638.01_1: Patient’s current medication list

Picture GS002638.01_2: Discontinued therapy with data about the time of discontinuation, the reason for discontinuation and the user who discontinued the therapy

GS002639.01 The system enables the user to modify health items, if legally admitted.

Scenario: Criteria: 1. Select a patient Check that the system rejects modifying health 2. Modify health items where legally admitted. data where updating is legally not admitted.

Try to modify health items where legally not admitted.

3. Display the updated health items.

Observations: Compliance: It is not possible to change previously entered health items if not legally admitted. The only way to change health data where updating is legally not admitted is to send a request to delete with the reason why. This request must be approved to be logically deleted from the patient’s record.

(Picture GS002639.01_1) For detailed explanations related to the correction of a health item, see GS001595.01.

Attachments:

Picture GS002639.01_1: Submitting a request to delete a contact

GS002655.02 The system has a timeout function, terminating a session after a configurable period of inactivity.

Scenario: Criteria: 1. User logs-in into the system Check that the system terminates a session after a 2. User remains inactive for a time period larger predefined period of inactivity. than the predefined value set for the timeout function.

Observations: Compliance: We set up an idle time interval in minutes (15 min.) after which the user will be required to log in (Picture GS002655.02_1). If a user is inactive for more than the idle time interval, the system will automatically log the user out.

After entering the correct password, the system will return the user to the application at the same point where they previously stopped working.

Attachments:

Picture GS002655.02_1: Dialog for setting up the idle time after which the users will be automatically logged out

GS003787.01 The system has a consistent way to present clinical alerts, e.g. red color for abnormally and/or high lab results.

Scenario: Criteria: 1. Select a patient#1 Check that the pathological data is presented in a 2. Insert at least two different pieces of clinical consistent way and are visualy distinguished from data with normal values. the normal/non-critical data.

3. Insert at least two different pieces of clinical data with pathological values.

4. Display the inserted health items.

Observations: Compliance: Laboratory results are marked in red color when the value is outside of the reference values. Otherwise, values are presented in blue color (picture GS003787.01_1).

Attachments:

Picture GS003787.01_1: Patient’s laboratory results

GS004729.02 A medication list presents at least the following elements: identification of the medicinal product (package), starting date, date of the latest prescription, dosing instructions (structured or as a textual expression) Scenario: Criteria: 1. Select a patient Check that the medication list presents all data 2. Insert all data rgd medicinal prescription requested medicinal product data.

3. Display data on a prescribed medicinal products.

Observations: Compliance: Picture GS004729.02_1 displays the dialog box for adding a new therapy.

We selected a drug whose commercial name is Pentrexyl. After choosing the drug, we provided additional therapy information: -

Dosing instructions: 1x1 - ‘Take one capsule a day’ before eating, orally

-

Quantity: 2 packs

-

Date of beginning of therapy and duration of therapy (4 days).

Picture GS004729.02_2 displays the patient’s medication list with all previously entered data (dosing, quantity, date of beginning, etc.).

Attachments:

Picture GS004729.02_1: The dialog box for adding a new therapy

Picture GS004729.02_2: A patient’s medication list