Icttoolsevaluation

ICT tools evaluation Criteria for tools that make something (presentations, posters ...)

There are five criteria that should be used to evaluate tools: ​ ease of use, sharing, maths integration, freedom of use and overall impression​ . As the tools are used for making the criteria apply equally to them, but when comparing it’s sensible only to compare tools in the same category, for example only compare the tools for presentations with other tools for presentations. Students should be given concrete tasks when evaluating, for example if evaluating tools for mind mapping they should all try to make a mindmap of a specific thing they are learning about. Also this tasks should all be the same so the evaluation grades can be comparable.

Every criteria has a scale of grades from 1 to 4, and has a name and a description connected to it so it’s easy to say what the grade is for. Based on this it will be possible to make a form for evaluation which should also have a field where students can give a short written comment in addition to grades.

What can be evaluated with this criteria: ● presentation tools ● poster tools ● comic tools ● video and animation tools ● mind­map tools Exception: ● quiz making tools ­ in addition to basic 5 criteria they also have an additional criteria: ​ Feedback (how does it give feedback to students) and ​ Question bank (is it possible to import questions and variety) ● Do we need additional exceptions?

Ease of use Evaluate how complicated is the tool ­ how complicated is the interface, can you find help etc.

1. Hard​ ­ The user interface is very complicated, it’s difficult to get anything done and its very hard to get help when you need it.

2. Medium​ ­ The user interface is complicated, but it’s possible to find help so you can learn how to use the tool. You can get things done but it takes a lot of time.

3. Easy​ ­ The user interface is either simple and intuitive or you have integrated help (tutorials) which takes you step by step of the basic usage even if things look unfamiliar. You can get things done and it’s possible to get up to speed.

4. Super easy​ ­ The user interface is simple and you have step by step help for everything if needed. You can get things done and it’s possible to be fast even if you just started to use it for the first time.

Sharing and collaboration Evaluate how easy is to share your work, are the end results exportable, is it possible for more people to work at the same time etc.

1. Closed​ ­ You can make things online, but it’s not possible to share online (you have to download the results first). Only one person can access the tool to edit the presentation.

2. Shareable​ ­ It’s possible to share you work on the website of the tool and download it, but it’s not possible to embed your work on other websites. Only one person can access the tool to edit the presentation.

3. Collaboration possible​ ­ It’s possible to share online, download and embed on other websites.

You can share your work for other people to edit but only one person can edit the work at one moment.

4. Easy collaboration​ ­ It’s possible to share online, download and embed on other websites. More people can work on the presentation at the same time.

Maths integration Evaluate how easy is it to input math notation when working with the tool and does it show correctly.

1. Bad​ ­ It’s possible only if you import maths as bitmap pictures (PNG, JPG etc.) so they show but equations look ugly.

2. Basic​ ­ It’s possible if you import maths in bitmap and vector pictures (SVG and PDF integration) so they show better.

3. Good​ ­ The tool has an option to input maths directly but it’s either limited (you can’t input all mathematical notation) or the maths shows weird when finished.

4. Made for maths​ ­ You can input all maths directly and it shows nicely when finished.

Freedom of use Evaluate how free is the tool ­ do you have to pay anything, do you have to sign up etc.

1. Closed​ ­ You have to pay even to try things out, sign up is mandatory.

2. Trial​ ­ You have a free trial period but have to pay later, sign up is mandatory.

3. Open​ ­ Tool has a free basic version but you have to sign up to try things out. You have to pay for additional options or there are integrated commercials.

4. Free​ ­ Tool is totally free to use and you don’t even have to sign up to start using it.

Overall impression Evaluate the general feel of the tool. Can you do the things you want to do with it? Does it have many options? Does it do something cool or is the interface very nice?

1. Meh​ ­ It has basic options, but it’s not possible to do everything you need to do. The tool has no special things that stand out.

2. It’s OK​ ­ It has options that you need to do everything you need to do. It has everything you expect it to have, but it doesn’t stand out in any way.

3. Good​ ­ You can do everything you need to do, and it also has some advanced options or the tool has a nice look and feel.

4. Best​ ­ You can do everything you need to and more. The tool has advanced options, or even something that no other tool has and it looks nice. The general feel is “wow, everybody should use this!”

Feedback This is a special criteria for quiz making tools. Evaluate is it possible to integrate feedback to students and how does the system give feedback to students.

1. True/false​ ­ It’s only possible to give basic feedback on simple questions, is the answer true or false. The feedback is only for the student who answers the quiz.

2. Basic​ ­ It’s possible to give more detailed feedback on partially correct answers for basic question types (eg. multiple choice). The feedback is only for the student who answers the quiz.

3. Good​ ­ It’s possible to give more detailed feedback on variety of question types or it’s possible to combine feedback for a whole class so every person has it’s feedback but you can discuss answers in a whole class.

4. Best​ ­ You can give detailed feedback on a per student basis and you can combine general results for a whole class so you can make a discussion.

Question types and bank This is a special criteria for quiz making tools. Evaluate what possible types of questions does it have and is it possible to import existing questions to make a question bank you can reuse.

1. Basic​ ­ You only have basic questions types (multiple choice, true/false, single word answers) and you can’t import questions so you have to enter everything by hand.

2. Simple import​ ­ You have only basic question types but it’s possible to import/export questions for reuse.

3. Complex import​ ­ You have various types questions enabling many possible type of answers and you can import/export questions for reuse.

Criteria for measuring tools on computers (analysis, simulations…)

As these tools have a different use the criteria are somewhat different. The end product of using measurement tools on computers is getting the results and learning something, not necessarily making something (although there are tools that can do both). The ​ “Ease of use”​ and ​ “Overall impression” ​ are similar, the rest are different.

What can be evaluated with this criteria: ● virtual experiments and simulations (these have a special criterion ​ “How real is it?”)​

● tools for analysis of real data on computers ● remote experiments (real experiments controlled over distance) Generally this should be added as a subtopic on etwinning forum.

Ease of use Evaluate how complicated is the tool ­ how complicated is the interface, can you find help and are there any materials to help with integration in teaching.

1. Hard​ ­ The user interface is very complicated, it’s difficult to get anything done and its very hard to get help when you need it. There are no examples of tasks that students can perform in class.

2. Medium​ ­ The user interface is complicated, but it’s possible to find help so you can learn how to use the tool. You can get things done but it takes a lot of time. There are a few examples of tasks for students.

3. Easy​ ­ The user interface is either simple and intuitive or you have integrated help (tutorials) which takes you step by step of the basic usage even if things look unfamiliar. You can get things done and it’s possible to get up to speed. There are detailed examples of tasks that students can work through in class.

4. Super easy​ ­ The user interface is simple and you have step by step help for everything if needed. You can get things done and it’s possible to be fast even if you just started to use it for the first time. There are many detailed examples of tasks that students can work through in class.

Variable control Evaluate the level of control that you can perform ­ can you choose what you will measure or are things predefined?

1. Scripted​ ­ everything is predefined, it’s not possible to choose what you want to measure and it’s more like watching a video than performing a measurement yourself.

2. Controllable​ ­ the thing you can measure is predefined but you can play with variables and see how things react by yourself.

3. Free use​ ­ it’s not that you can do literally everything (computers are limited) but you can choose what you will measure from many options and how you’ll do it by yourself.

Data analysis Evaluate the level of manipulation of data that is possible ­ is it even possible to do quantitative measurements, do you have different representations of data, is it easier than in real life…

1. Qualitative​ ­ It’s possible to manipulate things but it’s not really possible to get quantitative data.

You can generally describe what is happening but you can’t say exactly how much.

2. Basic​ ­ It’s possible to get quantitative data (numbers) but you feel that this measurement could be done with simpler tools and not on a computer. It’s complicated or impossible to analyse data inside the tool so you have to use something else.

3. Advanced​ ­ It’s possible to get quantitative data easier or better than with tools that you would use by hand. You can at least partially analyse data inside the tool (eg. it makes graphs automatically).

4. Made for computers​ ­ It wouldn’t be possible to make this measurements without this tool on a computer. Eg. you have so much data that it would be impractical to do it by hand. You can analyze the data in different formats (tables, graphs …) and even do modeling (it helps you calculate).

How real is it?

This is a special criterion for simulations. As they simulate reality you should try and evaluate how good of a yob do they do. Do they help you see things that you couldn’t see in real life?

1. Fake​ ­ It looks like that the simulation is not behaving like the real phenomenon that it’s supposed to simulate.

2. Limited​ ­ Simulation behaves as it should but it’s based on a very limited model, like it’s a very simplified version of a phenomenon that it simulates.

3. Realistic​ ­ Simulation behaves very similar to reality or it has special features that enables you to see some aspect of real world that would be very difficult to observe.

4. Complete​ ­ Simulation is realistic and it enables you to see some aspect of real world that would be very difficult to observe.

Overall impression Evaluate the general feel of the tool. Can you do the things you want to do with it? Does it have many options? Can you learn something unusual with it?

1. Meh​ ­ It has basic options, but it’s not possible to do everything you need to do. The tool has no special things that stand out, it would be better if you did this experiment in real life.

2. It’s OK​ ­ It has options that you need to do everything you need to do. It looks nice or makes measurement easier but you would still rather did this experiment in real life.

3. Good​ ­ You can do everything you need to do, and it shows you something that would be very hard to do with hands. The tool looks good and it has many options.

4. Best​ ­ The general feel is “wow, everybody should see this!”. You can understand what is happening better, it saves time and it feels like you are doing real science.