VINTAGE Project: Results of the Pilot Run1

Grant Agreement n. 2012-4192/001-001 Reference: 527349-LLP-1-2012-1-IT-GRUNDTVIG-GMP

PILOT RUN 1: THE RESULTS Synthesis of the results of the European experimentation of the VINTAGE framework for the self-evaluation of Key Competences

Associazione Formazione 80 This project has been funded with support from the European Commission. This publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

1 2 3

VINTAGE Project - www.vintage.euproject.org

CONTENTS Page 2 – About this Report Page 3 – Introduction Page 4 – Summary of the results of the focus groups Page 5 – Analytic results Page 10 – Possible use of the Vintage tool for adults Page 12 – The web survey Page 16 - Conclusions

ABOUT THIS REPORT This document collects and summarizes the results of the 7 local reports produced by the project partners in consequence of the Focus group sessions organized in their respective countries to complete the Pilot Run 1 phase. The aim of the Focus group was to test and gather useful information on the work develop by the Vintage Project up to the present phase.

The observations emerged during the focus group session are aimed at providing suggestions and recommendations on how to

adjust the project outcomes in order to make them more effective and to try to reach the wider possible range of target groups, as well as to gather a general opinion on their usefulness and employability in adult education.

The report is structured following the questions proposed to the participants during the focus groups sessions. Each question is followed by an observation that identifies and summarises the main elements expressed by the local reports on that issue.

These observations are supported, if the case, by short quotations from the local reports with the indication of the source (Partner and country).

VINTAGE 2014

Project,

February

4 1 2 3

3

Focus group sessions realized in the partner countries by the project partners: Austria, Die Berater Germany, Wisamar Ireland, Meath Partnership Italy, Learning Community (Rome) and Formazione 80 (Turin) The Netherlands, Plato Sweden, STPKC Web Survey gathered opinions from many

INTRODUCTION

European countries.

As envisaged by the WP4 Pilot Run 1 of the Vintage project between the months of December 2013 and February 2014 each Project partner realized a focus group session in its country. Experts were sent information materials, namely a presentation of the self-evaluation procedure developed by the Vintage project together with the frameworks of KC2 “Communication in foreign languages” and KC4 “Digital competence”, and were invited to analyse them, to try to experiment the procedure and finally to take part to the focus group session in order to discuss, openly and clearly, all relevant aspects of the project: point of strengths, weaknesses, possible changes, possible use within adult education. All focus group sessions were

organised by paying great attention in trying to have the wider possible variety of participants in terms of roles and positions covered within the field of adult education.

Among the participants it is possible to notice a prevalence of researchers, teachers and practitioners although significant is also the presence of people organizing and making decisions on various activities of adult education.

In general the panel of participants was composed by 28 males and 30 females with qualifications corresponding to tertiary education and higher.

All the received material and the website

participants have information about the project link to the project in order to read,

download and analyse the outputs of the project.

Together with the invitation to the Focus group sessions, they have also received a presentation of the self-assessment procedure, a guide to test the procedure and the frameworks of two KCs as a sample (KC2 and KC4).

Participants were asked to express and discuss their opinions on the general structure of the project, to test on themselves (if possible) the procedure and to study analytically two main questions: 1. Instructions and evaluation procedure;

self-

2. Possible use of the Vintage tool in Adult Education.

2 1 3

VINTAGE Project - www.vintage.euproject.org

COMPOSITION OF THE PANELS Adult Education Field of activity

Teacher trainer

Career guidance consultant

Professor Researcher expert 1

Manager decision maker 3

Total

Formal education

6

Non/formal education

8

1

5

2

16

Vocational training

6

4

3

2

15

13

4

17

22

11

58

Study and research Total

20

5

10

SUMMARY OF THE RESULTS OF FOCUS GROUPS An important premise to be noted is that despite the variety of the positions covered by the participants, their experiences in the field of adult education and their national background, the opinions emerged during the discussions and collected in the different local reports are generally the same for all partner countries. So, apart from some minor references to specific local situations or target groups, we can say that the observations and remarks that follows can be considered valid, unbiased and reflecting a common feeling on the procedure developed by the Vintage project.

GENERAL QUESTIONS Before reporting observations and

the the

contributions on the two specific issues object of the analysis (the self-evaluation procedure proposed by the Vintage project and its possible use within the field of adult education), it is important to underline the opinions expressed by the participants to the Focus group sessions on the Vintage project, particularly on its theoretical structure and on its aims.

To this respect the evaluation is very positive.

Most of the participants in the different national contexts agree that the theoretical structure (namely the concepts of competence and self evaluation) are central and of the utmost importance in adult education. The observations and remarks expressed can be summarized in four

points: 1. Tools and instruments promoting and fostering selfevaluation among adult people are necessary and highly valuable because whatever kind of learning in adult age has its origin in a process of self-evaluation.

This is the process bringing adult individuals to raise awareness of their training/education needs, it brings about their interest towards learning and it is at the basis of their decision to (re-)enter a learning route.

For these reasons, the attempt carried out by the Vintage project to develop and define a procedure of self-evaluation on Key Competences, to test it and verify its possible use is important and useful.

2. An effective self-evaluation

4 6 5 1 2 3

5

procedure, such as the Vintage project is trying to develop, cannot consist in a range of tests aimed at measuring knowledge but it should have the ambition to build a guided process of reflection, whose central figure is the adult person.

3. In the perspective of

such process of reflection, participants consider appropriate the choices made at the beginning of the project: • Analyzing competences focusing on Contexts and Behaviours; • Requesting the adult users to evaluate their own

performed behaviours in real life situations through quality indicators.

4. A self-evaluation tool presenting features such as those of the Vintage project demands a very complex procedure and requires adult users an investment in terms of time and attention.

ANALYTIC RESULTS language used are not suitable for adult people with lower skills or lower levels of education.

Many of the participants highlight the lack of an adequate introduction. “instructions very adequate for people with high education level, but most likely not for people with less formal training. The language is relatively advanced.”

STKC (SE)

Do you think the instructions of the selfevaluation procedure are clear?

Did you try to experiment the self-evaluation procedure? If yes, how much time did it take?

It is common opinion that instructions for the procedure are clear and understandable only for a target group with medium/high level of education. The concepts expressed and the

Most of the experts in the partner countries tested the procedure on themselves at least on one of the provided KC frameworks (KC 2 Communication in foreign languages, KC4 Digital Competence). For all those who

experimented the procedure the time necessary to complete it is about 1 hour and 30 minutes. It is definitely a long time which is likely to become even longer for the users with a lower level of education. This, according to all participants, can induce lower skilled users to quit the procedure without completing it.

What, in your opinion, are the key strengths of the procedure?

According to most of the participants the selfevaluation procedure works rather well for what concerns the choice of the mastery level and is acceptable for what concerns the articulation into clusters of the KCs.

4 6 5

VINTAGE Project - www.vintage.euproject.org Also the situations aimed at stimulating and recalling performed behaviours in real life are on the overall considered to be adequate.

“The self-evaluation model in itself is convincing. It is an active and thorough way of assessing the EU key competences. The user is asked to consider a known situation from different perspectives. The user has to make critical selfreflections”. PLATO (NL) “ at the centre of the procedure there is the real person, not the hypothetic one. The tool is built on the idea the person has about him/herself, and all the phases allow the emerging of his self-awareness.” LC (IT) “The procedure is impressing, the selfobservation at the beginning is an interesting perspective and is a stimulus for the self-motivation” LC (IT) “ the topic of the project is very important. It should be transferred to other areas, such as vocational training, as a professionalizing component that could allow us to be aligned to the other European labour systems.”

LC (IT) “The focus of Vintage on the performance of competences instead of a assessment focusing on skills and knowledge – and providing therefore long lists of multiple choice

questions concerned with knowledge – was thought to be a good alternative” WISAMAR (DE ) It is interesting because it is a totally different approach. DieBerater (AT) The format of the procedure, incorporating real-life situations, was seen as an advantage by some who believe that adult learners learn best in a contextualised setting.

Meath Partnership (IE).

A positive aspect is that the user is asked to recall in writing and proofs own performances in the portfolio – this was seen as real added value WISAMAR (DE)

What, in your opinion, are the critical points of the procedure?

From the analysis of the single local reports, there appear to be a common critical evaluation on some aspects of the procedure in terms of content and/or of their presentation: a) At present the procedure requires the user to read introduction and explication texts that are considered to be excessively long.

Moreover, they present a complex language which limits possible target groups to adult people with medium-high level of education.

b) The time required to complete the selfevaluation procedure (1hour 30 minutes for each KC, according to the participants) seems to be too long. This is likely to cause a loss of motivation in the potential user resulting in the abandon of the procedure before completing it. c) Many participants underline how, in some cases, the structure Mastery Level/Cluster/Items is not consistent: • Correspondence between item and mastery level is not always clear; • Level progression among the items within the same cluster, in some cases, presents a gap; • The user can have different mastery levels in the different clusters within the same KC. d) Recalling to the mind a real-life situation similar to the one presented in the items is possible and

7 8

7

rather simple but, passing from this memory to a detailed description of the behaviours performed in that situation can be rather difficult. e) The 10 quality dimensions identified as self-evaluation criteria are generally considered to be of difficult understanding and application. In fact, some of them are not

• • • • • • • • • • •

• •

• •

applicable to all clusters and/or to every mastery level. Moreover, they are likely to be interpreted more as personality tracts of the user rather than as qualities of the performed behaviours.

Most of the participants agree on the opportunity to reduce their number and to express them in a simpler way.

SOME REMARKS In paper form (the current state) there is too much text. DieBerater (AT) “in the different cluster I could have different mastery levels”; LC (IT) “I find a gap between the initial generic item (1/2) and the following items”; LC (IT) “The description of the levels of the items is not precise” LC (IT) “self-evaluation implies a perception of adequacy/inadequacy of oneself capacities (psychological dimension); but it requires a grid with explicit criteria, that is lacking in this procedure”: LC (IT) “target group will not be able to understand the questions correctly. And if they understand they will have problems in applying it”; STPK (SE) “quality grid seems to be too complex and difficult for the average learner”: STPK (SE) “not all quality dimensions are relevant to all clusters and all levels”; STPK (SE) “learners could have different mastery levels in the different clusters”; STPK (SE) “less advanced people need a very long time to complete the self-evaluation procedure; STPK (SE) “The tool is dependent on the ability of the user to self-reflect and think abstractly, so not everyone is capable to do that. For lower levels of working or thinking, the tool might be (too) difficult; PLATO (NL) It might be difficult for the user to understand the difference between ‘context’ and ‘quality” PLATO (NL) the approach might be suitable for people with higher education, for those with lower or middle education it might be too complex in several ways: the whole procedure, too much reading, abstract transfer from provided situations to corresponding own experiences, the qualities domains are too complex, the outcome might also be not considered to be worth the (time) effort WISAMAR (DE) “The identification and gathering of evidences to be included in a portfolio is not yet elaborated”.

PLATO (NL) Ten quality dimensions for the self-evaluation are too many. Moreover, many of them can result hardly applicable especially for mastery level 1 and 2. The risk is that the quality dimensions are interpreted and understood more as personality tracts rather than as qualities/characteristics of the performed behaviours. F80(IT)

1 2 3

VINTAGE Project - www.vintage.euproject.org What changes to procedure would recommend?

the you

Starting from the critical points listed in the previous paragraph, the focus group carried out in the various countries proposed some changes to the procedure, which could contribute to simplify its structure and, as a consequence, enlarge the number of potential users.

The suggestions provided can be summarised as follows: a) Simplifying the language used in the procedure and reduce the texts as much as possible, while explaining clearly all essential elements, and particularly the mastery levels; b) It would be important to explain from the very beginning to the potential user that the self-evaluation procedure focuses on – and consists of – a process of (self-)reflection and of raising awareness on the self, whose final outcome will not be expressed by a numeric score or by an objective result. c) The situations presented in the items tend to be too abstract for an average user; it would be better to add a list of actions/behaviours;

d) It would be important to: eliminate the titles of the quality dimensions and provide instead clearer descriptions; reduce the number of the qualities; diversify the quality grids of evaluation according to the different clusters and to the different mastery levels. e) Produce a step-by-step user guide to accompany the user all along the procedure by providing detailed information on how to describe and analyse the performed behaviours in real life situation recalled to the memory by the presented items. An example could be the use of the “STARmethod”. This would make the self-evaluation process smoother and simpler.

Besides this, it would be necessary to produce a guide for teachers/practicioners in order to make them able to support potential users to complete the procedure. f) Given the objective complexity of the procedure, it would be important to provide the

user with constant and clearly visible information on the progress achieved.

It would also be important for the user to have the possibility to interrupt the procedure at any given time (or after any step achieved) and to re-enter it at a later moment in order to complete it.

Most of the suggestions highlights how it would be important not just to advise possible users to write down notes on the real life situations recalled and on the behaviours performed, rather it would be necessary to verify the possibility to provide users with a detailed guide on how to describe that situation and those behaviours by giving them a checklist/scheme to follow (possibly accompanied by some examples).

Such a modification to the procedure would make the evaluation of the behaviours on the quality grid easier and smoother for all, particularly for adult people with lower levels of education. On the contrary, if this modification is not possible, it is recommended to reduce the number of the quality dimensions in relation to the chosen mastery level and to the items proposed.

9

•

• • •

• • • • • • • • •

•

• • • • • •

• •

SOME REMARKS provide some basic introduction to what is meant by a competence, and should explain some of the European jargon used throughout the framework in a glossary of terms. Meath Partnership (IE).

In paper form (the current state) there is too much text. DieBerater (AT) A list of behaviours could be a good alternative LC (IT) it would be better to have a list of actions/behaviours, something such “I’m able to do this, I can do that” before the choice of the profile. The objectivity in this procedure should be maintained visible; LC (IT) the titles of the quality should be removed: for example “critical thinking”. The terms are too difficult for the average user LC (IT) I should remove the labels in the 10 dimensions LC (IT) Clarify the mastery levels; STPK (SE) suggest to develop different (simplified) grids for different clusters and levels; STPK (SE) Make a handbook and guide for teachers / tutors; STPK (SE) The structure is complex. Maybe not everyone is capable of using the tool in a right way so the use of the tool should be supported in most cases; PLATO (NL) A step-by-step manual should accompany the tool; PLATO (NL) the progress achieved by the user in following the procedure must be clearly shown while using the tool, PLATO (NL) When users are asked to think of an own situation, the tool might ask them to write this down in the portfolio (as a proof) and provide information on how to describe it, for example by using the STAR-method: describe Situation, Task, Actions and Results. After this, the described situation must stay visible for the user to go through the self-evaluation. PLATO (NL) Maybe the instrument can be made more simple. It is important that the user understands what he is doing in the tool and for this reason the results of the user, after using the tool, must be very clear. PLATO (NL) it should be made clear from the start this tool is more about reflecting and awareness than about receiving a score or objective result WISAMAR (DE) to reduce the quality dimensions to autonomy/initiative, control/direction and success/impact PLATO (NL) Usability and design are extremely important; Die Berater (AT) In paper form (the current state) there is too much text. Die Berater (AT) The quality domains should be formulated in form of questions to help understanding WISAMAR (DE) It would be important to provide users with a guide on how to describe performed behaviours in the real life situation evoked and recalled through the items of the framework. This could take the form of a scheme enriched by some examples. F80 (IT) a user handbook or guide would be very beneficial to adult learners. As well as offering stepby-step advice on how to use the procedure Meath Partnership (IE).

The self-assessment process should be accompanied by trainer or teacher or embedded into a broader set of activities WISAMAR (DE)

1 2 3

VINTAGE Project - www.vintage.euproject.org • • • • • •

• •

it should be made clear from the start this tool is more about reflecting and awareness than about receiving a score or objective result WISAMAR (DE) to reduce the quality dimensions to autonomy/initiative, control/direction and success/impact PLATO (NL) Usability and design are extremely important; Die Berater (AT) In paper form (the current state) there is too much text. Die Berater (AT) The quality domains should be formulated in form of questions to help understanding WISAMAR (DE) It would be important to provide users with a guide on how to describe performed behaviours in the real life situation evoked and recalled through the items of the framework. This could take the form of a scheme enriched by some examples. F80 (IT) a user handbook or guide would be very beneficial to adult learners. As well as offering step-by-step advice on how to use the procedure Meath Partnership (IE).

The self-assessment process should be accompanied by trainer or teacher or embedded into a broader set of activities WISAMAR (DE)

POSSIBLE USE OF THE VINTAGE TOOL FOR ADULTS Thinking about your adult learners, what instructions and support would you suggest to complete the procedure?

Many of the participants to the focus groups work with adult people having a low educational level. They all insist on the need to make the instructions for the procedure easier to understand by using a simpler language and, possibly, to envisage a supported use of the tool (by teachers, practitioners or in a group of peers).

•

• • • •

the tool should contain more questions that guide to the choice of the level; LC (IT) Computer support; STPK (SE) Human support in a local context; STPK (SE) Instructions to teachers; STPK (SE) Using the tool in a

• •

context of Peer learning, produce a group profile, ex. my team at work, share ideas and learn from each other STPK (SE) Make the texts easier to understand; Die Berater (AT) basic training, assistance from a tutor or the use of a user-friendly handbook Meath Partnership (IE).

Do you think the selfevaluation process on each of the 8 KC proposed by the Vintage project can be useful and significant in adult education?

All experts acknowledge the self-evaluation procedure proposed by the project as very interesting for its approach and they all think that it could have a noteworthy impact on adult education. The most positive aspect is considered to be the process of reflection on the self and on one’s own behaviours.

This, if duly accomplished, produces an innovative

methodology of evaluation of KC’s mastery level and can be considered in itself an actual training activity.

Nonetheless, the prevailing opinion among the focus group participants is that the use of the tool/of the procedure should be conceived as a methodological instrument for teachers and practitioners to be embedded or integrated in a wider set of activities, rather than as an instrument for individual use.

In fact, the tool/procedure is considered to be very useful to carry out activities with small groups aimed at fostering peer-learning or to prepare adult learners to face the challenges of (re-)entering in the labour market, starting a personal development process or changing one’s career.

1 2 3

11

• • •

• •

• • •

SOME REMARKS this tool can be useful/significant in adult education, for purposes of empowerment, job interview preparation, formative evaluation of learning or career purposes; PLATO (NL) the complexity of the tool makes it not suitable for all adults. It can be useful for adults with a middle and higher education PLATO (NL) the Vintage tool can be widely used, provided it is embedded in structures (for example in plans for personal development) and there is a strategy to make the tool acquainted and available (for example by employment centres or Human Resources Developers, or Managers). PLATO (NL) può rappresentare un vero e proprio momento di formazione F 80 (IT) the self-reflective nature of the tool would be beneficial in some self-directed adult education training programmes, especially those facilitated through e-learning platforms.

Meath Partnership (IE).

Focus on competencies, so that the diagnoses lead to the desire to learn. The strength is that the individual takes a stand, have I learnt or not. STPK (SE) It features interesting points for a application conversation. Die Berater (AT) If it can be used than only within the setting of a coaching being an accompanied procedure WISAMAR (DE)

If you consider your specific job in the field of adult education, do you think the Vintage tool could be useful to your work?

According to the opinion of the participants, the vintage tool could be useful in their working activities.

As underlined in point 7, the tool is considered to be very interesting but it should be used with a small group (1 practitioner/teacher and 3-4 learners) where each learner

complete its own selfevaluation and at the end each one presents to the other members of the group the actions taken to self evaluate themselves and the result achieved. Actions and results should then be compared and discussed together with the trainer.

The HR department could get an overview of needs of courses or individual needs.

STKC (SE)

Most experts think they can use the tool in their work. Some say: ‘while working with groups of professionals’ of ‘it can be useful for every working and learning adult’; PLATO (NL) The tool can be used for example to estimate the start level of participants in a training, or as selfreflection tool in adult education contexts and

4 6 5 1 2 3

VINTAGE Project - www.vintage.euproject.org HRD. For more specific aims, more specific tools are needed. PLATO (NL) The tool was innovative, and that it would particularly suit formal vocational training and students of all ages involved in higher education. Meath Partnership (IE)

Feedback The opinions concerning the possible feedback of the tool emerged from the various focus group sessions are not unanimous. However, the prevalent opinion is that any possible numeric score should be accompanied by a descriptive competence profile capable of valorizing the positive aspects

expressed by the user while providing suggestions on how to improve any possible weakness resulting from the self-evaluation process.

their level of competence through formal and nonformal learning opportunities.

Meath Partnership (IE)

All agree that the construction of such a profile (its formulation in words) is a very complex challenge and have no clear idea on how it should look like.

The form should consist in a description of the competence level achieved by the user and should be formulated in an encouraging way. This can be achieved by highlighting the qualities under which the person has shown most confidence but at the same time by providing suggestions on those aspects (quality dimensions) on which it would be necessary to focus a personal development process. F80 (IT).

Results should be presented in form of a description with clear instructions about how to proceed from there WISAMAR (DE) Best way to present the results of this assessment for Irish adult learners is through a numeric result, coupled with a personalised description of their level of competence, and with some advice on how to improve

THE WEB SURVEY To involve practitioners and experts in the piloting also at a distance, a web survey has been realized.

It consisted in a questionnaire published on the web, using a free tool (surveymonkey.com); people were invited by email to read the project materials, try the selfassessment procedure and answer to the online questionnaire to supply a

feedback.

The web survey was structured in 10 questions; in each one the user was requested to express an opinion about the strengths and weaknesses, the significance of the tool for adult education, the ease or difficulty to identify oneself in the proposed levels, recall performed behaviours in real life situations, use the levels of quality, and the

clearness of the instruction and of the process for the adult learners.

Approximately 70 people returned back the web survey, whose results essentially confirmed the outputs of the focus groups.

The next three present the web questionnaire.

pages survey

1

13

VINTAGE PILOT RUN 1 WEB SURVEY 1 . P le ase pro vi de u s wi th s ome i nfo rm atio n abo ut yo u rsel f: Male Female Age: <36 Age: 36 - 50 Age: >50 Profession: Teacher Profession: Trainer Profession: Researcher/Expert/Professor Profession: Manager/Decision Maker Profession: Career guidance/ Consultant Comments:

2 . O n the ba sis o f th e in fo rmatio n mate rial s o n th e Vin tage se lf-e val uati on pro ce du re , wh at d o yo u thi nk are its ke y stren gth s?

1. Clear instructions of the self-evaluation procedure 2. Description and choice of 5 Mastery Level.

3. Evoking capability of the contextualised situations.

4. Reflection and description of one's behaviour performed in similar situations.

5. Analysis and evaluation of one's performed behaviour on the basis of 10 qualities.

2

VINTAGE Project - www.vintage.euproject.org

3 . O n th e basi s o f the i nfo rmatio n materi als on th e Vi ntag e se lf- e val uati on p ro ce d ure , wh at d o yo u thi nk are its we akn esse s?

1. Clear instructions of the self-evaluation procedure 2. Description and choice of 5 Mastery Level.

3. Evoking capability of the contextualised situations.

4. Reflection and description of one's behaviour performed in similar situations.

5. Analysis and evaluation of one's performed behaviour on the basis of 10 qualities.

4 . Do you thi nk the se lf -e val uati on pro ces s o n eac h o f the 8 KC p rop ose d b y th e V in tage pro je ct ca n b e u sef u l an d sign if ic ant in adu lt ed u catio n?

Yes, I do.

No, I don't.

Comments

5 . D id you try to e xpe rim en t yo urse lf th e Vin tage se lf -e val uati on pro ce du re o n on e of th e K ey C om pe ten ce s?

No, I didn't try.

Yes, I tried to experiment KC2 "Communication in foreign languages" Yes, I tried to experiment KC4 "Digital competence"

6 . Th e fol lo win g qu esti on s are on ly fo r th e partici pan ts wh o trie d to e xpe rie nc e the Vi ntag e se lf -e val uati on p roce d ure ( all oth e r parti cipan ts can pro ce ed to th e e nd o f the su rve y wi th o ut answ eri ng th em) : Di d yo u id e nti fy yo urse lf e asil y wi th o n e of th e prop os ed M aste ry Le ve ls? very easily somewhat slightly not at all Comments

3

15

7 . Are the co nte xtu ali sed situ atio ns su ffi cie ntl y e v ocati ve to re call to o ne 's me mo ry a pe rfo me d be ha vio u r in a re al li fe situ atio n ? very much somewhat slightly not at all Comments

8 . In yo ur o pi ni on , th e op erati on o f re -e nac tmen t/ reco n structi on o f a pe rfo rme d be h avi ou r, was it e asy to carry o u t o r did it req ui re an e xc essi ve effo rt of abstarctio n an d re co n textu ali satio n ? very easy somewhat easy somewhat difficult very difficult

9 . Was it easy to e val uate yo u r pe rforme d b eh avi or o n th e basi s o f th e te n qu ali tie s pro po se d b y the V in tage pro je ct? very easy somewhat easy somewhat difficult very difficult Comments

1 0. C on sid e rin g yo u r ad ul t l earn e rs, do yo u th in k th ey wo ul d be able to fol lo w th e in stru ctio ns pro vid e d an d co mpl ete th e pro ced u re i n an auto n omo u s way?

Yes, I do.

No, I don't.

Comments

1 2 3

VINTAGE Project - www.vintage.euproject.org

CONCLUSIONS The VINTAGE project received an overall positive evaluation from the national focus groups and a confirmation of its basic ideas.

Its basic choices: analysis of the competence in terms of behaviours and context and the evaluation of such behaviours in terms of quality is widely shared among the participant.

High is also the consensus on the attempt carried out by the project to consider self-evaluation as a guided process of reflection whose protagonist is the adult person.

However, it is important to notice that such choices lead to a tool whose operating procedure is very complex and that requires the potential user a strong commitment in terms of time and attention / concentration. From this considerations originates the prevalent opinion expressed by the participants to the focus group realised in the countries of the partnership: the Vintage procedure, more than a tool to be used individually, can work as a useful and important methodological instrument to be used in the various typical contexts of adult

education (defining a personal development plan, (re-)entering the labour market, etc.). In particular, it would be useful in a training contexts involving small groups (one trainer and 3-4 learners).

From this observation comes the suggestion of producing a handbook/guide for teachers explaining how to use the tool, also in a training environment. Moreover, the focus groups provided some precious suggestions and recommendation for the implementation of the online tool aimed at overcoming (as much as possible) the complexity of the procedure: 1. In order to allow a wider and successful access to the VINTAGE tool, including also adults with low level of education or low skilled, it is necessary to reduce the lenght of texts, simplify the language and use images and illustrations.

These measures would allow a fuller understanding of the steps and actions required to the user undergoing the selfevaluation procedure.

For what concerns the description of performed behaviours in real life situations evoked through the suggested items, it would be necessary to provide a guide for the description of the situation and the reconstruction of the behaviours perfomed.

The user should be given

detailed information on the best ways to describe and analyse situations and behaviours.

This would make easier the self-evaluation by using the quality grid.

2. It would be advisable to reduce the 10 quality dimensions in relation to the chosen mastery level and the proposed item. In addition to this, it would be better to eliminate the titles of the quality dimensions while substituting them with control questions.

3. Finally, given the objective complexity of the procedure, the user should always have a clear perception of the progress achieved through the selfevaluation procedure. It would also be very important to give the users the possibility to leave the procedure at any given time and to reenter it in a later moment to complete the process from where they left.