How Iconomix evaluates its units

The mission of Iconomix is to support economic and social studies education in Switzerland with an experience-oriented approach. The aim is to promote economic and financial literacy as a basic life skill.
To this end, Iconomix provides teaching units in four languages (D/F/I/E), which are available to teachers on a web platform. Specifically, Iconomix targets the more than 4,200 teachers in Switzerland who teach economics and social studies at the upper secondary level (vocational schools, upper secondary specialized schools and baccalaureate schools).

Iconomix is therefore not aimed directly at the learners, but at their teachers (as multipliers and gatekeepers). The advantage of this indirect approach is that, unlike the student body, the teachers remain quite stable over time. A disadvantage may be that Iconomix has no control over how the offer is used. 

However, we do not see this as a shortcoming since teaching is the core business of teachers, which they are trained and specialized for. Iconomix does not wish to interfere here but to support teachers in doing their best work (empowerment). Iconomix also sees itself as a complementary teaching and learning tool that teachers can use alongside existing teaching materials.

In addition, an Iconomix unit is hardly ever used by a teacher without adjustments in the classroom. Rather, a teacher has to make an initially unknown teaching unit her own - a demanding and sometimes time-consuming process ("fitting"). Ideally, this results in an individualized teaching unit that the teacher can use repeatedly over time.

Since slightly more than 90% of all young adults in Switzerland go through upper secondary education, the population of learners more or less reflects the heterogeneity of the population at large. In other words, there are big differences between learners in terms of their cognitive abilities, prior knowledge and background. Moreover, Iconomix is present in three language regions of Switzerland, and these regions differ not only in terms of language but also in terms of culture. This heterogeneity is reflected in the great diversity of the teaching units of Iconomix in terms of topics, didactic approaches and levels of difficulty. 

A serious impact analysis of the entire, diverse offer of Iconomix on the heterogeneous target group of teachers and learners at the Swiss upper secondary level would almost only be possible within the framework of a large-scale National Fund study. This may one day be the case. For the moment, however, we concentrate on the evaluation of the individual teaching units:
Each Iconomix teaching unit is evaluated either in the course of its development or in the course of its revision. However, we refrain from an actual determination of the "effectiveness" of the teaching units. The reason is that the teaching units are individually adapted by the teachers to fit their own needs or those of the class. It is hence only of limited practical value if we invest a lot of time and effort in proving the effectiveness of an ideally used teaching unit. For this reason, we do not use randomized controlled trials.

Instead, we focus on formative evaluation as an integral part of the quality assurance process. The aim is to obtain feedback on a new teaching unit already during its development in order to support its introduction and further development. 
Depending on the case, different methods are used:

• For more complex teaching units, we conduct practice tests. In a typical practice test, we support individual teachers in the use of the "beta version" of a new teaching unit in class. This is followed by a short interview or survey with the teacher on their experience with the teaching unit and a feedback survey among the learners. We are interested in questions such as: Does the new teaching unit promote the learning process? How is the relation between effort and learning gain? Are there any obstacles or pitfalls in lesson preparation or implementation? This allows us to identify and address problems that remained undiscovered during the development phase.
• Our digital educational media are reviewed by the independent evaluator Education Alliance Finland (EAF). This assessment compares the learners' activities with the learning objectives, the pedagogical quality and the engagement in learning. The result is a report and a certificate.
• Our worksheets – the actual core of a teaching unit – are developed in close collaboration with renowned didactics experts from Swiss universities. We receive detailed feedback on drafts for new worksheets, which are then implemented and re-evaluated. Over time, this process produces a collection of good practice examples that serve as guidelines.
• Not to be neglected is the feedback we receive from teachers after launching a new teaching unit. Feedback from committed teachers who share their experiences and insights with us helps us to continually improve a unit over its entire life cycle.

Iconomix currently works primarily with the following partners:

• Prof. Dr. Carmela Aprea, Chair of Economic and Business Education - Design and Evaluation of Instructional Systems, Faculty of Business Administration, University of Mannheim
• Dr. Daniel Schmuki, lecturer and didactics expert (for the subjects economics, law and politics), Swiss Federal University for Vocational Education and Training