Experiment design in education: measuring more than report grades

What exactly does a school report say about a child's situation? Admittedly, if there is anywhere where they have mastered the art of measurement, it is education.

‍

From your fourth to your eighteenth year of life, as a child, you can closely monitor your progress down to almost the decimal point. There is even consensus in the Netherlands on what is 'good' and what is 'insufficient', so you can also compare all students with each other. It's handy if you want to measure how everyone is doing individually; look at their grades compared to the standard, and you have an idea. 

But what if something big and unexpected disrupts your view (a pandemic, to name but one)? Then you want to measure more than numbers. In comes experiment design.

Check your assumptions

As an Experiment Designer, you constantly work with risky assumptions. Insights that form the foundation of a new initiative but have not been proven yet. Take a start-up with a new idea for a fitness app, for example. They make the assumption that users will find the app distinctive and, therefore, be willing to pay for it. Through experiment design, you look for evidence for these assumptions.

In education, you find these kinds of assumptions, too. 'The report cards reflect well on my students', for example. Or, 'There is no disadvantage caused by hybrid education.' These assumptions may well be true, but you want to prove that first. Because only then will you know if help is needed, and where exactly.

Experiment design in education: the practice

How do you go about searching for that evidence? First, you critically examine what you already know: the numbers. Continuously asking questions is the key here: what was it like for the teacher to teach? Were the tests easier? What happened to the standard? How did this child perform before home schooling became the norm? Has all the material been covered?

‍

Maybe a 6.5 for history this year is comparable to last year's 5.5, or vice versa. That makes quite a difference. 

You can come up with hundreds of questions for this theme, but they all focus on adding context to the 'bare' figures. With this additional perspective, your understanding becomes much sharper, but you can also clearly see what information is still missing.

Search for missing information

Perhaps the assumption has already been proven with this new perspective. However, often not everything important for a student has been measured. Consider, for example, group cohesion, concentration ability, reflective capacity, or the skills to plan homework.

‍

There are no grades for this. Perhaps these skills are not even measurable, but they can still convey a lot about the situation.

With a quick experiment or interview, you can already learn a lot. For example, have students write a review about their own learning process, or ask teachers which students were easily distracted behind the webcam. This won't provide you with standardized grades and norms, but it will give you an indication of whether there is a need for assistance in that area.

‍

Even in subjects that are difficult to measure, such as how someone is doing in the class or their mood. A noticeable, more ambiguous indication is more valuable than nothing at all. The picture is never entirely complete, but any additional information provides perspective!

urn the student into an experiment designer

Finally, respond to the needs you've spotted. A single student who is behind in math can simply be tutored. These methods are proven and not very exciting. The situation changes when you have 600 students who are behind in math. This requires a broader and more structural solution.

‍

Furthermore, you can't simply 'order' a concentration or group spirit tutoring. This requires something new.

Therefore, allow the student to shape the solution themselves. The best ideas often come from them. Ownership of their own learning process also fosters more enthusiasm and engagement.

‍

Combine this with the knowledge and expertise of their teachers and test the ideas on a small scale before implementing them. This not only provides internal solutions for identified gaps but also fosters a habit of promoting new initiatives within the school.

‍

And those gains are not quantifiable.

‍

Want to see how this works in practice? Check out the case about our collaboration with the Haags Montessori Lyceum.

‍