Motion Math Blog

Landmark study: Motion Math improves fractions knowledge, attitudes

[Updated: the study was recently accepted for publication after peer review by the Games and Culture journal.]

Very exciting news: today GameDesk has published the first experimental research on iPad learning. Professor Michelle Riconscente, an expert in educational technology and assessment at USC, studied 122 5th graders playing our iPad game Motion Math HD. The main findings:

• Students who played the game for 20 minutes for five days improved on a fractions test by an average of 15%. (The items on the test were taken from national and international standardized tests.)
• Students’ attitudes towards fractions improved 10%.
• Virtually all students rated the game as fun and that it helped them learn.

This is a very encouraging result that our engaged approach to math works. Read the full report, which includes an overview of how difficult it is for students to learn fractions, a description of the study’s rigorous design, some critiques of our game’s design, and recommendations for learning app producers and researchers.

What do you think is the best design and assessment of children’s apps? What do you think about the study? (please comment below) A few early reviewers have suggested to us that this study is overkill. Well, certainly not every kid’s app can afford the time and money it takes for a formal assessment. When designing an app, there are hundreds of design decisions to make, and certainly most of them can’t be guided by rigorous experimental data. A designer can follow his or her intuition, or listen to experts, or read about elements of good design, or read the learning literature, or, hopefully, do lots of user-testing to watch and see if students seem to be learning.

However, the history of educational technology is littered with many false promises and disappointing results, most recently given an overview by the NYT’s Matt Richtel. It’s easy to think, as you’re creating a learning game and as you watch a student use technology, that learning is happening. So it’s important that we sometimes hold our products up to scientific scrutiny.

We’re hoping this study raises the bar for learning apps, and encourages more studies, so that when parents and teachers buy our products, they can be confident they are getting a product that actually helps children learn. Read the study here! And then please comment below: What should the standards of evidence be for learning apps? How do you know if a learning app works?

7 Responses to “Landmark study: Motion Math improves fractions knowledge, attitudes”

  1. I haven’t read the full report – only your blog entry, gamedesk summary, and NYTimes article you refer to. But to answer your question about what standards of evidence should we use…IMO I hate the idea that effectiveness is solely tied to raising standardized test scores. It could be one criteria but not the only one.

    I was first drawn to ed tech in the early 90’s because it promised wider access to content to populations that wouldn’t have access. The internet and in particular online learning didn’t necessarily measure up to that promise but it was enough for me to get really excited about it. With the ipad and touch-based technology PLUS your visual approach to math, kids with LD, fine motor delays and ELL are able to ACCESS content in ways that haven’t been accessible to them before…or at least a better way (inclusive) to access content. A UDL model for effectiveness is more to my liking.

    The nice thing about gaming is that mastery can be built into the learning app by incorporating levels into the design. Once you reach level 9 (or save the city from doom and gloom…complete the challenge) then a student is considered having mastered the concept. The quirky thing about this idea is that students may not be able to transfer the skills or concepts offline. For my daughter just because she’s mastered the skill or concept with the iPad game doesn’t mean full mastery in the real world. But it does mean she’s well on her way and has been given a good foundation to progress even further. Many apps are like a sandbox for her to practice and gain confidence and confidence is something that can’t be tested BUT is absolutely necessary for the learning formula.

    Soooo happy you guys are working with GameDesk. I’ve never heard of them before now. I love MotionMath Zoom. I also love the Mind Research Institute – you guys remind me of them. I am sold on the visual and game-based approach to math and look forward to playing more of your games in the future.

  2. Thanks Anna for your response.

    I completely agree that content test scores are not the only criteria worth studying. That’s why our study also tested “fractions attitudes”, including measuring confidence. Can confidence (or “self-efficacy”) be perfectly measured? Definitely not; but you can approximate it with questions such as those in the study: “I am good at fractions”; “I can solve hard fractions problems”, etc. Students improved 10% in their attitude towards fractions as well after using our game – that’s one area where games should have an real advantage over other methods of teaching.

    Glad you’re sold on the visual and game-based approach –  we are too, but also want to have a healthy skepticism. The study gives us confidence that we’re on the right track, and that when a player reaches level 9 (as you say) that also means they’re gaining mastery that will transfer to outside the game.

  3. Great job on this guys! I am very glad you are taking the effort to submit Motion Math to some scientific scrutiny. I have been touting Motion Math as a great tool for students for a long time now, but I have often wondered if learning from apps could be measured in a study. Its great to see that our intuitions about these things can be backed up by data. I certainly hope you set an example for others in our field to follow, and I hope that as developers we strive to always back up our claims and work towards truly educational products for children. I would like to do the same for Math Evolve when I get the means and time to do so.

    Jacob, its unfortunate we didn’t get a chance to talk at the MWA workshop. I was looking forward to connecting, but I think I managed to meet everyone there except you. I’m looking forward to the new game, I am sure it will be as good as the others.

    -Adam Coccari

  4. […] Math’s GameDesk study on iPad learning – check it […]

  5. Fractions is probably the most hated word amongst mathematics students of all ages these days. I am not sure of the reasons why but I salute any attempt to make it easier for students to grasp. Fantastic article and report. Well done and thank you.

  6. […] Math] for 20 minutes for five days improved on a fractions test by an average of 15%.”  (link).  Pearson offers a free, generic framework (link) and many other similar resources […]

  7. i love motion math games!

Leave a Reply