Quiz results & responses not getting recorded

Analytics
1

Hi all,
I received the following query via email, so am posting here in case others are coming across a similar issue.

Quiz results and responses are not recorded for any of the CHAs from whom data was collected. I can see based on their activity tracker and points that these users completed the course, but no results show in the quiz_quizattempt and quiz_quizattemptresponse tables. As well for these users, on the cmedata dashboard, it will show quizzes completed “1/1” on the recent activity page, but then when you click “CHA Basics” for details, it shows “0 attempted, 0 passed quizzes” and no scores or attempts.

When I look at data from other users, I can see quiz results and response data is being recorded on the tables and is reflected in the dashboard. Why would the CHAs quiz scores and responses not be captured in the database? I thought given that other user quiz data is shown that maybe quizzes taken off-line are not being stored on the device, and so maybe only users who were connected to a network at the time they took the quiz have their results stored in the db?

First reply from me…

I’m not exactly sure what is going on here, until we’ve done some more investigation. However one possibility comes to mind…

The course installed on the device is different to the one that has been published on the Oppia server. For example, if the course is exported from Moodle and the course zip file is downloaded to ‘side-load’/pre-install on the device, but then the course (esp the quiz) has been edited in any way and then this version has been uploaded to the Oppia server.

Then the course quiz on the device will be different to the one on the server. The server won’t have any reference to the particular quiz and so can’t load in the quiz responses when the activity tracker log is uploaded.

The reason it may then show quizzes completed as 1/1 in one place and 0/0 in others is that it’s reading some information from the tracker table - so the tracker knows that the quiz has been completed (so shows 1/1), but when drilling down into the course activity it’s reading from the quiz attempt table, but of course there’s nothing there to read from.

Another option of course is that there is a bug in the activity log file upload.

We’ll look more into this and post any results or ideas up here,
Cheers,
Alex

2

To follow up, this shouldn’t be anything to do with the user being on/offline when the quiz was taken - even when offline the quiz attempt info is recorded and submitted to the server when a connection is available (or of course exported via the activity log file/s).

3

After a bit more investigation on this, it seems the reason that the quiz results were not being recorded was that the quiz id number in the Oppia server was different to that in the course that had been installed on the devices.

There are a couple of reasons why this might have happened:

  • the course (specifically the quiz) was updated and republished after it had been deployed onto the devices, so then the id numbers don’t match
  • the course zip file used to deploy onto the devices was one that had been published to a different server

This is probably an issue that is most likely to occur when the courses are side-loaded/pre-installed using a zip file that’s copied onto the device, rather than being downloaded directly from within the app.

I’m not sure there’s an absolute way to fix this (well not without removing functionality such being able to download and test courses before publishing etc), but some possibilities to help manage this include:

  • When a course is republished to the Oppia server, keep a database log table of the old/new activities (see: https://github.com/DigitalCampus/django-oppia/issues/441)
  • Adding logging in the Moodle block for when a course has been exported/re-published
  • Being able to ‘flag’ a course on the Oppia server as being the deployed version, then prevent it being republished without some extra confirmation

These are just some suggestions, and would welcome any other ideas to help with managing this,

Cheers,
Alex