Hi @alex
In the release notes you mentioned "
For the downloading of time spent in courses (#OPPIA-702), the update_summaries command will need to be run from the start - see the upgrading notes
"
Does that simply mean running #python manage update_summaries ?
Hi @hakimks ,
You’ll need to run with the --fromstart parameter too, ie. #python manage update_summaries --fromstart
It needs to do a full rebuild of the cache tables because of changes in how the dailyactiveuser cache tables divides up the data - as now it stores per course and user.
It can take a long time to run if there are a lot of tracker records, so you can run it in the background, using:
nohup python manage.py update_summaries --fromstart &
Then you can check the output/progress by viewing the nohup.out file
Cheers,
Alex
1 Like
I managed to successfully run that on the staging server. How do you access the time spend via the dashboard??
Hi @hakimks ,
It’s on the analytics/reports page… <oppia_url>/reports/
There are 3…
- total time spent - shows graph of total time spent per day (by all users)
- average time spent - shows graph or average time spent per user per day
- time spent by user and course - a csv download of the daily time spent, broken down by course and user
Cheers,
Alex
The update_summaries command run successfully but the graphs and csv file are all empty (at zero). I don’t know what might have gone wrong…
Hi @hakimks
I just had a quick look at this, and hope you don;t mind that I checked the Mesau staging server directly.
When I run the update_summaries from my local machine, the output is like this:
Updating DAUs for 2018-01-08 00:00:00+00:00 - Non-Communicable Diseases, Emergency Care and Mental Health Part 1
added staff
Updating DAUs for 2018-05-07 00:00:00+00:00 - Non-Communicable Diseases, Emergency Care and Mental Health Part 1
added demo
added staff
Updating DAUs for 2018-06-09 00:00:00+00:00 - Non-Communicable Diseases, Emergency Care and Mental Health Part 1
Reference course 1
but when I run from the mesau staging server , the output is like this:
Updating DAUs for 2021-02-09 00:00:00+03:00 - Testing Quiz
Updating DAUs for 2021-02-22 00:00:00+03:00 - Testing Quiz
Updating DAUs for 2021-04-07 00:00:00+03:00 - Testing Quiz
Updating DAUs for 2021-06-09 00:00:00+03:00 - Testing Quiz
So it seems that it’s not finding any of the users for that day/course. But looking closer, there’s also the timezone adjustment (+03:00), so I think this might be causing it not match to any users.
If you’re able to send me a (anonymised) dump of the staging mysql db, then I can try running on my local machine and see if the same issue comes up or not, and will also help me trying to fix it.
Cheers,
Alex
Hi @hakimks , actually there’s no need for you to send on a db dump, I can get one of the backup files, anonymise this and use for local testing.
hmmm… whilst trying to restore a backup on my local machine (from one of the backups from the mesau site), I got this error:
Unknown collation: utf8mb4_0900_ai_ci
I don;t think this will have anything to do with the issue you;re having, but likely would be something good to get fixed up at some point.
How I fixed this on my local machine was to (some extra steps here for when you’re doing to the actual db):
-
take site offline (eg stop apache), just so no new data comes in that then doesn;t get stored
-
dump db to (sql) plain text file (data.sql), alternatively do another backup using the backup.sh script, then extract the most recent .gz file to data.sql,
-
then:
sed -i 's/utf8mb4_0900_ai_ci/utf8mb4_general_ci/g' data.sql -
This will replace all the instances of the ‘odd’ collation in the db dump
-
Go into mysql command line and either (a) drop all the tables, or (b) drop and recreate the db (but in this instance will likely need to add back the db permissions)
-
Then load up the altered db file back to the database, eg
mysql$ source /path/to/data.sql -
Then you can start up apache again
Anyway, this is just for reference later on, and for sure best to test on your local machine or the staging server first!
And I’ll get back to looking at the actual issue you raised!
Alex
Hi again @hakimks
I think I have this figured out now…
On the servers I’m running (and on my local machine) I’m using the UTC timezone in the django settings. Of course, you have it set to Africa/Kampala to match your local timezone.
I realised that the update_summaries command for the daily active users (and time spent) was using TruncDay in the django model filtering, but this results in including the timezone info in the day (eg 2021-02-09 00:00:00+03:00), and is why (I think/presume) that nothing got matched when trying to filter by the date shown in the tracker table.
When I switched to using TruncDate, this just uses the day (eg 2021-02-09), and appears to be working now for whatever timezone I have in my django settings.
I’d like to do bit more testing/checking on this first, but should be able to push up the fix sometime tomorrow, and will let you know when I’ve pushed it.
Cheers,
Alex
Hi Alex
Thanks for all the replies:-
For the database restoration, I actually faced that same issue, when I was trying to restore a backup of the live database to my local machine. I actually solved it my editing the backup.sql file using notepad++ and replacing utf8mb4_0900_ai_ci with utf8mb4_general_ci and it worked.
But I will try and fix that up on the server.
I will wait for you to push the fix for the update_summaries error.
HI @hakimks ,
I’ve made a push to the release-0-12-19 and master branches with the fix in for this issue. So you should be able to merge this in and test, but please let me know if you;re still having problems with this.
Cheers,
Alex
Thanks Alex, Its now working and showing up in the dashboard.
1 Like