process all courses' assignments; misc. other cleaning up (iss. #6, #7, #8, #10)

[lsloan]

Resolves #6
Resolves #7
Resolves #8
Resolves #10

Test Plan

1. Follow the steps in README.md.

2. When configuring the .env file, set it up with Canvas-test and use the following for course IDs:

   • COURSE_IDS_CSV=545451
     Uses "ECON 101 200 FA 2022", which contains three peer-reviewed assignments.
   • COURSE_IDS_CSV=200,545451,461474
     Add other courses, which may or may not include peer-reviewed assignments.

[lsloan]

Implementing #6 is more complicated than I expected. Trying to save peer reviews for some of the other assignments in the test course is revealing more abnormalities in the Canvas data than expected.

[lsloan]

@ssciolla, @zqian: If you get a moment to review this PR again, I'd appreciate it. It's probably good enough for an approval now, as I've created issues for some of the points that were raised earlier. I could merge it tomorrow. (I'm away today.)

[ssciolla]

@lsloan, hi, sorry, it's been a busy week. I am off tomorrow. I probably won't be able to finish reviewing this until Monday. I'll get to it as soon as I can.

[ssciolla]

Sorry for the delay, @lsloan. This is a pretty meaty pull request, and I hadn't really seen this code before, so I basically felt like I was reviewing the whole project.

I'm approving, with some comments for further thought or future work. It would at least be good to see the GitHub Action fixed or removed before you merge. I don't know with a 100 percent certainty that this is working exactly as intended, but it did collect data, and in the poking around I did, things made sense. Thus I think it's okay to merge and work iteratively on improvements. Looks like some type hints could be tightened up, but I understand it's difficult in some cases since canvaspi doesn't seem to have them.

Couple more general suggestions:

1. You may want to add a knob for changing the level of canvasapi log messages, since they can be really noisy too like Django's. Something like this should work in settings.py:

  'loggers': {
      'django': {
          'handlers': ['console'],
          'level': os.getenv('LOG_LEVEL', 'INFO').upper(),
          'propagate': False
      },
      'canvasapi': {
          'handlers': ['console'],
          'level': os.getenv('CANVAS_API_LOG_LEVEL', 'WARN').upper()
      }
  }

2. Part of the difficulty in reviewing this is not knowing how to test the results. To what do I compare the data? What queries ensure that we can do the analysis we need later? I'm not sure what kind of tests this application needs, but selective unit tests and/or the ability to programmatically run certain queries could be useful. Here are some queries I ended up writing to so some of this work (in case they're helpful):

-- Courses, assignments, and submissions
select *
from course c
join assignment a
	on a.course_id=c.id
join submission s
	on s.assignment_id=a.id;

-- Assessor gave to submitter these comments about their work in this course
select
	assessor.login_id as `assessor_login_id`,
	submitter.login_id as `submitter_login_id`,
	cm.comments,
	s.id as `submission_id`
from course c
join assignment a
	on a.course_id=c.id
join submission s
	on s.assignment_id=a.id
join `user` submitter
	on s.user_id=submitter.id
join assessment am
	on am.submission_id=s.id
join `user` assessor 
	on am.assessor_id=assessor.id
join comment cm
	on cm.assessment_id=am.id
where c.id='XXXXX';

-- count peer review comments given by each student in a course
select
	assessor.login_id as `assessor_login_id`,
	count(*)
from course c
join assignment a
	on a.course_id=c.id
join submission s
	on s.assignment_id=a.id
join `user` as submitter
	on s.user_id=submitter.id
join assessment am
	on am.submission_id=s.id
join `user` assessor 
	on am.assessor_id=assessor.id
join comment cm
	on cm.assessment_id=am.id
where c.id='XXXXX'
group by assessor.id;

3. This process seemed to take about 2 and a half minutes for the large course you suggested we look at. I'm not sure how much we would/will scale this process, but it's possible we reach a point where performance is an issue. Adopting some async/await or doing bulk database actions could improve the time to complete. Maybe the async becomes less relevant if we switch to Canvas Data 2 queries, but something like Django's bulk_create would drastically reduce the number of database calls currently made.

[lsloan]

@ssciolla, thanks for the approval and further comments. I'll review them closely tomorrow morning before merging.

[lsloan]

@ssciolla, thanks again for the thoughtful, thorough code review. I really appreciate it. I understand the delay.

Yes, this PR has a lot going on. The earlier PRs were mostly about getting the base application going. Zhen had me pause progress on this project to work on deploying the Longhorn Open peer grading tool. So when I was able to resume work here, I've been trying to get as much done as possible and quickly before something else comes along to interrupt me.

Addressing the general suggestions in your top-of-review message:

1. Yes, I am dissatisfied with the current logging config. I think what you've suggested for differing general and canvasapi log levels may serve me well for handling enable separate log levels for app, framework, canvasapi #9. I wasn't sure how to set log levels for just parts of the application. I'll add this info to that issue.
2. Thanks for the SQL queries. I had it on my to-do list to make some. I should've had them in the test plan for this PR and added to the documentation. I'll be sure to do the latter.
   This data will be used by the MWrite AI analysis dashboard. That dashboard already runs using the MPR DB. This DB has a similar structure, so I think the dashboard's query, maybe somewhat modified, will be what we should use here. Your queries are similar to that one.
3. I'll make a new issue about improving performance, including a mention of bulk_create.