TODO.qmd

## TODO - Monday AM

- [x] from SQL get POS sizes for programmes
- [x] delete entire graph database
- [x] delete I400 files
- [x] new ipynb page for final load
- [x] screencapture I400 again
- [x] run extract/process for each programme in list
- [x] re-render html and pdf
- [x] review new html/pdf 
- [x] research graph explore/query
- [x] copy all sections so far to word doc - 6500 words
- [x] get word count - 6500 words
- [x] save all code as gists - new page

- [x] load remaining programmes to gdrive
- [x] load remaining programmes to neo4j

- [x] send/share zip with xiaodong
- [x] fix code gists page
- [x] techstack page
- [x] update pipe files - comments, etc.

- [x] copy all cypher queries to page and annotate
- [x] cypher query - QMD page
  - [x] cypher - clash QMD page
  - [x] annotate and tidy up cypher query pages (spaces, etc.)
  

- [x] dedication, thank you page - family colleagues supervisors
- [x] supervisor notes page
- [x] check other requirements and add

**Tuesday**


- [x] draft tte quality section -> see notes and cypher query section
- [x] draft conclusion section 
- [x] update references page 
- [x] add load times to somewhre
- [x] 2x word docs - 1x complete, 1x without headers, code, images, references, etc. (word count to be subtracted)
- [x] read ttq section
- [x] read and rewrite conclusion sections
- [x] fix soft constraints - explanation
- [x] appendix - rooms
- [x] reword bluesky oppos
- [x] reword perspectives
- [x] link bluesky and perspective to yaml
- [x] write absract
- [x] link bluesky and perspectives to main page 
- [x] check front page - vids, links

- [ ] review conclusion after render









- [ ] word todo list
  - [x] add Appendix or prefix to appendix titles, e.g. A: Random Graph, B: Tech Stack
  - [x] add page numbers
  - [x] fix footnotes
  - [x] add word count break down
  - [x] resize images
  - [ ] add alt text to images?
  - [x] fix reference page
  - [ ] add graph to summary page -> the journey?
  - [x] fix links within doc



- [ ] add cypher query insights page -> future questions which can be answered

- [ ] update intro and objectives - three goals, sql-graph ETL, open door to insights, spec how timetable 'quality' can be formalised







**later**

- perspectives:
  - programme leader
  - timetabling manager - use me as example with ttquality queries
  - adminstrator - room utilisation

- [ ] rewrite time model page with new images
- [ ] load demo archibus data
  - [ ] room cypher page
  - [ ] reference tt quality penalty

- [ ] rerender html
- [ ] redo quarto for pdf and word
- [ ] render word and pdf
  - [ ] fix word doc
  - [ ] fix pdf doc
- 
- [ ] README page - explaining html (index), word, pdf versions
  - [ ] html folder
  - [ ] docs folder for word and pdf
  - [ ] README in root explaining how this works -> designed to be viewed as a website (open ./html/index.html) -> also available as a word or pdf doc.  Github repo available here.  Any problems, contact me.

**WEDNESDAY**

- loom video

  



Learning OutcomesLearning Outcomes
The learning outcomes for the project are set out below. When you have completed the module, you should be able to:
Identify a research problem and scope research to suitably investigate it
Synthesise and critically evaluate recent research having a direct bearing on the problem space
Design and implement a technology solution or practical/analytical investigation to a high standard, with aspects of novelty and personal innovation.
Select and apply appropriate and contemporary best-of-breed tools, techniques and theoretical models
Critically and reflexively evaluate the outcome and personal performance to standards of research quality (validity, reliability) and professional competence.
Embody an ethical approach to research conduct in a specialist area in computer science
Communicate research outcomes and learning effectively to specialist and non-specialist audiences


Criterion
Percent
Poor/Fail (<50)
Good (50-59)
Very Good / Merit (60-69)
Excellent / Distinction (70+)
Rationale, Scope and Objectives
5

Very clear, SMART and research-based; convincing rationale.  Well addressed aims and objectives.
Novelty and Innovation
10

Highly innovative / creative. Substantial personal interpretation of the material and methods.  Project is relevant, up to date and using current techniques / Excellent demonstration of problem-solving skills.
Literature Review and Academic Content
20
Relevant, state-of-the-art literature understood and critically reviewed; Exceptional breadth & depth of academic content, understood & applied
Design and Approach & User requirements
15
Clear use of theory and appropriate process methodologies (e.g. test-driven design / reproducibility / model validation); highly suited to research objectives / Extensive, evidence-based user needs assessment
Implementation
15
Interface or analytical outputs clear, sophisticated and error free; objectives fully met or surpassed
Ethical Consideration and Approach
10

Clear and well-informed consideration of ethical issues with demonstrated impact on project approach and outputs
Evaluation and Reflection, evidence of planning and management
15

Structured, rigorous evaluation; Deep and honest reflection.  Excellent use made of feedback and supervision meetings; clear records kept and evidence supplied.
Presentation, Organisation, Documentation & Attribution
10

Logical layout of sections and narrative; Grammar and spelling correct; Accurate and complete referencing; Clear commenting of own code; Clear attribution of others' code

Viva
 	 	 	 	 
Criterion
Percent
Poor/Fail (<50)
Good (50-59)
Very Good / Merit (60-69)
Excellent / Distinction (70+)
Ability to demonstrate the artifact
50

Excellent demonstration of the artifact, discussion of issues and knowledge of the subject area.
Able to answer questions and discuss the project
50

Excellent understanding of the topic area; clear and nuanced awareness of the scope and limitations of the project outcomes


- [ ]use Loom for video - embed. 
  - a long time ago, i worked as a timetabler.  it was challenging, rewarding, frustrating, interesting, sometimes at the same time. 
  - timetabling is a vry complex university function which involves requirements gathering, data structuring, rules and constraints, solutionising, optimising against objectives...
  - only to release a timetable and wait for the complaints.  
  - the reality is that no news is good news, but you will hear about the things which are not good. 
  - and this is right.  the timetable is central to students' and staff experience at university.  it dictates where they need to be when, doing what.  when it is not favourable, you will know.
  - but what is a good timetable?  how do you know? 
  - this is at the heart of this project.  
  - university timetables are traditionally stored in highly structured, relational databases. 
  - given the highly relational nature of timetabling data - students attend classes, staff teach activities, activities belong to modules, modules contribute to programmes, activities take place in rooms, rooms are located in buildings, etc. 
  - I hypothesise that graph databases could answer timetabling use cases, in particular around gaining insight into timetable quality.
- my project has three parts:
  - 1. it is a data engineering prject - I have built an end-to-end ETL which extracts, transforms and loads timetabling data from SQL to Neo4j graph.  this involves modeling and documenting graph data structures suitable for timetables.
  - 2. this opens the door to exploring timetables using cypher querying language.  I have created proof-of-concept queries which query the database and provide insights to different personas - academic, planning, administrative staff.  I develop queries which identify where timetabling constraints have been violated such as - clashes, where a room is double booked, or maximum consecutive hours, or long gaps between activities or no lunch break.
  - 3. and i discuss how these could be used to derive a timetable quality score - an independent score based on penalties which takes the emotion out of timetables and provides a measurable, comparable quality index which can be used to compare individual or cohort timetables, or even whole year timebables.
  - This project is foundational and intended to be built on - it demonstrates the potential of restructuring timetables into graph and discovering insight. 

- [ ] perspectives and insights


- [ ] add word count to front page
- [ ] video to front page - and video script
- [ ] example supervisor notes

- [ ] delete csv from online repo
- [ ] check quarto repo for security issues


## Cypher


- [ ] project video - opening doors and peaking through







- [ ] explore graph 
  - [ ] write down interesting questions
  - [ ] screencapture interesting insights




## to check

- [ ] PDf version links
- [ ] make github public (graph-tt-quarto)
- [ ] add word count


## Later

- [ ] clipchamp for video
- [ ] quarto for presentation?
- [ ] script for video - combination of presentation slides, demo and neo4j capabilities - leave on cliffhanger



## overall




* trim references page
* write timetable-metrics section
* add end-to-end example for I400 -> appendix
* copy all cypher queries to a page
* presentation / appendix
  * show slicing/range example
  

### word/pdf doc
* add front page instruction about viewing as a website
* Put page 1 in front of Toc
* refresh toc





# 2 minute video

* fix references page not rendering from bib
* review learning outcomes on mod page 
* plan 2 min elevator pitch video:
  * timetables - a necessary evil
  * tell us where to go, why
  * but they are the source of complaints, they are never write, they cause issues, they're wrong.
  * they are complex to produce, difficult to collect data for, difficult to optimise
  * you can never please everyone, if you please anyone you are probably winning
  * but how do we know what is going on - what is good, what is the case
  * reporting in timetabling is challenging - you can count hard/soft constraint violations - examples
  * you can visually inspect a timetable - that looks good - that looks better than that - but why?
  * can it be quantified, can it be scored, can we move away from anecdotal conversations/evidence and individual views and into a more evidence based approach.
  * this project isn't about solving this problem - it's a big one - but it is about taking steps towards that possibility.
  * Timetabling - it is complex - it is relational - it has people (studnets, staff, instructors), resources (rooms, equipment), times (dates, hours, durations), it has curriculum requirements, it has rules and various outcomes to optimise.
  * currently timetables are structured and scheduled using relational databases - that is data organised into tables with relationship tables, primary keys, ids, etc.