03-06a-load.qmd

---
title: "Google Load"
---

The free cloud instance of Neo4j (Aura) requires that csv files are stored in [public cloud storage](https://neo4j.com/docs/aura/aurads/importing-data/load-csv/) like Google Drive or Dropbox.

Therefore, my project requires an intermediary step.

```{dot}
digraph google_drive_storage {
    //  top to bottom
    rankdir = TD;
    nodesep = 1.0;
    splines = false;
    
    // node styles
    node [shape=box, style="filled,rounded", fillcolor="#f0f0f0", fontname="Arial"];
    edge [fontname="Arial"];
    
    // nodes
    source_files [label="Local Files\n(./{hostkeys}/process)", shape=folder, fillcolor="#ffdead"];
    gdrive_api [label="Google Drive API", shape=component, fillcolor="#ffffcc"];
    get_folder [label="Get Folder Details"];
    create_folder [label="Create Folder if Needed"];
    upload_nodes [label="Upload Files\nMatching Node Pattern\nto './{hostkeys}/node'", shape=folder, fillcolor="#ffdead"];
    upload_relationships [label="Upload Files\nMatching Relationship Pattern\nto './{hostkeys}/relationships'", shape=folder, fillcolor="#ffdead"];
    process_done [label="Process Complete", shape=ellipse, fillcolor="#e0ffe0"];
    
    // edges and labels
    source_files -> gdrive_api [label="Connect via API"];
    gdrive_api -> get_folder;
    get_folder -> create_folder;
    create_folder -> upload_nodes;
    create_folder -> upload_relationships;
    upload_nodes -> process_done;
    upload_relationships -> process_done;
}

```

File storage and directories are controlled via Config. I created a publicly shared folder in Google drive which contains all project csvs:

```
- root: Google Drive folder
  - hostkeys (automatically created, unless override) 
    - nodes
    - relationships
```

![Screenshot of Google Drive folder and files for MSc Data Science (INB112): note that files were created by Google API](./images/google-drive-node.png)