|
1 | 1 | # taggr |
2 | 2 |
|
3 | | -Rediscover your **memories** while keeping your **privacy**. |
| 3 | +> Rediscover your **memories** while keeping your **privacy**. |
4 | 4 |
|
5 | | -Powered by machine learning. |
| 5 | +Powered by [TypeScript](https://www.typescriptlang.org/), [Electron](https://www.electronjs.org/), [React](https://reactjs.org/), [Redux](https://redux-toolkit.js.org/), [Node.js](https://nodejs.org/en/) and [TensorFlow.js](https://www.tensorflow.org/) 🚀 |
6 | 6 |
|
7 | | -## Architecture |
| 7 | + |
8 | 8 |
|
9 | | -`frontend` and `backend` are one. |
| 9 | +<p style="text-align: center;">👉 <a href="https://twitter.com/aperkaz">Keep up to date with my next side-projects</a> 👈</p> |
10 | 10 |
|
11 | | -**Modularized structure** for UI and backend, running on separated `BrowserWindow` processes: `renderer` for UI, `background` for backend. |
| 11 | +## Motivation |
12 | 12 |
|
13 | | -This allows to perform the long and resource intensive backend operations without blocking the UI thread (in development only). In production, the backend BrowserWindow is hidden. |
| 13 | +There is great software out there that provides image exploration capabilities using machine learning (Google Photos, iCloud), but generally is not build with privacy in mind. |
14 | 14 |
|
15 | | -**Message passing** interconnection betweeen modules, through [Broadcast Channel API](https://developer.mozilla.org/en-US/docs/Web/API/Broadcast_Channel_API). |
| 15 | +At the end of the day, you have to upload your pictures to a server (which perform the machine learning operations), so you have to trust a third party with your data. |
16 | 16 |
|
17 | | -Since the backend executes long running tasks, sync connections are not an option. The message passing acts as a the communication interfact between modules. Each module (FE/BE) implements a `message-handler`, which deal with the incomming (`message`) on a given topic. |
| 17 | +What if we could run image classification and tagging machine learning operations 100% locally? |
| 18 | +You dont have to trust a server if there is no server 😉 |
18 | 19 |
|
19 | | -Available messaging topics and action cretors are shared and centralized in `./shared/message-passing`. |
| 20 | +👇 _Under this premise, **taggr** was born_ 👇 |
20 | 21 |
|
21 | | -## Environments |
| 22 | +A photo explorer, which uses offline machine learning for enriched exploration. |
22 | 23 |
|
23 | | -3 app environments. `DEVELOP`, `BUILD_TEST` and `BUILD_PROD`. |
| 24 | +Build with privacy in mind, all the image processing is performed locally, and no data ever leaves you computer 😊 |
24 | 25 |
|
25 | | -Manually set the value in `src/shared/active-env.js`. |
| 26 | +## High-level architecture |
26 | 27 |
|
27 | | -## Publishing |
| 28 | +This is my first electron project, so I iterated multiple times until I settled on a general strucutere I was happy with (at developer experience and performance levels). |
28 | 29 |
|
29 | | -Run: |
| 30 | +In my case, I found the sweet spot by keeping as close to the web standard as possible, and leveraging the existing web / Node.js tooling that already exists. That mweant |
30 | 31 |
|
31 | | -```javascript |
32 | | -npm run publish |
33 | | -``` |
| 32 | +**taggr** is composed by two main modules (`frontend`, `backend`), a `shared` module, and a `communication bus`. |
| 33 | + |
| 34 | +The app is split into two distinct and independent processes, the `frontend` and the `backend` (mapping to the main modules), for the sake of separation of concerns. Each process runs in an [independent Electron process](https://blog.logrocket.com/advanced-electron-js-architecture/). |
| 35 | + |
| 36 | +### Message bus |
| 37 | + |
| 38 | +Before we cover the modules, lets discuss the communication layer. |
| 39 | + |
| 40 | +The `frontend` and `backend` modules communicate through an asynchonous and bidirectional message bus. |
| 41 | +Its implemented using Electron's [IPC module](https://www.electronjs.org/docs/latest/api/ipc-main/). |
| 42 | + |
| 43 | +The supported events are defined as types in the `shared` module, so type-safe handlers can be implemented in either side of the bus. For example in `frontend/src/message-bus/index.ts`. |
| 44 | + |
| 45 | +Since the message bus relies on the `BrowserWindow.id`, the Electron main process keeps a heartbeat with the render process ids. |
| 46 | + |
| 47 | +### Frontend → `./frontend` |
| 48 | + |
| 49 | +The 'face' of the app, this module takes care of all things UI. |
| 50 | + |
| 51 | +It does **not** hold business logic. It communicates with the `backend` for performing business logic operations (through the message bus). |
| 52 | + |
| 53 | +**Built with Typescript + React components**, following (loosely) the [Atomic Design Principles](https://bradfrost.com/blog/post/atomic-web-design/). I used [Storybook](https://storybook.js.org/) for that. |
| 54 | + |
| 55 | +The whole UI is **[controlled](https://www.robinwieruch.de/react-controlled-components)**, so it renders determinstically based on props, using [pure componets](https://www.geeksforgeeks.org/reactjs-pure-components/). Note that some state is kept local with Hooks, but thats UI state (ex. input contents before submission). |
| 56 | + |
| 57 | +Uses the **'smart' and 'dumb' component** [pattern](https://jaketrent.com/post/smart-dumb-components-react), only the `Page` component have side effects, passed as props by container components. The whole UI can be tested and migrated form Redux and Electron easily. Check `frontend/src/components/pages/**/WithStore.tsx` for examples. |
| 58 | + |
| 59 | +In order to deploy the app, the `frontend` gets build into static assets and copied over to the `backend` module. |
| 60 | + |
| 61 | +### Backend → `./electron` |
| 62 | + |
| 63 | +The 'brain' of the app, this module focuses on the business logic, processing and persistence of the data. |
| 64 | + |
| 65 | +The module also contains the Electron logic for bootstraping the render processes (one for `frontend` and another for the `backend`) and for packaging the app. |
34 | 66 |
|
35 | | -Generated prod buil and updates the `taggr-releases` repo. Generate build in windows and update it manually. Make sure that the `taggr` version in the package.json is updated. |
| 67 | +**Written in TypeScript**, it operates as a Node.js backed. Runs withing a [Electron renderer process](https://www.electronjs.org/docs/latest/tutorial/process-model), with the Node.js APIs enabled. Source code available in `./electron/renderer-backend/src` |
36 | 68 |
|
37 | | -1. Execute `npm run publish` |
38 | | -2. Increate the version in `electron/package.json` |
39 | | -3. Build windows and upload manually. |
| 69 | +The message bus handler triggers [transactional scripts](https://martinfowler.com/eaaCatalog/transactionScript.html), which composes the functionality provided by a service layer. |
40 | 70 |
|
41 | | -### Releases |
| 71 | +The service layer uses dependency injection through [factory functions](https://www.javascripttutorial.net/javascript-factory-functions/), so it can be easily tested with unit tests. |
| 72 | + |
| 73 | +The machine-learning uses classification and object recognition for extracting searchable tags from images, through [Tensorflow](https://github.com/tensorflow/tfjs). |
| 74 | + |
| 75 | +The storage of extracted tags is managed using [electron-store](https://github.com/sindresorhus/electron-store). |
| 76 | + |
| 77 | +### Shared → `./shared` |
| 78 | + |
| 79 | +A type-only module, helps keep type consistency between `frontend` and `backend`. |
| 80 | + |
| 81 | +It keeps shared data such as the available frontend routes, the supported message bus messages and typed representations of the shared domain entities (such as `Image`). |
| 82 | + |
| 83 | +This enables compile-time checks on the touch points at the message bus level. Also, it helps keep domain entities consistently typed accross the app. |
| 84 | + |
| 85 | +### Environments |
| 86 | + |
| 87 | +The app can be configured to run in `development` and `production` environtments, by setting a variable in the `shared` module. |
| 88 | + |
| 89 | +- `development`: the frontend runs in a separate process and is loaded into electron as a url. |
| 90 | + |
| 91 | +- `production`: the frontend is loaded from static files, and the backend window is hidden. All the debugging tools and extensions are not mounted. |
| 92 | + |
| 93 | +## Run it |
| 94 | + |
| 95 | +Requires `"node": ">=14.0.0"` and `"yarn": "^1.22.0"`. |
| 96 | + |
| 97 | +```bash |
| 98 | +# install dependencies |
| 99 | +yarn |
| 100 | + |
| 101 | +# run unit test |
| 102 | +yarn test:ci |
| 103 | + |
| 104 | +# start app |
| 105 | +yarn app |
| 106 | + |
| 107 | +# build app |
| 108 | +yarn build |
| 109 | +``` |
42 | 110 |
|
43 | | -https://github.com/aperkaz/taggr-releases/releases |
| 111 | +## Releases |
44 | 112 |
|
45 | | -## Future Features |
| 113 | +<https://github.com/aperkaz/taggr-releases/releases> |
46 | 114 |
|
47 | | -- Layout masonry: https://github.com/bvaughn/react-virtualized/issues/1366 |
48 | | -- Certificate trust increase: https://support.ksoftware.net/support/solutions/articles/215894-what-is-this-file-is-not-commonly-downloaded-and-could-harm-your-computer-message-smartscreen- |
49 | | -- Add github actions build: https://github.com/malept/electron-forge-demo123/actions/runs/116519042/workflow |
50 | | -- Some images are displayed rotated, example in thailand trip |
51 | | -- Replace gallery view with lazy loading: https://github.com/xiaolin/react-image-gallery |
52 | | -- Timeline with pictures https://github.com/rmariuzzo/react-chronos |
53 | | -- Timeline display of images per day http://tany.kim/quantify-your-year/#/ |
54 | | -- Add more ML: look into tensorflow alternatives: evaluate performance: with article https://learn.ml5js.org/docs/#/reference/face-api?id=demo |
55 | | -- Speed up app by paralelization. Example: https://github.com/aperkaz/tensorflow-playground |
56 | | -- Food classification: https://github.com/stratospark/food-101-keras/issues/14 |
57 | | -- File sharing options: |
58 | | - https://share.storewise.tech/upload |
59 | | - https://safenote.co/upload-file ?? |
| 115 | +## Future |
60 | 116 |
|
61 | | -### Not so relevant |
| 117 | +**taggr** has been a great side project for the past year, I learned plenty about how Electron works internally, how to structure controlled frontends and had lots of fun 🎉 |
62 | 118 |
|
63 | | -- micro animations: https://www.joshwcomeau.com/react/boop/ |
64 | | -- Add node_modules migration, to fix the known issues. |
65 | | -- Image editor: https://ui.toast.com/tui-image-editor/ |
| 119 | +I have other ideas I want to develop, so I dont plan on working on taggr any time soon. |
66 | 120 |
|
67 | | -## Known Issues |
| 121 | +Feel free to **fork** or open PRs!! |
68 | 122 |
|
69 | | -- **Windows / Mac build**: the tfjs bindings for windows are not located properly, they are built into `napi-v6`, it should be renamed to `napi-v5`. In `electron/node_modules/@tensorflow/tfjs-node/lib/napi-v6`, rename. |
| 123 | +## Credit |
70 | 124 |
|
71 | | -## Resources of interes |
| 125 | +Here are some of the great resources I have leveraged to build **taggr**, in no particular order: |
72 | 126 |
|
73 | | -- Modern electron apps: https://github.com/jlongster/electron-with-server-example |
74 | | -- High-level project structure: https://blog.axosoft.com/electron-things-to-know/ |
75 | | -- Window builds fail randomly due to problems with the cache. Try to clean cache and delete package-lock as in: https://github.com/cncjs/cncjs/issues/172 |
76 | | -- EU vat (local VAT up to 10.000E a year): https://europa.eu/youreurope/business/selling-in-eu/selling-goods-services/provide-services-abroad/index_es.htm#rules-annual-turnovers |
| 127 | +- [Great electron boilerplate](https://github.com/sindresorhus/electron-boilerplate) |
| 128 | +- [Modern electron apps](https://archive.jlongster.com/secret-of-good-electron-apps) |
| 129 | +- [Loading the Frontend from a separate process](https://medium.com/@kitze/%EF%B8%8F-from-react-to-an-electron-app-ready-for-production-a0468ecb1da3?p=a0468ecb1da3) |
| 130 | +- [awesome-electron](https://github.com/sindresorhus/awesome-electron) |
0 commit comments