All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
- Logging for Immich responses.
- Watcher now supports all data formats that can be imported via web interface.
- Imported points will now be reverse geocoded only after import is finished.
- Markers on the map are now being rendered with higher z-index than polylines. #577
- Cache cleaning and preheating upon application start.
PHOTON_API_KEY
env var to set Photon API key. It's an optional env var, but it's required if you want to use Photon API as a Patreon supporter.- 'X-Dawarich-Response' header to the
GET /api/v1/health
endpoint. It's set to 'Hey, I'm alive!' to make it easier to check if the API is working.
- Custom config for PostgreSQL is now optional in
docker-compose.yml
.
The dawarich_db
service now uses a custom postgresql.conf
file.
As @tabacha pointed out in #549, the default shm_size
for the dawarich_db
service is too small and it may lead to database performance issues. This release introduces a shm_size
parameter to the dawarich_db
service to increase the size of the shared memory for PostgreSQL. This should help database with peforming vacuum and other operations. Also, it introduces a custom postgresql.conf
file to the dawarich_db
service.
To mount a custom postgresql.conf
file, you need to create a postgresql.conf
file in the dawarich_db
service directory and add the following line to it:
dawarich_db:
image: postgres:14.2-alpine
shm_size: 1G
container_name: dawarich_db
volumes:
- dawarich_db_data:/var/lib/postgresql/data
- dawarich_shared:/var/shared
+ - ./postgresql.conf:/etc/postgresql/postgres.conf # Provide path to custom config
...
healthcheck:
test: [ "CMD-SHELL", "pg_isready -U postgres -d dawarich_development" ]
interval: 10s
retries: 5
start_period: 30s
timeout: 10s
+ command: postgres -c config_file=/etc/postgresql/postgres.conf # Use custom config
To ensure your database is using custom config, you can connect to the container (docker exec -it dawarich_db psql -U postgres
) and run SHOW config_file;
command. It should return the following path: /etc/postgresql/postgresql.conf
.
An example of a custom postgresql.conf
file is provided in the postgresql.conf.example
file.
- A button on a year stats card to update stats for the whole year. #466
- A button on a month stats card to update stats for a specific month. #466
- A confirmation alert on the Notifications page before deleting all notifications.
- A
shm_size
parameter to thedawarich_db
service to increase the size of the shared memory for PostgreSQL. This should help database with peforming vacuum and other operations.
...
dawarich_db:
image: postgres:14.2-alpine
+ shm_size: 1G
...
- In addition to
api_key
parameter,Authorization
header is now being used to authenticate API requests. #543
Example:
Authorization: Bearer YOUR_API_KEY
- The map borders were expanded to make it easier to scroll around the map for New Zealanders.
- The
dawarich_db
service now uses a custompostgresql.conf
file. - The popup over polylines now shows dates in the user's format, based on their browser settings.
- A point id is now being shown in the point popup.
- North Macedonia is now being shown on the scratch map. #537
- The app process is now bound to :: instead of 0.0.0.0 to provide compatibility with IPV6.
- The app was updated to use Rails 8.0.1.
- Setting
reverse_geocoded_at
for points that don't have geodata is now being performed in background job, in batches of 10,000 points to prevent memory exhaustion and long-running data migration.
GET /api/v1/points/tracked_months
endpoint added to get list of tracked years and months.GET /api/v1/countries/visited_cities
endpoint added to get list of visited cities.- A link to the docs leading to a help chart for k8s. #550
- A button to delete all notifications. #548
- A support for
RAILS_LOG_LEVEL
env var to change log level. More on that here: https://guides.rubyonrails.org/debugging_rails_applications.html#log-levels. The available log levels are::debug
,:info
,:warn
,:error
,:fatal
, and:unknown
, corresponding to the log level numbers from 0 up to 5, respectively. The default log level is:debug
. #540 - A devcontainer to improve developers experience. #546
- A point popup is no longer closes when hovering over a polyline. #536
- When polylines layer is disabled and user deletes a point from its popup, polylines layer is no longer being enabled right away. #552
- Paths to gems within the sidekiq and app containers. #499
- Months and years navigation is moved to a map panel on the right side of the map.
- List of visited cities is now being shown in a map panel on the right side of the map.
- Fixed a bug where upon deleting a point on the map, the confirmation dialog was shown multiple times and the point was not being deleted from the map until the page was reloaded. #435
- With the "Points" layer enabled on the map, points with negative speed are now being shown in orange color. Since Overland reports negative speed for points that might be faulty, this should help you to identify them.
- On the Points page, speed of the points with negative speed is now being shown in red color.
The dawarich_shared
volume now being mounted to /data
instead of /var/shared
within the container. It fixes Redis data being lost on container restart.
To change this, you need to update the docker-compose.yml
file:
dawarich_redis:
image: redis:7.0-alpine
container_name: dawarich_redis
command: redis-server
volumes:
+ - dawarich_shared:/data
restart: always
healthcheck:
Telemetry is now disabled by default. To enable it, you need to set ENABLE_TELEMETRY
env var to true
. For those who have telemetry enabled using DISABLE_TELEMETRY
env var set to false
, telemetry is now disabled by default.
- Flash messages are now being removed after 5 seconds.
- Fixed broken migration that was preventing the app from starting.
- Visits page is now loading a lot faster than before.
- Redis data should now be preserved on container restart.
- Fixed a bug where export files could have double extension, e.g.
file.gpx.gpx
.
- Places page is now accessible from the Visits & Places tab on the navbar.
- Exporting process is now being logged.
ENABLE_TELEMETRY
env var is now used instead ofDISABLE_TELEMETRY
to enable/disable telemetry.
- Fixed a bug where the map and visits pages were throwing an error due to incorrect approach to distance calculation.
The GET /api/v1/trips/:id/photos
endpoint now returns a different structure of the response:
{
id: 1,
latitude: 10,
longitude: 10,
localDateTime: "2024-01-01T00:00:00Z",
originalFileName: "photo.jpg",
city: "Berlin",
state: "Berlin",
country: "Germany",
type: "image",
+ orientation: "portrait",
source: "photoprism"
}
- Fixed a bug where the Photoprism photos were not being shown on the trip page.
- Fixed a bug where the Immich photos were not being shown on the trip page.
- Fixed a bug where the route popup was showing distance in kilometers instead of miles. #490
- A link to the Photoprism photos on the trip page if there are any.
- A
orientation
field in the Api::PhotoSerializer, hence theGET /api/v1/photos
endpoint now includes the orientation of the photo. Valid values areportrait
andlandscape
. - Examples for the
type
,orientation
andsource
fields in theGET /api/v1/photos
endpoint in the Swagger UI. DISABLE_TELEMETRY
env var to disable telemetry. More on telemetry: https://dawarich.app/docs/tutorials/telemetryreverse_geocoded_at
column added to thepoints
table.
- On the Stats page, the "Reverse geocoding" section is now showing the number of points that were reverse geocoded based on
reverse_geocoded_at
column, value of which is based on the time when the point was reverse geocoded. If no geodata for the point is available,reverse_geocoded_at
will be set anyway. Number of points that were reverse geocoded but no geodata is available for them is shown below the "Reverse geocoded" number.
- Refactored stats calculation to calculate only necessary stats, instead of calculating all stats
- Stats are now being calculated every 1 hour instead of 6 hours
- List of years on the Map page is now being calculated based on user's points instead of stats. It's also being cached for 1 day due to the fact that it's usually a heavy operation based on the number of points.
- Reverse-geocoding points is now being performed in batches of 1,000 points to prevent memory exhaustion.
- In-app notification about telemetry being enabled.
Dawarich now can collect usage metrics and send them to InfluxDB. Before this release, the only metrics that could be somehow tracked by developers (only @Freika, as of now) were the number of stars on GitHub and the overall number of docker images being pulled, across all versions of Dawarich, non-splittable by version. New in-app telemetry will allow us to track more granular metrics, allowing me to make decisions based on facts, not just guesses.
I'm aware about the privacy concerns, so I want to be very transparent about what data is being sent and how it's used.
Data being sent:
- Number of DAU (Daily Active Users)
- App version
- Instance ID (unique identifier of the Dawarich instance built by hashing the api key of the first user in the database)
The data is being sent to a InfluxDB instance hosted by me and won't be shared with anyone.
Basically this set of metrics allows me to see how many people are using Dawarich and what versions they are using. No other data is being sent, nor it gives me any knowledge about individual users or their data or activity.
The telemetry is enabled by default, but it can be disabled by setting DISABLE_TELEMETRY
env var to true
. The dataset might change in the future, but any changes will be documented here in the changelog and in every release as well as on the telemetry page of the website docs.
- Telemetry feature. It's now collecting usage metrics and sending them to InfluxDB.
- Sidekiq is now being correctly exported to Prometheus with
PROMETHEUS_EXPORTER_ENABLED=true
env var indawarich_sidekiq
service.
GET /api/v1/photos
endpoint now returns following structure of the response:
[
{
"id": "1",
"latitude": 11.22,
"longitude": 12.33,
"localDateTime": "2024-01-01T00:00:00Z",
"originalFileName": "photo.jpg",
"city": "Berlin",
"state": "Berlin",
"country": "Germany",
"type": "image", // "image" or "video"
"source": "photoprism" // "photoprism" or "immich"
}
]
- Photos from Photoprism are now can be shown on the map. To enable this feature, you need to provide your Photoprism instance URL and API key in the Settings page. Then you need to enable "Photos" layer on the map (top right corner).
- Geodata is now can be imported from Photoprism to Dawarich. The "Import Photoprism data" button on the Imports page will start the import process.
- z-index on maps so they won't overlay notifications dropdown
- Redis connectivity where it's not required
- Demo account. You can now login with
demo@dawarich.app
/password
to see how Dawarich works. This replaces previous default credentials.
- The login page now shows demo account credentials if
DEMO_ENV
env var is set totrue
.
- Fixed a bug where the trips interface was breaking when Immich integration is not configured.
- Flash messages are now being shown on the map when Immich integration is not configured.
You can now create, edit and delete trips. To create a trip, click on the "New Trip" button on the Trips page. Provide a name, date and time for start and end of the trip. You can add your own notes to the trip as well.
If you have points tracked during provided timeframe, they will be automatically added to the trip and will be shown on the trip map.
Also, if you have Immich integrated, you will see photos from the trip on the trip page, along with a link to look at them on Immich.
- The Trips feature. Read above for more details.
- Maps are now not so rough on the edges.
- Retrieving photos from Immich now using
takenAfter
andtakenBefore
instead ofcreatedAfter
andcreatedBefore
. WithcreatedAfter
andcreatedBefore
Immich was returning no items some years.
- Retrieving photos from Immich now correctly handles cases when Immich returns no items. It also logs the response from Immich for debugging purposes.
With this release, Dawarich can now show photos from your Immich instance on the map.
To enable this feature, you need to provide your Immich instance URL and API key in the Settings page. Then you need to enable "Photos" layer on the map (top right corner).
An important note to add here is that photos are heavy and hence generate a lot of traffic. The response from Immich for specific dates is being cached in Redis for 1 day, and that may lead to Redis taking a lot more space than previously. But since the cache is being expired after 24 hours, you'll get your space back pretty soon.
The other thing worth mentioning is how Dawarich gets data from Immich. It goes like this:
- When you click on the "Photos" layer, Dawarich will make a request to
GET /api/v1/photos
endpoint to get photos for the selected timeframe. - This endpoint will make a request to
POST /search/metadata
endpoint of your Immich instance to get photos for the selected timeframe. - The response from Immich is being cached in Redis for 1 day.
- Dawarich's frontend will make a request to
GET /api/v1/photos/:id/thumbnail.jpg
endpoint to get photo thumbnail from Immich. The number of requests to this endpoint will depend on how many photos you have in the selected timeframe. - For each photo, Dawarich's frontend will make a request to
GET /api/v1/photos/:id/thumbnail.jpg
endpoint to get photo thumbnail from Immich. This thumbnail request is also cached in Redis for 1 day.
- If you have provided your Immich instance URL and API key, the map will now show photos from your Immich instance when Photos layer is enabled.
GET /api/v1/photos
endpoint added to get photos from Immich.GET /api/v1/photos/:id/thumbnail.jpg
endpoint added to get photo thumbnail from Immich.
- Rate limit for the Photon API is now 1 request per second. If you host your own Photon API instance, reverse geocoding requests will not be limited.
- Requests to the Photon API are now have User-Agent header set to "Dawarich #{APP_VERSION} (https://dawarich.app)"
- Default number of Puma workers is now 2 instead of 1. This should improve the performance of the application. If you have a lot of users, you might want to increase the number of workers. You can do this by setting the
WEB_CONCURRENCY
env var in yourdocker-compose.yml
file. Example:
dawarich_app:
image: freikin/dawarich:latest
container_name: dawarich_app
environment:
...
WEB_CONCURRENCY: "2"
- Prometheus exporter is now bound to 0.0.0.0 instead of localhost
PROMETHEUS_EXPORTER_HOST
andPROMETHEUS_EXPORTER_PORT
env vars were added to thedocker-compose.yml
file to allow you to set the host and port for the Prometheus exporter. They should be added to bothdawarich_app
anddawarich_sidekiq
services Example:
dawarich_app:
image: freikin/dawarich:latest
container_name: dawarich_app
environment:
...
PROMETHEUS_EXPORTER_ENABLED: "true"
+ PROMETHEUS_EXPORTER_HOST: 0.0.0.0
+ PROMETHEUS_EXPORTER_PORT: "9394"
dawarich_sidekiq:
image: freikin/dawarich:latest
container_name: dawarich_sidekiq
environment:
...
PROMETHEUS_EXPORTER_ENABLED: "true"
+ PROMETHEUS_EXPORTER_HOST: dawarich_app
+ PROMETHEUS_EXPORTER_PORT: "9394"
- Dawarich now can export metrics to Prometheus. You can find the metrics at
your.host:9394/metrics
endpoint. The metrics are being exported in the Prometheus format and can be scraped by Prometheus server. To enable exporting, set thePROMETHEUS_EXPORTER_ENABLED
env var in your docker-compose.yml totrue
. Example:
dawarich_app:
image: freikin/dawarich:latest
container_name: dawarich_app
environment:
...
PROMETHEUS_EXPORTER_ENABLED: "true"
- Dawarich now uses
POST /api/search/metadata
endpoint to get geodata from Immich.
- Admins can now see all users in the system on the Users page. The path is
/settings/users
.
- Admins can now provide custom password for new users and update passwords for existing users on the Users page.
- The
bin/dev
file will no longer runbin/rails tailwindcss:watch
command. It's useful only for development and doesn't really make sense to run it in production.
- Exported files will now always have an extension when downloaded. Previously, the extension was missing in case of GPX export.
- Deleting and sorting points on the Points page will now preserve filtering and sorting params when points are deleted or sorted. Previously, the page was being reloaded and filtering and sorting params were lost.
- Make ActionCable respect REDIS_URL env var. Previously, ActionCable was trying to connect to Redis on localhost.
- Exported GPX file now being correctly recognized as valid by Garmin Connect, Adobe Lightroom and (probably) other services. Previously, the exported GPX file was not being recognized as valid by these services.
- Speed is now being recorded into points when a GPX file is being imported. Previously, the speed was not being recorded.
- GeoJSON file from GPSLogger now can be imported to Dawarich. Previously, the import was failing due to incorrect parsing of the file.
- The Vists suggestion job is disabled. It will be re-enabled in the future with a new approach to the visit suggestion process.
- New notifications are now being indicated with a blue-ish dot in the top right corner of the screen. Hovering over the bell icon will show you last 10 notifications.
- New points on the map will now be shown in real-time. No need to reload the map to see new points.
- User can now enable or disable Live Mode in the map controls. When Live Mode is enabled, the map will automatically scroll to the new points as they are being added to the map.
- Scale on the map now shows the distance both in kilometers and miles.
GET /api/v1/countries/borders
endpoint to get countries for scratch map feature
- Scratch map. You can enable it in the map controls. The scratch map highlight countries you've visited. The scratch map is working properly only if you have your points reverse geocoded.
- Importing Immich data on the Imports page now will trigger an attempt to write raw json file with the data from Immich to
tmp/imports/immich_raw_data_CURRENT_TIME_USER_EMAIL.json
file. This is useful to debug the problem with the import if it fails. #270
- New app version is now being checked every 6 hours instead of 1 day and the check is being performed in the background. #238
⚠️ The instruction to importRecords.json
from Google Takeout now mentionstmp/imports
directory instead ofpublic/imports
.⚠️ #326- Hostname definition for Sidekiq healtcheck to solve #344. See the diff:
dawarich_sidekiq:
image: freikin/dawarich:latest
container_name: dawarich_sidekiq
healthcheck:
- test: [ "CMD-SHELL", "bundle exec sidekiqmon processes | grep $(hostname)" ]
+ test: [ "CMD-SHELL", "bundle exec sidekiqmon processes | grep ${HOSTNAME}" ]
- Renamed directories used by app and sidekiq containers for gems cache to fix #339:
dawarich_app:
image: freikin/dawarich:latest
container_name: dawarich_sidekiq
volumes:
- - gem_cache:/usr/local/bundle/gems
+ - gem_cache:/usr/local/bundle/gems_app
...
dawarich_sidekiq:
image: freikin/dawarich:latest
container_name: dawarich_sidekiq
volumes:
- - gem_cache:/usr/local/bundle/gems
+ - gem_cache:/usr/local/bundle/gems_sidekiq
- Data migration that prevented the application from starting.
- Stats distance calculation now correctly calculates the daily distances.
- Refactored the stats calculation process to make it more efficient.
- User can now select between "Raw" and "Simplified" mode in the map controls. "Simplified" mode will show less points, improving the map performance. "Raw" mode will show all points.
- A bug where "RuntimeError: failed to get urandom" was being raised upon importing attempt on Synology.
- Import of Owntracks' .rec files now correctly imports points. Previously, the import was failing due to incorrect parsing of the file.
- Fixed a bug where Google Takeout import was failing due to unsupported date format with milliseconds in the file.
- Fixed a bug that prevented using the Photon API host with http protocol. Now you can use both http and https protocols for the Photon API host. You now need to explicitly provide
PHOTON_API_USE_HTTPS
to betrue
orfalse
depending on what protocol you want to use. Example is in thedocker-compose.yml
file.
- The Map page now by default uses timeframe based on last point tracked instead of the today's points. If there are no points, the map will use the today's timeframe.
- The map on the Map page can no longer be infinitely scrolled horizontally. #299
- Use static version of
geocoder
library that supports http and https for Photon API host. This is a temporary solution until the change is available in a stable release.
- Owntracks' .rec files now can be imported to Dawarich. The import process is the same as for other kinds of files, just select the .rec file and choose "owntracks" as a source.
- Owntracks' .json files are no longer supported for import as Owntracks itself does not export to this format anymore.
To expose the watcher functionality to the user, a new directory /tmp/imports/watched/
was created. Add new volume to the docker-compose.yml
file to expose this directory to the host machine.
...
dawarich_app:
image: freikin/dawarich:latest
container_name: dawarich_app
volumes:
- gem_cache:/usr/local/bundle/gems
- public:/var/app/public
+ - watched:/var/app/tmp/watched
...
dawarich_sidekiq:
image: freikin/dawarich:latest
container_name: dawarich_sidekiq
volumes:
- gem_cache:/usr/local/bundle/gems
- public:/var/app/public
+ - watched:/var/app/tmp/watched
...
volumes:
db_data:
gem_cache:
shared_data:
public:
+ watched:
- Watcher now looks into
/tmp/imports/watched/USER@EMAIL.TLD
directory instead of/tmp/imports/watched/
to allow using arbitrary file names for imports
linux/arm/v7
is added to the list of supported architectures to support Raspberry Pi 4 and other ARMv7 devices
The /public/imporst/watched/ directory is watched by Dawarich. Any files you put in this directory will be imported into the database. The name of the file must start with an email of the user you want to import the file for. The email must be followed by an underscore symbol (_) and the name of the file.
For example, if you want to import a file for the user with the email address "email@dawarich.app", you would name the file "email@dawarich.app_2024-05-01_2024-05-31.gpx". The file will be imported into the database and the user will receive a notification in the app.
Both GeoJSON and GPX files are supported.
- You can now put your GPX and GeoJSON files to
tmp/imports/watched
directory and Dawarich will automatically import them. This is useful if you have a service that can put files to the directory automatically. The directory is being watched every 60 minutes for new files.
- Monkey patch for Geocoder to support http along with https for Photon API host was removed becausee it was breaking the reverse geocoding process. Now you can use only https for the Photon API host. This might be changed in the future
- Disable retries for some background jobs
- Stats update is now being correctly triggered every 6 hours
- Now you can use http protocol for the Photon API host if you don't have SSL certificate for it
- For stats, total distance per month might have been not equal to the sum of distances per day. Now it's fixed and values are equal
- Mobile view of the map looks better now
GET /api/v1/points
can now accept optional?order=asc
query parameter to return points in ascending order by timestamp.?order=desc
is still available to return points in descending order by timestampGET /api/v1/points
now returnsid
attribute for each point
- Points imported from Google Location History (mobile devise) now have correct timestamps
GET /api/v1/points?slim=true
now returnsid
attribute for each point
- GPX export now finishes correctly and does not throw an error in the end
- Deleting points from the Points page now preserves
start_at
andend_at
values for the routes. #261 - Visits map now being rendered correctly in the Visits page. #262
- Fixed issue with timezones for negative UTC offsets. #194, #122
- Point page is no longer reloads losing provided timestamps when searching for points on Points page. #283
- Map layers from Stadia were disabled for now due to necessary API key
- GPX export now has time and elevation elements for each point
GET /api/v1/points
will no longer returnraw_data
attribute for each point as it's a bit too much
- "Slim" version of
GET /api/v1/points
: pass optional param?slim=true
to it and it will return only latitude, longitude and timestamp
- Optimize order of the dockerfiles to leverage layer caching by @JoeyEamigh
- Add support for alternate postgres ports and db names in docker by @JoeyEamigh
- Creating exports directory if it doesn't exist by @tetebueno
- Fixed a bug where the map was not loading due to invalid tile layer name
- 17 new tile layers to choose from. Now you can select the tile layer that suits you the best. You can find the list of available tile layers in the map controls in the top right corner of the map under the layers icon.
GET /api/v1/points
response now will includeX-Total-Pages
andX-Current-Page
headers to make it easier to work with the endpoint- The Pages point now shows total number of points found for provided date range
- Link to Visits page in notification informing about new visit suggestion
- Flatten geodata retrieved from Immich before processing it to prevent errors
- Links to view import points on the map and on the Points page on the Imports page.
- The Imports page now loading faster.
- Default value for
RAILS_MAX_THREADS
was changed to 10. - Visit suggestions background job was moved to its own low priority queue to prevent it from blocking other jobs.
- Fixed a bug preventing the application from starting, when there is no users in the database but a data migration tries to update one.
- Support for miles. To switch to miles, provide
DISTANCE_UNIT
environment variable with valuemi
in thedocker-compose.yml
file. Default value iskm
.
It's recommended to update your stats manually after changing the DISTANCE_UNIT
environment variable. You can do this by clicking the "Update stats" button on the Stats page.
DISTANCE_UNIT
is set to mi
.
dawarich_app:
image: freikin/dawarich:latest
container_name: dawarich_app
environment:
APPLICATION_HOST: "localhost"
APPLICATION_PROTOCOL: "http"
APPLICATION_PORT: "3000"
TIME_ZONE: "UTC"
+ DISTANCE_UNIT: "mi"
dawarich_sidekiq:
image: freikin/dawarich:latest
container_name: dawarich_sidekiq
environment:
APPLICATION_HOST: "localhost"
APPLICATION_PROTOCOL: "http"
APPLICATION_PORT: "3000"
TIME_ZONE: "UTC"
+ DISTANCE_UNIT: "mi"
- Default time range on the map is now 1 day instead of 1 month. It will help you with performance issues if you have a lot of points in the database.
- GeoJSON import now correctly imports files with FeatureCollection as a root object
- The Points page now have number of points found for provided date range
GET /api/v1/health
endpoint to check the health of the application with swagger docs
- Ruby version updated to 3.3.4
- Visits suggestion process now will try to merge consecutive visits to the same place into one visit.
The GPX and GeoJSON export release
Default exporting format is now GeoJSON instead of Owntracks-like JSON. This will allow you to use the exported data in other applications that support GeoJSON format. It's also important to highlight, that GeoJSON format does not describe a way to store any time-related data. Dawarich relies on the timestamp
field in the GeoJSON format to determine the time of the point. The value of the timestamp
field should be a Unix timestamp in seconds. If you import GeoJSON data that does not have a timestamp
field, the point will not be imported.
Example of a valid point in GeoJSON format:
{
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [13.350110811262352, 52.51450815]
},
"properties": {
"timestamp": 1725310036
}
}
- GeoJSON format is now available for exporting data.
- GPX format is now available for exporting data.
- Importing GeoJSON is now available.
- Default exporting format is now GeoJSON instead of Owntracks-like JSON. This will allow you to use the exported data in other applications that support GeoJSON format.
- Fixed a bug where the confirmation alert was shown more than once when deleting a point.
- Resource limits to docke-compose.yml file to prevent server overload. Feel free to adjust the limits to your needs.
deploy:
resources:
limits:
cpus: '0.50' # Limit CPU usage to 50% of one core
memory: '2G' # Limit memory usage to 2GB
- Importing geodata from Immich will now not throw an error in the end of the process
- A notification about an existing import with the same name will now show the import name
- Export file now also will contain
raw_dat
field for each point. This field contains the original data that was imported to the application.
PATCH /api/v1/settings
endpoint to update user settings with swagger docsGET /api/v1/settings
endpoint to get user settings with swagger docs- Missing
page
andper_page
query parameters to theGET /api/v1/points
endpoint swagger docs
- Map settings moved to the map itself and are available in the top right corner of the map under the gear icon.
- Fixed a bug that prevented data migration from working correctly
- With this release deployment, data migration will work, starting visits suggestion process for all users.
- After initial visit suggestion process, new suggestions will be calculated every 24 hours, based on points for last 24 hours.
- If you have enabled reverse geocoding and (optionally) provided Photon Api Host, Dawarich will try to reverse geocode your visit and suggest specific places you might have visited, such as cafes, restaurants, parks, etc. If reverse geocoding is not enabled, or Photon Api Host is not provided, Dawarich will not try to suggest places but you'll be able to rename the visit yourself.
- You can confirm or decline the visit suggestion. If you confirm the visit, it will be added to your timeline. If you decline the visit, it will be removed from your timeline. You'll be able to see all your confirmed, declined and suggested visits on the Visits page.
- A "Map" button to each visit on the Visits page to allow user to see the visit on the map
- Visits suggestion functionality. Read more on that in the release description
- Click on the visit name allows user to rename the visit
- Tabs to the Visits page to allow user to switch between confirmed, declined and suggested visits
- Places page to see and delete places suggested by Dawarich's visit suggestion process
- Importing a file will now trigger the visit suggestion process for the user
- Dawarich export was failing when attempted to be imported back to Dawarich.
- Imports page with a lot of imports should now load faster.
/api/v1/points
endpoint now returns 100 points by default. You can specify the number of points to return by passing theper_page
query parameter. Example:/api/v1/points?per_page=50
will return 50 points. Also,page
query parameter is now available to paginate the results. Example:/api/v1/points?per_page=50&page=2
will return the second page of 50 points.
- A user can now trigger the import of their geodata from Immich to Dawarich by clicking the "Import Immich data" button in the Imports page.
- A user can now provide a url and an api key for their Immich instance and then trigger the import of their geodata from Immich to Dawarich. This can be done in the Settings page.
- Table columns on the Exports page were reordered to make it more user-friendly.
- Exports are now being named with this pattern: "export_from_dd.mm.yyyy_to_dd.mm.yyyy.json" where "dd.mm.yyyy" is the date range of the export.
- Notification about any error now will include the stacktrace.
- The
api/v1/stats
endpoint to get stats for the user with swagger docs
- Redis and DB containers are now being automatically restarted if they fail. Update your
docker-compose.yml
if necessary
services:
dawarich_redis:
image: redis:7.0-alpine
command: redis-server
networks:
- dawarich
volumes:
- shared_data:/var/shared/redis
+ restart: always
dawarich_db:
image: postgres:14.2-alpine
container_name: dawarich_db
volumes:
- db_data:/var/lib/postgresql/data
- shared_data:/var/shared
networks:
- dawarich
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
+ restart: always
See the PR or Swagger docs (/api-docs
) for more information.
- Owntracks points are now being saved to the database with the full attributes
- Existing owntracks points also filled with missing data
- Definition of "reverse geocoded points" is now correctly based on the number of points that have full reverse geocoding data instead of the number of points that have only country and city
- Fixed a bug in gpx importing scipt (thanks, bluemax!)
- A bug where an attempt to import a Google's Records.json file was failing due to wrong object being passed to a background worker
- PHOTON_API_HOST env variable to set the host of the Photon API. It will allow you to use your own Photon API instance instead of the default one.
- Pagination to exports page
- Pagination to imports page
- GET
/api/v1/points
endpoint to get all points for the user with swagger docs - DELETE
/api/v1/points/:id
endpoint to delete a single point for the user with swagger docs - DELETE
/api/v1/areas/:id
swagger docs - User can now change route opacity in settings
- Points on the Points page can now be ordered by oldest or newest points
- Visits on the Visits page can now be ordered by oldest or newest visits
- Point deletion is now being done using an api key instead of CSRF token
- OpenStreetMap layer is now being selected by default in map controls
- Call to the background job to calculate visits
- Name of background job to calculate visits
- Map areas functionality
- A possibility to create areas. To create an area, click on the Areas checkbox in map controls (top right corner of the map), then in the top left corner of the map, click on a small circle icon. This will enable draw tool, allowing you to draw an area. When you finish drawing, release the mouse button, and the area will be created. Click on the area, set the name and click "Save" to save the area. You can also delete the area by clicking on the trash icon in the area popup.
- A background job to calculate your visits. This job will calculate your visits based on the areas you've created.
- Visits page. This page will show you all your visits, calculated based on the areas you've created. You can see the date and time of the visit, the area you've visited, and the duration of the visit.
- A possibility to confirm or decline a visit. When you create an area, the visit is not calculated immediately. You need to confirm or decline the visit. You can do this on the Visits page. Click on the visit, then click on the "Confirm" or "Decline" button. If you confirm the visit, it will be added to your timeline. If you decline the visit, it will be removed from your timeline.
- Settings for visit calculation. You can set the minimum time spent in the area to consider it as a visit. This setting can be found in the Settings page.
- POST
/api/v1/areas
and GET/api/v1/areas
endpoints. You can now create and list your areas via the API.
- A route popup now correctly shows distance made in the route, not the distance between first and last points in the route.
- A popup being shown when user clicks on a point now contains a link to delete the point. This is useful if you want to delete a point that was imported by mistake or you just want to clean up your data.
- Added
public/imports
andpublic/exports
folders to git to prevent errors when exporting data
- Some code from
maps_controller.js
was extracted into separate files
- Admin flag to the database. Now not only the first user in the system can create new users, but also users with the admin flag set to true. This will make easier introduction of more admin functions in the future.
- Route hover distance is now being rendered in kilometers, not in meters, if route distance is more than 1 km.
- Hover over a route does not move map anymore and shows the route tooltip where user hovers over the route, not at the end of the route. Click on route now will move the map to include the whole route.
- Fixed a bug where total reverse geocoded points were calculated based on number of imported points that are reverse geocoded, not on the number of total reverse geocoded points.
- Background jobs page. You can find it in Settings -> Background Jobs.
- Queue clearing buttons. You can clear all jobs in the queue.
- Reverse geocoding restart button. You can restart the reverse geocoding process for all of your points.
- Reverse geocoding continue button. Click on this button will start reverse geocoding process only for points that were not processed yet.
- A lot more data is now being saved in terms of reverse geocoding process. It will be used in the future to create more insights about your data.
- Point reference to a user is no longer optional. It should not cause any problems, but if you see any issues, please let me know.
⚠️ Calculation of total reverse geocoded points was changed.⚠️ Previously, the reverse geocoding process was recording only country and city for each point. Now, it records all the data that was received from the reverse geocoding service. This means that the total number of reverse geocoded points will be different from the previous one. It is recommended to restart the reverse geocoding process to get this data for all your existing points. Below you can find an example of what kind of data is being saved to your Dawarich database:
{
"place_id": 127850637,
"licence": "Data © OpenStreetMap contributors, ODbL 1.0. http://osm.org/copyright",
"osm_type": "way",
"osm_id": 718035022,
"lat": "52.51450815",
"lon": "13.350110811262352",
"class": "historic",
"type": "monument",
"place_rank": 30,
"importance": 0.4155071896625501,
"addresstype": "historic",
"name": "Victory Column",
"display_name": "Victory Column, Großer Stern, Botschaftsviertel, Tiergarten, Mitte, Berlin, 10785, Germany",
"address": {
"historic": "Victory Column",
"road": "Großer Stern",
"neighbourhood": "Botschaftsviertel",
"suburb": "Tiergarten",
"borough": "Mitte",
"city": "Berlin",
"ISO3166-2-lvl4": "DE-BE",
"postcode": "10785",
"country": "Germany",
"country_code": "de"
},
"boundingbox": [
"52.5142449",
"52.5147775",
"13.3496725",
"13.3505485"
]
}
- Added a logging config to the
docker-compose.yml
file to prevent logs from overflowing the disk. Now logs are being rotated and stored in thelog
folder in the root of the application. You can find usage example in the the repository'sdocker-compose.yml
file. Make sure to add this config to bothdawarich_app
anddawarich_sidekiq
services.
logging:
driver: "json-file"
options:
max-size: "100m"
max-file: "5"
- Visiting notifications page now marks this notifications as read
- Guide on how to setup a reverse proxy for Dawarich in the
docs/how_to_setup_reverse_proxy.md
file. This guide explains how to set up a reverse proxy for Dawarich using Nginx and Apache2.
MAP_CENTER
env var from thedocker-compose.yml
file. This variable was used to set the default center of the map, but it is not needed anymore, as the map center is now hardcoded in the application.⚠️ Feel free to remove this variable from yourdocker-compose.yml
file.⚠️
- Fixed a bug where Overland batch payload was not being processed due to missing coordinates in the payload. Now, if the coordinates are missing, the single point is skipped and the rest are being processed.
- Set
'localhost'
string as a default value forAPPLICATION_HOSTS
environment variable in thedocker-compose.yml
file instead of an array. This is necessary to prevent errors when starting the application.
- Support for multiple hosts. Now you can specify the host of the application by setting the
APPLICATION_HOSTS
(note plural form) environment variable in thedocker-compose.yml
file. Example:
dawarich_app:
image: freikin/dawarich:latest
container_name: dawarich_app
environment:
APPLICATION_HOSTS: "yourhost.com,www.yourhost.com,127.0.0.1"
Note, there should be no protocol prefixes in the APPLICATION_HOSTS
variable, only the hostnames.
APPLICATION_HOST
to APPLICATION_HOSTS
to avoid any issues in the future, as APPLICATION_HOST
will be deprecated in the nearest future.
- Support for HTTPS. Now you can specify the protocol of the application by setting the
APPLICATION_PROTOCOL
environment variable in thedocker-compose.yml
file. Default value ishttp
Example:
dawarich_app:
image: freikin/dawarich:latest
container_name: dawarich_app
environment:
APPLICATION_PROTOCOL: "https"
- Support for a
location-history.json
file from Google Takeout. It turned out, this file could contain not only an object with location data history, but also an array of objects with location data history. Now Dawarich can handle both cases and import the data correctly.
- Notifications system. Now you will receive a notification when an import or export is finished, when stats update is completed and if any error occurs during any of these processes. Notifications are displayed in the top right corner of the screen and are stored in the database. You can see all your notifications on the Notifications page.
- Swagger API docs for
/api/v1/owntracks/points
. You can find the API docs at/api-docs
.
- Google Takeout geodata, taken from a mobile devise, is now fully supported and can be imported to the Dawarich. The import process is the same as for other kinds of files, just select the JSON file and choose "Google Phone Takeout" as a source.
- Fixed a bug where an imported point was not being saved to the database if a point with the same timestamp and already existed in the database even if it was other user's point.
- First user in the system can now create new users from the Settings page. This is useful for creating new users without the need to enable registrations. Default password for new users is
password
.
- Registrations are now disabled by default. On the initial setup, a default user with email
user@domain.com
and passwordpassword
is created. You can change the password in the Settings page. - On the Imports page, now you can see the real number of points imported. Previously, this number might have not reflect the real number of points imported.
- New Settings page to change Dawarich settings.
- New "Fog of War" toggle on the map controls.
- New "Fog of War meters" field in Settings. This field allows you to set the radius in meters around the point to be shown on the map. The map outside of this radius will be covered with a fog of war.
- Order of points on Points page is now descending by timestamp instead of ascending.
In new Settings page you can now change the following settings:
- Maximum distance between two points to consider them as one route
- Maximum time between two points to consider them as one route
- New Settings page to change Dawarich settings.
- Settings link in user menu now redirects to the new Settings page.
- Old settings page is now available undeer Account link in user menu.
This release introduces support for GPX files to be imported. Now you can import GPX files from your devices to Dawarich. The import process is the same as for other kinds of files, just select the GPX file instead and choose "gpx" as a source. Both single-segmented and multi-segmented GPX files are supported.
/api/v1/points
endpoint is removed. Please use/api/v1/owntracks/points
endpoint to upload your points from OwnTracks mobile app instead.
- Support for GPX files to be imported.
- Couple of unnecessary params were hidden from route popup and now can be shown using
?debug=true
query parameter. This is useful for debugging purposes.
/exports/download
endpoint is removed. Now you can download your exports directly from the Exports page./api/v1/points
endpoint is removed.
- A link to Dawarich's website in the footer. It ain't much, but it's honest work.
- Fixed version badge in the navbar. Now it will show the correct version of the application.
- Default map center location was changed.
Please update your docker-compose.yml
file to include the following changes:
dawarich_sidekiq:
image: freikin/dawarich:latest
container_name: dawarich_sidekiq
volumes:
- gem_cache:/usr/local/bundle/gems
+ - public:/var/app/public
- Added a line with public volume to sidekiq's docker-compose service to allow sidekiq process to write to the public folder
- Fixed a bug where the export file was not being created in the public folder
This is a debugging release. No changes were made to the application.
- Exports page to list existing exports download them or delete them
- Exporting process now is done in the background, so user can close the browser tab and come back later to download the file. The status of the export can be checked on the Exports page.
ℹ️ Deleting Export file will only delete the file, not the points in the database. ℹ️
Volume, exposed to the host machine for placing files to import was changed. See the changes below.
Path for placing files to import was changed from tmp/imports
to public/imports
.
...
dawarich_app:
image: freikin/dawarich:latest
container_name: dawarich_app
volumes:
- gem_cache:/usr/local/bundle/gems
- - tmp:/var/app/tmp
+ - public:/var/app/public/imports
...
...
volumes:
db_data:
gem_cache:
shared_data:
- tmp:
+ public:
- A data migration to remove points with 0.0, 0.0 coordinates. This is necessary to prevent errors when calculating distance in Stats page.
- Reworked code responsible for importing "Records.json" file from Google Takeout. Now it is more reliable and faster, and should not throw as many errors as before.
- Test version of google takeout importing service for exports from users' phones
- Background jobs concurrency now can be set with
BACKGROUND_PROCESSING_CONCURRENCY
env variable indocker-compose.yml
file. Default value is 10. - Hand-made favicon
- Change minutes to days and hours on route popup
- Improved speed of "Stats" page loading by removing unnecessary queries
- New buttons to quickly move to today's, yesterday's and 7 days data on the map
- "Download JSON" button to points page
- For debugging purposes, now user can use
?meters_between_routes=500
and?minutes_between_routes=60
query parameters to set the distance and time between routes to split them on the map. This is useful to understand why routes might not be connected on the map. - Added scale indicator to the map
- Removed "Your data" page as its function was replaced by "Download JSON" button on the points page
- Hovering over a route now also shows time and distance to next route as well as time and distance to previous route. This allows user to understand why routes might not be connected on the map.
- Now user can hover on a route and see when it started, when it ended and how much time it took to travel
- Timestamps in export form are now correctly assigned from the first and last points tracked by the user
- Routes are now being split based both on distance and time. If the time between two consecutive points is more than 60 minutes, the route is split into two separate routes. This improves visibility of the routes on the map.
- Routes are now being split into separate one. If distance between two consecutive points is more than 500 meters, the route is split into two separate routes. This improves visibility of the routes on the map.
- Background jobs concurrency is increased from 5 to 10 to speed up the processing of the points.
- Point data, accepted from OwnTracks and Overland, is now being checked for duplicates. If a point with the same timestamp and coordinates already exists in the database, it will not be saved.
- Heatmap layer on the map to show the density of points
BREAKING CHANGES:
/api/v1/points
is still working, but will be deprecated in nearest future. Please use/api/v1/owntracks/points
instead.- All existing points recorded directly to the database via Owntracks or Overland will be attached to the user with id 1.
- Each user now have an api key, which is required to make requests to the API. You can find your api key in your profile settings.
- You can re-generate your api key in your profile settings.
- In your user profile settings you can now see the instructions on how to use the API with your api key for both OwnTracks and Overland.
- Added docs on how to use the API with your api key. Refer to
/api-docs
for more information. POST /api/v1/owntracks/points
endpoint.- Points are now being attached to a user directly, so you can only see your own points and no other users of your applications can see your points.
/api/v1/overland/batches
endpoint now requires an api key to be passed in the url. You can find your api key in your profile settings.- All existing points recorded directly to the database will be attached to the user with id 1.
- All stats and maps are now being calculated and rendered based on the user's points only.
- Default
TIME_ZONE
environment variable is now set to 'UTC' in thedocker-compose.yml
file.
- Fixed a bug where marker on the map was rendering timestamp without considering the timezone.
- Docker volume for importing Google Takeout data to the application
- Instruction on how to import Google Takeout data to the application
- Instruction on how to import Google Takeout data to the application
- Add Points page to display all the points as a table with pagination to allow users to delete points
- Sidekiq web interface to monitor background jobs is now available at
/sidekiq
- Now you can choose a date range of points to be exported
- Stop selecting
raw_data
column during requests toimports
andpoints
tables to improve performance.
- Rename PointsController to MapController along with all the views and routes
- Add Points page to display all the points as a table with pagination to allow users to delete points
- Stop ignoring
raw_data
column during requests toimports
andpoints
tables. This was preventing points from being created.
- In right sidebar you can now see the total amount of geopoints aside of kilometers traveled
- Improved overall performance if the application by ignoring
raw_data
column during requests toimports
andpoints
tables.
- Now you can import
records.json
file from your Google Takeout archive, not just Semantic History Location JSON files. The import process is the same as for Semantic History Location JSON files, just select therecords.json
file instead and choose "google_records" as a source.
- Swagger docs, can be found at
https:<your-host>/api-docs
- Cities, visited by user and listed in right sidebar now also have an active link to a date they were visited
- Dark/light theme switcher in navbar is now being saved in user settings, so it persists between sessions
Breaking changes:
This release changes how Dawarich handles a city visit threshold. Previously, the MINIMUM_POINTS_IN_CITY
environment variable was used to determine the minimum number of points in a city to consider it as visited. Now, the MIN_MINUTES_SPENT_IN_CITY
environment variable is used to determine the minimum minutes between two points to consider them as visited the same city.
The logic behind this is the following: if you have a lot of points in a city, it doesn't mean you've spent a lot of time there, especially if your OwnTracks app was in "Move" mode. So, it's better to consider the time spent in a city rather than the number of points.
In your docker-compose.yml file, you need to replace the MINIMUM_POINTS_IN_CITY
environment variable with MIN_MINUTES_SPENT_IN_CITY
. The default value is 60
, in minutes.
- A test for CheckAppVersion service class
- Replaced ActiveStorage with Shrine for file uploads
ActiveStorage::FileNotFoundError
error when uploading export files
- Set Redis as default cache store
- Consider timezone when parsing datetime params in points controller
- Add rescue for check version service class
- Application version badge to the navbar with check for updates button
- Npm dependencies install to Github build workflow
- Footer
- Disabled map points rendering by default to improve performance on big datasets
- Map controls to toggle polylines and points visibility
- Added content padding for mobile view
- Fixed stat card layout for mobile view
- Removed strong_params from POST /api/v1/points
ActiveStorage::FileNotFoundError: ActiveStorage::FileNotFoundError
error when uploading export files
You can now use Overland mobile app to track your location.
- Overland API endpoint (POST /api/v1/overland/batches)
You can now specify the host of the application by setting the APPLICATION_HOST
environment variable in the docker-compose.yml
file.
- Added version badge to navbar
- Added APPLICATION_HOST environment variable to docker-compose.yml to allow user to specify the host of the application
- Added CHANGELOG.md to keep track of changes
- Specified gem version in Docker entrypoint