Continued telemetry support

assets/support/configdefault.json

@@ -380,6 +380,12 @@
             },
             "stop_predator": {
             }
+        },
+        "telemetry": {
+            "enabled": false,
+            "target": "https://v0lttech.com/portal/ingest.php",
+            "vehicle_identifier": "",
+            "saved_failed_updates": true
         }
     },
     "developer": {

assets/support/configoutline.json

@@ -325,6 +325,12 @@
         "physical_controls": {
             "dashcam_saving": "dict",
             "stop_predator": "dict"
+        },
+        "telemetry": {
+            "enabled": "bool",
+            "target": "str",
+            "vehicle_identifier": "str",
+            "saved_failed_updates": "bool"
         }
     },
     "developer": {

docs/CONFIGURE.md

@@ -364,6 +364,12 @@ This document describes the configuration values found `config.json`.
                 - Each entry in this configuration section uses the GPIO pin number as a key, and contains the following values:
                     - `name` is a human friendly name for the pin, and can be set to any plain text string.
                     - `hold_time` is a floating point number that determines the number of seconds the button needs to be held to trigger the event.
+        - `telemetry` contains settings for configuring Predator to send telemetry data like dash-cam previous and GPS data to an external service (see [docs/INTEGRATION.md](docs/INTEGRATION.md) for more information).
+            - `enabled` is a boolean that determined whether telemetry sending is enabled.
+            - `target` is the network target that telemetry data will be submitted to.
+            - `vehicle_identifier` is the identifier used to authenticate with the external service.
+            - saved_failed_updates` is a boolean that determines whether Predator will store failed telemetry submissions, and attempt to automatically re-upload them later.
+                - When enabled, a file named `telemetry_backlog.json` will be created in the working directory.
 - `developer` contains technical configuration values that developers and experienced users might be interested in.
     - `ignore_list` contains settings for configuring a list of license plates that Predator will ignore.
         - `enabled` is a boolean that determines whether custom ignore lists are enabled or disabled.

docs/INTEGRATION.md

@@ -1,13 +1,13 @@
 # Integration
 
-This document explains how external local programs can interface with Predator.
+This document explains how external programs can interface with Predator.
 
 If you're interested in developing your own application to integrate with Predator, or you just want to understand how things work behind the scenes, then you're in the right place!
 
 
-## Collection
+## Collection - Local
 
-Predator uses various files to share information with external programs. External programs can read these files to collect information from Predator.
+Predator uses various files to share information with external programs. Programs running on the same system can read these files from the working directory to collect information from Predator in real-time.
 
 ### Heartbeat
 
@@ -49,8 +49,8 @@ Example file contents:
     "mode": "dashcam/parked_dormant",
     "gps": 3,
     "performance": {
-        "front": 49.571633251
-        "rear": 46.661928151
+        "front": 49.571633251,
+        "rear": 46.661928151,
         "cabin": 21.128199629
     }
 }
@@ -156,13 +156,55 @@ Example file contents:
 ```
 
 
-## Triggers
+## Triggers - Local
 
-In addition to the files for sharing information from Predator to external programs, external programs can create specific files to trigger certain events in Predator.
+In addition to the files for sharing information, programs running on the same system can create specific files to trigger certain events in Predator.
 
 ### Dashcam Saving
 
 When an important event happens while driving, a user may want to save a specific dashcam video segment to a separate folder so that it doesn't get accidentally deleted. This is referred to as "locking" or "saving" a dashcam video.
 
 Dashcam saving can be triggered by create the file specified by the `dashcam>saving>trigger` configuration value inside the interface directory. When this file is created, Predator will save the current dashcam video to the configured directory, then delete the trigger file. The trigger file does not need to contain any information. Only its existence is required to trigger Predator.
 
+
+## Telemetry - Remote
+
+Predator allows users to share telemetry with remote network targets.
+
+Predator sends telemetry as a POST request with two fields:
+- `"identifier"` is a unique identifier used to authenticate with the remote service.
+    - This value is defined in the configuration as `dashcam>telemetry>vehicle_identifier`
+- `"data"` contains the telemetry data as a JSON string with the following structure:
+    - `system` contains information about the Predator system. This field must be present.
+        - `timezone` is the timezone relative to UTC, which follows the format "UTC+00:00". This field must be present.
+    - `image` contains image information captured by each of the devices defined in the Predator configuration as `dashcam>capture>video>devices`.
+        - Each capture device has a field in this section, with the capture device as the key, and the image as the value (encoded in base64).
+        - This section can be omitted, and may not always be present in data submissions.
+    - `location` contains GPS location information. This section must be included, but can be set to all zeros if no GPS data is available.
+        - `time` is the Unix timestamp when this point was captured.
+        - `lat` is the latitude of this point.
+        - `lon` is the longitude of this point.
+        - `alt` is the altitude (in meters) of this point.
+        - `spd` is the speed (in meters per second) of this point.
+        - `head` is the heading of travel of this point.
+
+Below is an example of the "data" field, formatted for readability:
+```json
+{
+   "system": {
+        "timezone": "UTC-04:00"
+    },
+    "image": {
+        "front": "QECAgICAgQDAgICAgUEBAME...",
+        "rear": "SNVs45WGRagZLerHJ6nA4x2N..."
+    },
+    "location": {
+        "time": 1451606400,
+        "lat": 41.688906,
+        "lon": -81.208030,
+        "alt": 241.4,
+        "spd": 31.9,
+        "head": 40.0
+    }
+}
+```

utils.py

@@ -178,8 +178,8 @@ def process_timing(action, identifier):
         print(e)
         print("cv2 is required because the `developer>frame_count_method` value is set to 'manual' or 'opencv'")
 
-# TODO: Check if Base64 is needed (remote telemetry is enabled).
-import base64
+if ("telemetry" in config["dashcam"] and config["dashcam"]["telemetry"]["enabled"] == True): # Check to see if base64 is needed (for encoding images).
+    import base64
 
 
 
@@ -894,22 +894,42 @@ def count_frames(video):
         video_frame_count = 0
     return video_frame_count
 
+
+
+
+
+if ("telemetry" in config["dashcam"] and config["dashcam"]["telemetry"]["saved_failed_updates"]):
+    telemetry_backlog_file_location = config["general"]["interface_directory"] + "/telemetry_backlog.json"
+    if (os.path.exists(telemetry_backlog_file_location) == False): # If the backlog file doesn't exist, create it.
+        save_to_file(telemetry_backlog_file_location, "{}") # Save a blank placeholder dictionary to the backlog file.
+
+    telemetry_backlog_file = open(telemetry_backlog_file_location, "r") # Open the backlog log file for reading.
+    telemetry_backlog_contents = telemetry_backlog_file.read() # Read the raw contents of the backlog file as a string.
+    telemetry_backlog_file.close() # Close the backlog log file.
+
+    if (is_json(telemetry_backlog_contents) == True): # If the backlog file contains valid JSON data, then load it.
+        telemetry_backlog = json.loads(telemetry_backlog_contents) # Read and load the backlog log from the file.
+    else: # If the backlog file doesn't contain valid JSON data, then load a blank placeholder in it's place.
+        telemetry_backlog = json.loads("{}") # Load a blank placeholder dictionary.
 # This function is called for every frame, and uploads dash-cam telemetry information at regular intervals.
 last_telemetry_sent = 0 # This holds the timestamp of the last time telemetry was sent.
 def send_telemetry(data):
     global last_telemetry_sent
+    global telemetry_backlog
     data["system"] = {}
     data["system"]["timezone"] = timezone_offset_stamp # Add the system timezone to the data.
+
+    # Format:
     # data = {
-    #    "system": {
-    #         "timezone": "UTC+00:00"
+    #    "system": { # Must be present
+    #         "timezone": "UTC+00:00" # Must be present and valid.
     #     },
-    #     "image": {
+    #     "image": { # Can be omitted if no image data is present.
     #         "main": [OpenCV image],
     #         "rear": "[OpenCV image]",
     #         ...
     #     },
-    #     "location": {
+    #     "location": { # Must be present, but can be set to all zeros.
     #         "time": 0.0,
     #         "lat": 0.0,
     #         "lon": 0.0,
@@ -918,22 +938,62 @@ def send_telemetry(data):
     #         "head": 0.0
     #     }
     # }
-    if (time.time() - last_telemetry_sent >= 10): # Check to see if it has been at least 10 seconds since the last telemetry update.
-
+    if (time.time() - last_telemetry_sent >= 10): # Check to see if it has been at least 10 seconds since the last telemetry update. Note: most external receivers (such as V0LT Portal) will reject any data within 10 seconds of a previous datapoint.
+        print("Sending telemetry") # TODO: REMOVE
         for device in data["image"]:
+            # Rescale the image to 720p:
             original_height, original_width = data["image"][device].shape[:2]
             target_height = 720
             scale_factor = target_height / original_height
-            new_width = int(original_width * scale_factor)
+            if (scale_factor < 1): # Check to make sure we're shrinking the image, not upscaling it.
+                new_width = int(original_width * scale_factor)
+                data["image"][device] = cv2.resize(data["image"][device], (new_width, target_height)) # Resize the frame to the target size.
 
-            data["image"][device] = cv2.resize(data["image"][device], (new_width, target_height)) # Resize the frame to the target size.
+            # Encode the image to base64 for transmission:
             retval, buffer = cv2.imencode('.jpg', data["image"][device])
             data["image"][device] = base64.b64encode(buffer).decode("utf-8")
+
         last_telemetry_sent = time.time()
 
         submission = json.dumps(data) # Convert the image information into a string.
-        request = requests.post("http://192.168.0.137/portal/ingest.php", data={"identifier": "ebdbaf0781f03d54422b1321", "data": submission}, timeout=8) # TODO: Add configuration options.
-        # TODO: Check to see if the request was successful.
+        try:
+            request = requests.post(config["dashcam"]["telemetry"]["target"], data={"identifier": config["dashcam"]["telemetry"]["identifier"], "data": submission}, timeout=8) # Submit the data.
+            response = request.content
+            if (is_json(response)):
+                response = json.loads(response)
+                if ("success" in response and response["success"] == True):
+                    success = True
+                else:
+                    success = False
+                    if ("reason" in response):
+                        display_message("The remote telemetry provider responded with an error: " + str(response["reason"]), 2)
+                    else:
+                        display_message("The remote telemetry provider responded with an unknown error.", 2)
+            else:
+                display_message("The response from the remote telemetry provider was not valid JSON.", 2)
+                success = False
+        except:
+            display_message("The telemetry", 2)
+            success = False
+
+        if (success == True):
+            if (len(telemetry_backlog) > 0): # Check to see if there is at least one request in the back-log.
+                for timestamp in telemetry_backlog: # Iterate over each entry in the telemetry back-log.
+                    try:
+                        request = requests.post(config["dashcam"]["telemetry"]["target"], data={"identifier": config["dashcam"]["telemetry"]["identifier"], "data": json.dumps(telemetry_backlog[timestamp])}, timeout=8) # Submit the data.
+                        response = request.content
+                        if ("success" in response and response["success"] == True):
+                            success = True
+                        else:
+                            success = False
+                    except:
+                        success = False
+                    if (success == True): # Check to see if the data submission was successful.
+                        del telemetry_backlog[timestamp] # Delete this point from the back-log.
+                save_to_file(telemetry_backlog_file_location, json.dumps(telemetry_backlog)) # Save the updated back-log file.
+        else:
+            telemetry_backlog[data["location"]["time"]] = data # Add this datapoint to the back-log.
+            save_to_file(telemetry_backlog_file_location, json.dumps(telemetry_backlog)) # Save the updated back-log file.