Overview

A Python script that extracts System Activity data from the last 10 minutes, formats the data as Audit Logs, and exports the logs to Cloud Logging. The data formatting/mapping is best effort. See data mapping below.

NOTE: You can schedule this script to run every 10 minutes using a cron job or equivalent to continually create and export logs.

Requirements

• Looker Instance in which you have Admin or see_system_activity permission
• Google Cloud Project with Cloud Logging API enabled
• pyenv installed

Deployment

• Create Looker API credentials and set the below environment variables

  export LOOKERSDK_BASE_URL="<Your API URL>"
  export LOOKERSDK_CLIENT_ID="<Your Client ID>"
  export LOOKERSDK_CLIENT_SECRET="<Your Client Secret>"
  

• Create and configure a service account to write log entries to Cloud Logging and download the keys

  export GOOGLE_APPLICATION_CREDENTIALS="<Service Account Key Path>"
  

• Clone the repo

  git clone https://github.com/itodotimothy6/extract-looker-logs.git
  cd extract-looker-logs/
  

• Setup python virtual environment

  pyenv install 3.8.2
  pyenv local 3.8.2
  python -m venv .venv
  source .venv/bin/activate
  

• Install dependencies

  pip install looker-sdk
  pip install google-cloud-logging
  

• Run main.py

  python main.py
  

GCP Audit Log Fields to Looker System Activity Mapping

GCP Audit Log Field	Looker System Actvity Field or Value
logName	looker_system_activity_logs
timestamp	event.created
resource.type	looker_system_activity_logs
insertId	event.id
protoPayload.status	event.attribute.status
protoPayload.authenticationInfo	event.user_id, event.sudo_user_id
protoPayload.authorizationInfo	permission_set.permissions
protoPayload.methodName	event.name
protoPayload.response	event_attributes