skyline.vista package

Submodules

skyline.vista.agent module

class Vista[source]

Bases: object

Initializes Vista

run()[source]

run()[source]

Start the Vista.

Start the logger.

skyline.vista.bq_backfill module

bq_backfill.py

bq_backfill(bq_backfill_job_id)[source]: This function is used to get data from BigQuery, convert it to metrics and submit it to flux and Redis (if it would be dropped by Horizon due to settings.MAX_RESOLUTION)

skyline.vista.bq_fetcher module

skyline.vista.bq_update module

bq_update.py

bq_update(bq_update_job_id)[source]: This function is used to get data from BigQuery and compare it to the last data fetched from BigQuery for the period (if an archive is available, if not an archive is created using data from Redis and/or Graphite). The archive/current values are compared to the values for the timestamp from the latest BigQuery data for the period and if the BigQuery value has changed, the values are updated Graphite and Redis. The metrics that have changed are then reprocessed up to the timestamp of the changed data and if no anomaly is detected for the updated timestamp and previously an anomaly WAS detected for the timestamp the existing anomaly is updated with the new value and the previous value and new value for the anomaly is recorded in the database anomalies_updated table.

skyline.vista.fetcher module

class Fetcher(parent_pid)[source]

Bases: Thread

The fetcher thread asynchronisly retrieves the latest data points for metrics from multiple endpoints using asyncio and aiohttp and submits the data to the Redis set, vista.fetcher.metrics.json for the worker to process.

check_if_parent_is_alive()[source]: Self explanatory.

spawn_bq_backfill(bq_backfill_job_id)[source]

spawn_bq_update(bq_update_job_id)[source]

fetch_process(i, metrics_to_fetch)[source]

run()[source]

Called when the process intializes.
Determine if Redis is up and discover the number of unique metrics.
Wait for the processes to finish.
Send skyline.vista metrics to GRAPHITE_HOST

skyline.vista.worker module

class Worker(parent_pid)[source]

Bases: Process

The worker process retrieves time series published to the Vista Fetcher Redis set, vista.fetcher.metrics.json (and in future the vista.fetcher.metrics.csv Redis set) and validates the latest data points for each metric and submits it to Skyline Flux or directly to Graphite.

check_if_parent_is_alive()[source]: Self explanatory.

run()[source]: Called when the process intializes.