skyline.analyzer_dev package

Submodules

skyline.analyzer_dev.agent module

skyline.analyzer_dev.alerters module

skyline.analyzer_dev.algorithms_dev module

get_function_name()[source]
tail_avg(timeseries)[source]

This is a utility function used to calculate the average of the last three datapoints in the series as a measure, instead of just the last datapoint. It reduces noise, but it also reduces sensitivity and increases the delay to detection.

median_absolute_deviation(timeseries)[source]

A timeseries is anomalous if the deviation of its latest datapoint with respect to the median is X times larger than the median of deviations.

grubbs(timeseries)[source]

A timeseries is anomalous if the Z score is greater than the Grubb’s score.

first_hour_average(timeseries)[source]

Calcuate the simple average over one hour, FULL_DURATION seconds ago. A timeseries is anomalous if the average of the last three datapoints are outside of three standard deviations of this value.

stddev_from_average(timeseries)[source]

A timeseries is anomalous if the absolute value of the average of the latest three datapoint minus the moving average is greater than three standard deviations of the average. This does not exponentially weight the MA and so is better for detecting anomalies with respect to the entire series.

stddev_from_moving_average(timeseries)[source]

A timeseries is anomalous if the absolute value of the average of the latest three datapoint minus the moving average is greater than three standard deviations of the moving average. This is better for finding anomalies with respect to the short term trends.

mean_subtraction_cumulation(timeseries)[source]

A timeseries is anomalous if the value of the next datapoint in the series is farther than three standard deviations out in cumulative terms after subtracting the mean from each data point.

least_squares(timeseries)[source]

A timeseries is anomalous if the average of the last three datapoints on a projected least squares model is greater than three sigma.

histogram_bins(timeseries)[source]

A timeseries is anomalous if the average of the last three datapoints falls into a histogram bin with less than 20 other datapoints (you’ll need to tweak that number depending on your data)

Returns: the size of the bin which contains the tail_avg. Smaller bin size means more anomalous.

ks_test(timeseries)[source]

A timeseries is anomalous if 2 sample Kolmogorov-Smirnov test indicates that data distribution for last 10 minutes is different from last hour. It produces false positives on non-stationary series so Augmented Dickey-Fuller test applied to check for stationarity.

determine_median(timeseries)[source]

Determine the median of the values in the timeseries

is_anomalously_anomalous(metric_name, ensemble, datapoint)[source]

This method runs a meta-analysis on the metric to determine whether the metric has a past history of triggering. TODO: weight intervals based on datapoint

run_selected_algorithm(timeseries, metric_name)[source]

Filter timeseries and run selected algorithm.

skyline.analyzer_dev.analyzer_dev module

class AnalyzerDev(parent_pid)[source]

Bases: threading.Thread

The Analyzer class which controls the analyzer thread and spawned processes.

check_if_parent_is_alive()[source]

Self explanatory

spawn_alerter_process(alert, metric, context)[source]

Spawn a process to trigger an alert.

This is used by smtp alerters so that matplotlib objects are cleared down and the alerter cannot create a memory leak in this manner and plt.savefig keeps the object in memory until the process terminates. Seeing as data is being surfaced and processed in the alert_smtp context, multiprocessing the alert creation and handling prevents any memory leaks in the parent.

Added 20160814 relating to:

Parameters as per skyline.analyzer.alerters.trigger_alert

spin_process(i, unique_metrics)[source]

Assign a bunch of metrics for a process to analyze.

Multiple get the assigned_metrics to the process from Redis.

For each metric:

  • unpack the raw_timeseries for the metric.
  • Analyse each timeseries against ALGORITHMS to determine if it is anomalous.
  • If anomalous add it to the self.anomalous_metrics list
  • Add what algorithms triggered to the self.anomaly_breakdown_q queue
  • If settings.ENABLE_CRUCIBLE is True:
    • Add a crucible data file with the details about the timeseries and anomaly.
    • Write the timeseries to a json file for crucible.

Add keys and values to the queue so the parent process can collate for:

  • self.anomaly_breakdown_q
  • self.exceptions_q
run()[source]

Called when the process intializes.

Determine if Redis is up and discover the number of unique metrics.

Divide the unique_metrics between the number of ANALYZER_PROCESSES and assign each process a set of metrics to analyse for anomalies.

Wait for the processes to finish.

Process the Determine whether if any anomalous metrics require:

  • alerting on (and set EXPIRATION_TIME key in Redis for alert).
  • feeding to another module e.g. mirage.

Populated the webapp json the anomalous_metrics details.

Log the details about the run to the skyline log.

Send skyline.analyzer metrics to GRAPHITE_HOST,

Module contents