skyline.snab package
Submodules
skyline.snab.agent module
skyline.snab.algorithms module
skyline.snab.snab module
- class SNAB(parent_pid)[source]
Bases:
Thread
The SNAB class which controls the snab thread and spawned processes.
- spin_snab_process(i, check_details)[source]
Assign a metric and algorithm for a process to analyze.
- Parameters:
i – python process id
run_timestamp – the epoch timestamp at which this process was called
metric_name – the FULL_NAMESPACE metric name as keyed in Redis
last_analyzed_timestamp – the last analysed timestamp as recorded in the Redis key last_timestamp.basename key.
It is possible to want to run a metric through SNAB to check an algorithm in the normal anomaly detection workflow, in the algorithm testing context and concurrently. An algorithm could be added as part of the main detection process and another algorithm could be simply being tested. Although handling them separately is preferable as any testing algorithms will not slow down the real time processing. However having to load and preprocess that data for every algorithm is also inefficient, so there must be some trade offs. Therefore if a metric has multiple SNAB configurations, testing and realtime a check item must be added for each context. SNAB propritises realtime over testing.
check_details is a dict with the following structure: check_details = {
‘metric’: ‘<base_name>’|str, ‘labelled_metric_name’: ‘<labelled_metric_name>’|str, ‘timestamp’: anomaly_timestamp|int, ‘value’: datapoint|float, ‘full_duration’: full_duration|int, ‘anomaly_data’: ‘<absolute_path/filename>’|str, ‘source’: ‘<skyline_app>’|str, ‘added_at’: added_at_timestamp|int, ‘original_added_at’: original_added_at|int, ‘context’: ‘<testing|realtime>’|str, ‘algorithm’: ‘algorithm_1’|str, ‘algorithm_source’: ‘<absolute_path/filename>’|str’, ‘algorithm_parameters’: {}|dict, ‘max_execution_time’: max_seconds|ifloat, ‘debug_logging’: False|boolean, ‘analysed’: False|boolean, ‘processed’: False|boolean, ‘anomalous’: None|boolean, ‘anomalyScore’: 0.0|float, ‘alert_slack_channel’: ‘<channel_name>|None’|str or boolean, ‘snab_only’: False|boolean,
}
# @added 20201001 - food for thought Walk the dogs… Back to making the black hole. Does information have weight? At what point does the quantity of digital information in a space become to dense or too much? How much does a byte weigh? What about an EM wave/s? Could the pending singularity event not be an AI event, but black hole event? Could digital information or signals ever “supernova”? What is the equivalent to Hubbles Law for digital information and signals? Will all our digital information eventually cross an event horizon and vanish from our perception?
- Returns:
anomalous
- Return type:
boolean