skyline.snab package¶
Submodules¶
skyline.snab.agent module¶
skyline.snab.algorithms module¶
skyline.snab.snab module¶
-
class
SNAB(parent_pid)[source]¶ Bases:
threading.ThreadThe SNAB class which controls the snab thread and spawned processes.
-
spin_snab_process(i, check_details)[source]¶ Assign a metric and algorithm for a process to analyze.
Parameters: - i – python process id
- run_timestamp – the epoch timestamp at which this process was called
- metric_name – the FULL_NAMESPACE metric name as keyed in Redis
- last_analyzed_timestamp – the last analysed timestamp as recorded in the Redis key last_timestamp.basename key.
It is possible to want to run a metric through SNAB to check an algorithm in the normal anomaly detection workflow, in the algorithm testing context and concurrently. An algorithm could be added as part of the main detection process and another algorithm could be simply being tested. Although handling them separately is preferable as any testing algorithms will not slow down the real time processing. However having to load and preprocess that data for every algorithm is also inefficient, so there must be some trade offs. Therefore if a metric has multiple SNAB configurations, testing and realtime a check item must be added for each context. SNAB propritises realtime over testing.
check_details is a dict with the following structure: check_details = {
‘metric’: ‘<base_name>’|str, ‘timestamp’: anomaly_timestamp|int, ‘value’: datapoint|float, ‘full_duration’: full_duration|int, ‘anomaly_data’: ‘<absolute_path/filename>’|str, ‘source’: ‘<skyline_app>’|str, ‘added_at’: added_at_timestamp|int, ‘original_added_at’: original_added_at|int, ‘context’: ‘<testing|realtime>’|str, ‘algorithm’: ‘algorithm_1’|str, ‘algorithm_source’: ‘<absolute_path/filename>’|str’, ‘algorithm_parameters’: {}|dict, ‘max_execution_time’: max_seconds|ifloat, ‘debug_logging’: False|boolean, ‘analysed’: False|boolean, ‘processed’: False|boolean, ‘anomalous’: None|boolean, ‘anomalyScore’: 0.0|float, ‘alert_slack_channel’: ‘<channel_name>|None’|str or boolean, ‘snab_only’: False|boolean,}
# @added 20201001 - food for thought Walk the dogs… Back to making the black hole. Does information have weight? At what point does the quantity of digital information in a space become to dense or too much? How much does a byte weigh? What about an EM wave/s? Could the pending singularity event not be an AI event, but black hole event? Could digital information or signals ever “supernova”? What is the equivalent to Hubbles Law for digital information and signals? Will all our digital information eventually cross an event horizon and vanish from our perception?
Returns: anomalous Return type: boolean
-