skyline.luminosity package¶

Submodules¶

skyline.luminosity.agent module¶

skyline.luminosity.classify_anomalies module¶

classify_anomalies(i, classify_anomalies_set, start_timestamp, classify_for)[source]¶

skyline.luminosity.classify_metrics module¶

classify_metrics(i, start_timestamp, classify_for)[source]¶

skyline.luminosity.luminosity module¶

class PaulBourke[source]¶

Bases: object

# @added 20180526 - Branch #2270: luminosity The PaulBourke class pays homage to Paul Bourke who originally described the cross correlation method on which Linkedin’s luminol.correlate library is based upon, he described this in August 1996 and it is upon this which Skyline Luminosity is also based.

So Skyline has moved from: Shewhart’s statistical process control circa 1924 tech To tsfresh - circa 20161029 when it entered Skyline Now to 1996(2015), although 1996 ideas can still be very, very useful and work well.

Please do visit these following two URLs in order: http://paulbourke.net/miscellaneous/correlate/thanks_mr_bourke_from_skyline_for # this will 404 but hopefully he has something watching his 404 rate :) http://paulbourke.net/miscellaneous/correlate/ # this will 200

This is all part of the adventures in Skyline. If you enjoy this sort of thing, then I posit to you that Sir Walter Munk is one of the least known remarkable scientist of tthe current and previous millienia. Born: October 19, 1917 (age 100 years) His accomplishments abound, he is like Turing, Marconi, Berners-Lee and Einstein rolled into Oceangraphy, but few have ever heard of him.

If we are giving kudos to Shewhart, tsfresh and Paul Bourke, we can slip some Walther Munk in here too, although currently he nor his work has anything to do with Skyline, however there are some ideas, not necessarily desrcibed in the roadmap, that revolve a byproduct of monitoring surf conditions via a webcams, which could be used for monitoring tide levels too, which would be right up Dr Munk’s alley.

class Luminosity(parent_pid)[source]¶

Bases: threading.Thread

The Luminosity class which controls the luminosity thread and spawned processes.

check_if_parent_is_alive()[source]¶: Self explanatory

mysql_insert(insert)[source]¶

Insert data into mysql table

Parameters:	insert (str) – the insert string
Returns:	int
Return type:	int or boolean

Example usage:

query = 'insert into host (host) VALUES ('this_host')'
result = self.mysql_insert(query)

Note

If the MySQL query fails a boolean will be returned not a tuple
- False
- None

classify_anomalies(i, classify_anomalies_set, start_timestamp, max_run_seconds)[source]¶

Classify anomalies

Parameters:	i (object) – python process id classify_anomalies_set (set) – set from luminosity.classify_anomalies start_timestamp (int) – the process star timestamp max_run_seconds (int) – the max number of seconds to run for
Returns:	boolean
Return type:	boolean

classify_metrics(i, start_timestamp, max_run_seconds)[source]¶

Classify metrics

Parameters:	i (object) – python process id start_timestamp (int) – the process star timestamp max_run_seconds (int) – the max number of seconds to run for
Returns:	boolean
Return type:	boolean

spin_process(i, anomaly_id)[source]¶

Assign an anomalous metric and determine correlated metrics

Parameters:	i (object) – python process id anomaly_id (int) – the anomaly_id
Returns:	boolean
Return type:	boolean

run()[source]¶: Called when the process intializes.

skyline.luminosity.process_correlations module¶

Module contents¶

Luminosity

Determine correlations
Insert in correlations table (slow stream of inserts, not gluts, queue them)
- anomaly_id, mertic_id, coefficient, shift, shifted_coefficient
- a table per metric a la z_ts or;
- a table per month and a daily aggregated table?
In Ionosphere:
- Display Luminosity sorted_correlated_metrics in a block with a link to the anomaly page
In Panorama:
- Display a Luminosity link the anomaly page