skyline.webapp package¶
Submodules¶
skyline.webapp.backend module¶
-
panorama_request
()[source]¶ Gets the details of anomalies from the database, using the URL arguments that are passed in by the
request.args
to build the MySQL select query string and queries the database, parse the results and creates an array of the anomalies that matched the query and creates thepanaroma.json
file, then returns the array. The Webapp needs both the array and the JSONP file to serve to the browser for the client sidepanaroma.js
.Parameters: None – determined from request.args
Returns: array Return type: array Note
And creates
panaroma.js
for client side javascript
-
get_list
(thing)[source]¶ Get a list of names for things in a database table.
Parameters: thing (str) – the thing, e.g. ‘algorithm’ Returns: list Return type: list
-
luminosity_remote_data
(anomaly_timestamp)[source]¶ Gets all the unique_metrics from Redis and then mgets Redis data for all metrics. The data is then preprocessed for the remote Skyline luminosity instance and only the relevant fragments of the time series are returned. This return is then gzipped by the Flask Webapp response to ensure the minimum about of bandwidth is used.
Parameters: anomaly_timestamp (int) – the anomaly timestamp Returns: list Return type: list
skyline.webapp.gunicorn module¶
skyline.webapp.ionosphere_backend module¶
-
ionosphere_get_metrics_dir
(requested_timestamp, context)[source]¶ Get a list of all the metrics in timestamp training data or features profile folder
Parameters: - requested_timestamp (str) – the training data timestamp
- context (str) – the request context, training_data or features_profiles
Returns: tuple of lists
Return type: (list, list, list, list)
-
ionosphere_data
(requested_timestamp, data_for_metric, context)[source]¶ Get a list of all training data or profiles folders and metrics
Parameters: - requested_timestamp (str) – the training data or profile timestamp
- data_for_metric (str) – the metric base_name
- context (str) – the request context, training_data or features_profiles
Returns: tuple of lists
Return type: (list, list, list, list)
-
ionosphere_metric_data
(requested_timestamp, data_for_metric, context, fp_id)[source]¶ Get a list of all training data folders and metrics
-
features_profile_details
(fp_id)[source]¶ Get the Ionosphere details of a fetures profile
Parameters: fp_id (str) – the features profile id Returns: tuple Return type: (str, boolean, str, str)
-
ionosphere_search
(default_query, search_query)[source]¶ Gets the details features profiles from the database, using the URL arguments that are passed in by the
request.args
to build the MySQL select query string and queries the database, parse the results and creates an array of the features profiles that matched the query.Parameters: None – determined from request.args
Returns: array Return type: array
-
create_ionosphere_layers
(base_name, fp_id, requested_timestamp)[source]¶ Create a layers profile.
Parameters: None – determined from request.args
Returns: array Return type: array
-
feature_profile_layers_detail
(fp_layers_id)[source]¶ Get the Ionosphere layers details of a fetures profile
Parameters: fp_layers_id – the features profile layers_id Returns: tuple Return type: (str, boolean, str, str, object)
-
feature_profile_layer_alogrithms
(fp_layers_id)[source]¶ Get the Ionosphere layer algorithm details of a layer
Parameters: fp_layers_id – the features profile layers_id Returns: tuple Return type: (str, boolean, str, str)
-
metric_layers_alogrithms
(base_name)[source]¶ Get the Ionosphere layer algorithm details of a metric
Parameters: base_name (str) – the metric base_name Returns: tuple Return type: (str, boolean, str, str)
-
edit_ionosphere_layers
(layers_id)[source]¶ Edit a layers profile.
Parameters: layers_id – the layer id to edit Returns: array Return type: array
-
validate_fp
(update_id, id_column_name, user_id)[source]¶ Validate a single features profile or validate all enabled, unvalidated features profiles for a metric_id.
Parameters: - update_id (int) – the features profile id or metric_id to validate
- id_column_name (str) – the column name to select where on, e.g. id or metric_id
- user_id (int) – the user id of the user that is validating
Returns: tuple
Return type: (boolean, str, str)
-
save_training_data_dir
(timestamp, base_name, label, hdate)[source]¶ Save training_data and return details or just return details if exists
Parameters: - timestamp (str) – the Ionosphere training_data metric timestamp
- base_name (str) – metric base_name
- label (str) – the saved training_data label
- hdate (str) – human date for the saved training_data
Returns: saved_successful, details, fail_msg, trace
Return type: boolean, list, str, str
-
features_profile_family_tree
(fp_id)[source]¶ Returns the all features profile ids of the related progeny features profiles, the whole family tree.
Parameters: fp_id – the features profile id Returns: array Return type: array
-
disable_features_profile_family_tree
(fp_ids)[source]¶ Disable a features profile and all related progeny features profiles
Parameters: fp_ids – a list of the the features profile ids to disable Returns: array Return type: array
-
get_fp_matches
(metric, metric_like, get_fp_id, get_layer_id, from_timestamp, until_timestamp, limit, sort)[source]¶ Get all the matches.
Parameters: - metric – all or the metric name
- metric_like – False or the metric MySQL like string e.g statsd.%
- get_fp_id – None or int
- get_layer_id – None or int
- from_timestamp – timestamp or None
- until_timestamp – timestamp or None
- limit – None or number to limit to
- sort – DESC or ASC
Returns: list
Return type: list
-
get_matched_id_resources
(matched_id, matched_by, metric, requested_timestamp)[source]¶ Get the Ionosphere matched details of a features profile or layer
Parameters: - matched_id – the matched id
- matched_by – either features_profile or layers
- metric – metric base_name
- requested_timestamp – the timestamp of the features profile
Returns: tuple
Return type: (str, boolean, str, str)
-
get_features_profiles_to_validate
(base_name)[source]¶ Get the details for Ionosphere features profiles that need to be validated for a metric and returns a list of the details for each of the features profile including the ionosphere_image API URIs for all the relevant graph images for the weabpp Ionosphere validate_features_profiles page. For example:
[[ fp_id, metric_id, metric, fp_full_duration, anomaly_timestamp, fp_parent_id, parent_full_duration, parent_anomaly_timestamp, fp_date, fp_graph_uri, parent_fp_date, parent_fp_graph_uri, parent_parent_fp_id, fp_learn_graph_uri, parent_fp_learn_graph_uri, minimum_full_duration, maximum_full_duration, generation]]
Parameters: base_name (str) – metric base_name Returns: list of lists Return type: [[int, int, str, int, int, int, int, int, str, str, str, str, int, str, str, int, int]]
-
get_metrics_with_features_profiles_to_validate
()[source]¶ Get the metrics with Ionosphere features profiles that need to be validated and return a list of the details for each metric. [[metric_id, metric, fps_to_validate_count]]
Returns: list of lists Return type: [[int, str, int]]
-
ionosphere_show_graphs
(requested_timestamp, data_for_metric, fp_id)[source]¶ Get a list of all graphs
-
webapp_update_slack_thread
(base_name, metric_timestamp, value, message_context)[source]¶ Update slack threads with enabled events.
Parameters: - base_name (str) – metric base_name
- metric_timestamp (str or int) – the anomaly_timestamp
- value (int or None) – the features profile id, the validated_count or None
- message_context (str) – training_data_viewed or layers_created
Returns: True or False
Return type: boolean
-
validate_ionosphere_match
(match_id, validate_context, match_validated, user_id)[source]¶ Update the validated value in the DB for the match.
Parameters: - match_id (str) – the match id
- validate_context (str) – the context to validate either ionosphere_matched or ionosphere_layers_matched
- match_validated (int) – 1 for valid or 2 for invalid
- user_id (int) – the user id of the user validating
Returns: True or False
Return type: boolean
skyline.webapp.utilities module¶
-
alerts_matcher
(base_name, pattern, alerter, second_order_resolution_hours)[source]¶ Get a list of all the metrics that would match an ALERTS pattern
Parameters: - base_name (str) – The metric name
- pattern (str) – the ALERT pattern
- alerter (str) – the alerter name e.g. smtp, syslog, hipchat, pagerdaty
- second_order_resolution_hours (int) – (optional) The number of hours that Mirage should surface the metric timeseries for
Returns: matched_by
Return type: str
(‘metric3’, ‘hipchat’, 600), # Log all anomalies to syslog (‘stats.’, ‘syslog’, 1), # Wildcard namespaces can be used as well (‘metric4.thing.*.requests’, ‘stmp’, 900), # However beware of wildcards as the above wildcard should really be (‘metric4.thing..*..requests’, ‘stmp’, 900),
skyline.webapp.webapp module¶
-
limit_remote_addr
()[source]¶ This function is called to check if the requesting IP address is in the settings.WEBAPP_ALLOWED_IPS array, if not 403.
-
check_auth
(username, password)[source]¶ This function is called to check if a username / password combination is valid.
-
internal_error
(message, traceback_format_exc)[source]¶ Show traceback in the browser when running a flask app on a production server. By default, flask does not show any useful information when running on a production server. By adding this view, we output the Python traceback to the error 500 page and log.
As per: Show flask traceback when running on production server https://gist.github.com/probonopd/8616a8ff05c8a75e4601 - Python traceback rendered nicely by Jinja2
This can be tested by hitting SKYLINE_URL/a_500
-
get_client_details
()[source]¶ Gets the first X-Forwarded-For address and sets as the IP address. Gets the client_id by simply using a md5 hash of the client IP address and user agent. Determines whether the request was proxied. Determines the client protocol.
Returns: client_id, protocol, proxied, salt Return type: str, str, boolean, str
-
decode_token
(client_id)[source]¶ Use the app.secret, client_id and salt to decode the token JWT encoded payload and determine the Redis password.
Parameters: client_id (str) – the client_id string Returns: token, decoded_redis_password, fail_msg, trace Return type: str, str, str, str