skyline.horizon package

Submodules

skyline.horizon.agent module

class Horizon

Bases: object

Initializes Horizon

run()

Determine the MAX_QUEUE_SIZE for the listen process.

Determine if horizon should populate the mini redis store for Oculus.

Starts the defined number of WORKER_PROCESSES, with the first worker populating the canary metric.

Start the pickle (and UDP) listen processes.

Start roomba.

run()

Start the Horizon agent and logger.

skyline.horizon.listen module

class SafeUnpickler

Bases: _pickle.Unpickler

PICKLE_SAFE = {'__builtin__': {'object'}, 'copy_reg': {'_reconstructor'}}

find_class(module, name)

Return an object from a specified module.

If necessary, the module will be imported. Subclasses may override this method (e.g. to restrict unpickling of arbitrary classes and functions).

This method is called whenever a class or a function object is needed. Both arguments passed are str objects.

classmethod loads(pickle_string)

class Listen(port, queue, parent_pid, type='pickle')

Bases: multiprocessing.context.Process

The listener is responsible for listening on a port.

gen_unpickle(infile)

Generate a pickle from a stream

read_all(sock, n)

Read n bytes from a stream

check_if_parent_is_alive()

Self explanatory

listen_pickle()

Listen for pickles over tcp

listen_udp()

Listen over udp for MessagePack strings

run()

Called when process intializes.

skyline.horizon.roomba module

class Roomba(parent_pid, skip_mini)

Bases: threading.Thread

The Roomba is responsible for deleting keys older than DURATION.

check_if_parent_is_alive()

Self explanatory.

vacuum(i, namespace, duration)

Trim metrics that are older than settings.FULL_DURATION and purge old metrics.

run()

Called when process initializes.

skyline.horizon.worker module

worker.py

class Worker(queue, parent_pid, skip_mini, worker_number, canary=False)

Bases: multiprocessing.context.Process

The worker processes chunks from the queue and appends the latest datapoints to their respective timesteps in Redis.

check_if_parent_is_alive()

Self explanatory.

in_skip_list(metric_name)

Check if the metric is in SKIP_LIST.

# @added 20170319 - Feature #1978: worker - DO_NOT_SKIP_LIST The SKIP_LIST allows for a string match or a match on dotted elements within the metric namespace.

in_shard(metric_name)

Check if the metric belongs to the Horizon instance shard.

run()

Called when the process intializes.

Module contents