======
1.2.16
======

v1.2.16 - May 4, 2019

This is a features and dependencies update release.

Changes in v1.2.16
------------------

- Dependencies updated (2964)
- Report defaulted settings to log.  Added all the globally declared settings to
  enable reporting in the log the state of each setting (2950)
- Added Skyline slack thread updates on alerts (2646)

Update notes
------------

- These update instruction apply to upgrading from v1.2.15 to v1.2.16 only.
  However as with all Skyline updates it is possible to go through the update
  notes for each version and make your own update notes/process to take you from
  version x to version y.
- There are changes to the DB in v1.2.16
- There are changes to settings.py in v1.2.16

.. note: the requirements.txt specifies urllib3>=1.25.1 however requests needs
  this fix at <=1.24.3.  Please amend the requirements.txt

How to update from v1.2.15
--------------------------

- Download the new release tag or clone/update to get it to a temp location,
  ready to be deployed.
- Diff your current running version skyline/settings.py with the new version of
  skyline/settings.py and create a new settings.py in the temp location
  appropriate for Skyline v1.2.16 and your instance with the new
  :mod:`settings.SLACK_OPTS` dictionary options, ready to be deployed.  The new
  options that have been added to the `SLACK_OPTS` dictionary are shown below.
  Replace the `YOUR_` variables with your channel and channel_id strings.  You
  can set the message_on options for each event type.  If you do not want
  Skyline to update slack threads, set `thread_updates: False`:

.. code-block:: python

      # Your default slack Skyline channel name e.g. '#skyline'
      'default_channel': 'YOUR_default_slack_channel',
      # Your default slack Skyline channel id e.g. 'C0XXXXXX'
      'default_channel_id': 'YOUR_default_slack_channel_id',
      # Whether to update slack message threads on any of the below events
      'thread_updates': True,
      # You can disable or enable each message_on event
      'message_on_training_data_viewed': True,
      'message_on_training_data_viewed_reaction_emoji': 'eyes',
      'message_on_features_profile_created': True,
      'message_on_features_profile_created_reaction_emoji': 'thumbsup',
      'message_on_features_profile_learnt': True,
      'message_on_features_profile_learnt_reaction_emoji': 'heavy_check_mark',
      'message_on_features_profile_disabled': True,
      'message_on_features_profile_disabled_reaction_emoji': 'x',
      'message_on_validated_features_profiles': True,


- Stop Skyline Ionosphere and Panorama services.
- Backup your Skyline database.
- Apply the SQL update to your Skyline database as per described below, change
  the below variables as appropriate to your set up.
- Stop all the Skyline services and ensure any service controls such as monit,
  supervisor or systemd do not start them again, (change as appropriate for
  your set up)

.. code-block:: bash

    NEW_SKYLINE_VERSION="v1.2.16"    # Your new Skyline version
    OLD_SKYLINE_VERSION="v1.2.15"    # Your old Skyline version

    CURRENT_SKYLINE_PATH="/opt/skyline/github/skyline"                 # Your Skyline path
    NEW_SKYLINE_PATH="${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"  # Your new Skyline path

    mkdir -p "${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"
    cd "${CURRENT_SKYLINE_PATH}.${NEW_SKYLINE_VERSION}"
    git clone https://github.com/earthgecko/skyline .
    git checkout "$NEW_SKYLINE_VERSION"

    cp "$NEW_SKYLINE_PATH/skyline/settings.py" "$NEW_SKYLINE_PATH/skyline/settings.py.${NEW_SKYLINE_VERSION}.bak"
    cat "${CURRENT_SKYLINE_PATH}/skyline/settings.py" > "$NEW_SKYLINE_PATH/skyline/settings.py"

    # ADD the new SLACK_OPTS to settings
    vi "$NEW_SKYLINE_PATH/skyline/settings.py"

    # Stop Skyline DB related services
    SKYLINE_SERVICES="ionosphere
    luminosity
    panorama"
    for i in $SKYLINE_SERVICES
    do
      /etc/init.d/$i stop
    done
    # And stop any service controls like monit or systemd

    # BACKUP THE DB AND APPLY THE NEW SQL
    BACKUP_DIR="/tmp"  # Where you want to backup the DB to
    MYSQL_USER="<YOUR_MYSQL_USER>"
    MYSQL_HOST="127.0.0.1"  # Your MySQL IP
    MYSQL_DB="skyline"  # Your MySQL Skyline DB name

    # Backup DB
    mkdir -p $BACKUP_DIR
    mysqldump -u$MYSQL_USER -p $MYSQL_DB > $BACKUP_DIR/pre.$NEW_SKYLINE_VERSION.$MYSQL_DB.sql

    # Update DB
    mysql -u$MYSQL_USER -p $MYSQL_DB < "${NEW_SKYLINE_PATH}/updates/sql/v1.2.16-stable.sql"

    # Stop all other Skyline services
    SKYLINE_SERVICES="horizon
    analyzer
    mirage
    crucible
    boundary
    ionosphere
    luminosity
    panorama
    webapp"
    for i in $SKYLINE_SERVICES
    do
      /etc/init.d/$i stop
    done

- Move your current Skyline directory to a backup directory and move the new
  Skyline v1.2.16 with your new settings.py from the temp location to your
  working Skyline directory, (change your paths as appropriate) e.g.

.. code-block:: bash

    mv "$CURRENT_SKYLINE_PATH" "${CURRENT_SKYLINE_PATH}.${OLD_SKYLINE_VERSION}"
    mv "$NEW_SKYLINE_PATH" "$CURRENT_SKYLINE_PATH"

- Update the dependencies

.. code-block:: bash

    PYTHON_MAJOR_VERSION="2.7"
    PYTHON_VIRTUALENV_DIR="/opt/python_virtualenv"
    # Handle 2.7.14 and 2.7.16
    PROJECT="skyline-py2716"
    if [ ! -d "${PYTHON_VIRTUALENV_DIR}/projects/${PROJECT}" ]; then
      PROJECT="skyline-py2714"
    fi
    cd "${PYTHON_VIRTUALENV_DIR}/projects/${PROJECT}"
    source bin/activate
    # This can take lots of minutes with the new pandas
    bin/"pip${PYTHON_MAJOR_VERSION}" install $(cat "${CURRENT_SKYLINE_PATH}/requirements.txt" | grep "^numpy\|^scipy\|^patsy\|^pandas" | tr '\n' ' ')
    # Remove the luminol egg as it will already be installed and pip will moan
    cat "${CURRENT_SKYLINE_PATH}/requirements.txt" | grep -v "luminol" > /tmp/requirements.txt

    # IF YOU ARE ON CENTOS 6 and running mainstream mysql-server-5.1 uncomment
    # the following line and run it to fix to mysql-python-connector 8.0.6
    #cat "${CURRENT_SKYLINE_PATH}/requirements.txt" | grep -v "luminol\|mysql-connector-python" > /tmp/requirements.txt

    # With v1.2.16 a urllib version of 1.25.2 was available but it does not work
    # with requests
    cat /tmp/requirements.txt | sed -e 's/urllib3.*/urllib3==1.24.3/g' > /tmp/requirements.2.1.16.txt
    cat /tmp/requirements.2.1.16.txt > /tmp/requirements.txt

    bin/"pip${PYTHON_MAJOR_VERSION}" install -r /tmp/requirements.txt

- Start the all Skyline services (change as appropriate for your set up) e.g.

.. code-block:: bash

    # Start all other Skyline services
    SKYLINE_SERVICES="panorama
    luminosity
    horizon
    analyzer
    mirage
    crucible
    boundary
    ionosphere
    webapp"
    for i in $SKYLINE_SERVICES
    do
      /etc/init.d/$i start
    done

- Check the logs

.. code-block:: bash

    # How are they running
    tail -n 20 /var/log/skyline/*.log

    # Any errors - each app
    find /var/log/skyline -type f -name "*.log" | while read skyline_logfile
    do
      echo "#####
    # Checking for errors in $skyline_logfile"
      cat "$skyline_logfile" | grep -B2 -A10 -i "error ::\|traceback" | tail -n 60
      echo ""
      echo ""
    done