Merge branch 'release-v0.2.31'

.dockerignore

@@ -25,4 +25,3 @@ notebooks
 examples
 docs
 build
-coverage.xml

.github/workflows/pythontest.yaml

@@ -29,12 +29,14 @@ jobs:
     steps:
       - name: Checkout code
         uses: actions/checkout@v2
-      - name: Build and test panoptes-utils in container
+      - name: Build panoptes-utils in container
         run: |
           docker build -t panoptes-utils:testing -f tests/Dockerfile .
           mkdir -p logs && chmod -R 777 logs
           mkdir -p build && chmod -R 777 build
-          docker run --rm -i -v "${PWD}/build:/var/panoptes/panoptes-utils/build" -v "${PWD}/logs:/var/panoptes/logs" panoptes-utils:testing
+      - name: Test panoptes-utils
+        run: |
+          docker run --rm -i -v "${PWD}/build:/app/build" -v "${PWD}/logs:/app/logs" panoptes-utils:testing
       - name: Upload coverage report to codecov.io
         uses: codecov/codecov-action@v1
         if: success()

CHANGELOG.rst

@@ -2,6 +2,38 @@
 Changelog
 =========
 
+0.2.31 - 2020-01-31
+-------------------
+
+Added
+^^^^^
+
+* Docker musical chairs:
+
+  * Add Dockerfile to be ``panoptes-utils`` but don't use for testing. (#264)
+  * Generic ``panoptes-utils`` with ``panoptes-config-server`` as default command example. (#264)
+  * Move conda ``environment`` file into ``docker`` folder. (#264)
+
+Bugs Fixed
+^^^^^^^^^^
+
+* ``parse_config_directories`` no longer modifies dictionary in place. (#264)
+
+Changed
+^^^^^^^
+
+* Clean out the Contributing guide to point to POCS. (#264)
+* Removing ``pillow<7`` requirement. (#264)
+
+Removed
+^^^^^^^
+
+* Removing unused new cli experiment. (#264)
+* Removed all ``PANDIR`` and ``PANLOG`` references. Closes #263. (#264)
+* Removed ``astroplan`` from dependencies. (#264)
+
+
+
 0.2.30 - 2021-01-14
 -------------------
 

CONTRIBUTING.md

@@ -1,146 +1,4 @@
-Please see the
-[code of conduct](https://github.com/panoptes/POCS/blob/develop/CODE_OF_CONDUCT.md)
-for our playground rules and follow them during all your contributions.
+# Contributing
 
-# Getting Started
-
-We prefer that all changes to POCS have an associated
-[GitHub Issue in the project](https://github.com/panoptes/POCS/issues)
-that explains why it is needed. This allows us to debate the best
-approach to address the issue before folks spend a lot of time
-writing code. If you are unsure about a possible contribution to
-the project, please contact the project owners about your idea;
-of course, an [issue](https://github.com/panoptes/POCS/issues) is a
-good way to do this.
-
-# Pull Request Process
-_This is a summary of the process. See
-[the POCS wiki](https://github.com/panoptes/POCS/wiki/PANOPTES-Feature-Development-Process)
-for more info._
-
-* Pre-requisites
-   - Ensure you have a [github account.](https://github.com/join)
-   - If the change you wish to make is not already an
-     [Issue in the project](https://github.com/panoptes/POCS/issues),
-     please create one specifying the need.
-* Process
-   - Create a fork of the repository and use a topic branch within your fork to make changes.
-      - All of our repositories have a default branch of `develop` when you first clone them, but
-      your work should be in a separate branch.
-      - Create a branch with a descriptive name, e.g.:
-         - `git checkout -b new-camera-simulator`
-         - `git checkout -b issue-28`
-   - Ensure that your code meets this project's standards (see Testing and Code Formatting below).
-         - Run `python setup.py test` from the `$POCS` directory before pushing to github
-   - Squash your commits so they only reflect meaningful changes.
-   - Submit a pull request to the repository, be sure to reference the issue number it
-      addresses.
-
-
-# Setting up Local Environment
-  - Follow instructions in the [README](https://github.com/panoptes/POCS/blob/develop/README.md)
-    as well as the [Coding in PANOPTES](https://github.com/panoptes/POCS/wiki/Coding-in-PANOPTES)
-    document.
-
-
-# Testing
- - All changes should have corresponding tests and existing tests should pass after
-    your changes.
- - For more on testing see the
- [Coding in PANOPTES](https://github.com/panoptes/POCS/wiki/Coding-in-PANOPTES) page.
-
-# Code Formatting
-
-- All Python should use [PEP 8 Standards](https://www.python.org/dev/peps/pep-0008/)
-   - Line length is set at 100 characters instead of 80.
-   - It is recommended to have your editor auto-format code whenever you save a file
-      rather than attempt to go back and change an entire file all at once.
-   - You can also use
-     [yapf (Yet Another Python Formatter)](https://github.com/google/yapf)
-     for which POCS includes a style file (.style.yapf). For example:
-     ```bash
-     # cd to the root of your workspace.
-     cd $(git rev-parse --show-toplevel)
-     # Format the modified python files in your workspace.
-     yapf -i $(git diff --name-only | egrep '\.py$')
-     ```
-- Do not leave in commented-out code or unnecessary whitespace.
-- Variable/function/class and file names should be meaningful and descriptive.
-- File names should be lower case and underscored, not contain spaces. For example, `my_file.py`
-instead of `My File.py`.
-- Define any project specific terminology or abbreviations you use in the file you use them.
-- Use root-relative imports (i.e. relative to the POCS directory). This means that rather
-  than using a directory relative imports such as:
-  ```python
-  from panoptes.utils.base import PanBase
-  from panoptes.utils.time import current_time
-  ```
-  Import from the top-down instead:
-  ```python
-  from pocs.base import PanBase
-  from panoptes.utils.time import current_time
-  ```
-  The same applies to code inside of `peas`.
-- Test imports are slightly different because `pocs/tests` and `peas/tests` are not Python
-  packages (those directories don't contain an `__init__.py` file). For imports of `pocs` or
-  `peas` code, use root-relative imports as described above. For importing test packages and
-  modules, assume the test doing the imports is in the root directory.
-
-# Log Messages
-
-Use appropriate logging:
-- Log level:
-   - DEBUG (i.e. `self.logger.debug()`) should attempt to capture all run-time
-      information.
-   - INFO (i.e. `self.logger.info()`) should be used sparingly and meant to convey
-      information to a person actively watching a running unit.
-   - WARNING (i.e. `self.logger.warning()`) should alert when something does not
-      go as expected but operation of unit can continue.
-   - ERROR (i.e. `self.logger.error()`) should be used at critical levels when
-      operation cannot continue.
-- The logger supports variable information without the use of the `format` method.
-- There is a `say` method available on the main `POCS` class that is meant to be
-used in friendly manner to convey information to a user. This should be used only
-for personable output and is typically displayed in the "chat box"of the PAWS
-website. These messages are also sent to the INFO level logger.
-
-#### Logging examples:
-
-_Note: These are meant to illustrate the logging calls and are not necessarily indicative of real
-operation_
-
-```py
-self.logger.info("PANOPTES unit initialized: {}", self.config['name'])
-
-self.say("I'm all ready to go, first checking the weather")
-
-self.logger.debug("Setting up weather station")
-
-self.logger.warning('Problem getting wind safety: {}'.format(e))
-
-self.logger.debug("Rain: {} Clouds: {} Dark: {} Temp: {:.02f}",
-   is_raining,
-   is_cloudy,
-   is_dark,
-   temp_celsius
-)
-
-self.logger.error('Unable to connect to AAG Cloud Sensor, cannot continue')
-```
-
-#### Viewing log files
-
-- You typically want to follow an active log file by using `tail -F` on the command line.
-- The [`grc`](https://github.com/garabik/grc) (generic colouriser) can be used with
-`tail` to get pretty log files.
-
-```
-(panoptes-env) $ grc tail -F $PANDIR/logs/pocs_shell.log
-```
-
-The following screenshot shows commands entered into a `jupyter-console` in the top
-panel and the log file in the bottom panel.
-
-<p align="center">
-   <img src="http://www.projectpanoptes.org/images/log-example.png" width="600">
-</p>
+See [the POCS wiki](https://github.com/panoptes/POCS/wiki/PANOPTES-Feature-Development-Process)
+for more info on how to contribute to the various PANOPTES repositories.

README.md

@@ -43,10 +43,36 @@ To install type:
 pip install panoptes-utils
 ```
 
-Full options for local install:
+Full options for install:
 
 ```bash
 pip install -e ".[config,docs,images,testing,social]"
 ```
 
 See the full documentation at: https://panoptes-utils.readthedocs.io
+
+Docker Service
+==============
+
+The `docker` folder defines an image that can be used as the base for other  
+PANOPTES services.
+
+The `Dockerfile` is built by the `cloudbuild.yaml` and stored in Google  
+Registry as `gcr.io/panoptes-exp/panoptes-utils:latest`.
+
+You can pull the image like any other docker image:
+
+```
+docker pull gcr.io/panoptes-exp/panoptes-utils:latest
+```
+
+Config Server
+-------------
+
+There is also a service defined in `docker-compose.yaml` that will run the
+`panoptes-config-server` cli tool.
+
+```bash
+PANOPTES_CONFIG_FILE=/path/to/config.yaml docker-compose \
+    -f docker/docker-compose.yaml up
+```

conftest.py

@@ -25,8 +25,7 @@
           "<lvl>{message}</lvl>"
 
 # Put the log file in the tmp dir.
-log_dir = os.getenv('PANLOG', '/var/panoptes/logs')
-log_file_path = os.path.realpath(f'{log_dir}/panoptes-testing.log')
+log_file_path = os.path.realpath(f'logs/panoptes-testing.log')
 startup_message = f' STARTING NEW PYTEST RUN - LOGS: {log_file_path} '
 logger.add(log_file_path,
            enqueue=True,  # multiprocessing
@@ -75,7 +74,7 @@ def pytest_addoption(parser):
 
 @pytest.fixture(scope='session')
 def config_path():
-    return os.getenv('PANOPTES_CONFIG_FILE', '/var/panoptes/panoptes-utils/tests/testing.yaml')
+    return os.getenv('PANOPTES_CONFIG_FILE', 'tests/testing.yaml')
 
 
 @pytest.fixture(scope='function', params=_all_databases)
@@ -106,7 +105,7 @@ def save_environ():
 
 @pytest.fixture(scope='session')
 def data_dir():
-    return os.path.expandvars('/var/panoptes/panoptes-utils/tests/data')
+    return os.path.expandvars('tests/data')
 
 
 @pytest.fixture(scope='function')

docker/.gcloudignore

@@ -0,0 +1,2 @@
+cloudbuild.yaml
+docker-compose.yaml

docker/Dockerfile

@@ -0,0 +1,70 @@
+ARG image_url=ubuntu
+ARG image_tag=latest
+FROM ${image_url}:${image_tag} AS pocs-utils
+
+LABEL description="Installs the dependencies for panoptes-utils."
+LABEL maintainers="developers@projectpanoptes.org"
+LABEL repo="github.com/panoptes/panoptes-utils"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
+
+ARG panuser=pocs-user
+ARG userid=1000
+ARG pip_install_extras="[config]"
+
+ENV PANUSER $panuser
+ENV USERID $userid
+
+# Install system dependencies.
+RUN echo "Building from ${image_name}:${image_tag}" && \
+    apt-get update && apt-get install --no-install-recommends --yes \
+        bzip2 ca-certificates \
+        wget gcc git pkg-config sudo less udev wait-for-it \
+        dcraw exiftool \
+        astrometry.net \
+        libcfitsio-dev libcfitsio-bin \
+        libfreetype6-dev libpng-dev libjpeg-dev libffi-dev && \
+    useradd -u ${USERID} -o -c "Captain POCS" \
+        -p panoptes -m -G plugdev,dialout,users,sudo ${PANUSER} && \
+    # Allow sudo without password.
+    echo "%sudo ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers && \
+    # Setup SSH so localhost works without password
+    mkdir -p "/home/${panuser}/.ssh" && \
+    echo "Host localhost\n\tStrictHostKeyChecking no\n" >> "/home/${panuser}/.ssh/config"
+
+USER "${userid}"
+
+# Miniconda
+WORKDIR /tmp
+RUN echo "Installing conda via miniforge" && \
+    sudo mkdir -p /conda && \
+    sudo chown -R "${PANUSER}:${PANUSER}" /conda && \
+    # Miniforge
+    wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-$(uname -m).sh" \
+        -O install-miniforge.sh && \
+    /bin/sh install-miniforge.sh -b -f -p /conda && \
+    # Initialize conda for the shells.
+    /conda/bin/conda init bash
+
+ENV PATH "/home/${PANUSER}/.local/bin:$PATH"
+
+COPY docker/environment.yaml .
+RUN /conda/bin/conda env update -n base -f environment.yaml
+
+RUN echo "Installing panoptes-pocs module with ${pip_install_extras}" && \
+    /conda/bin/pip install "panoptes-utils${pip_install_extras}" && \
+    # Cleanup
+    /conda/bin/pip cache purge && \
+    /conda/bin/conda clean -tipy && \
+    sudo apt-get autoremove --purge --yes && \
+    sudo apt-get autoclean --yes && \
+    sudo apt-get --yes clean && \
+    sudo rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+COPY docker/docker-compose.yaml .
+
+# We are still the PANUSER.
+ENTRYPOINT [ "/usr/bin/env", "bash", "-ic" ]
+CMD [ "panoptes-config-server", "--help" ]