LIU-439: Use translator avahi approach for engine.

Also added quickstart for README.rst to help people get started ASAP, based on user feedback.

README.rst

@@ -33,17 +33,59 @@ It originated from a prototyping activity as part of the `SDP Consortium
 <https://www.skatelescope.org/sdp/>`_ called Data Flow Management System (DFMS). DFMS aimed to 
 prototype the execution framework of the proposed SDP architecture.
 
-For more information about the installation and usage of the system please refer to the `documentation <https://daliuge.readthedocs.io>`_
-
-
 Development and maintenance of |daliuge| is currently hosted at ICRAR_
 and is performed by the `DIA team <http://www.icrar.org/our-research/data-intensive-astronomy/>`_.
 
-See the ``docs/`` directory for more information, or visit `our online
-documentation <https://daliuge.readthedocs.io/>`_
+Quickstart
+==========
+
+Most users will need to use DALiuGE locally only when prototyping their workflow as it 
+is designed in EAGLE. This repository provides a `Makefile` that simplifies the setup
+options for running DALiuGE locally, without needing to take into account the distributed 
+and flexible nature of the ecosystem. 
+
+.. note:: It is recommended to have Docker installed when running DALiuGE as a workflow 
+prototyping tool. Please review the installation material for Docker `here <https://docs.docker.com/engine/install/>`.  
+
+The following steps are recommended for quickstarting your DALiuGE install: 
+
+1. Create and enterthe virtual environment
+2. Build the docker images
+3. Run the docker images
+4. Confirm the images are running and accessible from EAGLE using a web browser. 
+
+Creating the virtual environment is as simple as running:: 
+
+    make virtualenv
+    source .venv/bin/activate 
+
+Now, in the ``.venv`` environment, run:: 
 
+    make docker-install 
 
+This will install a development version of the DALiuGE images and is appropriate for local
+installation (*not* a production environment). 
 
+Running both the Engine and the Translator is as a simple as:: 
+
+    make docker-run
+
+It is possible to confirm the running of both by accessing the following links in a browser
+(Opera, Firefox, or Chrome are recommended):: 
+
+    http://dlg-trans.local:8084/ # Translator
+    http://dlg-engine.local:8000/ # Node Manager 
+    http://dlg-engine.local:8001/ # Data Island Manager
+
+
+With this setup running, it is now possible to translate and deploy a prototype EAGLE workflow
+on your local machine. 
+
+For more information about the installation and usage of the system please refer to the `documentation <https://daliuge.readthedocs.io>`_
+
+
+See the ``docs/`` directory for more information, or visit `our online
+documentation <https://daliuge.readthedocs.io/>`_
 
 
 .. |daliuge| replace:: DALiuGE

daliuge-common/docker/Dockerfile.dev

@@ -9,15 +9,15 @@ LABEL stage=builder
 LABEL build=$BUILD_ID
 RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
 RUN apt-get update && \
-    apt-get install -y avahi-utils gcc python3 python3.8-venv python3-pip python3-distutils python3-appdirs libmetis-dev curl git sudo && \
+    apt-get install -y avahi-daemon avahi-utils gcc python3.9 python3.9-venv python3-pip python3-distutils python3-appdirs libmetis-dev curl git sudo && \
     apt-get clean
 
 COPY / /daliuge
 
-RUN cd / && python3 -m venv dlg && cd /daliuge && \
+RUN cd / && python3.9 -m venv dlg && cd /daliuge && \
     . /dlg/bin/activate && \
     pip install --upgrade pip && \
     pip install wheel numpy && \
     pip install .
 
-# we don't clean this up, will be done in the derived images
+# we don't clean this up, will be done in the derived images

daliuge-engine/docker/Dockerfile.dev

@@ -4,7 +4,11 @@ FROM icrar/daliuge-common:${VCS_TAG:-latest}
 
 # RUN sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends tzdata \
 #     gcc g++ gdb casacore-dev clang-tidy-10 clang-tidy libboost1.71-all-dev libgsl-dev
-RUN service dbus start && service avahi-daemon start && avahi-set-host-name dlg-engine
+RUN service avahi-daemon stop && \
+    service dbus start && \
+    service avahi-daemon start && \
+    avahi-set-host-name dlg-engine
+
 COPY / /daliuge
 RUN . /dlg/bin/activate && pip install --upgrade pip && pip install wheel && cd /daliuge && \
     pip install . 

daliuge-engine/docker/Dockerfile.devall

@@ -5,10 +5,16 @@ FROM icrar/daliuge-common:${VCS_TAG:-latest}
 # RUN sudo apt-get update && sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends tzdata \
 #     gcc g++ gdb casacore-dev clang-tidy-10 clang-tidy libboost1.71-all-dev libgsl-dev
 
+RUN apt install -y git python3.9-dev
+
 RUN apt-get update &&\
     DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends gcc curl python3-pip python3-numpy
 
-RUN service dbus start && service avahi-daemon start && avahi-set-host-name dlg-engine
+RUN service avahi-daemon stop && \
+    service dbus start && \
+    service avahi-daemon start && \
+    avahi-set-host-name dlg-engine
+
 # USER ${USER}
 COPY / /daliuge
 RUN . /dlg/bin/activate && pip install wheel && cd /daliuge && \
@@ -26,7 +32,6 @@ ENV VIRTUAL_ENV=/dlg
 ENV PATH="$VIRTUAL_ENV/bin:$PATH"
 ENV DLG_ROOT="/tmp/dlg"
 
-RUN apt install -y git python3-dev
 
 # Nifty
 #RUN pip install --prefix=$PYTHON_PREFIX git+https://gitlab.com/ska-telescope/sdp/ska-gridder-nifty-cuda.git

daliuge-engine/docker/Dockerfile.ray

@@ -6,7 +6,11 @@ FROM icrar/daliuge-common:${VCS_TAG:-latest}
 #     gcc g++ gdb casacore-dev clang-tidy-10 clang-tidy libboost1.71-all-dev libgsl-dev
 
 RUN sudo apt update && sudo DEBIAN_FRONTEND=noninteractive apt install -y --no-install-recommends gcc python3-pip curl
-RUN service dbus start && service avahi-daemon start && avahi-set-host-name dlg-engine
+
+RUN service avahi-daemon stop && \
+    service dbus start && \
+    service avahi-daemon start && \
+    avahi-set-host-name dlg-engine
 
 COPY / /daliuge
 RUN . /home/ray/dlg/bin/activate && \