Airflow set up, part 3: Fixing issue with docker compose syntax

[tskir]

Closes opentargets/issues#3143.

Third and final part of updates for opentargets/issues#3143, split for readability and easier review.

For me, on Ubuntu, docker-compose (with hyphen) works, while docker compose (with space) does not:

docker: 'compose' is not a docker command.
See 'docker --help'

I installed it in a standard way via apt (sudo apt install docker.io).

It looks like this distinction has a long history: docker/compose#8630; and certainly there are fixes and workarounds about this.

However, I was wondering: @d0choa @ireneisdoomed, does docker-compose (with hyphen) work for you on your machines?

• If yes, we can leave this option as seemingly the most compatible.
• If no, then I'll amend this PR and specify this discrepancy and potential workarounds.

[d0choa]

I have no issue using docker-compose:

❯ ls /usr/local/bin/ | grep docker
com.docker.cli
docker
docker-compose
docker-compose-v1
docker-credential-desktop
docker-credential-ecr-login
docker-credential-gcloud
docker-credential-osxkeychain
docker-diagnose
docker-image-tool.sh
docker-index
kubectl.docker

Update: But the fact that the official Airflow documentation is written using the docker compose syntax makes me wonder:
https://airflow.apache.org/docs/apache-airflow/stable/howto/docker-compose/index.html#initialize-the-database

Update: it seems docker-compose is deprecated. You might need to look at your setup @tskir
https://stackoverflow.com/questions/66514436/difference-between-docker-compose-and-docker-compose

[tskir]

Thank you very much for the pointers @d0choa! I investigated further, and apparently the issue was that my Docker installation on Ubuntu was done years ago, before it even had official Ubuntu packages. So all this time I was using an unofficial package docker.io, which, while it kept updating the actual Docker versions, didn't port the compose command to new syntax.

I've reverted syntax back to "docker compose" and added a note in case anyone encounters this in the future.

[codecov-commenter]

Codecov Report

Merging #223 (528136e) into tskir-3143-2-fix-for-uid (ebc383e) will not change coverage.
The diff coverage is n/a.

@@                    Coverage Diff                    @@
##           tskir-3143-2-fix-for-uid     #223   +/-   ##
=========================================================
  Coverage                     85.64%   85.64%           
=========================================================
  Files                            81       81           
  Lines                          1853     1853           
=========================================================
  Hits                           1587     1587           
  Misses                          266      266           

[d0choa]

We can live without this warning. We can probably assume someone is using a moderately modern version of Docker.

[tskir]

@d0choa Here's the thing though: I did have the latest version of Docker installed and still ran into this error. It's not about the version, it's about the packages which were used to install it.

Docker only started officially providing Linux packages two years ago. So everyone who installed it on Debian/Ubuntu more than two years ago using the docker.io package (which was the recommended way to install at the time), and then kept updating to the latest version as I did, will still run into this problem.

My opinion remains that having this warning, which might apply to a sizable portion of Debian/Ubuntu users, would be helpful. Especially seeing as we do keep a platform specific warning about macOS memory issues 🙃