Build failure: postgresql with pythonSupport = true

[hrdinka]

Hi everyone,

since 445371f (which added strictDeps = true;) the build of PostgreSQL is broken if python support is enabled.

Steps To Reproduce

Here is a minimal example using flakes:

{
  # offending commit
  inputs.nixpkgs.url = "github:nixos/nixpkgs?ref=445371f309a62e9107ad78fb3c65fff768efb43c";

  # commit before (on master), that works
  # inputs.nixpkgs.url = "github:nixos/nixpkgs?ref=31faab38bee31d3692ca976a599185d189fb60f9";

  # and parent, which also works
  # inputs.nixpkgs.url = "github:nixos/nixpkgs?ref=5c74641e44678cfcec3eb8b1bf184e2f1ac1caae";

  outputs =
    { self, nixpkgs }:
    {
      packages.x86_64-linux.default = nixpkgs.legacyPackages.x86_64-linux.postgresql_16.override {
        pythonSupport = true;
      };
    };
}

Save it in flake.nix in a git repo and call nix build. Change the system (x86_64-linux) in case needed.

Build log

Build Log

structuredAttrs is enabled
Running phase: unpackPhase
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking source archive /nix/store/ppk11f9397ykkqaj2v3pajilrg6zjckd-postgresql-16.6.tar.bz2
source root is postgresql-16.6
setting SOURCE_DATE_EPOCH to timestamp 1731962883 of file postgresql-16.6/INSTALL
Running phase: patchPhase
@nix { "action": "setPhase", "phase": "patchPhase" }
applying patch /nix/store/2hippmr9naclnhlk1w0ddz4prhk1ggih-relative-to-symlinks-16+.patch
patching file src/common/exec.c
applying patch /nix/store/ipcrcsn4chil3dw70rijydkxspp2wdka-empty-pg-config-view-15+.patch
patching file src/backend/utils/misc/pg_config.c
patching file src/test/regress/expected/sysviews.out
patching file src/test/regress/sql/sysviews.sql
applying patch /nix/store/dzx9iq7nnh3r66kn76cwjq49kzgvvssf-less-is-more.patch
patching file src/include/fe_utils/print.h
Hunk #1 succeeded at 20 (offset 2 lines).
applying patch /nix/store/046vri33dmh9z8sl1l77mbyh2rc9wjis-paths-for-split-outputs.patch
patching file src/common/config_info.c
Hunk #1 succeeded at 117 (offset -1 lines).
patching file src/Makefile.global.in
applying patch /nix/store/zp3wdx74ahiqyx97djx285im92sg881q-paths-with-postgresql-suffix.patch
patching file src/Makefile.global.in
Hunk #1 succeeded at 101 (offset -1 lines).
Hunk #2 succeeded at 135 (offset -1 lines).
applying patch /nix/store/1q9whcmr694qncagy53zqwy5hmlx0ql6-locale-binary-path.patch
patching file src/backend/commands/collationcmds.c
Hunk #1 succeeded at 829 (offset 218 lines).
applying patch /nix/store/5103krcj82l78xz6dkqw8scwf510wfz3-socketdir-in-run-13+.patch
patching file src/include/pg_config_manual.h
Hunk #1 succeeded at 206 (offset 5 lines).
Running phase: updateAutotoolsGnuConfigScriptsPhase
@nix { "action": "setPhase", "phase": "updateAutotoolsGnuConfigScriptsPhase" }
Updating Autotools / GNU config script to a newer upstream version: ./config/config.sub
Updating Autotools / GNU config script to a newer upstream version: ./config/config.guess
Running phase: configurePhase
@nix { "action": "setPhase", "phase": "configurePhase" }
patching script interpreter paths in ./configure
./configure: interpreter directive changed from "#! /bin/sh" to "/nix/store/bvnsygak2cmxqpgvckv5brg1xy7rkxy4-bash-5.2p37/bin/sh"
configure flags: --prefix=/nix/store/83q6p4kp43nm7dpbl735i8gcrv7iya09-postgresql-16.6 --bindir=/nix/store/83q6p4kp43nm7dpbl735i8gcrv7iya09-postgresql-16.6/bin --sbindir=/nix/store/83q6p4kp43nm7dpbl735i8gcrv7iya09-postgresql-16.6/sbin --includedir=/nix/store/1fyvlg7m8a32d4krik1764yi29h32wcr-postgresql-16.6-dev/include --oldincludedir=/nix/store/1fyvlg7m8a32d4krik1764yi29h32wcr-postgresql-16.6-dev/include --mandir=/nix/store/d6cfffmpjpw2jj9f8jszsf6vg87rrssg-postgresql-16.6-man/share/man --infodir=/nix/store/83q6p4kp43nm7dpbl735i8gcrv7iya09-postgresql-16.6/share/info --docdir=/nix/store/93xlva0hrqg8cm6n9vgldg5clnmq3mr4-postgresql-16.6-doc/share/doc/postgresql --libdir=/nix/store/76k7j73nmkgdg4kb3wl7fwhahzm5x5hm-postgresql-16.6-lib/lib --libexecdir=/nix/store/76k7j73nmkgdg4kb3wl7fwhahzm5x5hm-postgresql-16.6-lib/libexec --localedir=/nix/store/76k7j73nmkgdg4kb3wl7fwhahzm5x5hm-postgresql-16.6-lib/share/locale --with-openssl --with-libxml --with-icu --sysconfdir=/etc --with-system-tzdata=/nix/store/qj0qhvccmi563cwggfsgc2xy21cpla4r-tzdata-2024b/share/zoneinfo --enable-debug --with-systemd --with-uuid=e2fs --with-lz4 --with-zstd --with-gssapi --with-python --with-pam
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking which template to use... linux
checking whether NLS is wanted... no
checking for default port number... 5432
checking for block size... 8kB
checking for segment size... 1GB
checking for WAL block size... 8kB
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for gcc option to accept ISO C99... none needed
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for gawk... gawk
checking whether gcc supports -Wdeclaration-after-statement, for CFLAGS... yes
checking whether gcc supports -Werror=vla, for CFLAGS... yes
checking whether gcc supports -Werror=unguarded-availability-new, for CFLAGS... no
checking whether g++ supports -Werror=unguarded-availability-new, for CXXFLAGS... no
checking whether gcc supports -Wendif-labels, for CFLAGS... yes
checking whether g++ supports -Wendif-labels, for CXXFLAGS... yes
checking whether gcc supports -Wmissing-format-attribute, for CFLAGS... yes
checking whether g++ supports -Wmissing-format-attribute, for CXXFLAGS... yes
checking whether gcc supports -Wimplicit-fallthrough=3, for CFLAGS... yes
checking whether g++ supports -Wimplicit-fallthrough=3, for CXXFLAGS... yes
checking whether gcc supports -Wcast-function-type, for CFLAGS... yes
checking whether g++ supports -Wcast-function-type, for CXXFLAGS... yes
checking whether gcc supports -Wshadow=compatible-local, for CFLAGS... yes
checking whether g++ supports -Wshadow=compatible-local, for CXXFLAGS... yes
checking whether gcc supports -Wformat-security, for CFLAGS... yes
checking whether g++ supports -Wformat-security, for CXXFLAGS... yes
checking whether gcc supports -fno-strict-aliasing, for CFLAGS... yes
checking whether g++ supports -fno-strict-aliasing, for CXXFLAGS... yes
checking whether gcc supports -fwrapv, for CFLAGS... yes
checking whether g++ supports -fwrapv, for CXXFLAGS... yes
checking whether gcc supports -fexcess-precision=standard, for CFLAGS... yes
checking whether g++ supports -fexcess-precision=standard, for CXXFLAGS... yes
checking whether gcc supports -funroll-loops, for CFLAGS_UNROLL_LOOPS... yes
checking whether gcc supports -ftree-vectorize, for CFLAGS_VECTORIZE... yes
checking whether gcc supports -Wunused-command-line-argument, for NOT_THE_CFLAGS... no
checking whether gcc supports -Wcompound-token-split-by-macro, for NOT_THE_CFLAGS... no
checking whether gcc supports -Wformat-truncation, for NOT_THE_CFLAGS... yes
checking whether gcc supports -Wstringop-truncation, for NOT_THE_CFLAGS... yes
checking whether gcc supports -Wcast-function-type-strict, for NOT_THE_CFLAGS... no
checking whether gcc supports -fvisibility=hidden, for CFLAGS_SL_MODULE... yes
checking whether g++ supports -fvisibility=hidden, for CXXFLAGS_SL_MODULE... yes
checking whether g++ supports -fvisibility-inlines-hidden, for CXXFLAGS_SL_MODULE... yes
checking whether the C compiler still works... yes
checking how to run the C preprocessor... gcc -E
checking pkg-config is at least version 0.9.0... yes
checking allow thread-safe client libraries... yes
checking whether to build with ICU support... yes
checking for icu-uc icu-i18n... yes
checking whether to build with Tcl... no
checking whether to build Perl modules... no
checking whether to build Python modules... yes
checking whether to build with GSSAPI support... yes
checking whether to build with PAM support... yes
checking whether to build with BSD Authentication support... no
checking whether to build with LDAP support... no
checking whether to build with Bonjour support... no
checking whether to build with SELinux support... no
checking whether to build with systemd support... yes
checking whether to build with XML support... yes
checking for libxml-2.0 >= 2.6.23... yes
checking whether to build with LZ4 support... yes
checking for liblz4... yes
checking whether to build with ZSTD support... yes
checking for libzstd >= 1.4.0... yes
checking for strip... strip
checking whether it is possible to strip libraries... yes
checking for ar... ar
checking for a BSD-compatible install... /nix/store/g066plklksv7jrrqvbwdkhl4d09zizzg-coreutils-9.5/bin/install -c
checking for tar... /nix/store/q7c79zajqbqaj5gz6491hnxak6f35i19-gnutar-1.35/bin/tar
checking whether ln -s works... yes
checking for a thread-safe mkdir -p... /nix/store/g066plklksv7jrrqvbwdkhl4d09zizzg-coreutils-9.5/bin/mkdir -p
checking for bison... no
configure: WARNING:
*** Without Bison you will not be able to build PostgreSQL from Git nor
*** change any of the parser definition files.  You can obtain Bison from
*** a GNU mirror site.  (If you are using the official distribution of
*** PostgreSQL then you do not need to worry about this, because the Bison
*** output is pre-generated.)
checking for flex... no
configure: WARNING:
*** Without Flex you will not be able to build PostgreSQL from Git nor
*** change any of the scanner definition files.  You can obtain Flex from
*** a GNU mirror site.  (If you are using the official distribution of
*** PostgreSQL then you do not need to worry about this because the Flex
*** output is pre-generated.)
checking for perl... no
configure: WARNING:
*** Without Perl you will not be able to build PostgreSQL from Git.
*** You can obtain Perl from any CPAN mirror site.
*** (If you are using the official distribution of PostgreSQL then you do not
*** need to worry about this, because the Perl output is pre-generated.)
checking for python3... no
checking for python... no
configure: error: Python not found

Metadata

Notify maintainers

@wolfgangwalther (author of offending commit)
@thoughtpolice
@Ma27

---

Note for maintainers: Please tag this issue in your PR.

---

Add a 👍 reaction to issues you find important.

[eclairevoyant]

Yeah, python3 should be in nativeBuildInputs, not buildInputs.

[wolfgangwalther]

Yeah, python3 should be in nativeBuildInputs, not buildInputs.

I'd say it should be in both, right? We still need it at run-time, I guess.

[eclairevoyant]

mkDerivation wouldn't do anything with it in buildInputs.

[wolfgangwalther]

mkDerivation wouldn't do anything with it in buildInputs.

It would link to it. So for cross-compiling this is important, imho.

Or to put it the other way around: The fact that putting python in nativeBuildInputs makes the build succeed is not because python is a build tool - it's because PostgreSQL's configure script is really bad at cross-compiling and detecting dependencies properly.

PostgreSQL needs python at run-time for plpython3u. So it needs to link the hostPlatform's variant, not the buildPlatform's one.

[hrdinka]

There is a PYTHON environment variable to set the path of the interpreter:

https://www.postgresql.org/docs/current/install-make.html#CONFIGURE-ENVVARS-PYTHON

Further PostgreSQL should be wrapped to also set PYTHONPATH, or else modules don’t work. I wrapped it in the past in an override, but currently I am supplying it as an ENV variable via systemd due changes in the derivation, which is not the best solution.

I will try to make some free time to test this out in the following days.