test failures on Strawberry Perl 5.40.0 #522
Comments
|
the |
|
The GD tests are part of the PDL::IO::GD distro. Are you running tests from an old PDL, and yet don't have that old PDL available to execute those tests? If so, what could anyone change in new software to make that scenario not horribly fail? |
|
Also, as far as I know, nothing tests GD capability without first making sure it's available. I was pretty careful about that in the recent PDL::IO::Pic updates to enable using GD to read JPEGs so Windows could run the TriD demo. I'm looking at line 97 of P:IO:Pic, and it's only a little below line 89, which puts that behind an |
|
I'm running from a git checkout from my fork, synched to PDL master. There are version checks but they need to be more stringent. This seems to get past the flexraw and pic test failures: diff --git a/lib/PDL/IO/Pic.pm b/lib/PDL/IO/Pic.pm
index d3ac2aac..7a84f941 100644
--- a/lib/PDL/IO/Pic.pm
+++ b/lib/PDL/IO/Pic.pm
@@ -86,7 +86,7 @@ sub init_converter_table {
$Dflags = '';
%converter = ();
- if (eval {require PDL::IO::GD; 1}) {
+ if (eval {require PDL::IO::GD; $PDL::IO::GD::VERSION ge '2.101' ? 1 : 0}) {
$converter{JPEG} = {referral => {
put => sub {
my $pdl = $_[0];
diff --git a/t/flexraw_fortran.t b/t/flexraw_fortran.t
index 7923c257..cbf6d270 100644
--- a/t/flexraw_fortran.t
+++ b/t/flexraw_fortran.t
@@ -26,7 +26,7 @@ $Verbose |= $PDL::Verbose;
my $null = ' 2>' . File::Spec->devnull;
my $datalen = length($data);
-eval "use PDL::Slatec";
+eval "use PDL::Slatec 2.096";
plan skip_all => "Skipped tests as no Slatec" if $@;
plan skip_all => "temp file path too long for f77 ($datalen chars), skipping all tests"
if $datalen > 70; |
|
I won't accept those, I'm afraid, without properly understanding how your system can partially load those things and have to be stopped by a version check. Routinely accepting that sort of change would quickly turn into a rickety maintenance nightmare. That won't be happening. |
|
However! It has just occurred to me that the IO::Pic bit of code could do a |
|
These fix the symptoms, but of course might not be a solution. To paraphrase the original comment, this is a system which already has When When |
|
Updated diff for The diff --git a/lib/PDL/IO/Pic.pm b/lib/PDL/IO/Pic.pm
index d3ac2aac..c36d9697 100644
--- a/lib/PDL/IO/Pic.pm
+++ b/lib/PDL/IO/Pic.pm
@@ -86,7 +86,7 @@ sub init_converter_table {
$Dflags = '';
%converter = ();
- if (eval {require PDL::IO::GD; 1}) {
+ if (eval {require PDL::IO::GD; PDL::IO::GD->can ('to_rpic') && PDL::IO::GD->can ('write_Jpeg')}) {
$converter{JPEG} = {referral => {
put => sub {
my $pdl = $_[0]; |
|
FWIW, the |
|
I'll make your IO::Pic bit into a commit (with your name on it) now, thank you for the effort.
I am somewhat concerned about the Compression failure, and will see if I can repro locally. |
|
As shown by the above commits, the IO::Pic module is now fixed (I repro-ed on my system, and checked the fix worked), and Once I have more information on that Compression failure, then I can comment better. Could you add some instrumentation to it, to make it dump the full |
Thanks for sorting that out. Removing flexraw makes sense given PDL:Slatec is deprecated anyway.
The current version of I extracted the current I can't see any functional changes to There have, however, been changes to |
|
More historical info: The |
|
Sorry, neglected to add the debugging steps. These are the dump and info of |
|
The failure is in the |
To avoid confusion: IO::FlexRaw is useful beyond Fortran, and it's staying. But there's another test for it that doesn't use Fortran. |
|
The newer test failing with older PDL-with-that-Compression is no surprise, because I found and fixed a bug in Compression recently. Can you confirm when you're running the latest test against the latest PDL, that it's not loading the old Compression? |
|
I checked |
|
OK. I was asking about PDL::Compression, not about either of those things? |
|
All are from |
|
Updated diagnostics given it was the second call that failed, not the first. |
|
Another data point, but the issue disappears when the failing input ndarray is converted from For reference: diff --git a/t/compression.t b/t/compression.t
index 51de80a1..7c12ef79 100755
--- a/t/compression.t
+++ b/t/compression.t
@@ -20,7 +20,13 @@ my $compressed_correct = pdl(byte, '[[126 48 24 0 96 48 14 179 32 54 219 109 147
my $got = eval { $compressed_correct->rice_expand($compressed_correct->dim(0), 64) };
is $@, '', 'no error';
is_pdl $got, $expected, 'decompress correct version gives right answer';
+
+$expected = pdl long, $expected->unpdl;
+diag $expected;
+diag $expected->info;
+
($y, $xsize, undef, $len) = $expected->rice_compress(32);
$got = eval { $y->rice_expand($len, $xsize) };
is $@, '', 'no error';
is_pdl $got, $expected, 'decompress same as original (2)'; |
|
OK, that feels like progress. Does the same thing happen on different versions of Perl? Which compiler is being used? I can imagine some numerical edge-case being what triggered the bug that the test-change captured and fixed (which I did just by updating the RICE code from CFITSIO), and that being treated differently by different compilers. I'm using Strawperl 5.32's MinGW, it has GCC 8.3.0. By the way, a slightly less roundabout way of converting that |
I can provide more details if needed.
I figured there was an easier way. (I dropped into the docs a bit below where that's documented). Testing further, the code also fails for types |
That's an answer to the second question. How about the first? :-) Especially, do you get the same results with 5.32 as I do (i.e. it passes with latest
This does sound a bit like some C compiler stuff. Let's see what the results of the previous point indicate. |
|
SP 5.32 passes with git master. SP 5.36 fails (it uses If anyone following along wants to try replicating the issue, Strawberry Perl can be downloaded from https://strawberryperl.com/releases.html. Download and unzip a portable version, and then run |
|
Some more data using SP 5.40. Tests fail using OPTIMIZE set to SP 5.40 uses |
|
As regards t/pic_16bit.t on SP-5.40.0, I notice that |
The bug was that PDL::IO::Pic was loading PDL::IO::GD, and assuming that it was a very recent one with new functionality that enabled JPEGs to be read and written on platforms without NetPBM, including Strawberry. That has now been fixed. If you want JPEGs readable by PDL post 2.096, you have to install PDL::IO::GD which is now an external module. Until I added that to PDL::IO::GD, Strawberry could not read JPEGs at all. |
Thanks. I just had a look at https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html, and it says That will inform what adjustments are likely to fix things. I wouldn't be surprised at all if a |
|
The Of those extra ones used for Strawberry Perl, the one I can see missing from the list in the GCC docs is Will have a go at the compilation. Hopefully it is not an interaction between two or more optimisations. |
I am definitely hoping that also! But we can check that as a further step. My thought for how to go quickly is to binary-search, so make a full list of the optimisation flags If there are interactions, then that first one could be moved to the start, and the binary search could be done again between indices 1 (instead of 0) and |
|
It looks like the cause is The compression tests pass on my system when using the commands below. It would be good if others could try to replicate. Hopefully someone has an idea why this argument triggers the fail under Interestingly, all PDL tests pass when using For those interested in the process, I ended up running the full Makefile.pl/gmake process with |
It still fails for 5.36, with the default Its GCC is |
I tried (a copy'n'paste of) that mantra using freshly unpacked PDL-2.098 source, but it still failed for me: That was running SP-5.40.0.1, on Windows 11 : I can see the |
Drat. Same here. It might have been an insufficiently clean tree. I was using |
|
I've updated my script to use I then ran a few combinations with the following results:
Which suggests something amiss related to how Edit: In the event it it useful, I've uploaded the script as a gist at https://gist.github.com/shawnlaffan/ed9a0c7d7026e6f1d7b5014d309f51e2 |
|
@shawnlaffan Thank you for this investigative work. It's difficult for me to see what I could do in PDL to avoid what looks a great deal like a compiler bug. Do you agree? With this picture of things, what am I as PDL person supposed to do about this? |
|
It could well be a compiler bug but confirmation of that is beyond me. There are several possibilities I see to get PDL to install "out of the box" on Strawberry Perl. These are not mutually exclusive.
The third option is a potential maintenance burden, although it would be a "set once" thing if the first option works. It would be good to try the second option if it is possible. If it works then that part could be conditional on versions of SP rather than all of PDL in option three. Pointers welcome. |
|
We switched from using -O2 to -Os because -O2 started failing (threads-related) tests. It will be most noteworthy if you find that you can successfully get perl to pass its test suite using -O2 and a recent version of gcc. ASIDE: "-O2" is fine with unthreaded builds, and also with 32-bit builds. The third option works for me, and strikes me as being the best solution if the first option is unworkable. I think it should apply to all Windows builds, irrespective of gcc version. If "OPTIMIZE" doesn't include "-Os", I think it will already be -O2 anyway. Perhaps, instead of building perl with "-Os plus allowable additions" we should be looking at building with "-O2 minus necessary deletions" ? I don't usually build PDL, but I've just built PDL-2.098 on my I've now just done a second build using Maybe the optimization bug SP is up against has been fixed in gcc-14. |
That makes sense, particularly given there is only one I suspect you're right that the bug is fixed in gcc-14. I'm reluctant to shift to gcc-14 for SP 5.40, partly for continuity and also given Perl needs to be patched (I think?), but will contemplate a more recent gcc-13. Discussion of that can be done on the relevant Strawberry Perl issue tracker. |
|
Since this looks like it's not a PDL issue as such, please could you close this issue? That's so that you could reopen it if it turns out PDL can be changed. If I close it, you can't. |
|
There still needs to be some resolution of items 2 and 3 from #522 (comment). Both of those would involve changes to PDL but would allow it to build without needing command line arguments. |
|
I saw those. I'm not accepting that as a responsibility that PDL has to maintain. If Strawberry needs to do a local patch to the Makefile.PL and/or adjust Makefile or command line in order to build on a visibly broken compiler, then Strawberry can do that. There isn't anything wrong with the code as distributed. |
|
OK. The SP build system can set optimisation levels and other parameters so does not need to patch anything for future builds. If anyone has suggestions for option 2 then I'd still be interested for general knowledge. |
Building using Strawberry Perl 5.40.0 PDL edition, using current head (b7980e4) generates the following failures.
Note that this version of Strawberry Perl comes with PDL 2.089.
PDL::IO::GDwas split off recently but has not yet been separately installed on this system. Hence the available version is from PDL 2.089. Perhaps the GD tests need to be version-gated to only run withPDL-IO-GD2.101 or later?I'm not sure about the other two. I'll add more details if I get a chance to drill into them.
The text was updated successfully, but these errors were encountered: