Jigsaw - error when OUTADJUSTMENTH5 = yes and images are read only

[lwellerastro]

ISIS version(s) affected: 9.0.0-RC1

Description
Running jigsaw where the new parameter OUTADJUSTMENTH5 = yes (which is the default value) results in the following error:

  Group = Error
    Program = jigsaw
    Class   = "USER ERROR"
    Code    = 2
    Message = "Unable to bundle adjust network
               [KTC_Morning_LQ07_CombinedImageValDNAddFix_GCP2.net]"
    File    = jigsaw.cpp
    Line    = 339
    Program = jigsaw
  End_Group

  Group = Error
    Program = jigsaw
    Class   = "I/O ERROR"
    Code    = 4
    Message = "Failed to open
               [...Kaguya_TC/Global/Morning/Lev0/TC1W2B0_01_00720N621E14-
               19.lev0.cub] with read/write access"
    File    = Cube.cpp
    Line    = 714
    Program = jigsaw
  End_Group

My data are write protected for a variety of reasons - very large dataset with multiple users and too easy to write over/ modify original processing. I remove the write protection when I am ready to update, but I was not updating at this time, just running a new bundle to test and create the new .h5 file.

Permissions on my files:
-r--r--r-- 1 lweller flagstaf 37M Oct 9 10:21 .../Kaguya_TC/Global/Morning/Lev0/TC1W2B0_01_02896N410E1376.lev0.cub

Why does jigsaw need write access to these data when OUTADJUSTMENTH5 = yes ?

How to reproduce
Run jigsaw without updates against images that are read only using the following parameters:

jigsaw froml=KTC_Morning_LQ07_CombinedImageValDNAddFix_GCP2.lis
cnet=KTC_Morning_LQ07_CombinedImageValDNAddFix_GCP2.net
onet=JigOut_I900RC1.net
radius=yes update=no
sigma0=1.0e-10 maxits=10
camsolve=accelerations twist=yes overexisting=yes
spsolve=position overhermite=yes
spacecraft_position_sigma=1000
camera_angles_sigma=0.25
camera_angular_velocity_sigma=0.1
camera_angular_acceleration_sigma=0.01
point_radius_sigma=100
OUTADJUSTMENTH5=yes
file_prefix=RadAccelTwist_SpkPos_I900RC1

This is a very large dataset, so I recommend changing the image permissions to read only to mimic.

This bundle runs fine under older versions of isis using read only images. It also runs to completion when OUTADJUSTMENTH5=no under the latest RC.

I've run some tests and comparisons on a smaller set of images and the images are modified somehow because the date/time change on them after running jigsaw with the new parameter set to yes, but I'm having a hard time seeing what has changed.

I've compared the label and caminfo output between the original images (in a different area and copied over) and a local version that jigsaw pointed to when update=no and OUTADJUSTMENTH5=yes but they appear to be identical. Why are the times changing? This is really puzzling. Nothing should be changing about the data from my perspective yet something has. Jigsaw only wrote out a new data file along with the expected bundleout files, nothing more.

Although I can see why some might wish OUTADJUSTMENTH5 default to yes, I think it should be set to no. Users are likely to run a bundle adjustment numerous times before getting to the point of updating and it seems extra time is needed to create the .h5 file and it requires extra space. I would be inclined to start turning it on when my errors get smaller and change less frequently, maybe 1 or 2 bundles before I call it good enough and get ready for updating. The bundleout files are already rather large for big global datasets and creating an extra file that you can't use quite yet is not preferred.

See my work user area Isis3Tests/Jigsaw/I900RC1_HD5Format/ for a small network, image list and command line for jigsaw (found in proc.scr). These images are not write protected and already have "modified" somehow using the new parameter. proc.scr points to the location where the files were copied from for reference. Those images have a date of Aug 7 2023, but after running a copy through the latest jigsaw with update=no, OUTADJUSTMENTH5=yes the images have today as a time stamp.

[lwellerastro]

Wondering how OUTADJUSTMENTH5=yes is not a breaking change. As mentioned, I will most likely always set the new parameter to no and will have to remember to do so now. The defaults have changed, so to me that seems like it's a breaking change. Maybe I don't quite understand breaking changes. Or perhaps the leap to 9.0.0 makes it ok?

Either way, I'm also concerned about the size the h5 output file and the extra time needed to create it by default for something like my Kaguya global network which has about 170,000 images, takes 2 days to bundle and already creates massive bundleout files:

-rwxr-x--- 1 lweller nobody  12G Oct  4 09:03 ../../GlobalNetwork/Global/Iterations/3_Updates/RadAccelTwist_SpkPos_ValDNAddGCP_Fix2_residuals.csv*
-rwxr-x--- 1 lweller nobody 2.3G Oct  4 08:57 ../../GlobalNetwork/Global/Iterations/3_Updates/RadAccelTwist_SpkPos_ValDNAddGCP_Fix2_bundleout_points.csv*
-rwxr-x--- 1 lweller nobody  65M Oct  4 08:55 ../../GlobalNetwork/Global/Iterations/3_Updates/RadAccelTwist_SpkPos_ValDNAddGCP_Fix2_bundleout_images_KAGUYA_TC2.csv*
-rwxr-x--- 1 lweller nobody  54M Oct  4 08:51 ../../GlobalNetwork/Global/Iterations/3_Updates/RadAccelTwist_SpkPos_ValDNAddGCP_Fix2_bundleout_images_KAGUYA_TC1.csv*
-rwxr-x--- 1 lweller nobody  11G Oct  4 08:50 ../../GlobalNetwork/Global/Iterations/3_Updates/RadAccelTwist_SpkPos_ValDNAddGCP_Fix2_bundleout.txt*

I don't know how large the h5 file will be for a network this size, but I will get an area set up for a quad sized network that takes about an hour bundle and let jigsaw create the new file as a test. I'll also check the processing time (and changes to memory need if applicable).