Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spatial flexure options #1855

Open
wants to merge 74 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 70 commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
86d34a2
update for spatial flexure
rcooke-ast Aug 16, 2024
eb7e76e
update for spatial flexure
rcooke-ast Aug 16, 2024
de4d97a
argument rename: flexure -> spat_flexure
rcooke-ast Aug 16, 2024
b2bcc5f
argument rename: flexure -> spat_flexure
rcooke-ast Aug 16, 2024
9790a33
update spatial flexure
rcooke-ast Aug 16, 2024
3e83ecc
update spatial flexure arguments
rcooke-ast Aug 16, 2024
da607cd
update spatial flexure tests
rcooke-ast Aug 16, 2024
0a3affd
update spatial flexure tests
rcooke-ast Aug 17, 2024
41cad85
update spatial flexure
rcooke-ast Aug 17, 2024
da18826
masked value for metadata
rcooke-ast Aug 17, 2024
888a754
allow ndarray for flexure
rcooke-ast Aug 17, 2024
e3f6122
update spec2d for flexure
rcooke-ast Aug 17, 2024
65b84b9
array spatial flexure during findobj
rcooke-ast Aug 17, 2024
0802abf
speed up fit2tilts when only calculating one slit at a time.
rcooke-ast Aug 17, 2024
8f67040
speed up fit2tilts when only calculating one slit at a time.
rcooke-ast Aug 17, 2024
233a03f
speed up fit2tilts when only calculating one slit at a time.
rcooke-ast Aug 17, 2024
b2f56e2
fix None spatflex
rcooke-ast Aug 18, 2024
c86977d
fix spatial flexure in spectrographs
rcooke-ast Aug 18, 2024
0183a91
allow fits.gz
rcooke-ast Aug 18, 2024
d8fe596
Merge branch 'filesearch' into spatflex_options
rcooke-ast Aug 18, 2024
70d2925
fix tests
rcooke-ast Aug 19, 2024
a8c6940
update docstring
rcooke-ast Aug 20, 2024
e4d5ed3
spatial flexure for skymask
rcooke-ast Aug 20, 2024
c4c01a9
Merge branch 'filesearch' into spatflex_options
rcooke-ast Aug 20, 2024
6dfb0cd
update release notes for spatial flexure
rcooke-ast Aug 20, 2024
0588d2e
fix for when spat_flexure is None
rcooke-ast Aug 20, 2024
6d687c4
fix tests
rcooke-ast Aug 21, 2024
e9221d3
Merge branch 'develop' into spatflex_options
rcooke-ast Sep 2, 2024
f2d31fd
update for edge specific spatial flexure
rcooke-ast Sep 2, 2024
fe676ae
update for edge specific spatial flexure
rcooke-ast Sep 2, 2024
b059e9a
Merge branch 'bh3_support' into spatflex_options
rcooke-ast Sep 3, 2024
3e3eaa6
Merge branch 'uves_popler' into spatflex_options
rcooke-ast Sep 3, 2024
0fe9901
UVES_popler updates
rcooke-ast Sep 4, 2024
f399958
Merge branch 'develop' into spatflex_options
rcooke-ast Sep 11, 2024
6bfa9fa
updated users
rcooke-ast Sep 11, 2024
13daa62
corrected spatial flexure in flatfield
rcooke-ast Sep 11, 2024
e4f42d1
doc update
kbwestfall Sep 20, 2024
25a0737
PR cleanup
rcooke-ast Sep 20, 2024
73d0afb
fix tilt shape
rcooke-ast Sep 20, 2024
78120d8
update users
rcooke-ast Sep 20, 2024
4e8f629
Merge branch 'develop' into spatflex_options
rcooke-ast Oct 2, 2024
14c55cb
Merge remote-tracking branch 'origin/order_sync' into spatflex_options
rcooke-ast Oct 2, 2024
4f5fa09
merge with order_sync
rcooke-ast Oct 2, 2024
7755dee
review comments addressed
rcooke-ast Oct 2, 2024
77cf107
updated flexure docs
rcooke-ast Oct 2, 2024
16847ef
update for flexure refactor
rcooke-ast Oct 2, 2024
be45f42
Merge branch 'order_sync' into spatflex_options
rcooke-ast Oct 9, 2024
2f3eb13
masked flexure
rcooke-ast Oct 10, 2024
731bb3f
manual flexure
rcooke-ast Oct 10, 2024
09c5497
mv manual flexure to rawimage
rcooke-ast Oct 10, 2024
872dbec
mv manual flexure to rawimage
rcooke-ast Oct 10, 2024
e8f1426
flexure only for when slits is not None
rcooke-ast Oct 10, 2024
e5112b1
flexure only for when slits is not None
rcooke-ast Oct 10, 2024
f48c767
deal with masked values
rcooke-ast Oct 11, 2024
11e2100
masked manual extraction
rcooke-ast Oct 11, 2024
30022e3
testing
rcooke-ast Oct 12, 2024
4e10c8b
masked header cards
rcooke-ast Oct 12, 2024
e80f42e
masked header cards
rcooke-ast Oct 12, 2024
0056842
masked header cards
rcooke-ast Oct 12, 2024
06b95d5
Update pypeit/calibrations.py
rcooke-ast Oct 15, 2024
8dbf228
update docstrings
rcooke-ast Oct 15, 2024
0dce8e9
consistent setting of pixel indices
rcooke-ast Oct 15, 2024
d8f5293
Merge pull request #1860 from pypeit/spatflex_redoshift
rcooke-ast Oct 15, 2024
deae010
Merge branch 'order_sync' into spatflex_options
kbwestfall Oct 29, 2024
66aa7c4
Merge branch 'release' into spatflex_options
kbwestfall Nov 6, 2024
668e6d5
doc update
kbwestfall Nov 6, 2024
ab688f4
Merge branch 'dev2rel' into spatflex_options
rcooke-ast Nov 17, 2024
3fefeab
Merge branch 'develop' into spatflex_options
rcooke-ast Nov 21, 2024
7dd2fa6
Merge branch 'develop' into spatflex_options
kbwestfall Dec 19, 2024
373e3c9
Merge branch 'develop' into spatflex_options
rcooke-ast Dec 20, 2024
797f20e
fix merge problems
rcooke-ast Dec 20, 2024
c954a3f
no method variable needed
rcooke-ast Dec 20, 2024
ea6961d
fix merge tilts bug
rcooke-ast Dec 20, 2024
5a4c0c4
convert to spat_flexure
rcooke-ast Dec 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions deprecated/find_objects.py
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ def illum_profile_spatial(self, skymask=None, trim_edg=(0, 0), debug=False):
skymask_now = skymask if (skymask is not None) else np.ones_like(self.sciImg.image, dtype=bool)
hist_trim = 0 # Trim the edges of the histogram to take into account edge effects
gpm = self.sciImg.select_flag(invert=True)
slitid_img_init = self.slits.slit_img(pad=0, initial=True, flexure=self.spat_flexure_shift)
slitid_img_init = self.slits.slit_img(pad=0, initial=True, spat_flexure=self.spat_flexure_shift)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My guess is this was because of a global find/replace, but should we be editing the deprecated code?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost... it was a global find, but I manually replaced all instances. I figured that it's important to make it clear that this is a spatial flexure as opposed to a spectral flexure (even for those that look back at the code). Granted... it's not absolutely necessary in this case.

spatScaleImg = np.ones_like(self.sciImg.image)
# For each slit, grab the spatial coordinates and a spline
# representation of the spatial profile from the illumflat
Expand Down Expand Up @@ -290,7 +290,7 @@ def illum_profile_spectral(self, global_sky, skymask=None):
# relative spectral sensitivity is calculated at a given wavelength for all slits simultaneously.
scaleImg = flatfield.illum_profile_spectral(self.sciImg.image.copy(), self.waveimg, self.slits,
slit_illum_ref_idx=sl_ref, model=global_sky, gpmask=gpm,
skymask=skymask, trim=trim, flexure=self.spat_flexure_shift,
skymask=skymask, trim=trim, spat_flexure=self.spat_flexure_shift,
smooth_npix=smooth_npix)
# Now apply the correction to the science frame
self.apply_relative_scale(scaleImg)
Expand Down
2 changes: 1 addition & 1 deletion deprecated/flat.py
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ def fit_flat(flat, tilts_dict, tslits_dict_in, slit, inmask = None,
slitmask_pad = pixels.tslits2mask(tslits_dict_in, pad = pad)
thismask = (slitmask_pad == slit) # mask enclosing the wider slit bounadries
# Create a tilts image using this padded thismask, rather than using the original thismask_in slit pixels
tilts = tracewave.fit2tilts(shape, tilts_dict['coeffs'], tilts_dict['func2d'])
tilts = tracewave.fit2tilts(tilts_dict['coeffs'], tilts_dict['func2d'], shape=shape)
piximg = tilts * (nspec-1)
pixvec = np.arange(nspec)

Expand Down
2 changes: 1 addition & 1 deletion deprecated/ql_keck_lris.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ def config_lines(args):
cfg_lines += [' refframe = observed']
cfg_lines += ['[scienceframe]']
cfg_lines += [' [[process]]']
cfg_lines += [' spat_flexure_correct = False']
cfg_lines += [' spat_flexure_method = skip']
if not args.mask_cr:
cfg_lines += [' mask_cr = False']
cfg_lines += ['[reduce]']
Expand Down
2 changes: 1 addition & 1 deletion deprecated/ql_multislit.py
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ def config_lines(args):
cfg_lines += [' refframe = observed']
cfg_lines += ['[scienceframe]']
cfg_lines += [' [[process]]']
cfg_lines += [' spat_flexure_correct = False']
cfg_lines += [' spat_flexure_method = skip']
if not args.mask_cr:
cfg_lines += [' mask_cr = False']
cfg_lines += ['[reduce]']
Expand Down
14 changes: 7 additions & 7 deletions deprecated/reduce.py
Original file line number Diff line number Diff line change
Expand Up @@ -222,10 +222,10 @@ def initialise_slits(self, initial=False):
self.slits = self.caliBrate.slits
# Select the edges to use
self.slits_left, self.slits_right, _ \
= self.slits.select_edges(initial=initial, flexure=self.spat_flexure_shift)
= self.slits.select_edges(initial=initial, spat_flexure=self.spat_flexure_shift)

# Slitmask
self.slitmask = self.slits.slit_img(initial=initial, flexure=self.spat_flexure_shift,
self.slitmask = self.slits.slit_img(initial=initial, spat_flexure=self.spat_flexure_shift,
exclude_flag=self.slits.bitmask.exclude_for_reducing+['BOXSLIT'])
# Now add the slitmask to the mask (i.e. post CR rejection in proc)
# NOTE: this uses the par defined by EdgeTraceSet; this will
Expand Down Expand Up @@ -339,7 +339,7 @@ def run_objfind(self, std_trace=None, show_peaks=False):
else:
tilt_flexure_shift = self.spat_flexure_shift
msgs.info("Generating tilts image")
self.tilts = self.waveTilts.fit2tiltimg(self.slitmask, flexure=tilt_flexure_shift)
self.tilts = self.waveTilts.fit2tiltimg(self.slitmask, spat_flexure=tilt_flexure_shift)
#

# First pass object finding
Expand Down Expand Up @@ -375,7 +375,7 @@ def prepare_extraction(self, global_sky):
(np.invert(self.slits.bitmask.flagged(self.slits.mask,
flag=self.slits.bitmask.exclude_for_reducing)))
# Update Slitmask to remove `BOXSLIT`, i.e., we don't want to extract those
self.slitmask = self.slits.slit_img(flexure=self.spat_flexure_shift,
self.slitmask = self.slits.slit_img(spat_flexure=self.spat_flexure_shift,
exclude_flag=self.slits.bitmask.exclude_for_reducing)
# use the tweaked traces if they exist - DP: I'm not sure this is necessary
self.sciImg.update_mask_slitmask(self.slitmask)
Expand All @@ -391,7 +391,7 @@ def prepare_extraction(self, global_sky):
else:
tilt_flexure_shift = self.spat_flexure_shift
msgs.info("Generating tilts image")
self.tilts = self.waveTilts.fit2tiltimg(self.slitmask, flexure=tilt_flexure_shift)
self.tilts = self.waveTilts.fit2tiltimg(self.slitmask, spat_flexure=tilt_flexure_shift)

# Wavelengths (on unmasked slits)
msgs.info("Generating wavelength image")
Expand Down Expand Up @@ -1605,7 +1605,7 @@ def illum_profile_spatial(self, skymask=None, trim_edg=(0, 0)):
skymask_now = skymask if (skymask is not None) else np.ones_like(self.sciImg.image, dtype=bool)
hist_trim = 0 # Trim the edges of the histogram to take into account edge effects
gpm = self.sciImg.select_flag(invert=True)
slitid_img_init = self.slits.slit_img(pad=0, initial=True, flexure=self.spat_flexure_shift)
slitid_img_init = self.slits.slit_img(pad=0, initial=True, spat_flexure=self.spat_flexure_shift)
spatScaleImg = np.ones_like(self.sciImg.image)
# For each slit, grab the spatial coordinates and a spline
# representation of the spatial profile from the illumflat
Expand Down Expand Up @@ -1701,7 +1701,7 @@ def illum_profile_spectral(self, global_sky, skymask=None):
gpm = self.sciImg.select_flag(invert=True)
scaleImg = flatfield.illum_profile_spectral(self.sciImg.image.copy(), self.waveimg, self.slits,
slit_illum_ref_idx=ref_idx, model=global_sky, gpmask=gpm,
skymask=skymask, trim=trim, flexure=self.spat_flexure_shift)
skymask=skymask, trim=trim, spat_flexure=self.spat_flexure_shift)
# Now apply the correction to the science frame
self.apply_relative_scale(scaleImg)

Expand Down
21 changes: 16 additions & 5 deletions doc/calibrations/flexure.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,22 @@ Spatial Flexure
The code has a simple yet relatively robust method to cross-correlate
the slits against any input image to determine a rigid, spatial offset.
This algorithm is performed for any frame type by setting
``spat_flexure_correct = True`` in the ``process`` block
of :ref:`processimagespar`.
``spat_flexure_method = <method>`` in the ``process`` block
of :ref:`processimagespar`, where you should replace ``<method>`` with one of the
following valid methods:
``skip`` - this will skip the spatial flexure correction
``detector`` - this will calculate one spatial flexure value for all slits on the detector
``slit`` - this will calculate the spatial flexure for each slit separately
``edge`` - this will calculate the spatial flexure for each slit edge separately

We have made our own determinations for which instruments to enable this as the
default. Inspect the :ref:`instr_par` list to see if your instrument is included
(search for the value of ``spat_flexure_method``). We recommend that you use the
``detector`` method for most instruments, unless you have good reason to believe
that the spatial flexure is different for different slits or slit edges. Also note
that some spectrographs post-process the spatial flexure values when using the ``slit``
or ``edge`` methods (e.g. perform a linear fit to the spatial flexure values) to improve
the result. Consult the documentation for your instrument to see if this is the case.
(search for the value of ``spat_flexure_correct``).
A QA plot is generated for each frame for which the spatial flexure correction
is applied. We recommend the user to inspect these plots to ensure the flexure
Expand All @@ -52,11 +63,11 @@ add this to your PypeIt file:

[scienceframe]
[[process]]
spat_flexure_correct = True
spat_flexure_method = detector
[calibrations]
[[standardframe]]
[[[process]]]
spat_flexure_correct = True
spat_flexure_method = detector

This will:

Expand All @@ -78,7 +89,7 @@ following to your :doc:`../pypeit_file`:
[calibrations]
[[tiltframe]]
[[[process]]]
spat_flexure_correct = True
spat_flexure_method = detector

This will:

Expand Down
4 changes: 2 additions & 2 deletions doc/calibrations/image_proc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -288,8 +288,8 @@ Spatial Flexure Shift

A spatial shift in the slit positions due to instrument flexure is calculated
using :func:`~pypeit.core.flexure.spat_flexure_shift` if the
``spat_flexure_correct`` parameter is true. See :ref:`flexure` for additional
discussion.
``spat_flexure_method`` parameter is set to an option other than ``skip``.
See :ref:`flexure` for additional discussion.

Flat-fielding
-------------
Expand Down
Loading
Loading