Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ExtractorProcessor.__call__() does not retain the order given by the members filter #457

Open
carlocastoldi opened this issue Jan 10, 2025 · 0 comments
Labels
bug Report a problem that needs to be fixed

Comments

@carlocastoldi
Copy link

Description of the problem:
When I use an ExtractorProcessor (e.g. Unzip), I would expect that if I specify a list of files i want to extract, the order of the extracted paths was retained.
Instead, ExtractorProcessor uses os.walk(self.extract_dir)

for path, _, files in os.walk(self.extract_dir):
and then filters the extracted files based on the given members, resulting in an arbitrary order.

I would like to suggest to invert the check: loop over the members (if any), and filter on the extracted files.

Also: why not use pathlib.Path?

Example

>>> pooch.Unzip(
                extract_dir=".",
                members=[
                        'E11.5/E11-5_DevCCF_Annotations_31.5um.nii.gz',
                        'E11.5/E11-5_MRI-adc_31.5um.nii.gz',
                ])

Expected output

'/home/castoldi/brainglobe_workingdir/kim_dev_mouse/downloads/./E11.5/E11-5_DevCCF_Annotations_31.5um.nii.gz']
'/home/castoldi/brainglobe_workingdir/kim_dev_mouse/downloads/./E11.5/E11-5_MRI-adc_31.5um.nii.gz',

Actual output

'/home/castoldi/brainglobe_workingdir/kim_dev_mouse/downloads/./E11.5/E11-5_MRI-adc_31.5um.nii.gz',
'/home/castoldi/brainglobe_workingdir/kim_dev_mouse/downloads/./E11.5/E11-5_DevCCF_Annotations_31.5um.nii.gz']

System information

  • Operating system: Ubuntu 24.04
  • Python installation (Anaconda, system, ETS): system
  • Version of Python: 3.12.3
  • Version of this package: 1.8.2
@carlocastoldi carlocastoldi added the bug Report a problem that needs to be fixed label Jan 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Report a problem that needs to be fixed
Projects
None yet
Development

No branches or pull requests

1 participant