Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix no such key #75

Merged
merged 3 commits into from
Jan 8, 2025
Merged

Fix no such key #75

merged 3 commits into from
Jan 8, 2025

Conversation

mdellabitta
Copy link
Contributor

@mdellabitta mdellabitta commented Jan 8, 2025

Important

Handle missing S3 keys by returning None and update processing logic to track missing items, adding types-boto3-s3 dependency.

  • Behavior:
    • Handle NoSuchKey exception in get_item_file() in s3.py, returning None if key is missing.
    • Update get_item_metadata() and get_file_list() in s3.py to return None or empty list if file is missing.
    • Modify process_item() in uploader.py to increment Result.ITEM_NOT_PRESENT if metadata is missing.
  • Enums:
    • Add ITEM_NOT_PRESENT to Result enum in tracker.py.
  • Dependencies:
    • Add types-boto3-s3 to pyproject.toml and uv.lock for type checking support.
  • Misc:
    • Fix platform marker for colorama in uv.lock.

This description was created by Ellipsis for 706c19a. It will automatically update as commits are pushed.

Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Looks good to me! Reviewed everything up to 706c19a in 1 minute and 1 seconds

More details
  • Looked at 162 lines of code in 5 files
  • Skipped 0 files when reviewing.
  • Skipped posting 4 drafted comments based on config settings.
1. ingest_wikimedia/s3.py:136
  • Draft comment:
    The return type hint for get_item_file should be updated to Optional[str] to reflect that it can return None. This applies to get_item_metadata as well.
  • Reason this comment was not posted:
    Comment looked like it was already resolved.
2. ingest_wikimedia/s3.py:105
  • Draft comment:
    Check if get_item_file returns None before calling split to avoid AttributeError.
  • Reason this comment was not posted:
    Marked as duplicate.
3. tools/uploader.py:186
  • Draft comment:
    Ensure consistency in handling None return values from get_item_metadata and get_file_list. Consider checking for None before further processing.
  • Reason this comment was not posted:
    Marked as duplicate.
4. ingest_wikimedia/s3.py:133
  • Draft comment:
    Avoid hardcoding sensitive information like S3 bucket names. Consider using environment variables or a secure vault.
  • Reason this comment was not posted:
    Comment was not on a valid diff hunk.

Workflow ID: wflow_A6qcegKAYMoHz3oc


You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@mdellabitta mdellabitta merged commit dea8a76 into main Jan 8, 2025
7 checks passed
@mdellabitta mdellabitta deleted the fix-no-such-key branch January 8, 2025 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant