Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-1873529: Add support for empty semi-structured StructType and ArrayType to ast #2834

Closed

Conversation

sfc-gh-jrose
Copy link
Contributor

@sfc-gh-jrose sfc-gh-jrose commented Jan 8, 2025

  1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

    Fixes SNOW-1873529

  2. Fill out the following pre-review checklist:

    • I am adding a new automated test(s) to verify correctness of my new code
      • If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
    • I am adding new logging messages
    • I am adding a new telemetry message
    • I am adding new credentials
    • I am adding a new dependency
    • If this is a new feature/behavior, I'm adding the Local Testing parity changes.
    • I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: Thread-safe Developer Guidelines
  3. Please describe how your code solves the related issue.

This change adds support for ArrayType and StructType having empty elements/fields in the AST. An ArrayType without an element_type represents a semi-structured Array. A StructType with fields set to None represents a semi-structured OBJECT. This semantics change is gated behind a client-side parameter in snowflake.snowpark.context.

With the new semantics enabled, OBJECTS are handled differently now. Previously semi-structred objects were represented with MapType(StringType(), StringType()) while structured objects were represented with StructType([...]). After this change both are represented with StructType and MapType only represents MAP columns.

@sfc-gh-jrose sfc-gh-jrose added the NO-CHANGELOG-UPDATES This pull request does not need to update CHANGELOG.md label Jan 8, 2025
@sfc-gh-jrose sfc-gh-jrose changed the title SNOW-1873529: Add support for empty element_type ArrayType to ast SNOW-1873529: Add support for empty semi-structred StructType and ArrayType to ast Jan 13, 2025
@sfc-gh-jrose sfc-gh-jrose force-pushed the jrose_snow_1873529_implement_empty_array_type_ast branch from 02f086e to 8520146 Compare January 13, 2025 18:52
@sfc-gh-jrose sfc-gh-jrose changed the title SNOW-1873529: Add support for empty semi-structred StructType and ArrayType to ast SNOW-1873529: Add support for empty semi-structured StructType and ArrayType to ast Jan 13, 2025
@sfc-gh-jrose sfc-gh-jrose force-pushed the jrose_snow_1873529_implement_empty_array_type_ast branch from 180ba3f to accf4cf Compare January 13, 2025 20:11
@github-actions github-actions bot locked and limited conversation to collaborators Jan 23, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
NO-CHANGELOG-UPDATES This pull request does not need to update CHANGELOG.md
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant