Foundry Spatial Transform Proposal

[reinecke]

At the August 14th, 2024 TSC Meeting, the team from Foundry presented their schema enhancement proposal for spatial transforms (slides).

The proposed effect schema would look like this in JSON form:

{
  "OTIO_SCHEMA": "Transform. 1",
  "metadata": {},
  "center_x": 0.5,
  "center_y": 0.5,
  "rotate": 0.0,
  "scale_x": 1.0,
  "scale_y": 1.0,
  "skew x": 0.0,
  "skew_y": 0.0,
  "translate_x": 0.0,
  "translate_y": 0.0,
  "filter": "cubic"
}

Because transform operations are generally not commutative, this proposal includes a proposed order:

• Translate by center translation
• Translate by translation
• Rotate
• Skew
• Scale
• Subtract center translation

Related Issues: #1089

Let's have a discussion here to iterate and find a consensus schema!

[reinecke]

We have a document proposing a spatial coordinate system from Autodesk dating to when they added available_image_bounds to MediaReference in #1219.

Something notable about the older documentation is that it doesn't handle the full set of transformation, though it does discuss a lot more about how define the composition coordinate space. It feels drawing from those semantics to define how to correctly apply the transform values from this proposal could land us in a really good place.

[reinecke]

Some thoughts from the TSC meeting discussion:

• Can the _x/_y pairs be replaced with the Imath vector types available in OTIO?
• Should order be part of the schema, or should OTIO have a canonical order with utility functions to convert data sets between orders (a potentially lossy operation?)
• There was general consensus that the value for filter should be any string the user wants, however we should provide a reasonable set of pre-defined filter types - this would mirror what we do with track kind. In this scheme, we should also define fallback behavior if the specified filter is unrecognized by the application.

[jminor]

Here's a revision with Imath vectors instead:

{
  "OTIO_SCHEMA": "Transform.1",
  "metadata": {},
  "center": {
    "OTIO_SCHEMA":"V2d.1",
    "x": 0.5,
    "y": 0.5
  },
  "rotate": 0.0,
  "scale": {
    "OTIO_SCHEMA":"V2d.1",
    "x": 1.0,
    "y": 1.0
  },
  "skew": {
    "OTIO_SCHEMA":"V2d.1",
    "x": 0.0,
    "y": 0.0
  },
  "translate": {
    "OTIO_SCHEMA":"V2d.1",
    "x": 0.0,
    "y": 0.0
  },
  "filter": "cubic"
}

[jminor]

Also, can I suggest that we call it TransformEffect.1 or SpatialTransformEffect.1 instead of just Transform.1 to keep it clearly separated from other uses of the term "transform" in other contexts. One downside of the OTIO schema mechanism is that we need schema names to be globally unique.

[BrandonBussinger]

@reinecke Have you looked at ASC-FDL use case? This would allow you to imped framing decisions from camera into an OTIO and tell you how they correspond to current media source (Region Of Interest A in Media A = Region Of Interest A in Media B). It is not intended as a rendering implementation only an interchange so can probably live side X side with this. Also +1 to not using 'Transform' without qualifying or establishing a parent/child to specific transforms (color, motion....)

• https://theasc.com/society/ascmitc/asc-framing-decision-list
• https://github.com/ascmitc/fdl