@JsonInclude content inclusion does not work

[jefftt]

Jackson version 2.9.1

I would like to serialized Some(List.empty) to []

import java.io.StringWriter

import com.fasterxml.jackson.annotation.JsonInclude
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper

case class Test(opt: Option[List[String]] = Some(List.empty))

val mapper = new ObjectMapper with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
mapper.setDefaultPropertyInclusion(JsonInclude.Value.construct(JsonInclude.Include.NON_ABSENT, JsonInclude.Include.ALWAYS))


val nonEmptyList = new StringWriter
mapper.writeValue(nonEmptyList, Test(Some(List("hello", "world"))))
nonEmptyList.toString // {"opt":["hello","world"]}

val emptyList = new StringWriter
mapper.writeValue(emptyList, Test(Some(List())))
emptyList.toString // {} should be {"opt": []} !?

val emptyOption = new StringWriter
mapper.writeValue(emptyOption, Test(None))
emptyOption.toString // {}

[jefftt]

Willing to work on this if someone could point me in the right direction. Thanks!

[jefftt]

Looks like this broke in 2.9

Works fine with 2.8.10 using
mapper.setSerializationInclusion(JsonInclude.Include.NON_ABSENT)

[cowtowncoder]

@jefftt Ok, so Some should be ReferenceType, and NON_ABSENT should be fine: it'd only exclude "none" case.

What are unexpected outputs for these tests?

[jefftt]

Sorry for not being clear:

An example case class:
case class Test(opt: Option[List[String]])

In 2.9.x:
Test(Some(List.empty)) is being serialized to {} however i expect the output to be { "opt": [] }

[cowtowncoder]

Ok. So for some reason filtering criteria matches in this case.

Standard reference type deserializer:

src/main/java/com/fasterxml/jackson/databind/deser/std/ReferenceTypeDeserializer.java

has implementations for Java 8 and Guava Optionals, as well as AtomicReference.
Scala version probably hasn't been retrofitted since this module hasn't had active developer for quite a while now.

[brharrington]

I hit this issue recently and was looking into a fix. It was relatively straightforward to get it working by changing the OptionSerializer to use ReferenceTypeSerializer (brharrington@f6448e8), basically copying the implementation of OptionalSerializer from the java 8 module and adjusting it a bit for the scala Option class.

However, after making this change three of the other test cases started failing. Specifically the three tests for using JsonTypeInfo with a generic collection that is in the option (L219). It isn't honoring the JsonTypeInfo annotation on the interface:

org.scalatest.exceptions.TestFailedException: "{"fruits":[{"[]color":"green"}]}" was not equal to "{"fruits":[{"[type":"Apple","]color":"green"}]}"

As a sanity check I also added a test to see what the java8 module would do with an Optional for the same scenario (brharrington/jackson-modules-java8@9178ca9) and it also fails honor the type info annotation:

org.junit.ComparisonFailure:
Expected :{"strategy":[{"type":"foo","foo":42}]}
Actual   :{"strategy":[{"foo":42}]}

Any ideas as to how to go about fixing this?

[cowtowncoder]

Hmmh. I am not 100% sure if collection of polymorphic values via Option(al) is tested properly, so it is possible this could be a subtle bug in ReferenceTypeSerializer. But then again it could also be something within scala module's interaction, possibly within refinement of Optional into JavaType.

One thing I could do regarding first possibility would be if this could be reproduce with jackson-databind -- type that is equivalent in many ways is AtomicReference: it is the only JDK type (before Java 8...) that is considered ReferenceType. So if test could be modified to use AtomicReference, but use polymorphic type, this would either reproduce issue (and I could maybe fix it), or confirm it is something on Scala side.

[brharrington]

I'll try to reproduce with AtomicReference, but the sanity check test case I mentioned for java8 Optional ((brharrington/jackson-modules-java8@9178ca9)) does not use Scala at all. So I don't think this issue is specific to Scala in any way.

[cowtowncoder]

@brharrington Ok that makes sense. I'd still like test that can be included directly in jackson-databind, for convenience (and quicker catching of regression). In a way it's good problem is there since Scala handling might then work ok once this issue is resolved.

[brharrington]

@cowtowncoder Here is a test case on jackson-databind showing the same issue with AtomicReference: brharrington/jackson-databind@1549473

[david-convey]

@brharrington This issue affects us as well; thank you for getting to the bottom of it. It looks like your fix using ReferenceTypeSerializer is the right solution; is it just blocked for now by FasterXML/jackson-databind#1673?

[brharrington]

As far as I know that is the only blocker.

[cowtowncoder]

@david-convey @brharrington Fwtw I was able to fix the issue on jackson-databind side, for 2.9.4.

There is still #357 that blocks Scala module release, but it would be good if this could be verified from Scala module side.

[brharrington]

@cowtowncoder I confirmed that using a jackson-databind jar built with the fix for #1673 fixes the test cases that were failing.

[cowtowncoder]

@brharrington Thank you. I will then close this issue, assuming it would work for 2.9.4 of Scala module, once it gets released.

[brharrington]

@cowtowncoder This issue is different and would still need a change (brharrington/jackson-module-scala@f6448e8). But the fix to jackson-databind means that the proposed fix to this issue will not break other use-cases. I can submit a PR once 2.9.4 is used by jackson-module-scala.

[cowtowncoder]

Ah. So a patch is needed to change Scala Option handler to be based on (now) shared reference-type (de)serializer. Gotcha.