Getting unexpected EOF when field is missing on struct definition

[vytautas-karpavicius]

Getting unexpected EOF error when trying to deserialise message that was encoded with earlier schema version that does not contain new field f2 (look at repro case below).

It works fine, if I add f2 field into go struct. However this field is not used, and it is unexpected that I need to list it. For example I can remove f1, keep f2 and it works fine.

It should be possible to not list unused fields and still deserialise message successfully.

Reproduction:

package server

import (
	hambavro "github.com/hamba/avro/v2"
	"github.com/stretchr/testify/require"
	"testing"
)

var schemaV1 = `{
  "name": "MySchema",
  "type": "record",
  "fields": [
    {
      "name": "f1",
      "type": "long"
    }
  ]
}`

var schemaV2 = `{
  "name": "MySchema",
  "type": "record",
  "fields": [
    {
      "name": "f1",
      "type": "long"
    },
    {
      "name": "f2",
      "type": "long",
      "default": 0
    }
  ]
}`

type SerializeType struct {
	F1 int64 `avro:"f1"`
}

type DeserializeType struct {
	F1 int64 `avro:"f1"`
	//F2 int64 `avro:"f2"` //<--- FAILS WHEN THIS IS NOT PRESENT
}

func Test_Hambavro(t *testing.T) {
	serializeSchema, err := hambavro.Parse(schemaV1)
	require.NoError(t, err)

	msg, err := hambavro.Marshal(serializeSchema, SerializeType{F1: 123})
	require.NoError(t, err)

	deserializeSchema, err := hambavro.Parse(schemaV2)
	require.NoError(t, err)

	var deserializeType DeserializeType
	err = hambavro.Unmarshal(deserializeSchema, msg, &deserializeType)
	require.NoError(t, err)
}

Tested with version: v2.27.0

[nrwiersma]

The issue is not the struct, but the schema. You cannot decode the a schema v1 encoding with schema v2.

[vytautas-karpavicius]

The issue is not the struct, but the schema. You cannot decode the a schema v1 encoding with schema v2.

Shouldn't it be backwards compatible? It has a default set.
This seems like a common thing, during schema evolution.

[nrwiersma]

I suggest you read the avro spec, as the misunderstanding is with how avro works and what schema evolution is in Avro. Just putting a different schema is not schema evolution.