Can I store and search data in bytes?

[oguzdelioglu]

I'm storing as bytes, but I can't search for bytes.

<Error index error: argument must be a string: []byte{0xc6, 0x1b, 0x9b, 0xb3, 0xa7, 0xa0, 0x76, 0x7e, 0x31, 0x79, 0x71, 0x3f, 0x3a, 0x5c, 0x7a, 0x9a, 0xed, 0xce, 0x19, 0x3c}

[oguzdelioglu]

type DBAddress struct {
	address []byte
}

schema := &memdb.DBSchema{
Tables: map[string]*memdb.TableSchema{
	"addressList": {
		Name: "addressList",
		Indexes: map[string]*memdb.IndexSchema{
			"id": {
				Name:    "id",
				Unique:  true,
				Indexer: &memdb.StringFieldIndex{Field: "address"},
			},
		},
	},
},
}


it, err := txn.Get("addressList", "id", hash)

[dnephin]

Thank you for your interesting in go-memdb!

I believe the problem is the use of StringFieldIndex. This indexer uses https://pkg.go.dev/reflect#Value.String when creating an index from the struct. As it says on that docstring reflect.Value.String() is able to convert the []byte into a string, so it doesn't error when you index.
However the StringFieldIndex expects the args passed to txn.Get to always be a string, it doesn't do the same conversion. It looks like #58 is the same issue.

This failure scenario is a bit surprising (I would have expected it to error when trying to index, instead of when trying to query), but otherwise I believe is behaving correctly. StringFieldIndex is only intended to work with string fields, not []byte.

I think to index a []byte you'll want to build an indexer that implements both https://pkg.go.dev/github.com/hashicorp/go-memdb#SingleIndexer and https://pkg.go.dev/github.com/hashicorp/go-memdb#Indexer, optionally it could also implement https://pkg.go.dev/github.com/hashicorp/go-memdb#PrefixIndexer if you need to do prefix lookups.

I cover the details of writing a customer indexer a bit more in this comment: #97 (comment)

[oguzdelioglu]

Thank you for your interesting in go-memdb!

I believe the problem is the use of StringFieldIndex. This indexer uses https://pkg.go.dev/reflect#Value.String when creating an index from the struct. As it says on that docstring reflect.Value.String() is able to convert the []byte into a string, so it doesn't error when you index. However the StringFieldIndex expects the args passed to txn.Get to always be a string, it doesn't do the same conversion. It looks like #58 is the same issue.

This failure scenario is a bit surprising (I would have expected it to error when trying to index, instead of when trying to query), but otherwise I believe is behaving correctly. StringFieldIndex is only intended to work with string fields, not []byte.

I think to index a []byte you'll want to build an indexer that implements both https://pkg.go.dev/github.com/hashicorp/go-memdb#SingleIndexer and https://pkg.go.dev/github.com/hashicorp/go-memdb#Indexer, optionally it could also implement https://pkg.go.dev/github.com/hashicorp/go-memdb#PrefixIndexer if you need to do prefix lookups.

I cover the details of writing a customer indexer a bit more in this comment: #97 (comment)

There are about 70 million pieces of data.
I have to convert them to bytes and add them to the database like this.

In this way, it provides both less ram consumption and faster search.

Thank you for your interest.
Unfortunately, I couldn't find what I wanted in the codes you sent.