[FEATURE] Add support for Apache Spark 3.5.1 (streaming)

[braniq]

Is your feature request related to a problem?

I try to use Opensearch with Spark 3.5.1 and I found out that this version of Spark is not yet supported. This is valid for Spark streaming (writing stream to Opensearch) because there were changes made on side of Spark API. Batch processing seems to be working.

What solution would you like?

It would be good if next release would contain also support for Apache Spark 3.5.1 (streaming)

What alternatives have you considered?

There is not alternative as I know I have to use previous version of Spark (3.4.3)

Do you have any additional context?

The same problem is with Elasticsearch where the ticket is already created:
elastic/elasticsearch-hadoop#2210
I get the same exeption:
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 8) : java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.encoders.RowEncoder$.apply(Lorg/apache/spark/sql/types/StructType;)Lorg/apache/spark/sql/catalyst/encoders/ExpressionEncoder;