Hyperspace CreateIndex failed

[soyarym]

Hello!
I use pyspark 2.4.7 and on-premise Hadoop cluster, and also Hyperspace v 0.4.0 by adding param "spark.jars.packages": {"com.microsoft.hyperspace:hyperspace-core_2.11": "0.3.0"} to the spark-session conf.
When I try to reproduce the example from the quick start guide:

sample_data = [(1, "name1"), (2, "name2")]

spark.createDataFrame(sample_data, ['id', 'name'])\
    .write\
    .mode("overwrite")\
    .parquet("warehouse/something_sb.db/hyperspace_test_1")

df = spark.read.parquet("warehouse/something_sb.db/hyperspace_test_1")

from hyperspace import Hyperspace

hs = Hyperspace(spark)

from hyperspace import IndexConfig

idxConfig = IndexConfig("test_index", ["id"], ["name"])
hs.createIndex(df, idxConfig)

I get this error:

Py4JJavaError: An error occurred while calling o324.createIndex.
: com.microsoft.hyperspace.HyperspaceException: Could not acquire proper state
	at com.microsoft.hyperspace.actions.Action$class.saveEntry(Action.scala:79)
	at com.microsoft.hyperspace.actions.Action$class.begin(Action.scala:53)
	at com.microsoft.hyperspace.actions.Action$class.run(Action.scala:90)
	at com.microsoft.hyperspace.actions.CreateAction.run(CreateAction.scala:30)
	at com.microsoft.hyperspace.index.IndexCollectionManager.create(IndexCollectionManager.scala:43)
	at com.microsoft.hyperspace.index.CachingIndexCollectionManager.create(CachingIndexCollectionManager.scala:77)
	at com.microsoft.hyperspace.Hyperspace.createIndex(Hyperspace.scala:42)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
	at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
	at py4j.Gateway.invoke(Gateway.java:282)
	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
	at py4j.commands.CallCommand.execute(CallCommand.java:79)
	at py4j.GatewayConnection.run(GatewayConnection.java:238)
	at java.lang.Thread.run(Thread.java:748)

What did I do wrong?
Thank you!

[soyarym]

Sorry, this was accidentally posted in that repo instead of Hyperspace and I couldn't delete it. :(

[dbeavon]

@soyarym Too bad...I was excited to see another participant in this repo

Come back soon!