Cross-language workflows

[purcell]

Hi, and thanks so much for sharing this SDK!

I'm evaluating Temporal to help with orchestration within a polyglot Clojure + Python system, so I'm particularly considering interop between different languages, and I recognise this might not be on your list of priorities at all.

From what I can tell, cross-language workflow interop is currently tricky with this SDK for a couple of reasons:

1. Only activities defined in clojure with defactivity can be invoked, so an activity that may be implemented by a worker in another language is inaccessible from this SDK. (As a workaround one can declare a stub activity in Clojure that never gets executed, but this feels a bit hokey.)
2. All activity params/results are encoded with nippy rather than JSON, so by default workflow data will be unreadable by SDKs in other languages. I guess this choice was made so that Clojure-to-Clojure activity interop works smoothly, with seamless serialisation of :keywords etc. For interop beyond Clojure, would you expect that a Data Converter could be written to convert nippy data to/from JSON or similar?

I'd rather not drop down to the Java SDK if I can avoid it, so I thought I'd check in here to see if you have any tips — thanks!

[ghaskins]

Hi,

Yeah, at the moment the SDK is fairly opinionated that its clojure-to-clojure for the reasons you highlighted.

Re (1), its probably a small lift to update the invoke api so that it takes an interop-friendly string, rather than a clojure symbol that was created by the defactivity macro.

Re (2), it's a little trickier, and I don't think a data converter will work. This is because we need to encode things before serializing them into Java types like strings/bytes, and the data encoder would sit too low in the stack.It's simply too late to capture what we need to capture there.

What can probably be done is add our own "data converter" type mechanism at the right point of the stack, such as a defprotocol. Then, the SDK can supply a nippy-based default implementation, or users may swap it out for their own.

[kpassapk]

I have been using Biff recently, and I like the author's approach of providing a very opinionated implementation but documenting alternatives. A defprotocol + how-to might work nicely here.

[Vaelatern]

I feel like this purpose would be served "ok" for inter-language workflows with edn. Unlike nippy, edn can be used in other languages.

Would a PR to this effect, making it possible, be welcomed? Is it even an OK idea or am I missing something?