[Question] Wrapped Exchange Array - put projects together

[casabre]

@emmt I really like your smart and lean implementation of the WrappedArrays. I had to exchange data between Julia and Python and thus, the WrappedArray was the way to go for me. In the end, I decided to provide a package which replicates your WrappedArrays in Python. I named it wea which stands for Wrapped Exchanges Array.

In order to have the same meaning in Julia and Python ( + more languages ), I want to ask you if you are willing to cut out your WrappedArrays from InterProcessCommunication and put it into my wea.jl because in the end, the WrappedArrays are ready to support shared memory as well as a byte buffer for sharing data.

I would take over the porting the WrappedArrays into wea.jl and all necessary adaptations. Does this sounds good to you?

Looking forward hearing from you.

Thanks and best regards
Carsten

[emmt]

I am happy to see WrappedArrays used elsewhere. From what you propose, I realize that WrappedArrays in themselves have some interest out of InterProcessCommunication.jl. You however seem to need some adaptations / specializations, can you be more specific? It is not completely clear for me what you mean by "porting" WrappedArrays into your wea.jl. Do you just need the facilities offered by WrappedArrays (i.e., just an array-like structure built on top of a buffer) or do you also need the shared memory stuff offered by InterProcessCommunication.jl?

If having WrappedArrays alone have some interest, it may then be a good idea to make WrappedArrays a standalone package from which InterProcessCommunication.jl and your wea.jl depend. This is surely better than duplicating the code and I fully admit that if you just need the capabilities of WrappedArrays (which are portable) you may prefer not to load the whole InterProcessCommunication.jl package (which probably only work on Unix). Having a small package used by several other packages is certainly a better practice in Julia (or in any other language in fact).

[casabre]

I am glad to hear that this sound reasonable to you!

Hm, maybe "porting" is a too strong word. I am seeing wea.jl as the lean package you have proposed. No InterProcessCommunication sources involved at all → at most as include. In the end, I just want to extend your WrappedArray.jl with some high-level create and load functions respectively a getter function for the prepared binary buffer including the header in order to be close to the Python implementation.

Respectively having an overall interface which is more or less similar in different languages because it should have the same look and feel, somehow. But I am also fine with including a WrappedArray.jl package in order to put the high-level functions on top of that.

[emmt]

OK that's more clear for me but my question was not so much about the code (which you are always free to copy of course) but about whether you will need other fonctionalities provided by InterProcessCommunication.jl such as shared memory? In other words, if WrappedArrays becomes a standalone package is that sufficient for wea.jl or do you need extra stuff from InterprocessCommunication.jl?

Perhaps the best thing to do is that you first write wea.jl (or at least a skeleton of it) with a dependency on InterProcessCommunication.jl and then we can figure out which parts of InterProcessCommunication.jl can be put apart to satisfy both InterProcessCommunication.jl and weal.jl.

[casabre]

Sure, I know that I can copy it anytime but I believe having something more collaborative helps to avoid duplicated code. Finally, there is no direct need of other references of InterProcessCommunication.jl. A package reference is quite sufficient 😊.

Yes, I would start with a draft, including also the WrappedArray.jl file in order to show you my idea about the package. I'll keep you posted and share my first implementation 😃 .

[casabre]

@emmt I have setup a draft branch in order to get it running → https://github.com/casabre/wea.jl/tree/draft. Unit tests are running ok and it seems fine to me. In the end, I re-used your wrappedarrays.jl files and renamed it.

There are still some links to InterProcessCommunication

• throw_argument_error
• roundup
• checkdims
• get_memory_parameters
  and I just would leave it that way in order to avoid duplicated code.

I am looking forward hearing your feedback.