Umbrella crate issues

[seunlanlege]

There are quite a few problems with the umbrella crate although it is a good step in the right direction. There are a few issues

1. Umbrella crate doesn't work with macros

I recently updated one of our libraries (polytope-labs/sc-simnode#48) to the umbrella crate only to discover it is incompatible with the macros in:

• sp-runtime-interface
• cumulus-pallet-parachain-system

2. Missing features for individual crates

The umbrella crate does not allow for using crate-specific features, for example sp-core has the feature: full_crypto, sp-io has the feature: disable_panic_handler. And there are a lot more crates with their own features which are critical to downstream dependencies. None of these features are included in the umbrella crate. There needs to be script which pulls in all of the features of the individual crates in to the umbrella crate.

3. Umbrella crate has a very weird semver on crates.io

This is to me, perhaps the most important issue. The latest umbrella crate has a version v0.7.0 which corresponds to stable2409. But this versioning strategy is uncorrellated with the actual github versioning which would be 1.16.0. It's important that we use the same version everywhere to avoid confusion and dependency hell for downstream users. The umbrella crate should follow the official github release versioning. This would mean publishing the correct version for the polkadot-sdk crate that corresponds with it's github tag. Eg instead of 0.8.0 for stable2412 this should be v1.17.0.

This also allows for publishing patch versions, in between stable releases.

[aurexav]

3 also be reported here

https://substrate.stackexchange.com/questions/12099/versioning-convention

[boundless-forest]

I strongly support 3.

It's very confusing to understand the relationship between the various version numbers and the polkadot-sdk version when examining a project.

For example, can you tell me which version of polkadot-sdk is used for this project if you see the following dependencies in the Cargo.toml file:

sc-network-common = { version = "0.44.0" }
sc-network-sync = { version = "0.44.0" }
sp-api = { version = "34.0.0", default-features = false }
sp-database = { version = "10.0.0" }
sp-genesis-builder = { version = "0.15.1", default-features = false }

Currently, I prefer using the git dependency to specify the polkadot-sdk version. It's more explicit and easier to understand.

[ggwpez]

About 3:

We tried to get rid of the numbers (like 1.16), but it is not possible since many systems expect version numbers. So now we are stuck with those legacy numbers and the new naming schema (stableYYMM).
Aligning them seems like a good idea, although we cannot do it in a way that respects semver, since the umbrella crate will mostly always have breaking changes.
@EgorPopelyaev @Morganamilo, ideas? We can bump major on that thing, but then it would be out of sync with the GitHub release number again...

2:

There needs to be script which pulls in all of the features of the individual crates in to the umbrella crate.

Yea we can do this.

1:

umbrella crate only to discover it is incompatible with the macros in

These macros are buggy then and should know where to find their dependencies, should be able to fix it.

[ggwpez]

We could also go the other way and do 24.09.0 as version number for the umbrella crate to link it up to stable2409. The we could also still do patch releases.

[seunlanlege]

We could also go the other way and do 24.09.0 as version number for the umbrella crate to link it up to stable2409. The we could also still do patch releases.

The issue is, this versioning doesn't make sense for 3rd party crates that depend on polkadot-sdk. I think we need a solution that serves both.

3rd party crates should be able to communicate what Polkadot-sdk version they support and still make minor and patch releases