From e17f1fb027911e8d22d0707157cbb018cf70a4ca Mon Sep 17 00:00:00 2001 From: mab <140427262+Mark-Swirlds@users.noreply.github.com> Date: Wed, 23 Oct 2024 12:44:16 -0700 Subject: [PATCH 1/2] Update hip-1046.md Based on feedback I've edited the following: * Updated Abstract to make problem statement clear. * Change entries containing "grpc_web nodes" to "grpc_web proxies" Signed-off-by: mab <140427262+Mark-Swirlds@users.noreply.github.com> --- HIP/hip-1046.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/HIP/hip-1046.md b/HIP/hip-1046.md index 6b952f20f..f828a8714 100644 --- a/HIP/hip-1046.md +++ b/HIP/hip-1046.md @@ -15,11 +15,11 @@ updated: 2024-10-15 ## Abstract -Frontend applications using the Hedera JavaScript SDK face challenges due to browser security sandbox rules, which block mixed content (i.e., HTTP requests from HTTPS pages) and do not permit direct gRPC requests.  Browsers also require TLS endpoints to be signed by trusted certificate authorities, necessitating fully qualified domain names (FQDNs). +Frontend applications using the Hedera JavaScript SDK face challenges because of lack of support for direct gRPC requests. -Currently, gRPC-Web servers work around these limitations, but the endpoints are not supported in the Hedera Address Book. To circumvent this, gRPC-Web endpoints are hard-coded in the JavaScript SDK, which is neither scalable nor efficient. This approach requires manual updates and coordination, which will become increasingly problematic as Hedera decentralizes and more independent nodes operate. +Currently, gRPC-Web servers bypass these restrictions, but their endpoints are not included in the Hedera Address Book. As a result, the JavaScript SDK uses hard-coded gRPC-Web endpoints, which is inefficient and requires manual updates. This becomes increasingly unsustainable as Hedera decentralizes and more independent nodes are added. -To address this issue, we propose enhancing the Hedera address book to include gRPC-Web endpoint information, allowing the JavaScript SDK to dynamically determine the appropriate endpoints +We propose enhancing the Hedera Address Book to include gRPC-Web endpoints, enabling the JavaScript SDK to dynamically select the appropriate ones, improving scalability and efficiency. ## Motivation @@ -63,18 +63,18 @@ Given that this proposal involves an incremental change to the already existing - Node operator can update gRPC-Web endpoint in the address book - The updated endpoint is propagated to all frontend applications using the Hedera JavaScript SDK. - The process is simple and does not require extensive coordination with SDK maintainers -3. As an Hedera community member I want to run my own gRPC-Web node for my consensus node and make it visible to the community to assist in growing the network. +3. As an Hedera community member I want to run my own gRPC-Web proxy for my consensus node and make it visible to the community to assist in growing the network. Acceptance Criteria: - - A community member can launch a gRPC-Web node without external interaction. - - A community member can submit a request to have their gRPC-Web node added to the network address book. + - A community member can launch a gRPC-Web proxy without external interaction. + - A community member can submit a request to have their gRPC-Web proxy added to the network address book. ## Specification -The dynamic address book stores an entry in state for each node (only consensus, currently). This HIP proposes to extend that “address book” concept to include gRPC-Web node endpoints as an additional field of the `Node` entry. The expectation is that these additional node endpoints will be managed with the same `nodeCreate`, `nodeUpdate`, and `nodeDelete` transactions that manage the other endpoints for consensus nodes. The only changes needed are one additional field in the `NodeCreateTransactionBody`, the same field in `NodeUpdateTransactionBody`, and a matching additional field in the `Node` entry in state. +The dynamic address book stores an entry in state for each node (only consensus, currently). This HIP proposes to extend that “address book” concept to include gRPC-Web proxy endpoints as an additional field of the `Node` entry. The expectation is that these additional node endpoints will be managed with the same `nodeCreate`, `nodeUpdate`, and `nodeDelete` transactions that manage the other endpoints for consensus nodes. The only changes needed are one additional field in the `NodeCreateTransactionBody`, the same field in `NodeUpdateTransactionBody`, and a matching additional field in the `Node` entry in state. -Any entity wishing to operate a gRPC-Web proxy node would request to have that node added to the network address book via a `nodeCreate` (which, initially, requires council approval), and would manage that node via `nodeUpdate` as needed to maintain current and up-to-date information for the service endpoints of that node. +Any entity wishing to operate a gRPC-Web proxy would request to have that node added to the network address book via a `nodeCreate` (which, initially, requires council approval), and would manage that node via `nodeUpdate` as needed to maintain current and up-to-date information for the service endpoints of that node. ### Protobufs @@ -140,9 +140,9 @@ new NodeCreateTransaction() ## Backwards Compatibility -Initially, there should be no change or impact to existing users of the SDKs and gRPC-Web nodes. -The current set of nodes will remain available at the current addresses. After all SDKs are migrated to use dynamic lookup of the addresses via the mirror node APIs, the existing nodes are expected to remain at the current addresses for some time, but may, eventually, begin to migrate. Clients using SDKs should adopt the new SDK versions that support dynamic address lookup within this time period. Clients not using SDKs to access the gRPC-Web nodes should also migrate to look up the node addresses via the mirror node APIs in the same time period. -Any client that chooses to continue using the legacy approach of manually maintaining a fixed static list of gRPC-Web nodes may continue to do so, with the constraint that those entities may need to make more frequent updates to their static lists, but the information for those updates will be available from the mirror node. +Initially, there should be no change or impact to existing users of the SDKs and gRPC-Web proxies. +The current set of nodes will remain available at the current addresses. After all SDKs are migrated to use dynamic lookup of the addresses via the mirror node APIs, the existing nodes are expected to remain at the current addresses for some time, but may, eventually, begin to migrate. Clients using SDKs should adopt the new SDK versions that support dynamic address lookup within this time period. Clients not using SDKs to access the gRPC-Web proxies should also migrate to look up the node addresses via the mirror node APIs in the same time period. +Any client that chooses to continue using the legacy approach of manually maintaining a fixed static list of gRPC-Web proxies may continue to do so, with the constraint that those entities may need to make more frequent updates to their static lists, but the information for those updates will be available from the mirror node. ## Security Implications @@ -154,7 +154,7 @@ actions is extremely low. ## How to Teach This -Address Book documentation will need to be updated to reflect the new field. +The Address Book documentation will be updated to include the new field. Additionally, a blog post or video will be created to explain the change, ensuring developers understand it and are aware that it will not affect existing SDK-based applications. ## Reference Implementation From 8d7df76b326ddf53ce2538ebfbc566d6a11a5020 Mon Sep 17 00:00:00 2001 From: Joseph Sinclair Date: Tue, 29 Oct 2024 14:43:50 -0700 Subject: [PATCH 2/2] Updates to gRPC-WEB HIP to address design considerations raised. Signed-off-by: Joseph Sinclair --- HIP/hip-1046.md | 234 ++++++++++++++++++++++++++++++++---------------- 1 file changed, 156 insertions(+), 78 deletions(-) diff --git a/HIP/hip-1046.md b/HIP/hip-1046.md index f828a8714..9bd34ecd9 100644 --- a/HIP/hip-1046.md +++ b/HIP/hip-1046.md @@ -2,7 +2,7 @@ hip: 1046 title: Adding support for gRPC-Web proxy endpoints to the Address Book author: Joseph Sinclair , Mark Blackman , Simi Hunjan -working-group: Mike Cave , Alex Popowycz +working-group: Mike Cave , Alex Popowycz , Steven Sheehy <@steven-sheehy> requested-by: Hedera type: Standards Track category: Core @@ -10,139 +10,211 @@ needs-council-approval: Yes status: Last Call last-call-date-time: 2024-10-29T07:00:00Z created: 2024-09-18 -updated: 2024-10-15 +updated: 2024-10-29 --- ## Abstract -Frontend applications using the Hedera JavaScript SDK face challenges because of lack of support for direct gRPC requests. +Frontend applications using the Hedera JavaScript SDK face challenges because of +lack of support for direct gRPC requests. -Currently, gRPC-Web servers bypass these restrictions, but their endpoints are not included in the Hedera Address Book. As a result, the JavaScript SDK uses hard-coded gRPC-Web endpoints, which is inefficient and requires manual updates. This becomes increasingly unsustainable as Hedera decentralizes and more independent nodes are added. +Currently, gRPC-Web servers bypass these restrictions, but their endpoints are +not included in the Hedera Address Book. As a result, the JavaScript SDK uses +hard-coded gRPC-Web endpoints, which is inefficient and requires manual updates. +This becomes increasingly unsustainable as Hedera decentralizes and more +independent nodes are added. -We propose enhancing the Hedera Address Book to include gRPC-Web endpoints, enabling the JavaScript SDK to dynamically select the appropriate ones, improving scalability and efficiency. +We propose enhancing the Hedera Address Book to include gRPC-Web endpoints, +enabling the JavaScript SDK to dynamically select the appropriate ones, +improving scalability and efficiency. ## Motivation -Hedera offers a gRPC API for various transactions, accessible through SDKs. These SDKs support both backend applications (e.g., node.js) and frontend web applications and plug-ins. Backend applications can connect directly to the network proxies' gRPC endpoint. However, frontend applications must comply with browser security sandbox rules, which block mixed content (i.e., HTTP requests from HTTPS pages) and do not permit the low-level HTTP2 access required for direct gRPC requests. - -Therefore frontend applications and web plug-ins using the Hedera JavaScript SDK face challenges due to browser security restrictions. This necessitates the use of gRPC-Web proxy servers with TLS endpoints signed by trusted certificate authorities, which require fully qualified domain names (FQDNs). - -Presently, gRPC-Web server endpoints are not supported in the Hedera Address Book.  To work around this gRPC-Web endpoints are hard-coded in the JavaScript SDK.  This is not a scalable or efficient solution as it requires manual updates and coordination.  The problem will only get worse as Hedera continues decentralization of the network and independent node operation, where this coordination is not possible. - -We propose enhancing the Hedera address book to include gRPC-Web endpoint information, allowing the JavaScript SDK to dynamically determine the appropriate endpoints. This will be achieved by: - -1. **Extending the Address Book Schema**: Modify the address book schema to include entries for gRPC-Web endpoints. -2. **Dynamic Discovery**: Update the JavaScript SDK to dynamically retrieve the gRPC-Web endpoints from the address book. -3. **Query Availability**: Update the Mirror Node to recognize gRPC-Web entries in the address book and make those entries available for the SDK to query. - -By implementing these changes, we can ensure that frontend applications can securely and efficiently connect to the Hedera network using dynamically discovered, decentralized TLS endpoints. This will improve the network's resilience, scalability, and overall user experience. +Hedera offers a gRPC API for various transactions, accessible through SDKs. +These SDKs support both backend applications (e.g., node.js) and frontend web +applications and plug-ins. Backend applications can connect directly to the +network proxies' gRPC endpoint. However, frontend applications must comply with +browser security sandbox rules, which block mixed content (i.e., HTTP requests +from HTTPS pages) and do not permit the low-level HTTP2 access required for +direct gRPC requests. + +Therefore frontend applications and web plug-ins using the Hedera JavaScript SDK +face challenges due to browser security restrictions. This necessitates the use +of gRPC-Web proxy servers with TLS endpoints signed by trusted certificate +authorities, which require fully qualified domain names (FQDNs). + +Presently, gRPC-Web server endpoints are not supported in the Hedera Address +Book. To work around this gRPC-Web endpoints are hard-coded in the JavaScript +SDK. This is not a scalable or efficient solution as it requires manual updates +and coordination. The problem will only get worse as Hedera continues +decentralization of the network and independent node operation, where this +coordination is not possible. + +We propose enhancing the Hedera address book to include gRPC-Web endpoint +information, allowing the JavaScript SDK to dynamically determine the +appropriate endpoints. This will be achieved by: + +1. **Extending the Address Book Schema**: Modify the address book schema to + include entries for gRPC-Web endpoints. +2. **Dynamic Discovery**: Update the JavaScript SDK to dynamically retrieve the + gRPC-Web endpoints from the address book. +3. **Query Availability**: Update the Mirror Node to recognize gRPC-Web entries + in the address book and make those entries available for the SDK to query. + +By implementing these changes, we can ensure that frontend applications can +securely and efficiently connect to the Hedera network using dynamically +discovered, decentralized TLS endpoints. This will improve the network's +resilience, scalability, and overall user experience. ## Rationale -Given that this proposal involves an incremental change to the already existing Hedera Address Book, the design is straightforward. The primary goal is to enhance the address book to include gRPC-Web endpoint information, allowing frontend applications to dynamically determine the appropriate endpoints. +Given that this proposal involves an incremental change to the already existing +Hedera Address Book, the design is straightforward. The primary goal is to +enhance the address book to include gRPC-Web endpoint information, allowing +frontend applications to dynamically determine the appropriate endpoints. -*Why:* Updating the JavaScript SDK to dynamically retrieve gRPC-Web endpoints from the address book eliminates the need for hard-coded endpoints. This design decision enhances flexibility and scalability, allowing frontend applications to adapt to changes in the network configuration without requiring manual updates. +*Why:* Updating the JavaScript SDK to dynamically retrieve gRPC-Web endpoints +from the address book eliminates the need for hard-coded endpoints. This design +decision enhances flexibility and scalability, allowing frontend applications to +adapt to changes in the network configuration without requiring manual updates. -*Alternatives Considered:* Another alternative was to continue with the current practice of hard-coding endpoints within the SDK. However, this method is not sustainable as it requires frequent updates and coordination, particularly as Hedera continues to decentralize. Dynamic discovery provides a more robust and future-proof solution. +*Alternatives Considered:* Another alternative was to continue with the current +practice of hard-coding endpoints within the SDK. However, this method is not +sustainable as it requires frequent updates and coordination, particularly as +Hedera continues to decentralize. Dynamic discovery provides a more robust and +future-proof solution. ## User stories -## +1. As a Frontend Developer, I want the Hedera JavaScript SDK to dynamically + retrieve gRPC-Web endpoints, so that my web application can securely connect + to the Hedera network even as gRPC-Web endpoints change over time. -1. As a Frontend Developer, I want the Hedera JavaScript SDK to dynamically retrieve gRPC-Web endpoints, so that my web application can securely connect to the Hedera network even as gRPC-Web endpoints change over time. - Acceptance Criteria: - + - The SDK retrieves the gRPC-Web endpoints from the address book automatically. - The application connects securely using the latest TLS endpoints. - Any changes in the network configuration are reflected without requiring SDK updates. -2. As a gRPC-Web Node Operator, I want to update the FQDN for my node’s endpoint in the address book, so that developers can automatically use the correct, secure endpoint without needing manual intervention. - + +2. As a gRPC-Web Node Operator, I want to update the FQDN for my node’s endpoint + in the address book, so that developers can automatically use the correct, + secure endpoint without needing manual intervention. + Acceptance Criteria: - + - Node operator can update gRPC-Web endpoint in the address book - - The updated endpoint is propagated to all frontend applications using the Hedera JavaScript SDK. - - The process is simple and does not require extensive coordination with SDK maintainers -3. As an Hedera community member I want to run my own gRPC-Web proxy for my consensus node and make it visible to the community to assist in growing the network. - + - The updated endpoint is propagated to all frontend applications using the + Hedera JavaScript SDK. + - The process is simple and does not require extensive coordination with SDK + maintainers + +3. As an Hedera community member I want to run my own gRPC-Web proxy for my + consensus node and make it visible to the community to assist in growing + the network. + Acceptance Criteria: - + - A community member can launch a gRPC-Web proxy without external interaction. - - A community member can submit a request to have their gRPC-Web proxy added to the network address book. - -## Specification + - A community member can submit a request to have their gRPC-Web proxy added + to the network address book. -The dynamic address book stores an entry in state for each node (only consensus, currently). This HIP proposes to extend that “address book” concept to include gRPC-Web proxy endpoints as an additional field of the `Node` entry. The expectation is that these additional node endpoints will be managed with the same `nodeCreate`, `nodeUpdate`, and `nodeDelete` transactions that manage the other endpoints for consensus nodes. The only changes needed are one additional field in the `NodeCreateTransactionBody`, the same field in `NodeUpdateTransactionBody`, and a matching additional field in the `Node` entry in state. +## Specification -Any entity wishing to operate a gRPC-Web proxy would request to have that node added to the network address book via a `nodeCreate` (which, initially, requires council approval), and would manage that node via `nodeUpdate` as needed to maintain current and up-to-date information for the service endpoints of that node. +The dynamic address book stores an entry in state for each node (only consensus, +currently). This HIP proposes to extend that “address book” concept to include +gRPC-Web proxy endpoints as an additional entry in the existing `Node` +`service_endpoint` list The expectation is that these additional node endpoints +will be managed with the same `nodeCreate`, `nodeUpdate`, and `nodeDelete` +transactions that manage the other endpoints for consensus nodes. The only +changes needed are one additional field in the `ServiceEndpoint` message and +one enumeration to describe the service endpoint purpose. + +Any entity wishing to operate a gRPC-Web proxy would request to have that +endpoint added to the network address book via a `nodeUpdate` (which, initially, +requires council approval), and would manage that endpoint via `nodeUpdate` as +needed to maintain current and up-to-date information for the service endpoint(s) +of the consensus node behind that proxy. ### Protobufs Add a new field to `NodeCreateTransactionBody`. ```protobuf -message NodeCreateTransactionBody { +message ServiceEndpoint { ... /** - * An administrative key controlled by the node operator. - */ - proto.Key admin_key = 7; - - /** - * A web proxy for gRPC from non-gRPC clients. + * A purpose for this service endpoint.
+ * This is one of several options, including a gossip endpoint, + * a gRPC endpoint, or a gRPC-Web Proxy endpoint, among others. + *

+ * This field is REQUIRED.
+ * This field MUST match the actual purpose of this endpoint.
+ * There MUST NOT be more than one "gossip" endpoint for each `Node`.
+ * There MUST NOT be more than one "secure gRPC" endpoint for + * each `Node`.
+ * If a `Node` has a "secure gRPC" endpoint, it SHOULD NOT have an + * "insecure gRPC endpoint. */ - proto.ServiceEndpoint grpc_proxy_endpoint = 8;** + ServiceEndpointPurpose purpose = 4; } ``` -Add a new field to `NodeUpdateTransactionBody`. +Add a new enumeration for service endpoint purpose. ```protobuf -message NodeUpdateTransactionBody { -... +/** + * An enumeration of possible "purpose" values.
+ * This enumeration is used to describe the purpose of each `ServiceEndpoint` + * entry in a list of service endpoints. The canonical use is to describe + * the service endpoint purpose for values in the `service_endpoint` field + * of a `Node`. + */ +enum ServiceEndpointPurpose { /** - * An administrative key controlled by the node operator. + * An endpoint used to gossip transactions or other data. */ - proto.Key admin_key = 8; + GOSSIP = 0; /** - * A web proxy for gRPC from non-gRPC clients. + * An endpoint that provides a TLS secured gRPC API. */ - proto.ServiceEndpoint grpc_proxy_endpoint = 9;** -} - -``` + SECURE_GRPC = 1; -Partial detail for updates to the `Node` message in state. + /** + * An endpoint that provides an un-secured gRPC API. + */ + INSECURE_GRPC = 2; -```protobuf -/** - * A single address book node in the network state. - */ -message Node { -... /** - * An administrative key controlled by the node operator. + * An endpoint that provides a TLS secured web proxy for gRPC requests + * from clients that cannot directly connect to gRPC endpoints. */ - proto.Key admin_key = 10; + SECURE_GRPC_WEB_PROXY = 3; /** - * A web proxy for gRPC from non-gRPC clients. + * An endpoint that provides an un-secured web proxy for gRPC requests + * from clients that cannot directly connect to gRPC endpoints. */ - proto.ServiceEndpoint grpc_proxy_endpoint = 11;** + INSECURE_GRPC_WEB_PROXY = 4; } - ``` -new NodeCreateTransaction() - .... - .setNodeType () - ... ## Backwards Compatibility -Initially, there should be no change or impact to existing users of the SDKs and gRPC-Web proxies. -The current set of nodes will remain available at the current addresses. After all SDKs are migrated to use dynamic lookup of the addresses via the mirror node APIs, the existing nodes are expected to remain at the current addresses for some time, but may, eventually, begin to migrate. Clients using SDKs should adopt the new SDK versions that support dynamic address lookup within this time period. Clients not using SDKs to access the gRPC-Web proxies should also migrate to look up the node addresses via the mirror node APIs in the same time period. -Any client that chooses to continue using the legacy approach of manually maintaining a fixed static list of gRPC-Web proxies may continue to do so, with the constraint that those entities may need to make more frequent updates to their static lists, but the information for those updates will be available from the mirror node. +Initially, there should be no change or impact to existing users of the SDKs and +gRPC-Web proxies. +The current set of nodes will remain available at the current addresses. After +all SDKs are migrated to use dynamic lookup of the addresses via the mirror node +APIs, the existing nodes are expected to remain at the current addresses for +some time, but may, eventually, begin to migrate. Clients using SDKs should +adopt the new SDK versions that support dynamic address lookup within this time +period. Clients not using SDKs to access the gRPC-Web proxies should also migrate +to look up the node addresses via the mirror node APIs in the same time period. +Any client that chooses to continue using the legacy approach of manually +maintaining a fixed static list of gRPC-Web proxies may continue to do so, with +the constraint that those entities may need to make more frequent updates to +their static lists, but the information for those updates will be available +from the mirror node. ## Security Implications @@ -154,11 +226,16 @@ actions is extremely low. ## How to Teach This -The Address Book documentation will be updated to include the new field. Additionally, a blog post or video will be created to explain the change, ensuring developers understand it and are aware that it will not affect existing SDK-based applications. +The Address Book documentation will be updated to include the new service +endpoint options. +Additionally, a blog post or video will be created to explain the change, +ensuring developers understand it and are aware that it will not affect +existing SDK-based applications. ## Reference Implementation -The reference implementation must be complete before any HIP is given the status of “Final”. The final implementation must include test code and documentation. +The reference implementation must be complete before any HIP is given the status +of “Final”. The final implementation must include test code and documentation. ## Rejected Ideas @@ -177,4 +254,5 @@ https://github.com/hashgraph/hedera-grpcWeb-proxy ## Copyright/license -This document is licensed under the Apache License, Version 2.0 -- see [LICENSE](../LICENSE) or (https://www.apache.org/licenses/LICENSE-2.0) +This document is licensed under the Apache License, Version 2.0 -- +see [LICENSE](../LICENSE) or (https://www.apache.org/licenses/LICENSE-2.0)