-
Notifications
You must be signed in to change notification settings - Fork 142
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update hip-1046.md #1066
base: main
Are you sure you want to change the base?
Update hip-1046.md #1066
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,147 +2,219 @@ | |
hip: 1046 | ||
title: Adding support for gRPC-Web proxy endpoints to the Address Book | ||
author: Joseph Sinclair <Joseph Sinclair<@jsync-swirlds>, Mark Blackman <[email protected]>, Simi Hunjan <[email protected]> | ||
working-group: Mike Cave <[email protected]>, Alex Popowycz <[email protected]> | ||
working-group: Mike Cave <[email protected]>, Alex Popowycz <[email protected]>, Steven Sheehy <@steven-sheehy> | ||
requested-by: Hedera | ||
type: Standards Track | ||
category: Core | ||
needs-council-approval: Yes | ||
status: Last Call | ||
last-call-date-time: 2024-10-29T07:00:00Z | ||
created: 2024-09-18 | ||
updated: 2024-10-15 | ||
updated: 2024-10-29 | ||
--- | ||
|
||
## Abstract | ||
|
||
Frontend applications using the Hedera JavaScript SDK face challenges due to browser security sandbox rules, which block mixed content (i.e., HTTP requests from HTTPS pages) and do not permit direct gRPC requests. Browsers also require TLS endpoints to be signed by trusted certificate authorities, necessitating fully qualified domain names (FQDNs). | ||
Frontend applications using the Hedera JavaScript SDK face challenges because of | ||
lack of support for direct gRPC requests. | ||
|
||
Currently, gRPC-Web servers work around these limitations, but the endpoints are not supported in the Hedera Address Book. To circumvent this, gRPC-Web endpoints are hard-coded in the JavaScript SDK, which is neither scalable nor efficient. This approach requires manual updates and coordination, which will become increasingly problematic as Hedera decentralizes and more independent nodes operate. | ||
Currently, gRPC-Web servers bypass these restrictions, but their endpoints are | ||
not included in the Hedera Address Book. As a result, the JavaScript SDK uses | ||
hard-coded gRPC-Web endpoints, which is inefficient and requires manual updates. | ||
This becomes increasingly unsustainable as Hedera decentralizes and more | ||
independent nodes are added. | ||
|
||
To address this issue, we propose enhancing the Hedera address book to include gRPC-Web endpoint information, allowing the JavaScript SDK to dynamically determine the appropriate endpoints | ||
We propose enhancing the Hedera Address Book to include gRPC-Web endpoints, | ||
enabling the JavaScript SDK to dynamically select the appropriate ones, | ||
improving scalability and efficiency. | ||
|
||
## Motivation | ||
|
||
Hedera offers a gRPC API for various transactions, accessible through SDKs. These SDKs support both backend applications (e.g., node.js) and frontend web applications and plug-ins. Backend applications can connect directly to the network proxies' gRPC endpoint. However, frontend applications must comply with browser security sandbox rules, which block mixed content (i.e., HTTP requests from HTTPS pages) and do not permit the low-level HTTP2 access required for direct gRPC requests. | ||
|
||
Therefore frontend applications and web plug-ins using the Hedera JavaScript SDK face challenges due to browser security restrictions. This necessitates the use of gRPC-Web proxy servers with TLS endpoints signed by trusted certificate authorities, which require fully qualified domain names (FQDNs). | ||
|
||
Presently, gRPC-Web server endpoints are not supported in the Hedera Address Book. To work around this gRPC-Web endpoints are hard-coded in the JavaScript SDK. This is not a scalable or efficient solution as it requires manual updates and coordination. The problem will only get worse as Hedera continues decentralization of the network and independent node operation, where this coordination is not possible. | ||
|
||
We propose enhancing the Hedera address book to include gRPC-Web endpoint information, allowing the JavaScript SDK to dynamically determine the appropriate endpoints. This will be achieved by: | ||
|
||
1. **Extending the Address Book Schema**: Modify the address book schema to include entries for gRPC-Web endpoints. | ||
2. **Dynamic Discovery**: Update the JavaScript SDK to dynamically retrieve the gRPC-Web endpoints from the address book. | ||
3. **Query Availability**: Update the Mirror Node to recognize gRPC-Web entries in the address book and make those entries available for the SDK to query. | ||
|
||
By implementing these changes, we can ensure that frontend applications can securely and efficiently connect to the Hedera network using dynamically discovered, decentralized TLS endpoints. This will improve the network's resilience, scalability, and overall user experience. | ||
Hedera offers a gRPC API for various transactions, accessible through SDKs. | ||
These SDKs support both backend applications (e.g., node.js) and frontend web | ||
applications and plug-ins. Backend applications can connect directly to the | ||
network proxies' gRPC endpoint. However, frontend applications must comply with | ||
browser security sandbox rules, which block mixed content (i.e., HTTP requests | ||
from HTTPS pages) and do not permit the low-level HTTP2 access required for | ||
direct gRPC requests. | ||
|
||
Therefore frontend applications and web plug-ins using the Hedera JavaScript SDK | ||
face challenges due to browser security restrictions. This necessitates the use | ||
of gRPC-Web proxy servers with TLS endpoints signed by trusted certificate | ||
authorities, which require fully qualified domain names (FQDNs). | ||
|
||
Presently, gRPC-Web server endpoints are not supported in the Hedera Address | ||
Book. To work around this gRPC-Web endpoints are hard-coded in the JavaScript | ||
SDK. This is not a scalable or efficient solution as it requires manual updates | ||
and coordination. The problem will only get worse as Hedera continues | ||
decentralization of the network and independent node operation, where this | ||
coordination is not possible. | ||
|
||
We propose enhancing the Hedera address book to include gRPC-Web endpoint | ||
information, allowing the JavaScript SDK to dynamically determine the | ||
appropriate endpoints. This will be achieved by: | ||
|
||
1. **Extending the Address Book Schema**: Modify the address book schema to | ||
include entries for gRPC-Web endpoints. | ||
2. **Dynamic Discovery**: Update the JavaScript SDK to dynamically retrieve the | ||
gRPC-Web endpoints from the address book. | ||
3. **Query Availability**: Update the Mirror Node to recognize gRPC-Web entries | ||
in the address book and make those entries available for the SDK to query. | ||
|
||
By implementing these changes, we can ensure that frontend applications can | ||
securely and efficiently connect to the Hedera network using dynamically | ||
discovered, decentralized TLS endpoints. This will improve the network's | ||
resilience, scalability, and overall user experience. | ||
|
||
## Rationale | ||
|
||
Given that this proposal involves an incremental change to the already existing Hedera Address Book, the design is straightforward. The primary goal is to enhance the address book to include gRPC-Web endpoint information, allowing frontend applications to dynamically determine the appropriate endpoints. | ||
Given that this proposal involves an incremental change to the already existing | ||
Hedera Address Book, the design is straightforward. The primary goal is to | ||
enhance the address book to include gRPC-Web endpoint information, allowing | ||
frontend applications to dynamically determine the appropriate endpoints. | ||
|
||
*Why:* Updating the JavaScript SDK to dynamically retrieve gRPC-Web endpoints from the address book eliminates the need for hard-coded endpoints. This design decision enhances flexibility and scalability, allowing frontend applications to adapt to changes in the network configuration without requiring manual updates. | ||
*Why:* Updating the JavaScript SDK to dynamically retrieve gRPC-Web endpoints | ||
from the address book eliminates the need for hard-coded endpoints. This design | ||
decision enhances flexibility and scalability, allowing frontend applications to | ||
adapt to changes in the network configuration without requiring manual updates. | ||
|
||
*Alternatives Considered:* Another alternative was to continue with the current practice of hard-coding endpoints within the SDK. However, this method is not sustainable as it requires frequent updates and coordination, particularly as Hedera continues to decentralize. Dynamic discovery provides a more robust and future-proof solution. | ||
*Alternatives Considered:* Another alternative was to continue with the current | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This 80 character limit seems too restrictive and conflicts with our Java coding standards to use 120. The standards just mention java so not sure about other formats. Is it documented somewhere that we should use 80 for markdown? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There is no standard for markdown that I know of, but github enforces 80 characters in much of the review interface (particularly suggestions), so I habitually reformat HIP markdown to 80 characters for ease of review. |
||
practice of hard-coding endpoints within the SDK. However, this method is not | ||
sustainable as it requires frequent updates and coordination, particularly as | ||
Hedera continues to decentralize. Dynamic discovery provides a more robust and | ||
future-proof solution. | ||
|
||
## User stories | ||
|
||
## | ||
1. As a Frontend Developer, I want the Hedera JavaScript SDK to dynamically | ||
retrieve gRPC-Web endpoints, so that my web application can securely connect | ||
to the Hedera network even as gRPC-Web endpoints change over time. | ||
|
||
1. As a Frontend Developer, I want the Hedera JavaScript SDK to dynamically retrieve gRPC-Web endpoints, so that my web application can securely connect to the Hedera network even as gRPC-Web endpoints change over time. | ||
|
||
Acceptance Criteria: | ||
|
||
- The SDK retrieves the gRPC-Web endpoints from the address book automatically. | ||
- The application connects securely using the latest TLS endpoints. | ||
- Any changes in the network configuration are reflected without requiring SDK updates. | ||
2. As a gRPC-Web Node Operator, I want to update the FQDN for my node’s endpoint in the address book, so that developers can automatically use the correct, secure endpoint without needing manual intervention. | ||
|
||
|
||
2. As a gRPC-Web Node Operator, I want to update the FQDN for my node’s endpoint | ||
in the address book, so that developers can automatically use the correct, | ||
secure endpoint without needing manual intervention. | ||
|
||
Acceptance Criteria: | ||
|
||
- Node operator can update gRPC-Web endpoint in the address book | ||
- The updated endpoint is propagated to all frontend applications using the Hedera JavaScript SDK. | ||
- The process is simple and does not require extensive coordination with SDK maintainers | ||
3. As an Hedera community member I want to run my own gRPC-Web node for my consensus node and make it visible to the community to assist in growing the network. | ||
|
||
- The updated endpoint is propagated to all frontend applications using the | ||
Hedera JavaScript SDK. | ||
- The process is simple and does not require extensive coordination with SDK | ||
maintainers | ||
|
||
3. As an Hedera community member I want to run my own gRPC-Web proxy for my | ||
consensus node and make it visible to the community to assist in growing | ||
the network. | ||
|
||
Acceptance Criteria: | ||
|
||
- A community member can launch a gRPC-Web node without external interaction. | ||
- A community member can submit a request to have their gRPC-Web node added to the network address book. | ||
|
||
## Specification | ||
|
||
The dynamic address book stores an entry in state for each node (only consensus, currently). This HIP proposes to extend that “address book” concept to include gRPC-Web node endpoints as an additional field of the `Node` entry. The expectation is that these additional node endpoints will be managed with the same `nodeCreate`, `nodeUpdate`, and `nodeDelete` transactions that manage the other endpoints for consensus nodes. The only changes needed are one additional field in the `NodeCreateTransactionBody`, the same field in `NodeUpdateTransactionBody`, and a matching additional field in the `Node` entry in state. | ||
- A community member can launch a gRPC-Web proxy without external interaction. | ||
- A community member can submit a request to have their gRPC-Web proxy added | ||
to the network address book. | ||
|
||
Any entity wishing to operate a gRPC-Web proxy node would request to have that node added to the network address book via a `nodeCreate` (which, initially, requires council approval), and would manage that node via `nodeUpdate` as needed to maintain current and up-to-date information for the service endpoints of that node. | ||
## Specification | ||
|
||
The dynamic address book stores an entry in state for each node (only consensus, | ||
currently). This HIP proposes to extend that “address book” concept to include | ||
gRPC-Web proxy endpoints as an additional entry in the existing `Node` | ||
`service_endpoint` list The expectation is that these additional node endpoints | ||
will be managed with the same `nodeCreate`, `nodeUpdate`, and `nodeDelete` | ||
transactions that manage the other endpoints for consensus nodes. The only | ||
changes needed are one additional field in the `ServiceEndpoint` message and | ||
one enumeration to describe the service endpoint purpose. | ||
|
||
Any entity wishing to operate a gRPC-Web proxy would request to have that | ||
endpoint added to the network address book via a `nodeUpdate` (which, initially, | ||
requires council approval), and would manage that endpoint via `nodeUpdate` as | ||
needed to maintain current and up-to-date information for the service endpoint(s) | ||
of the consensus node behind that proxy. | ||
|
||
### Protobufs | ||
|
||
Add a new field to `NodeCreateTransactionBody`. | ||
|
||
```protobuf | ||
message NodeCreateTransactionBody { | ||
message ServiceEndpoint { | ||
... | ||
/** | ||
* An administrative key controlled by the node operator. | ||
*/ | ||
proto.Key admin_key = 7; | ||
|
||
/** | ||
* A web proxy for gRPC from non-gRPC clients. | ||
* A purpose for this service endpoint.<br/> | ||
* This is one of several options, including a gossip endpoint, | ||
* a gRPC endpoint, or a gRPC-Web Proxy endpoint, among others. | ||
* <p> | ||
* This field is REQUIRED.<br/> | ||
* This field MUST match the actual purpose of this endpoint.<br/> | ||
* There MUST NOT be more than one "gossip" endpoint for each `Node`.<br/> | ||
* There MUST NOT be more than one "secure gRPC" endpoint for | ||
* each `Node`.<br/> | ||
* If a `Node` has a "secure gRPC" endpoint, it SHOULD NOT have an | ||
* "insecure gRPC endpoint. | ||
Comment on lines
+153
to
+156
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These statements are not really true. A node can have more than one secure port on different proxy IPs. A node can also have both secure and insecure on the same IP since they are different ports. I imagine in the future with permissionless some nodes might even want multiple secure per IP or multiple insecure per IP. Here's node 0.0.3 service endpoints on mainnet right now: "service_endpoints":[
{"domain_name":"","ip_address_v4":"34.239.82.6","port":50211},
{"domain_name":"","ip_address_v4":"34.239.82.6","port":50212},
{"domain_name":"","ip_address_v4":"35.237.200.180","port":50211},
{"domain_name":"","ip_address_v4":"35.237.200.180","port":50212}] There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It is permitted to do so (SHOULD is a recommendation keyword). It is not best practice, however. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I was commenting on both statements. There is a MUST NOT statement that is not true. And I disagree with your SHOULD recommendation as it can't be considered a bad practice if every node in production does it. The statements should reflect the reality on mainnet. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I see. That is not ideal (having both secure and insecure endpoints is bad security practice), but I can remove the specification text (once the author returns and can accept a PR). |
||
*/ | ||
proto.ServiceEndpoint grpc_proxy_endpoint = 8;** | ||
ServiceEndpointPurpose purpose = 4; | ||
} | ||
``` | ||
|
||
Add a new field to `NodeUpdateTransactionBody`. | ||
Add a new enumeration for service endpoint purpose. | ||
|
||
```protobuf | ||
message NodeUpdateTransactionBody { | ||
... | ||
/** | ||
* An enumeration of possible "purpose" values.<br/> | ||
* This enumeration is used to describe the purpose of each `ServiceEndpoint` | ||
* entry in a list of service endpoints. The canonical use is to describe | ||
* the service endpoint purpose for values in the `service_endpoint` field | ||
* of a `Node`. | ||
*/ | ||
enum ServiceEndpointPurpose { | ||
/** | ||
* An administrative key controlled by the node operator. | ||
* An endpoint used to gossip transactions or other data. | ||
*/ | ||
proto.Key admin_key = 8; | ||
GOSSIP = 0; | ||
|
||
/** | ||
* A web proxy for gRPC from non-gRPC clients. | ||
* An endpoint that provides a TLS secured gRPC API. | ||
*/ | ||
proto.ServiceEndpoint grpc_proxy_endpoint = 9;** | ||
} | ||
|
||
``` | ||
SECURE_GRPC = 1; | ||
|
||
Partial detail for updates to the `Node` message in state. | ||
/** | ||
* An endpoint that provides an un-secured gRPC API. | ||
*/ | ||
INSECURE_GRPC = 2; | ||
|
||
```protobuf | ||
/** | ||
* A single address book node in the network state. | ||
*/ | ||
message Node { | ||
... | ||
/** | ||
* An administrative key controlled by the node operator. | ||
* An endpoint that provides a TLS secured web proxy for gRPC requests | ||
* from clients that cannot directly connect to gRPC endpoints. | ||
*/ | ||
proto.Key admin_key = 10; | ||
SECURE_GRPC_WEB_PROXY = 3; | ||
|
||
/** | ||
* A web proxy for gRPC from non-gRPC clients. | ||
* An endpoint that provides an un-secured web proxy for gRPC requests | ||
* from clients that cannot directly connect to gRPC endpoints. | ||
*/ | ||
proto.ServiceEndpoint grpc_proxy_endpoint = 11;** | ||
INSECURE_GRPC_WEB_PROXY = 4; | ||
} | ||
|
||
``` | ||
new NodeCreateTransaction() | ||
.... | ||
.setNodeType () | ||
... | ||
|
||
## Backwards Compatibility | ||
|
||
Initially, there should be no change or impact to existing users of the SDKs and gRPC-Web nodes. | ||
The current set of nodes will remain available at the current addresses. After all SDKs are migrated to use dynamic lookup of the addresses via the mirror node APIs, the existing nodes are expected to remain at the current addresses for some time, but may, eventually, begin to migrate. Clients using SDKs should adopt the new SDK versions that support dynamic address lookup within this time period. Clients not using SDKs to access the gRPC-Web nodes should also migrate to look up the node addresses via the mirror node APIs in the same time period. | ||
Any client that chooses to continue using the legacy approach of manually maintaining a fixed static list of gRPC-Web nodes may continue to do so, with the constraint that those entities may need to make more frequent updates to their static lists, but the information for those updates will be available from the mirror node. | ||
Initially, there should be no change or impact to existing users of the SDKs and | ||
gRPC-Web proxies. | ||
The current set of nodes will remain available at the current addresses. After | ||
all SDKs are migrated to use dynamic lookup of the addresses via the mirror node | ||
APIs, the existing nodes are expected to remain at the current addresses for | ||
some time, but may, eventually, begin to migrate. Clients using SDKs should | ||
adopt the new SDK versions that support dynamic address lookup within this time | ||
period. Clients not using SDKs to access the gRPC-Web proxies should also migrate | ||
to look up the node addresses via the mirror node APIs in the same time period. | ||
Any client that chooses to continue using the legacy approach of manually | ||
maintaining a fixed static list of gRPC-Web proxies may continue to do so, with | ||
the constraint that those entities may need to make more frequent updates to | ||
their static lists, but the information for those updates will be available | ||
from the mirror node. | ||
|
||
## Security Implications | ||
|
||
|
@@ -154,11 +226,16 @@ actions is extremely low. | |
|
||
## How to Teach This | ||
|
||
Address Book documentation will need to be updated to reflect the new field. | ||
The Address Book documentation will be updated to include the new service | ||
endpoint options. | ||
Additionally, a blog post or video will be created to explain the change, | ||
ensuring developers understand it and are aware that it will not affect | ||
existing SDK-based applications. | ||
|
||
## Reference Implementation | ||
|
||
The reference implementation must be complete before any HIP is given the status of “Final”. The final implementation must include test code and documentation. | ||
The reference implementation must be complete before any HIP is given the status | ||
of “Final”. The final implementation must include test code and documentation. | ||
|
||
## Rejected Ideas | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would have expected a new field in
There are probably good reasons why this approach was not taken, even though it seems less error-prone to use. Can we please add an entry under "Rejected Ideas" explaining why this approach was not taken? |
||
|
||
|
@@ -177,4 +254,5 @@ https://github.com/hashgraph/hedera-grpcWeb-proxy | |
|
||
## Copyright/license | ||
|
||
This document is licensed under the Apache License, Version 2.0 -- see [LICENSE](../LICENSE) or (https://www.apache.org/licenses/LICENSE-2.0) | ||
This document is licensed under the Apache License, Version 2.0 -- | ||
see [LICENSE](../LICENSE) or (https://www.apache.org/licenses/LICENSE-2.0) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand the new proposal. The
NodeCreateTransactionBody
for example, which adds a node to the address book, hasServiceEndpoints
such asgossip_endpoint
andservice_endpoint
. This HIP doesn't change that. But it does addServiceEndpointPurpose
. So what happens if I set a ServiceEndpoint ongossip_endpoint
with a purpose ofSECURE_GRPC
?I'm assuming the node transaction will have to be updated in an incompatible way? That isn't a problem since it is not currently used, but I didn't see that change in the specification section.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After this HIP, we would need to enforce the correct
purpose
value fornodeCreate
andnodeUpdate
endpoint lists in consensus node software.All changes needed would be compatible from the perspective of protobuf. The node transaction definitions do not need to be changed here as we're just adding a new field.
During upgrade we might need perform a state migration to set the purpose for each endpoint (which can be determined from the containing list and port, as Mirror Node does currently).