docs: migrate adrs and rfcs across from master (tendermint#9115)

informalsystems · Jul 30, 2022 · e32b86b · e32b86b
1 parent 85636f6
commit e32b86b
Show file tree

Hide file tree

Showing 120 changed files with 18,135 additions and 0 deletions.
diff --git a/docs/architecture/README.md b/docs/architecture/README.md
@@ -0,0 +1,120 @@
+---
+order: 1
+parent:
+  order: false
+---
+
+# Architecture Decision Records (ADR)
+
+This is a location to record all high-level architecture decisions in the tendermint project.
+
+You can read more about the ADR concept in this [blog post](https://product.reverb.com/documenting-architecture-decisions-the-reverb-way-a3563bb24bd0#.78xhdix6t).
+
+An ADR should provide:
+
+- Context on the relevant goals and the current state
+- Proposed changes to achieve the goals
+- Summary of pros and cons
+- References
+- Changelog
+
+Note the distinction between an ADR and a spec. The ADR provides the context, intuition, reasoning, and
+justification for a change in architecture, or for the architecture of something
+new. The spec is much more compressed and streamlined summary of everything as
+it stands today.
+
+If recorded decisions turned out to be lacking, convene a discussion, record the new decisions here, and then modify the code to match.
+
+Note the context/background should be written in the present tense.
+
+## Table of Contents
+
+### Implemented
+
+- [ADR-001: Logging](./adr-001-logging.md)
+- [ADR-002: Event-Subscription](./adr-002-event-subscription.md)
+- [ADR-003: ABCI-APP-RPC](./adr-003-abci-app-rpc.md)
+- [ADR-004: Historical-Validators](./adr-004-historical-validators.md)
+- [ADR-005: Consensus-Params](./adr-005-consensus-params.md)
+- [ADR-008: Priv-Validator](./adr-008-priv-validator.md)
+- [ADR-009: ABCI-Design](./adr-009-ABCI-design.md)
+- [ADR-010: Crypto-Changes](./adr-010-crypto-changes.md)
+- [ADR-011: Monitoring](./adr-011-monitoring.md)
+- [ADR-014: Secp-Malleability](./adr-014-secp-malleability.md)
+- [ADR-015: Crypto-Encoding](./adr-015-crypto-encoding.md)
+- [ADR-016: Protocol-Versions](./adr-016-protocol-versions.md)
+- [ADR-017: Chain-Versions](./adr-017-chain-versions.md)
+- [ADR-018: ABCI-Validators](./adr-018-ABCI-Validators.md)
+- [ADR-019: Multisigs](./adr-019-multisigs.md)
+- [ADR-020: Block-Size](./adr-020-block-size.md)
+- [ADR-021: ABCI-Events](./adr-021-abci-events.md)
+- [ADR-025: Commit](./adr-025-commit.md)
+- [ADR-026: General-Merkle-Proof](./adr-026-general-merkle-proof.md)
+- [ADR-033: Pubsub](./adr-033-pubsub.md)
+- [ADR-034: Priv-Validator-File-Structure](./adr-034-priv-validator-file-structure.md)
+- [ADR-043: Blockchain-RiRi-Org](./adr-043-blockchain-riri-org.md)
+- [ADR-044: Lite-Client-With-Weak-Subjectivity](./adr-044-lite-client-with-weak-subjectivity.md)
+- [ADR-046: Light-Client-Implementation](./adr-046-light-client-implementation.md)
+- [ADR-047: Handling-Evidence-From-Light-Client](./adr-047-handling-evidence-from-light-client.md)
+- [ADR-051: Double-Signing-Risk-Reduction](./adr-051-double-signing-risk-reduction.md)
+- [ADR-052: Tendermint-Mode](./adr-052-tendermint-mode.md)
+- [ADR-053: State-Sync-Prototype](./adr-053-state-sync-prototype.md)
+- [ADR-054: Crypto-Encoding-2](./adr-054-crypto-encoding-2.md)
+- [ADR-055: Protobuf-Design](./adr-055-protobuf-design.md)
+- [ADR-056: Light-Client-Amnesia-Attacks](./adr-056-light-client-amnesia-attacks.md)
+- [ADR-059: Evidence-Composition-and-Lifecycle](./adr-059-evidence-composition-and-lifecycle.md)
+- [ADR-062: P2P-Architecture](./adr-062-p2p-architecture.md)
+- [ADR-063: Privval-gRPC](./adr-063-privval-grpc.md)
+- [ADR-066: E2E-Testing](./adr-066-e2e-testing.md)
+- [ADR-072: Restore Requests for Comments](./adr-072-request-for-comments.md)
+- [ADR-077: Block Retention](./adr-077-block-retention.md)
+- [ADR-078: Non-zero Genesis](./adr-078-nonzero-genesis.md)
+- [ADR-079: ED25519 Verification](./adr-079-ed25519-verification.md)
+- [ADR-080: Reverse Sync](./adr-080-reverse-sync.md)
+
+### Accepted
+
+- [ADR-006: Trust-Metric](./adr-006-trust-metric.md)
+- [ADR-024: Sign-Bytes](./adr-024-sign-bytes.md)
+- [ADR-035: Documentation](./adr-035-documentation.md)
+- [ADR-039: Peer-Behaviour](./adr-039-peer-behaviour.md)
+- [ADR-060: Go-API-Stability](./adr-060-go-api-stability.md)
+- [ADR-061: P2P-Refactor-Scope](./adr-061-p2p-refactor-scope.md)
+- [ADR-065: Custom Event Indexing](./adr-065-custom-event-indexing.md)
+- [ADR-068: Reverse-Sync](./adr-068-reverse-sync.md)
+- [ADR-067: Mempool Refactor](./adr-067-mempool-refactor.md)
+- [ADR-075: RPC Event Subscription Interface](./adr-075-rpc-subscription.md)
+- [ADR-076: Combine Spec and Tendermint Repositories](./adr-076-combine-spec-repo.md)
+- [ADR-081: Protocol Buffers Management](./adr-081-protobuf-mgmt.md)
+
+### Deprecated
+
+None
+
+### Rejected
+
+- [ADR-023: ABCI-Propose-tx](./adr-023-ABCI-propose-tx.md)
+- [ADR-029: Check-Tx-Consensus](./adr-029-check-tx-consensus.md)
+- [ADR-058: Event-Hashing](./adr-058-event-hashing.md)
+
+### Proposed
+
+- [ADR-007: Trust-Metric-Usage](./adr-007-trust-metric-usage.md)
+- [ADR-012: Peer-Transport](./adr-012-peer-transport.md)
+- [ADR-013: Symmetric-Crypto](./adr-013-symmetric-crypto.md)
+- [ADR-022: ABCI-Errors](./adr-022-abci-errors.md)
+- [ADR-030: Consensus-Refactor](./adr-030-consensus-refactor.md)
+- [ADR-036: Empty Blocks via ABCI](./adr-036-empty-blocks-abci.md)
+- [ADR-037: Deliver-Block](./adr-037-deliver-block.md)
+- [ADR-038: Non-Zero-Start-Height](./adr-038-non-zero-start-height.md)
+- [ADR-040: Blockchain Reactor Refactor](./adr-040-blockchain-reactor-refactor.md)
+- [ADR-041: Proposer-Selection-via-ABCI](./adr-041-proposer-selection-via-abci.md)
+- [ADR-042: State Sync Design](./adr-042-state-sync.md)
+- [ADR-045: ABCI-Evidence](./adr-045-abci-evidence.md)
+- [ADR-050: Improved Trusted Peering](./adr-050-improved-trusted-peering.md)
+- [ADR-057: RPC](./adr-057-RPC.md)
+- [ADR-064: Batch Verification](./adr-064-batch-verification.md)
+- [ADR-069: Node Initialization](./adr-069-flexible-node-initialization.md)
+- [ADR-071: Proposer-Based Timestamps](./adr-071-proposer-based-timestamps.md)
+- [ADR-073: Adopt LibP2P](./adr-073-libp2p.md)
+- [ADR-074: Migrate Timeout Parameters to Consensus Parameters](./adr-074-timeout-params.md)
diff --git a/docs/architecture/adr-001-logging.md b/docs/architecture/adr-001-logging.md
@@ -0,0 +1,216 @@
+# ADR 1: Logging
+
+## Context
+
+Current logging system in Tendermint is very static and not flexible enough.
+
+Issues: [358](https://github.com/tendermint/tendermint/issues/358), [375](https://github.com/tendermint/tendermint/issues/375).
+
+What we want from the new system:
+
+- per package dynamic log levels
+- dynamic logger setting (logger tied to the processing struct)
+- conventions
+- be more visually appealing
+
+"dynamic" here means the ability to set smth in runtime.
+
+## Decision
+
+### 1) An interface
+
+First, we will need an interface for all of our libraries (`tmlibs`, Tendermint, etc.). My personal preference is go-kit `Logger` interface (see Appendix A.), but that is too much a bigger change. Plus we will still need levels.
+
+```go
+# log.go
+type Logger interface {
+    Debug(msg string, keyvals ...interface{}) error
+    Info(msg string, keyvals ...interface{}) error
+    Error(msg string, keyvals ...interface{}) error
+
+	  With(keyvals ...interface{}) Logger
+}
+```
+
+On a side note: difference between `Info` and `Notice` is subtle. We probably
+could do without `Notice`. Don't think we need `Panic` or `Fatal` as a part of
+the interface. These funcs could be implemented as helpers. In fact, we already
+have some in `tmlibs/common`.
+
+- `Debug` - extended output for devs
+- `Info` - all that is useful for a user
+- `Error` - errors
+
+`Notice` should become `Info`, `Warn` either `Error` or `Debug` depending on the message, `Crit` -> `Error`.
+
+This interface should go into `tmlibs/log`. All libraries which are part of the core (tendermint/tendermint) should obey it.
+
+### 2) Logger with our current formatting
+
+On top of this interface, we will need to implement a stdout logger, which will be used when Tendermint is configured to output logs to STDOUT.
+
+Many people say that they like the current output, so let's stick with it.
+
+```
+NOTE[2017-04-25|14:45:08] ABCI Replay Blocks                       module=consensus appHeight=0 storeHeight=0 stateHeight=0
+```
+
+Couple of minor changes:
+
+```
+I[2017-04-25|14:45:08.322] ABCI Replay Blocks            module=consensus appHeight=0 storeHeight=0 stateHeight=0
+```
+
+Notice the level is encoded using only one char plus milliseconds.
+
+Note: there are many other formats out there like [logfmt](https://brandur.org/logfmt).
+
+This logger could be implemented using any logger - [logrus](https://github.com/sirupsen/logrus), [go-kit/log](https://github.com/go-kit/kit/tree/master/log), [zap](https://github.com/uber-go/zap), log15 so far as it
+
+a) supports coloring output<br>
+b) is moderately fast (buffering) <br>
+c) conforms to the new interface or adapter could be written for it <br>
+d) is somewhat configurable<br>
+
+go-kit is my favorite so far. Check out how easy it is to color errors in red https://github.com/go-kit/kit/blob/master/log/term/example_test.go#L12. Although, coloring could only be applied to the whole string :(
+
+```
+go-kit +: flexible, modular
+go-kit “-”: logfmt format https://brandur.org/logfmt
+
+logrus +: popular, feature rich (hooks), API and output is more like what we want
+logrus -: not so flexible
+```
+
+```go
+# tm_logger.go
+// NewTmLogger returns a logger that encodes keyvals to the Writer in
+// tm format.
+func NewTmLogger(w io.Writer) Logger {
+  return &tmLogger{kitlog.NewLogfmtLogger(w)}
+}
+
+func (l tmLogger) SetLevel(level string() {
+  switch (level) {
+    case "debug":
+      l.sourceLogger = level.NewFilter(l.sourceLogger, level.AllowDebug())
+  }
+}
+
+func (l tmLogger) Info(msg string, keyvals ...interface{}) error {
+  l.sourceLogger.Log("msg", msg, keyvals...)
+}
+
+# log.go
+func With(logger Logger, keyvals ...interface{}) Logger {
+  kitlog.With(logger.sourceLogger, keyvals...)
+}
+```
+
+Usage:
+
+```go
+logger := log.NewTmLogger(os.Stdout)
+logger.SetLevel(config.GetString("log_level"))
+node.SetLogger(log.With(logger, "node", Name))
+```
+
+**Other log formatters**
+
+In the future, we may want other formatters like JSONFormatter.
+
+```
+{ "level": "notice", "time": "2017-04-25 14:45:08.562471297 -0400 EDT", "module": "consensus", "msg": "ABCI Replay Blocks", "appHeight": 0, "storeHeight": 0, "stateHeight": 0 }
+```
+
+### 3) Dynamic logger setting
+
+https://dave.cheney.net/2017/01/23/the-package-level-logger-anti-pattern
+
+This is the hardest part and where the most work will be done. logger should be tied to the processing struct, or the context if it adds some fields to the logger.
+
+```go
+type BaseService struct {
+    log     log15.Logger
+    name    string
+    started uint32 // atomic
+    stopped uint32 // atomic
+...
+}
+```
+
+BaseService already contains `log` field, so most of the structs embedding it should be fine. We should rename it to `logger`.
+
+The only thing missing is the ability to set logger:
+
+```
+func (bs *BaseService) SetLogger(l log.Logger) {
+  bs.logger = l
+}
+```
+
+### 4) Conventions
+
+Important keyvals should go first. Example:
+
+```
+correct
+I[2017-04-25|14:45:08.322] ABCI Replay Blocks                       module=consensus instance=1 appHeight=0 storeHeight=0 stateHeight=0
+```
+
+not
+
+```
+wrong
+I[2017-04-25|14:45:08.322] ABCI Replay Blocks                       module=consensus appHeight=0 storeHeight=0 stateHeight=0 instance=1
+```
+
+for that in most cases you'll need to add `instance` field to a logger upon creating, not when u log a particular message:
+
+```go
+colorFn := func(keyvals ...interface{}) term.FgBgColor {
+		for i := 1; i < len(keyvals); i += 2 {
+			if keyvals[i] == "instance" && keyvals[i+1] == "1" {
+				return term.FgBgColor{Fg: term.Blue}
+			} else if keyvals[i] == "instance" && keyvals[i+1] == "1" {
+				return term.FgBgColor{Fg: term.Red}
+			}
+		}
+		return term.FgBgColor{}
+	}
+logger := term.NewLogger(os.Stdout, log.NewTmLogger, colorFn)
+
+c1 := NewConsensusReactor(...)
+c1.SetLogger(log.With(logger, "instance", 1))
+
+c2 := NewConsensusReactor(...)
+c2.SetLogger(log.With(logger, "instance", 2))
+```
+
+## Status
+
+Implemented
+
+## Consequences
+
+### Positive
+
+Dynamic logger, which could be turned off for some modules at runtime. Public interface for other projects using Tendermint libraries.
+
+### Negative
+
+We may loose the ability to color keys in keyvalue pairs. go-kit allow you to easily change foreground / background colors of the whole string, but not its parts.
+
+### Neutral
+
+## Appendix A.
+
+I really like a minimalistic approach go-kit took with his logger https://github.com/go-kit/kit/tree/master/log:
+
+```
+type Logger interface {
+    Log(keyvals ...interface{}) error
+}
+```
+
+See [The Hunt for a Logger Interface](https://web.archive.org/web/20210902161539/https://go-talks.appspot.com/github.com/ChrisHines/talks/structured-logging/structured-logging.slide#1). The advantage is greater composability (check out how go-kit defines colored logging or log-leveled logging on top of this interface https://github.com/go-kit/kit/tree/master/log).