raft: plumbing crdb version handle into raft

[lyang24]

We need the ability to safely ship backward incompatible changes in the raft module. This commit plumbs version handle in to raft package to enable version gate infrastructure in raft.

Epic: None

Release note: None

[blathers-crl]

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

Thank you for contributing to CockroachDB. Please ensure you have followed the guidelines for creating a PR.

My owl senses detect your PR is good for review. Please keep an eye out for any test failures in CI.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

[cockroach-teamcity]

This change is 

[pav-kv]

Probably need to run dev generate bazel to update the BUILD files.

[blathers-crl]

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

Thank you for updating your pull request.

My owl senses detect your PR is good for review. Please keep an eye out for any test failures in CI.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

[pav-kv]

LGTM, this will do. Let's keep this uncommitted for now, it's better to include this commit into a PR that would use the gate.

[blathers-crl]

Thank you for updating your pull request.

Before a member of our team reviews your PR, I have some potential action items for you:

• We notice you have more than one commit in your PR. We try break logical changes into separate commits, but commits such as "fix typo" or "address review commits" should be squashed into one commit and pushed with --force
• Please ensure your git commit message contains a release note.
• When CI has completed, please ensure no errors have appeared.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

[blathers-crl]

Thank you for updating your pull request.

Before a member of our team reviews your PR, I have some potential action items for you:

• We notice you have more than one commit in your PR. We try break logical changes into separate commits, but commits such as "fix typo" or "address review commits" should be squashed into one commit and pushed with --force
• Please ensure your git commit message contains a release note.
• When CI has completed, please ensure no errors have appeared.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

[lyang24]

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @pav-kv)

---

pkg/raft/raft_test.go line 1149 at r3 (raw file):

		// Increase the commit index only if the log is in sync with the leader.
		{pb.Message{From: 2, To: 1, Type: pb.MsgHeartbeat, Term: 2, Commit: commit + 1}, false, commit},

I copied code from this commit etcd-io/raft@eb6bfc6
This test case will fail I see both leader's commit index and follower's last index is at 3 and we are expecting 2
cc @pav-kv

[pav-kv]

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @lyang24)

---

pkg/raft/raft_test.go line 1149 at r3 (raw file):

Previously, lyang24 (Lanqing Yang) wrote…

I copied code from this commit etcd-io/raft@eb6bfc6
This test case will fail I see both leader's commit index and follower's last index is at 3 and we are expecting 2
cc @pav-kv

This test was buggy initially, let's clean it up first (see the other comment).

---

pkg/raft/raft_test.go line 1158 at r3 (raw file):

		require.NoError(t, storage.Append(index(1).terms(1, 2, 3)))
		sm := newTestRaft(1, 5, 1, storage)
		sm.becomeFollower(2, 2)

This test was incorrect. The sm.becomeFollower(2, 2) call makes the node a follower at term 2 with the leader node ID = 2. But the log in the Storage, initialized 2 lines above, ends with an entry at term 3. This can't be the case: the invariant is that Term >= log.Entry(lastIndex).Term.

We should probably:

• Call storage.SetHardState() with the HardState at term 3, and commit index at commit.
• Then the newTestRaft, while creating the raft node, will read the hard state from storage and init it correctly.
• Then we can remove the commitTo invocation in the next line.
• And we probably can remove the logSynced initialization as well, because it should be initialized when the raft node is created in newTestRaft.

[blathers-crl]

Thank you for updating your pull request.

Before a member of our team reviews your PR, I have some potential action items for you:

• We notice you have more than one commit in your PR. We try break logical changes into separate commits, but commits such as "fix typo" or "address review commits" should be squashed into one commit and pushed with --force
• Please ensure your git commit message contains a release note.
• When CI has completed, please ensure no errors have appeared.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

[pav-kv]

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @lyang24)

---

pkg/raft/raft.go line 358 at r5 (raw file):

	// candidate. For a follower, becomes true the first time a MsgApp append to
	// the log succeeds.
	logSynced bool

Let's replace this field with accTerm uint64 (the latest "accepted term").

The code for tracking this field will be simpler because accTerm will be update only in 2 places: when accepting a log append, or when installing a snapshot (which wipes the log, and is equivalent to simultaneously accepting and committing the state represented by this snapshot).

In contrast, the code for updating logSynced is a bit less explicit, and there are more conditions under which it has to be updated (for example, we need to update it also when r.Term is bumped).

Having the accTerm field will be aligned with #122446, and eventually this field will become persistent.

[pav-kv]

Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @lyang24)

---

pkg/raft/raft.go line 358 at r5 (raw file):

Previously, pav-kv (Pavel Kalinnikov) wrote…

Let's replace this field with accTerm uint64 (the latest "accepted term").

The code for tracking this field will be simpler because accTerm will be update only in 2 places: when accepting a log append, or when installing a snapshot (which wipes the log, and is equivalent to simultaneously accepting and committing the state represented by this snapshot).

In contrast, the code for updating logSynced is a bit less explicit, and there are more conditions under which it has to be updated (for example, we need to update it also when r.Term is bumped).

Having the accTerm field will be aligned with #122446, and eventually this field will become persistent.

You could draw some inspiration from the earlier version of the upstream PR: etcd-io/raft@87ac09e.