Rewrapping Commit Message Text Using 'git filter-repo'

[RandomDSdevel]

     I forgot to set Git's 'core.editor' configuration variable globally for/on the machine I'm working on at the moment to 'nano --fill=80' and had accumulated some commits on a new, unpushed repository where their messages then lacked proper line wrapping, so I wanted to fix that using 'git filter-repo.' I tried running:

…$ git filter-repo --force --message-callback '
  import textwrap

  return textwrap.fill(message, width=80)
'

at the root of my repository, but this resulted in the following error:

Traceback (most recent call last):  
  File "/usr/bin/git-filter-repo", line 4004, in <module>
    main()
  File "/usr/bin/git-filter-repo", line 4001, in main
    filter.run()
  File "/usr/bin/git-filter-repo", line 3936, in run
    self._parser.run(self._input, self._output).
  File "/usr/bin/git-filter-repo", line 1409, in run
    self._parse_commit()
  File "/usr/bin/git-filter-repo", line 1260, in _parse_commit
    self._commit_callback(commit, aux_info)
  File "/usr/bin/git-filter-repo", line 3413, in _tweak_commit
    commit.message = self._message_callback(commit.message)
  File "<string>", line 5, in callback
  File "/usr/lib/python3.9/textwrap.py", line 391, in fill
    return w.fill(text)
  File "/usr/lib/python3.9/textwrap.py", line 363, in fill
    return "\n".join(self.wrap(text))
  File "/usr/lib/python3.9/textwrap.py", line 351, in wrap
    chunks = self._split_chunks(text)
  File "/usr/lib/python3.9/textwrap.py", line 337, in _split_chunks
    text = self._munge_whitespace(text)
  File "/usr/lib/python3.9/textwrap.py", line 156, in _munge_whitespace
    text = text.translate(self.unicode_whitespace_trans)
TypeError: a bytes-like object is required, not 'dict'
fatal: stream ends early
fast-import: dumping crash report to .git/fast_import_crash_32668

and that internal 'git fast-import' crash dump with the following contents:

fast-import crash report:
    fast-import process: 32668
    parent process     : 32662
    at 2022-06-06 23:08:36 +0000

fatal: stream ends early

Most Recent Commands Before Crash
---------------------------------
* feature done

Active Branch LRU
-----------------
    active_branches = 0 cur, 5 max

  pos  clock name
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Inactive Branches
-----------------

Marks
-----

-------------------
END OF CRASH REPORT

This error suggests I'm missing a type conversion (via 'bytearray()') somewhere, but I couldn't figure out where that might be. What am I doing wrong here?

[newren]

None of the data in git-filter-repo is a string; it uses bytestrings everywhere.

Try this:

textwrap.fill(b'foo bar', width=3)

You'll see the same basic python stack trace. To fix the problem, you need to convert message from bytestring to string, then do the textwrap.fill, then convert back to bytestring. This would look something like (warning: untested):

git filter-repo --force --message-callback '
  import textwrap

  return textwrap.fill(message.decode('utf-8', 'backslashreplace'), width=80).encode()
'

That would work with about all but one of the repositories I've ever worked on. However...the assumption that all commit messages are utf-8 could byte you; if it's wrong, the backslashreplace will put escape codes into the resulting string, and they'll be transliterated back into ascii characters of the escape sequence -- i.e. it will not be the same as the original.

If your commit messages are not all utf-8, you could do a little better here by using a commit callback together with both commit.message and commit.encoding. So long as any non-utf-8 commit message has the correct encoding recorded in the commit object and has only valid characters from such encodings in the commit message, then you'd be safe.

Personally, I've only ever seen one repo use a non-utf-8 encoding for any commits, and it only did so for I think 1, maybe 2 commits in its many thousands of commits history. So, in practice, just assuming utf-8 is likely good enough.

[RandomDSdevel]

     I had to use double quotes instead of single quotes for my arguments to 'decode(),' but this mostly worked after I made that small change, though it merged my commit message headers and bodies by removing the line breaks between them. Thank you very much. My commit messages should all be in UTF-8.