Ruff fixes

[akx]

This PR applies a bunch of fixes suggested by the Ruff linter (which has been configured since 2c605d0 (#896), just never really run).

Some of these are mechanical auto-fixes (--fix --select=I,F401 style), but most of them are manual.

The commits themselves are atomic and small enough to be able to be reviewed one by one.

It would probably be a good idea to enforce linting with Ruff in CI; if the maintainers like, I can contribute a the requisite GitHub Actions configuration too :)

[Titus-von-Koeller]

Thanks a lot for this contribution. Yes, I had contributed the Ruff configuration (which might still need some tweaking, just put this as a starting point).

I agree that having proper formatting + linting in place is a definite must for a project like ours. Anyways, the issue why I haven't still followed through with this is that, unfortunately, our test suite is still flaky. The bounds on non-deterministic operations also are different based on GPU model, which makes the whole situation tricky as so far we depend on contributors running their tests locally or using our integration test pipeline on the HuggingFace side which has T4 GPU runners, but which we don't run commit for commit and which can't integrate directly with the PRs as bitsandbytes is purposefully not part of the HF organization.

Right now, this leaves us without GPU runners to run our test suite on every PR (or even commit) and we've shied away from implementing this as we're still rushing to get on top of the big maintenance load that had gone relatively unattended for quite a while. Our biggest topics right now are the FSDP + QLoRA and cross-platform, for the latter there's a couple of outstanding PRs.

I'm mentioning PRs, because if we merge these fixes here first, then all 20ish open PRs might get merge collisions because of this. And for each fix, we also have to validate that it's not breaking anything as some of the Ruff fixes are not guaranteed to not introduce issues based on my current knowledge (correct me if I'm wrong). I'm afraid to do this because we have 2+ million downloads a month and given a certain degree of uncertainty so far my judgement has been to hold off with this, until other more high impact topics have been solved.

However, I'm open to discuss this and agree that linting + formatting should be solved (preferably asap). The CI topic is also close to my heart, but currently the team has decided to hold off on that for the time being based on our priorities.

Did you run the full test suite after applying the fixes? Maybe you could post the results?

Yes, if you'd contribute a GitHub Action, I would definitely merge it as soon as we decide it's safe.

What are your thoughts?

Thanks for your initiative, I really appreciate having so many active contributors on this project! 🙏🏻

[akx]

I'm mentioning PRs, because if we merge these fixes here first, then all 20ish open PRs might get merge collisions because of this.

That's the case for any PR, though, isn't it? 😅 Merge conflicts will just need to be fixed one way or the other.

This PR does not run a full ruff-format (or black) re-format on the entire codebase, so it will not wholesale invalidate all open PRs. (Such a reformat should be done as a separate, single, mechanical commit in a separate PR, so it can be cleanly applied and added to .git-blame-ignore-revs.)

And for each fix, we also have to validate that it's not breaking anything as some of the Ruff fixes are not guaranteed to not introduce issues based on my current knowledge (correct me if I'm wrong).

As mentioned in the pull request description, these fixes are mostly manual, i.e. suggested or flagged by Ruff, and applied (or marked noqa) by hand. And similarly, as mentioned, the fixes have been applied in separate commits to make reviewing even somewhat possible 😄 You can scroll through each commit in the GitHub review view.

As for whether issues can be introduced – any software can have bugs, Ruff included, but the Ruff team (disclaimer: I've contributed to Ruff but am not otherwise affiliated) has taken quite some care to make possibly-unsafe autofixes opt-in (--unsafe-fixes).

I'm afraid to do this because we have 2+ million downloads a month and given a certain degree of uncertainty so far my judgement has been to hold off with this, until other more high impact topics have been solved.

I'm the only current maintainer of Babel which has 28+ million downloads a month, so I'm aware of how that feels... 😄

Either way, even if changes are merged to main, that doesn't mean they'll need to be released in a new version on PyPI right away (and risk production issues and the ire of those downloaders). As evident from #981 and #982, there are intrepid people who are using the repo as-is directly from main, not a tagged release, and that's a good thing in my books; issues should surface, and hopefully get fixes, before they're released in a tagged version.

Did you run the full test suite after applying the fixes? Maybe you could post the results?

I wish I could – my primary dev machine is Apple Silicon, so it's not exactly CUDA conducive, nor even make cpuonly conducive by default since there's a bunch of x86/x64-expecting code to be built.

Running in a linux/amd64 docker container doesn't help much either – there's a ton of tests that actually require CUDA but aren't marked as such (see #985), so there's a lot of false "failed"s that should be skips:

706 failed, 75 passed, 2339 skipped, 1 warning, 16 errors in 42.86s

[Titus-von-Koeller]

Really cool, great work on Babel! That kind of stuff keeps the community thriving 🙌🏻

Ok, thanks for your input. All valid points and thanks in your interest in bitsandbytes.

Also thanks for all the good tips + pointers, really appreciated 🤗

I've been wanting to improve our Hugging Face side integration pipeline to also run bitsandbytes tests and not just the HF integration tests anyways. Let me take this as an occasion and take a look these days and then see what our tests say and review your PR.

Getting the tests to be non-flaky on that pipeline / GPU model will be another big win.

You're right, better to get this merged :)

Thanks again for your work and kind words. If you feel like contributing the GH action, that would also be very appreciated. We've so much going on to get on top maintenance since we took over that having that solved would be a big help!

[akx]

I've been wanting to improve our Hugging Face side integration pipeline to also run bitsandbytes tests and not just the HF integration tests anyways. Getting the tests to be non-flaky on that pipeline / GPU model will be another big win.

Good luck 😅 I hope the pipeline won't remain just an "integration pipeline on our side" that contributors at large won't have any visibility to, or that tests devolve into "oh yeah, it works for our in-house developers but is red for external contributions 🤷" (looking at Gradio's CI right now...). On that note, I think it would be super beneficial to get #949 merged.

Would be nice if HF could donate CPU + CUDA GPU time in the form of Github Actions runners!

If you feel like contributing the GH action, that would also be very appreciated.

Rebased + added a commit to add it (as a pre-commit configuration).

We've so much going on to get on top maintenance since we took over that having that solved would be a big help!

Ah, is HF (if I understood who "we" is correctly) maintaining bitsandbytes now? Nice.

[akx]

@Titus-von-Koeller Could this get merged? I rebased it to fix conflicts (and added a commit to fix a bug introduced in #873 that would have been spotted by Ruff... 😄)

[Titus-von-Koeller]

Yes, with HF I refer to Hugging Face. I was with bitsandbytes already before, but after using my savings to work on BNB full-time, they allowed me to continue my work by paying me and they now support bitsandbytes otherwise as well.

Yes, they already said that they'd sponsor compute for the runners, but there's no managed solution available and with the cross-platform effort we need a whole matrix of accelerated runners with Linux, Windows, Mac, Nvidia, AMD, Intel, Apple Silicon.. We're still shying away from the pipeline logic of spinning up runners on demand, but that's definitely gonna happen within the coming 3 months or so.

The integration pipeline on the HF side was just because we have easy access to accelerated runners there (HF Github organization) and needed a quick fix to have visibility of the integrations. It was not to exclude anyone. Anyways, I totally agree that visibility on PRs is super important and we'll try to get that working soon enough. It's just that we have a ginormous amount of work to get on top of and the GPU runners was decided to not be the most urgent.

By us, I mean BNB maintainers btw. Right now that is pretty much mainly me, with @younesbelkada chipping in when he finds some time and Tim Dettmers not available for the next 3 months due to his applications in accademia, but consulting every now and then.

I should also be working on high impact stuff as the FSDP + Qlora integration, etc, so right now it's a lot on my plate. But I'm trying to get to a point where I'm not blocking community contributions anymore and we have a plan that allows the community to chip in as much as possible, also in regard to which technical solutions we go for.

[Titus-von-Koeller]

Very helpful contribution! Thanks for being so considerate and the initiative of tackling this heads-on. Reviewed it thoroughly and happy to merge it :)

[younesbelkada]

Thanks a lot for addressing this @akx - very nice contribution ! Thanks @Titus-von-Koeller for the detailed review!
The CI is failing due to some files not correctly formatted, could you have a look before merging? 🙏

[Titus-von-Koeller]

@younesbelkada Yeah, just fixed it. Had to go to do sports first, so you caught me in the middle of messing things up by trying to fix the merge conflict from before in the web editor 😅

Anyways, @younesbelkada please be sure to also

pip install pre-commit
cd bnb
pre-commit install

We have to add that to the docs as well, once we have those up.

[younesbelkada]

Awesome - thanks so much @Titus-von-Koeller ! 🚀

[Titus-von-Koeller]

Hey @akx,

Unfortunately a bunch of the integration tests from the Hugging Face side broke based on some of the changes that we merged within the last day.

Would you be able to look into this? Unfortunately, this week is very tight for me, as I'm going to be off from this Saturday for 9 days. Actually, that's why we'll also hold off on a release until I'm back (of course we shall also work toward CD in the mid/long-term).

I agree that it would be nicer to trigger this pipeline for PRs in this repo, but that's what we currently have to work with for the time being. Definitely sth to improve sooner rather than later, but for now it works.

Find the CI logs below:
transformers-integration-tests_multi-gpu.log
transformers-integration-tests_single-gpu.log

If you're too busy, please let me know and we'll figure out another way.

[akx]

@Titus-von-Koeller I can take a look, but those logs don't help an awful lot, unfortunately. I'll try to set up WSL2 on my only Nvidia-equipped machine to try and reproduce, but of course you can also try to git bisect to find out where things went wrong.

As for releasing a new version, is there pressure from... somewhere to do that? AFAICS there aren't very exciting things in main since 0.42.0 – the Cmake and/or Apple Silicon work would definitely be exciting to release in a new version :)

[Titus-von-Koeller]

We should run those tests for every commit on main anyways (as long as we don't have proper CI for PRs within bnb). But for now it's manual trigger or nightly. I wasn't mindful enough to trigger after every PR, lesson learned.

Yeah, setting things up on WSL2 with Nvidia would be cool, especially since you're so involved with the repo now :) WSL2 support is also important, so if you come across anything, let us know.

Nono, currently there's no pressure to release. No worries. I would aim to have a release in 2-3 weeks.

I don't think I'll get around to Git bisect this week. Todo is already too long.

[akx]

You did mention earlier that some tests are flaky – can you double-check if those tests that supposedly fail due to recent changes had ever failed flakily in the past?

WSL2 support is also important, so if you come across anything, let us know.

Well, now that you mention it, to begin with:

• There's no check in the makefile for a missing CUDA_HOME, so make attempting /bin/nvcc was a bit confusing 😁 (but that should be fixed by the CMake efforts in Cmake + workflows #908, Make native code portable and add GitHub workflow for building #949)
• The output of python -m bitsandbytes is long, seems to conflict with itself, starts with an inexplicable False, followed by BUG REPORT, followed by UserWarning: Welcome to bitsandbytes. (why is that a warning? I'm a bit scared already... 😁 )... 😂 But well, we're already working on it, aren't we? [RFC] cross-platform: Refactoring bitsandbytes/cuda_setup #918, CUDA setup cleanup #996...
• Running pytest after pip install -r requirements.txt simply doesn't work (missing einops, transformers) (this is taken care of in Make native code portable and add GitHub workflow for building #949)
• Some tests (matmullt and igemmlt at least) segfault py.test in File "/home/akx/bitsandbytes/bitsandbytes/functional.py", line 1930 in igemmlt, the cigemmlt_turing_32 call. For the time being, I patched that function out of my copy. I think it's my Geforce 1070 and the assert(false) in the C++ code – it should probably just signal a Not Implemented error to the Python code instead of crashing the whole damn thing 😁 (see Don't crash Python interpreter via assert(false) when missing cublaslt #998 for a fix)
• (Failing, maybe all) tests can be made to run a whole lot faster with --assert=plain (because otherwise pytest attempts to call torch's repr on a huge vector to give a human-readable comparison)...
• ... and finally, just at 99% of tests run, at tests/test_optim.py::test_stream_optimizer_bench[dim1_2048_gtype_torch.float16_optim_paged_adamw_mode_bnb] the pytest process crashes hard and I get

  [Jan30 21:36] misc dxg: dxgk: dxgvmb_send_create_allocation: send_create_allocation failed ffffffb5
  [  +0.002524] misc dxg: dxgk: dxgkio_create_allocation: Ioctl failed: -75
  [  +0.064831] misc dxg: dxgk: dxgvmb_send_create_allocation: send_create_allocation failed ffffffb5
  [  +0.000503] misc dxg: dxgk: dxgkio_create_allocation: Ioctl failed: -75
  [  +0.517778] misc dxg: dxgk: dxgkio_make_resident: Ioctl failed: -12
  [  +5.554598] hv_vmbus: Failed to establish GPADL: err = 0xc0000044
  [  +0.000415] misc dxg: dxgk: create_existing_sysmem: establish_gpadl failed: -122
  [  +0.000442] misc dxg: dxgk: dxgkio_create_allocation: Ioctl failed: -12
  [  +0.007347] hv_vmbus: Failed to establish GPADL: err = 0xc0000044
  [  +0.000305] misc dxg: dxgk: create_existing_sysmem: establish_gpadl failed: -122
  [  +0.000528] misc dxg: dxgk: dxgkio_create_allocation: Ioctl failed: -12
  

  in dmesg, which I'll assume is the test suite attempting to allocate more VRAM than I have available.

I didn't get around to running transformers's tests just yet...

[akx]

@Titus-von-Koeller Based on transformers-integration-tests_single-gpu.log, I ran env RUN_SLOW=1 py.test tests/quantization/bnb/test_4bit.py -ra -svv --assert=plain with my bitsandbytes working copy and the tests that fail are

FAILED tests/quantization/bnb/test_4bit.py::BaseSerializationTest::test_serialization - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_4bit.py::ExtendedSerializationTest::test_fp4_double_safe - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_4bit.py::ExtendedSerializationTest::test_fp4_double_unsafe - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_4bit.py::ExtendedSerializationTest::test_fp4_single_safe - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_4bit.py::ExtendedSerializationTest::test_fp4_single_unsafe - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_4bit.py::ExtendedSerializationTest::test_nf4_double_unsafe - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_4bit.py::ExtendedSerializationTest::test_nf4_single_safe - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_4bit.py::ExtendedSerializationTest::test_nf4_single_unsafe - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_4bit.py::ExtendedSerializationTest::test_serialization - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_4bit.py::BloomSerializationTest::test_serialization - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_4bit.py::GPTSerializationTest::test_serialization - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']

and I really don't think that has anything to do with my changes 😁 Instead, #991 seems like it could be related...

Also: practically every test in transformers raises a

The load_in_4bit and load_in_8bit arguments are deprecated and will be removed in the future versions. Please, pass a BitsAndBytesConfig object in quantization_config argument instead.

warning. Would be nice if they didn't.

EDIT: I also finished a run of env RUN_SLOW=1 py.test tests/quantization/bnb/test_mixed_int8.py -ra -svv --assert=plain:

SKIPPED [1] tests/quantization/bnb/test_mixed_int8.py:634: test requires multiple GPUs
SKIPPED [1] tests/quantization/bnb/test_mixed_int8.py:746: test requires multiple GPUs
SKIPPED [1] tests/quantization/bnb/test_mixed_int8.py:773: test requires multiple GPUs
SKIPPED [1] tests/quantization/bnb/test_mixed_int8.py:719: test requires multiple GPUs
SKIPPED [1] tests/quantization/bnb/test_mixed_int8.py:671: test requires multiple GPUs
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8Test::test_generate_quality - AssertionError: 'Hello my name is John and I am a senior at the University of' not found in {'Hello my name is John.\nI am a friend of the family.\n'}
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8Test::test_generate_quality_config - ValueError:
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8Test::test_int8_from_pretrained - AssertionError: 'Hello my name is John and I am a senior at the University of' not found in {'Hello my name is John.\nI am a friend of the family.\n'}
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8Test::test_int8_serialization - ValueError:
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8Test::test_int8_serialization_regression - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8Test::test_int8_serialization_sharded - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8T5Test::test_inference_with_keep_in_fp32_serialized - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8TestPipeline::test_pipeline - AssertionError: 'Hello my name is John and I am a senior at the University of' not found in {'Hello my name is John.\nI am a friend of the family.\n'}
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8TestTraining::test_training - AssertionError: False is not true
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8GPT2Test::test_generate_quality - AssertionError: "Hello my name is John. I'm a writer. I'm a" not found in {"Hello my name is John Doe, and I'm a big fan of", "Hello my name is John Doe, and I'm a fan of the"}
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8GPT2Test::test_generate_quality_config - AssertionError: "Hello my name is John. I'm a writer. I'm a" not found in {"Hello my name is John Doe, and I'm a big fan of", "Hello my name is John Doe, and I'm a fan of the"}
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8GPT2Test::test_int8_from_pretrained - AssertionError: "Hello my name is John. I'm a writer. I'm a" not found in {"Hello my name is John Doe, and I'm a big fan of", "Hello my name is John Doe, and I'm a fan of the"}
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8GPT2Test::test_int8_serialization - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8GPT2Test::test_int8_serialization_regression - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8GPT2Test::test_int8_serialization_sharded - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
======= 15 failed, 23 passed, 5 skipped, 11 warnings in 428.12s (0:07:08) 

These don't seem very related either.

For completeness, I did git checkout v0.44.0 in bitsandbytes, pip install -e ../bitsandbytes'd it and ran the last battery of tests again with --lf, and the tests still fail:

FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8Test::test_generate_quality - AssertionError: 'Hello my name is John and I am a senior at the University of' not found in {'Hello my name is John.\nI am a friend of the family.\n'}
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8Test::test_generate_quality_config - ValueError:
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8Test::test_int8_from_pretrained - AssertionError: 'Hello my name is John and I am a senior at the University of' not found in {'Hello my name is John.\nI am a friend of the family.\n'}
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8Test::test_int8_serialization - ValueError:
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8Test::test_int8_serialization_regression - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8Test::test_int8_serialization_sharded - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8T5Test::test_inference_with_keep_in_fp32_serialized - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8TestPipeline::test_pipeline - AssertionError: 'Hello my name is John and I am a senior at the University of' not found in {'Hello my name is John.\nI am a friend of the family.\n'}
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8TestTraining::test_training - AssertionError: False is not true
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8GPT2Test::test_generate_quality - AssertionError: "Hello my name is John. I'm a writer. I'm a" not found in {"Hello my name is John Doe, and I'm a big fan of", "Hello my name is John Doe, and I'm a fan of the"}
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8GPT2Test::test_generate_quality_config - AssertionError: "Hello my name is John. I'm a writer. I'm a" not found in {"Hello my name is John Doe, and I'm a big fan of", "Hello my name is John Doe, and I'm a fan of the"}
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8GPT2Test::test_int8_from_pretrained - AssertionError: "Hello my name is John. I'm a writer. I'm a" not found in {"Hello my name is John Doe, and I'm a big fan of", "Hello my name is John Doe, and I'm a fan of the"}
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8GPT2Test::test_int8_serialization - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8GPT2Test::test_int8_serialization_regression - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
FAILED tests/quantization/bnb/test_mixed_int8.py::MixedInt8GPT2Test::test_int8_serialization_sharded - ValueError: Unknown quantization type, got bitsandbytes - supported types are: ['awq', 'bitsandbytes_4bit', 'bitsandbytes_8bit', 'gptq']
15 failed, 28 deselected, 5 warnings in 165.83s (0:02:45) 

[younesbelkada]

@akx sorry for the trouble ! huggingface/transformers#28788 should fix the failing tests