Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

numpy ufunc reduce should expect built in upcasting (remove setbufsize logic) #17

Open
tdimitri opened this issue Oct 1, 2020 · 0 comments
Labels
numpy changes fixing this requires support from numpy

Comments

@tdimitri
Copy link
Collaborator

tdimitri commented Oct 1, 2020

When an .sum() operation is performed (which is a popular and important operation) on an int16 array it appears that numpy

  1. upcasts int16 to int32 based on setbufsize
  2. then sets the output value to 0
  3. then the reduce routine adds up all the int32 and produced a result
  4. another upcast from int16 to int32 occurs based on setbufsize
  5. the previous output value is expected as the first input to add up all the numbers

I presume this is done due to either handle overflow, or round off error in sum (when it is a float32).

I recommend that this change because a sum routine can internally upcast and return the result of the upcast.
For instance int16 can be upcast to int64 (not sure why it is int32 on windows) at no cost since the int64 is held in registers.

Further the signature of the reduce for sum is coupled to the signature for add even though they should be different.
For example the reduce signature of add on int16 is really one input: dtype: int16 and outputs dtype:int64.

@jack-pappas jack-pappas added the numpy changes fixing this requires support from numpy label Dec 16, 2020
@mattip mattip removed their assignment Jul 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
numpy changes fixing this requires support from numpy
Projects
None yet
Development

No branches or pull requests

3 participants