Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks to check performance #518

Open
6 tasks
duffee opened this issue Jan 7, 2025 · 1 comment
Open
6 tasks

Add benchmarks to check performance #518

duffee opened this issue Jan 7, 2025 · 1 comment

Comments

@duffee
Copy link

duffee commented Jan 7, 2025

Test PDL against Julia's loop fusion

Comparison with Programming Language Benchmark v2 will require an implementation of nqueens, sudoku and bedcov. I assume we can already do matmult out of the box. Rolling this out will go some ways towards covering our collective blushes when it comes to Perl being the slowest on the graph.

We have timing code under Basic/examples/Benchmark/ already. I think the new examples could live there, unless there's a reason to build more infrastructure that doesn't belong in the core project. Also, follow up on TODOs in documentation for PDL::MatrixOps determinant v det and all of PDL::Matrix and try to address the benchmarking comment mentioned in PDL::Indexing.

Assign this to me. I've used Hyperfine before, but I'll be in touch for the more esoteric bits of C and XS.

TODO

Write PDL benchmarks for:

  • nqueens
  • matmult
  • sudoku
  • bedcov

and then

  • Identify functions with the longest dwell times
  • Write a guide for how we generate our benchmarks

Stretch goals

@mohawk2
Copy link
Member

mohawk2 commented Jan 7, 2025

I'm assuming that last one is a Freudian slip? You put "pdl2" ;-)

There's an instant win for the one Perl one I looked at, nqueens. Near the start, it first initialises some Perl arrays, then loops through initialising them (even though they already have the right values). Just deleting that loop can only save time.

And yes, the whole point of making these benchmarks in PDL (that perform well) would be to PR them into that repo. I'm pretty sure with auto-pthreading, the PDL version would (for algorithms that can be parallelised, which my first glance at nqueens did not show) actually be faster than C. That was my finding with Fourmilab/floating_point_benchmarks#1

See #451 for performance-measuring tools and techniques.

Another target: https://github.com/FPBench/FPBench

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants