faster bidirectional dijkstra using pairing heap

[dcoudert]

We improve method bidirectional_dijkstra from c_graph.py by using the new pairing heap data structure (see #39046) instead of a priority_queue.

For the duration of the review, I have renamed the former method bidirectional_dijkstra_old and it can be accessed using G._backend.bidirectional_dijkstra_old(...). It will be removed at the end of the review process.
I also plan to do similar changes to bidirectional_dijkstra_special in a follow-up PR.

📝 Checklist

• The title is concise and informative.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation and checked the documentation preview.

⌛ Dependencies

[dcoudert]

a few timings to see the advantage of the new implementation

sage: G = graphs.PetersenGraph()
sage: %timeit G._backend.bidirectional_dijkstra_old(0, 1)
10.6 µs ± 31.9 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)
sage: %timeit G._backend.bidirectional_dijkstra(0, 1)
10.4 µs ± 10.4 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)
sage: G = graphs.RandomGNP(10, .5)
sage: %timeit G._backend.bidirectional_dijkstra_old(0, 1)
23.2 µs ± 65 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each)
sage: %timeit G._backend.bidirectional_dijkstra(0, 1)
20.6 µs ± 105 ns per loop (mean ± std. dev. of 7 runs, 10,000 loops each)
sage: G = graphs.RandomGNP(10, .3)
sage: %timeit G._backend.bidirectional_dijkstra_old(0, 1)
17.2 µs ± 41.8 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)
sage: %timeit G._backend.bidirectional_dijkstra(0, 1)
15.6 µs ± 41.6 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)
sage: G = graphs.RandomGNP(100, .05)
sage: %timeit G._backend.bidirectional_dijkstra_old(0, 1)
309 µs ± 908 ns per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
sage: %timeit G._backend.bidirectional_dijkstra(0, 1)
259 µs ± 677 ns per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
sage: G = graphs.RandomGNP(100, .1)
sage: %timeit G._backend.bidirectional_dijkstra_old(0, 1)
587 µs ± 1.48 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
sage: %timeit G._backend.bidirectional_dijkstra(0, 1)
437 µs ± 1.2 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
sage: G = graphs.RandomGNP(100, .3)
sage: %timeit G._backend.bidirectional_dijkstra_old(0, 1)
1.61 ms ± 3.18 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
sage: %timeit G._backend.bidirectional_dijkstra(0, 1)
1.2 ms ± 1.67 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
sage: G = graphs.RandomGNP(10000, .0005)
sage: %timeit G._backend.bidirectional_dijkstra_old(0, 1)
35.6 ms ± 170 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
sage: %timeit G._backend.bidirectional_dijkstra(0, 1)
31.7 ms ± 311 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
sage: G = graphs.RandomGNP(10000, .001)
sage: %timeit G._backend.bidirectional_dijkstra_old(0, 1)
61.8 ms ± 137 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
sage: %timeit G._backend.bidirectional_dijkstra(0, 1)
52.5 ms ± 615 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
sage: G = graphs.RandomGNP(10000, .005)
sage: %timeit G._backend.bidirectional_dijkstra_old(0, 1)
119 ms ± 542 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
sage: %timeit G._backend.bidirectional_dijkstra(0, 1)
103 ms ± 1 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

[github-actions]

Documentation preview for this PR (built with commit 13c7a89; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

[gmou3]

It does seem to be faster indeed. Is there a theoretical guarantee for this?

Can you remove the _old method so we can review the code changes more easily?

[dcoudert]

With a priority queue, insertion is done in $O(\log{n})$-time and extraction of the top item (a max) in $O(\log{n})$-time. Consequently, the overall time complexity of the algorithm was in $O((n + m)\log{n})$.

With a pairing heap, insertion is done in $O(1)$-time and extraction of the top item (a min) in $O(\log{n})$-time. Consequently, the overall time complexity of the algorithm is in $O(m + n\log{n})$. This is the correct way to implement the Dijkstra algorithm.