Alternative solutions for searching that fit existing system requirements

[NateWr]

The search functionality in OJS and OMP has not had much attention over the years. It may be worth exploring what third-party solutions we can integrate which would provide a better search experience, but which wouldn't have steep system requirements [like Lucene/Solr].

Also, the smarty functions for pagination are only used on search pages. If these pages are brought in line with the changes in #1922, those functions could be removed.

[asmecher]

@lilients, just FYI, as I believe you're working on modernizing the Lucene plugin? We have no concrete goals here, just a sense that the built-in search engine could be improved, and that there may be good alternatives that we can integrate rather than maintaining our own.

[asmecher]

(See also #321 -- searching should be extended to issue galleys etc.)

[Glottotopia]

Speaking only for OMP, the amount of metadata in a catalog is very small, and searching/arranging/filtering that data could completely be done client side. That excludes the full texts of course. See here for an example with actual (static) data from an OMP installation: http://langsci.github.io/catalog

[asmecher]

Speaking only for OMP, ...

There's a related thread about this in the forum: https://forum.pkp.sfu.ca/t/omp-filtering-for-book-catalog/39618

[NateWr]

An alternative to our own searching toolset that wouldn't require additional server dependencies: https://github.com/teamtnt/tntsearch

[asmecher]

^ Interesting! It does require SQLite, which it uses for its own storage, so I'd want to verify that it has very wide uptake in general -- but I suspect it does. It's a shame it can't just use the same PDO toolset it uses for a data source as the indexing storage as well.

[jonasraoni]

If SQLite is still a requirement for this library, it will need an extra note for sysadmins to be cautious when sharing the index folder (e.g. "cloud" environment) with other nodes, given the underlying OS might not be that great with file locks 👀

Another issue related to the locks, is the lack of concurrency, so larger journals will probably face issues if the indexing process is too slow... So we'll also need to change how the indexing is working, and offload it to a single-running task.

[NateWr]

Ah, that's a shame. I saw a couple of issues (1, 2) talking about alternate db implementations. I wonder if they'd accept a PR going forward...

[jonasraoni]

According to the comments, the issue 1 should be done, as the maintainer said it would be part of the v2 (they've just released the v3).

[Glottotopia]

For reference, here is a client side implementation of metadata search in OMP
https://langsci-press.org/catalogSearch

Here is the code https://github.com/langsci/catalogSearchPage