1287 remote rsd search

[dmijatovic]

Search for software in multiple RSD instances

Closes #1287

Changes proposed in this pull request:

• rsd administrators can specify the instance name to be used as "label" for the software from this instance. New section "RSD info" is created for this purpose. Initially the default "Not defined" value is added to rsd_info table.
• rsd_info table/endpoint is created to store the info about rsd instance. Basic endpoint information and remote_name property is inserted in the table. remote_name prop is used in aggregated_software_overview.
• rsd admin can add remote RSD to be included in software search on software overview page.
• all users can search and filter software in all defined RSD instances from the software overview page.
• the "remote" software opens in the new tab and the "local" software opens in the same tab. The remote software has "opens in new tab" icon at the top right of the card
• if at least one remote RSD is defined, the additional filter is shown on the software overview page.
• global search includes remote software. Under the software name the domain name is shown to indicate "remote" source

How to test:

• make start to build and generate test data
• navigate to software overview page and confirm that no RSD hosts filter is present (no remotes are defined yet).
• login as rsd-admin, navigate to Rsd info page. Change remote_name to your liking. At some other values. You can read this values from the api at: /api/v1/rsd_info
• navigate to Remotes page (Admin section)
• add Helmholtz remote: https://helmholtz.software/
• add RSD Saas remote: https://research-software-directory.org/
• add RSD dev remote: https://research-software.dev/ . RSD dev is updated to this PR version to return RSD remote name. This suggestion is shown in the form of the icon (see last image).
• try to add not existing RSD remote. The validation should raise error and prevent you from adding invalid remote.
• run scrapers manually or wait for scrapers to run. You can stop & restart RSD to see the logs docker compose down & docker compose up
• after scrapers run examine the software overview. It should include software from remote RSD's too.
• you can also filter all software, including remote software. The counts shown in the filters should reflect all software including remote software.
• confirm that additional filter is shown on the software overview page and you can use filter.
• confirm that local software is open in the same tab and remote software in the new tab.
• navigate to highlights page and confirm that additional filter "RSD Hosts" is not shown on the page and that filtering works properly

Example rsd info section

Example add remote RSD

Example remotes overview

Example software overview cards (showing source, remote icon and the additional filter)

Example software overview list (with source)

Example suggested remote RSD name (based on info received from remote)

Example global search includes domain for the remotes

PR Checklist:

• Increase version numbers in docker-compose.yml
• Link to a GitHub issue
• Update documentation
• Tests

[jmaassen]

The fetching of the remote RSD name seems to be a bit buggy?

When I add the dev server URL, the remote RSD name does not show up automatically. Instead I have to click the "remote control icon" first. Why not fetch it and allow the admin to change it later?

After fetching the name, the text box containing it seems messed up? Also, the save icon is still disabled.

When clicking on the name text box, it is cleaned up, but the save is still disabled.

Clicking on the update interval or URL box activates the save button.

[jmaassen]

When selecting by keyword, the selected keyword does not always show up. For example, when harvesting from the RSD SAAS and Helmholtz RSD, selecting "3D" works as expected:

However, if I select "Data Analysis" instead, the filter works as expected, but isn't shown in the sidebar:

All others I've tried seem to work as expected, so I'm not sure what the problem is with this one?

[dmijatovic]

@jmaassen Well spotted! I have fixed both remarks. I did force push so please remote old branch and pull again.

[cmeessen]

When adding research-software.dev as a remote I get this error message:

[dmijatovic]

When adding research-software.dev as a remote I get this error message:

@cmeessen Thanks for notifying us. It seems that rsd-dev api failed for uknown reason. I restarted docker-compose and the rsd-dev is working now.

[ewan-escience]

Nice work, I have some comments on the database setup.

[cmeessen]

Using the global search does not include the results from remote RSDs.

[jmaassen]

When looking at the NWO website, I noticed that they used the following method for indicating that clicking on a tile leads to an external website:

So a similar "exit" icon that we discussed during out meeting last friday. They do highlight this icon by surrounding it using a colored box.

[sonarqubecloud]

Quality Gate passed for 'rsd-database'

Issues
4 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
8.2% Duplication on New Code

See analysis details on SonarQube Cloud

[sonarqubecloud]

Quality Gate passed for 'scrapers'

Issues
5 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[sonarqubecloud]

Quality Gate passed for 'rsd-frontend'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
19.6% Coverage on New Code
2.3% Duplication on New Code

See analysis details on SonarQube Cloud