Enhance pkg_deps data.frame functionalities

[maksymiuks]

Hi all

I'd like to follow up on our discussion at r-lib/remotes#790 and suggest some new functionalities for the pkg_deps. These would greatly help our enterprise use and allow us to migrate from the remotes package.

1. Make pkg_deps display all dependencies regardless of remotes sources. Currently, if a remote source is specified, it overwrites the version available in repos if it exists. We suggest a parameter (or even default behaviour) that allows us to keep both - the version identified in remotes and the one from repos. In our enterprise application, depending on the stage and dependency type, we want to either include or exclude remote sources, so having them both in the data.frame would allow us to implement dynamic filters on our own and build them on top of the pak. The same could be applied to supplied repositories (and, by extension, Additional_repositoires provided we use the source package). If a package is available in multiple repositories, it would be great if the data.frame had a different row for each latest version and repository pair. Again, bringing our needs into context, we need to filter packages available in multiple repositories based on some conditions (like the order of repos, similar to the package search path). Similar to remotes packges request, had we had all these versions in the data.frame, we could start from there and build our installation (filtering) mechanism based on that.
2. Implement something similar to remotes:::update.package_deps(). Bascially a function which would accept a subset of rows from the pkg_deps data frame and install the specified package.

Please, let me know what you think about these suggestions and whether you see them as part of the pak package in the future. :)

[maksymiuks]

Hi, I wanted to ask if there is any work planned in regards to that issue?

[gaborcsardi]

Here is how you can see both versions:

dps <- pkgdepends::new_pkg_deps("local::.", config = list(dependencies = TRUE))
dps$resolve()
res <- dps$get_resolution()
res[res$package == "sass", ]

This package has Remotes: rstudio/sass, so I get:

# A data frame: 3 × 31
  ref     type  direct directpkg status package version license needscompilation
* <chr>   <chr> <lgl>  <lgl>     <chr>  <chr>   <chr>   <chr>   <lgl>
1 sass    stan… FALSE  FALSE     OK     sass    0.4.9   MIT + … FALSE
2 sass    stan… FALSE  FALSE     OK     sass    0.4.9   MIT + … TRUE
3 rstudi… gith… FALSE  FALSE     OK     sass    0.4.9.… MIT + … TRUE

I guess we could have a function that only does the resolution. We could also have an option to ignore Remotes fields, but maybe what you want is already possible, if you use a custom field, instead of Remotes. E.g. you could use Config/Needs/xyz and then specify dependencies = c("hard", "Config/Needs/xyz"). E.g. for pak itself:

pak::pkg_deps("local::.", dependencies = c("hard", "Config/needs/dependencies"))[,1:4]

# A data frame: 16 × 4
   ref              type     direct directpkg
   <chr>            <chr>    <lgl>  <lgl>
 1 local::.         local    TRUE   TRUE
 2 R6               standard FALSE  FALSE
 3 callr            standard FALSE  FALSE
 4 curl             standard FALSE  FALSE
 5 desc             standard FALSE  FALSE
 6 filelock         standard FALSE  FALSE
 7 jsonlite         standard FALSE  FALSE
 8 lpSolve          standard FALSE  FALSE
 9 pkgbuild         standard FALSE  FALSE
10 processx         standard FALSE  FALSE
11 ps               standard FALSE  FALSE
12 r-hub/pkgsearch  github   FALSE  FALSE
13 r-lib/pkgdepends github   FALSE  FALSE
14 zip              standard FALSE  FALSE
15 r-lib/cli        github   FALSE  FALSE
16 r-lib/pkgcache   github   FALSE  FALSE

At some point we'll have an update function that updates the whole library or a subset of it.

[maksymiuks]

Hi @gaborcsardi

First of all, thank you for your response! I really appreciate your time investment into helping us!

I tried to use the first code snippet you provided, and unfortunately, I cannot reproduce the results. The Remotes packages are still overwriting those available in standard repositories. Perhaps there is some option I should enable? I'm using this dummy DESCRIPTION file

Package: dummy.description
Title: Some package
Version: 2.1.3
Authors@R: c(
      person(
        given = "Szymon",
        family = "Maksymiuk",
        role = "aut"
      )
    )
Description:
    Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod
    tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
    quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.
Depends:
    R (>= 3.2.0)
Imports:
    checked (>= 0.2.3.9004),
    covr (>= 3.5.1.9002)
Remotes:
    dgkf/covr,
    git::https://github.com/genentech/checked.git@main
Encoding: UTF-8
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.3.2
Config/testthat/edition: 3
License: file LICENSE

and by running your code:

# remotes::install_git("https://github.com/r-lib/pkgdepends.git@main")
dps <- pkgdepends::new_pkg_deps("local::.", config = list(dependencies = TRUE))
dps$resolve()
res <- dps$get_resolution()

I'm getting :

> res[res$package == "covr", c("ref", "type", "package", "version")]
# A data frame: 1 × 4
  ref       type   package version   
* <chr>     <chr>  <chr>   <chr>     
1 dgkf/covr github covr    3.6.4.9003

> res[res$package == "checked", c("ref", "type", "package", "version")]
# A data frame: 1 × 4
  ref                                                type  package version
* <chr>                                              <chr> <chr>   <chr>  
1 git::https://github.com/genentech/checked.git@main git   checked 0.2.5  

Contrary to your results with sass.

Also, could you please elaborate on that second suggestion? Because in the cited results, I don't see standard equivalents of github packages.

[gaborcsardi]

I cannot reproduce the results.

Yeah, my mistake.

Also, could you please elaborate on that second suggestion? Because in the cited results, I don't see standard equivalents of github packages.

Yes, the point is that if you use the dependencies argument with Config/Needs/..., then you'll get the GH package, if you don't then you get the CRAN package. So this is a way to choose between the released and the dev packages.

[maksymiuks]

Yes, the point is that if you use the dependencies argument with Config/Needs/..., then you'll get the GH package, if you don't then you get the CRAN package. So this is a way to choose between the released and the dev packages.

Oh, I see; now I understand, thank you. However, that does not necessarily solve the case as we often analyze packages we did not develop and are only provided with source code. These packages almost exclusively use the Remotes field and not the Config/Needs/..., meaning it would be of no use to us, unfortunately.

[gaborcsardi]

At some point we'll have a general override mechanism, which will let you do that. Until then you'll need to update the DESCRIPTION, I am afraid.