Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dependencies to rpm #4084

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

alok1304
Copy link

@alok1304 alok1304 commented Jan 10, 2025

Added dependencies to rpm.

Reference: #649

Fixes #649

Tasks

  • Reviewed contribution guidelines
  • PR is descriptively titled 📑 and links the original issue above 🔗
  • Tests pass -- look for a green checkbox ✔️ a few minutes after opening your PR
    Run tests locally to check for errors.
  • Commits are in uniquely-named feature branch and has no merge conflicts 📁
  • Updated documentation pages (if applicable)
  • Updated CHANGELOG.rst (if applicable)

Signed-off-by: Alok Kumar [email protected]

@alok1304 alok1304 force-pushed the 649-add-dependencies-to-rpm branch from f9c5a17 to 7b69733 Compare January 10, 2025 11:28
@alok1304
Copy link
Author

@pombredanne please review this PR, also all test cases have been successfully executed and passed.

@alok1304 alok1304 force-pushed the 649-add-dependencies-to-rpm branch from 1c0e2b1 to 9bb48db Compare January 10, 2025 12:33
Added dependencies to rpm.

Reference: aboutcode-org#649
Signed-off-by: Alok Kumar <[email protected]>

Signed-off-by: Alok Kumar <[email protected]>
@alok1304 alok1304 force-pushed the 649-add-dependencies-to-rpm branch from 9bb48db to e16372a Compare January 10, 2025 14:26
Copy link
Member

@pombredanne pombredanne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this effort. Please see comments for feedback:

  1. This code cannot and does not work as it is: .rpm archives and installed RPM dbs are not the same.
  2. There is a lot of code duplication
  3. A package cannot be its own dependency
  4. Please start by crafting proper tests first so you can set expectations with carefully reviewed cases.

src/packagedcode/rpm.py Show resolved Hide resolved
src/packagedcode/rpm.py Show resolved Hide resolved
src/packagedcode/rpm.py Show resolved Hide resolved
src/packagedcode/rpm.py Show resolved Hide resolved
src/packagedcode/rpm.py Show resolved Hide resolved
src/packagedcode/rpm.py Show resolved Hide resolved
src/packagedcode/rpm.py Show resolved Hide resolved
)

# Prepare the dependent package model
dependencies.append(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since when there is a single dependency?

Copy link
Author

@alok1304 alok1304 Jan 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pombredanne Can you suggest how I proceed to find multiple dependencies, I tried by Adding RPMTAG_REQUIRES and RPMTAG_REQUIREVERSION in RPMtags in pyrpm.py but in RPMTAG_REQUIRES I got this like eg: ['/bin/sh', '/bin/sh', '/bin/sh', '/bin/sh', '/bin/sh', 'rpmlib(PayloadFilesHavePrefix)', 'rpmlib(CompressedFileNames)', 'rpmlib(PayloadIsBzip2)']
corresponding I got require_version=[None, None, None, None, None, '4.0-1', '3.0.4-1', '3.0.5-1']
source: http://ftp.rpm.org/max-rpm/ch-queryformat-tags.html (For rpm tages)

can you tell how I proceed to find out their package name and their version, in rmp_requires, these represent capabilities or libraries, not actual packages

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since when there is a single dependency?

I do for single dependency only.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do for single dependency only.

This cannot work. We need all of them.

  1. You can use "get_rpm_tags" only for a .rpm file, but then you do not need to call it because this is already called
  2. for installed RPM databases, you can use exclusively the data in parse_rpm_xmlish https://github.com/aboutcode-org/scancode-toolkit/pull/4084/files#diff-17fc898047e57c63a40599213e50fb03f8d853b60ea7790133ace1c6ab2b9709R170 and if it does not return what you need, parse_rpm_xmlish needs to be updated in https://github.com/aboutcode-org/scancode-toolkit/blob/develop/src/packagedcode/rpm_installed.py. Look at require*, provide* and conflict* for instance in https://github.com/aboutcode-org/scancode-toolkit/edit/develop/tests/packagedcode/data/rpm_installed/distro-xmlish/rhel-rpms.xmlish

Copy link
Member

@pombredanne pombredanne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alok1304 you cannot use get_rpm_tags for installed packages. And its dope not make sense to collect "self" as a dependency and to further consider that there is only one dependency which is the package itself.
See also my comments inline

src/packagedcode/rpm.py Show resolved Hide resolved
src/packagedcode/rpm.py Show resolved Hide resolved
)

# Prepare the dependent package model
dependencies.append(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do for single dependency only.

This cannot work. We need all of them.

  1. You can use "get_rpm_tags" only for a .rpm file, but then you do not need to call it because this is already called
  2. for installed RPM databases, you can use exclusively the data in parse_rpm_xmlish https://github.com/aboutcode-org/scancode-toolkit/pull/4084/files#diff-17fc898047e57c63a40599213e50fb03f8d853b60ea7790133ace1c6ab2b9709R170 and if it does not return what you need, parse_rpm_xmlish needs to be updated in https://github.com/aboutcode-org/scancode-toolkit/blob/develop/src/packagedcode/rpm_installed.py. Look at require*, provide* and conflict* for instance in https://github.com/aboutcode-org/scancode-toolkit/edit/develop/tests/packagedcode/data/rpm_installed/distro-xmlish/rhel-rpms.xmlish

@alok1304
Copy link
Author

alok1304 commented Jan 13, 2025

@alok1304 you cannot use get_rpm_tags for installed packages. And its dope not make sense to collect "self" as a dependency and to further consider that there is only one dependency which is the package itself. See also my comments inline

Yes, I got it. I will sure see this and get some idea of this.
thanks : )

@alok1304
Copy link
Author

alok1304 commented Jan 13, 2025

@pombredanne I checked .rpm(rpm_archive) for dependencies, there are four things in dependencies.

  1. RPMTAG_PROVIDES, RPMTAG_PROVIDEVERSION
  2. RPMTAG_REQUIRES, RPMTAG_REQUIREVERSION
  3. RPMTAG_CONFLICTS, RPMTAG_CONFLICTVERSION
  4. RPMTAG_OBSOLETES, RPMTAG_OBSOLETEVERSION

what I do, I add this in RPMTAGS in pyrmp.py
and getting its name and version
I scan for tests/packagedcode/data/rpm/header/zziplib-0.11.15-3sf.i586.rpm
I got this output in dependencies

"dependencies": [
    {
      "purl": "pkg:rpm/libzzip-0.so.11",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/libzzip-0.so.11?uuid=f9d951c0-b971-48cb-8c63-1fbe62f7db35",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/zziplib@2:0.11.15-3sf",
      "extracted_requirement": "2:0.11.15-3sf",
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": true,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=cb1b8ea8-96d9-485b-8044-4ae59748fab6",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/libz",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/libz?uuid=66e6c90d-27e8-4a1b-b56b-9e4515eb5043",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/sbin/ldconfig",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/sbin/ldconfig?uuid=0d7c0b46-4104-48ae-97bc-97eeaa6f5d14",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/sbin/ldconfig",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/sbin/ldconfig?uuid=9d824e07-f01d-4d95-ad37-6f4b19566d9e",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/rpmlib%28PayloadFilesHavePrefix%[email protected]",
      "extracted_requirement": "4.0-1",
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": true,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/rpmlib%28PayloadFilesHavePrefix%[email protected]?uuid=7b5e02b0-da80-4e56-8e7b-8fafae47b478",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/rpmlib%28CompressedFileNames%[email protected]",
      "extracted_requirement": "3.0.4-1",
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": true,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/rpmlib%28CompressedFileNames%[email protected]?uuid=a3667b77-e392-44b1-aa7f-d955ce604640",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/ld-linux.so.2",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/ld-linux.so.2?uuid=e2df7012-671c-43e7-8ae6-b608357006f4",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/libc.so.6",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/libc.so.6?uuid=a2d6d367-5f5f-4dc4-9780-6b48ed78e186",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/libz.so.1",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/libz.so.1?uuid=c3c3c20b-782c-424e-8ddd-ab79e5221b7b",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    },
    {
      "purl": "pkg:rpm/libc.so.6%28GLIBC_2.0%29",
      "extracted_requirement": null,
      "scope": "dependencies",
      "is_runtime": true,
      "is_optional": false,
      "is_pinned": false,
      "is_direct": true,
      "resolved_package": {},
      "extra_data": {},
      "dependency_uid": "pkg:rpm/libc.so.6%28GLIBC_2.0%29?uuid=e17e3466-25a5-4044-a8c2-317b8a75cca9",
      "for_package_uid": "pkg:rpm/zziplib@2:0.11.15-3sf?uuid=765830b0-cce8-4560-a409-d016e1bce00d",
      "datafile_path": "zziplib-0.11.15-3sf.i586.rpm",
      "datasource_id": "rpm_archive"
    }
  ]

is this right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add dependencies to RPMs
2 participants