Run typing on orm/{comments,computers}.py and orm/implementation/storage_backend.py #6704

danielhollas · 2025-01-14T15:03:18Z

I am not sure if the typing changes here make sense, would be good to get reviewed by someone familiar with the orm implementation.

~~Depends on #6703~~

codecov · 2025-01-14T15:20:48Z

Codecov Report

Attention: Patch coverage is 54.54545% with 5 lines in your changes missing coverage. Please review.

Project coverage is 78.00%. Comparing base (b356950) to head (042e513).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
src/aiida/orm/computers.py	50.00%	3 Missing ⚠️
src/aiida/orm/comments.py	33.34%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #6704   +/-   ##
=======================================
  Coverage   78.00%   78.00%           
=======================================
  Files         563      563           
  Lines       41766    41766           
=======================================
  Hits        32574    32574           
  Misses       9192     9192

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

danielhollas · 2025-01-14T16:15:58Z

src/aiida/orm/computers.py

@@ -54,7 +54,7 @@ def get_or_create(self, label: Optional[str] = None, **kwargs) -> Tuple[bool, 'C

    def list_labels(self) -> List[str]:
        """Return a list with all the labels of the computers in the DB."""
-        return self._backend.computers.list_names()
+        return self._backend.computers.list_names()  # type: ignore[attr-defined]


I am not sure why mypy complains here

danielhollas · 2025-01-14T16:17:30Z

src/aiida/orm/computers.py

@@ -224,7 +224,7 @@ def _mpirun_command_validator(self, mpirun_cmd: Union[List[str], Tuple[str, ...]
        """Validates the mpirun_command variable. MUST be called after properly
        checking for a valid scheduler.
        """
-        if not isinstance(mpirun_cmd, (tuple, list)) or not all(isinstance(i, str) for i in mpirun_cmd):
+        if not isinstance(mpirun_cmd, (tuple, list)) or not all(isinstance(i, str) for i in mpirun_cmd):  # type: ignore[redundant-expr]


Mypy complains that the isinstance check should be always true since the mpirun_cmd is typed at list or tuple, but we obviously want to preserve the runtime check since we don't control all the callers.

Or I think the better change would be we move the mpirun_cmd runtime check when set up the computer? But it seems out the scope of this PR.

danielhollas · 2025-01-14T16:19:01Z

src/aiida/orm/implementation/storage_backend.py

@@ -456,7 +456,7 @@ def get_orm_entities(self, detailed: bool = False) -> dict:
        """
        from aiida.orm import Comment, Computer, Group, Log, Node, QueryBuilder, User

-        data = {}
+        data: dict[str, Any] = {}


Without this change I get these errors:

src/aiida/orm/implementation/storage_backend.py:465: error: Incompatible types in assignment (expression has type "list[Any]", target has type "int") [assignment] src/aiida/orm/implementation/storage_backend.py:470: error: Incompatible types in assignment (expression has type "list[Any]", target has type "int") [assignment] src/aiida/orm/implementation/storage_backend.py:478: error: Incompatible types in assignment (expression has type "list[Any]", target has type "int") [assignment] src/aiida/orm/implementation/storage_backend.py:486: error: Incompatible types in assignment (expression has type "list[Any]", target has type "int") [assignment] src/aiida/orm/implementation/storage_backend.py:491: error: Incompatible types in assignment (expression has type "list[Any]", target has type "int") [assignment]

I don't know if there is a better solution.

I'd do the same.

src/aiida/transports/transport.py

danielhollas · 2025-01-14T16:30:55Z

src/aiida/orm/comments.py

@@ -18,7 +18,7 @@

 if TYPE_CHECKING:
    from aiida.orm import Node, User
-    from aiida.orm.implementation import StorageBackend
+    from aiida.orm.implementation import BackendComment, BackendNode, StorageBackend  # noqa: F401


If I don't import BackendComment I get the following error:

src/aiida/orm/comments.py:63: error: Name "BackendComment" is not defined [name-defined]

From this code:

aiida-core/src/aiida/orm/comments.py

Line 63 in 199a027

class Comment(entities.Entity['BackendComment', CommentCollection]):

The noqa is needed because ruff complains about an unused import. I think this is a bug in ruff, I'll file an issue.

In seems a bit strange the type in the square of Entity is mixed, did you try entities.Entity['BackendComment', 'CommentCollection']? I remember my basedpyright gave something that can not use string and class for typing at same time. But doesn't matter, let's use noqa for the moment.

Just look at the type annotation, I am wondering if it can be just have a generic type to hold both backendComment and CommentCollection?

But is it even technically a type annotation? I don't know how the entities.Entity is even supposed to work, since this is somehow specifying the parent class for Comment.

I don't think I'd want to touch this in this PR in any case tbh.

Fair, this typing annotation is quite confuse. But let's keep as it is. We can come back and check all noqa. I'd say, the complex typing usually come from improper design, but it requires way more effort to touch some core parts of storage.

unkcpz · 2025-01-14T16:42:52Z

would be good to get reviewed by someone familiar with the orm implementation.

Then my review will not count ;p

Just out of curious, the goal of the PR is to add more mypy checking to code base (I am totally in it, thanks! I was thinking maybe we can even have it as a project for this year's GSoC to find someone can working on it ) or you are targeting to bring some new features that may benefit from adding type?

danielhollas · 2025-01-14T16:45:23Z

Then my review will not count ;p

Haha, fair. I am mostly confused about this code:

class Comment(entities.Entity['BackendComment', CommentCollection]):

Mypy complains about the missing BackendComment type, but when I import it, I get an error from ruff that the import is unused. I think it might be a limitation of ruff so I just added a noqa comment.

danielhollas · 2025-01-14T16:47:31Z

Just out of curious, the goal of the PR is to add more mypy checking to code base (I am totally in it, thanks! I was thinking maybe we can even have it as a project for this year's GSoC to find someone can working on it ) or you are targeting to bring some new features that may benefit from adding type?

Just adding type checking. These files just seemed like low-hanging fruit, they were already mostly typed!

What I really don't like is that we have a list of ignored types in pre-commit config file. It's very easy to miss, and to mess with types in files that are not even checked.

I think it would be better to have type: ignore comments at the top of the ignored files, which makes it super clear that they are not checked.

EDIT: What originally prompted me was the other PR which add checking for the orm/caching file since I was looking at it while working on aiida-test-cache.

unkcpz · 2025-01-14T17:10:12Z

Mypy complains about the missing BackendComment type, but when I import it, I get an error from ruff that the import is unused.

How you import the BackendComment? Since in the typing annotation, you use single-quotes you need to import:

if typing.TYPE_CHECKING:
    from aiida.xxx import BackendComment

unkcpz · 2025-01-14T17:21:42Z

I think it would be better to have type: ignore comments at the top of the ignored files, which makes it super clear that they are not checked.

I agree, but it is not easy to make typing correct in python. No actually compiler will help us (yes, I am comparing it with Rust 🙈 ). But I am also trying to bring typing to engine part of the code base module by module (like this one #6641).

danielhollas · 2025-01-14T17:33:39Z

How you import the BackendComment? Since in the typing annotation, you use single-quotes you need to import:

Yes, that's what I am doing in this PR.

unkcpz

I gave my go on this PR, but it would be nice that someone more familiar with orm can give it a look. I pin @edan-bainglass because your name is in the storage part of team responsibilities.

danielhollas force-pushed the comments-typing branch from cf1cf48 to 8d0fa45 Compare January 14, 2025 16:13

danielhollas marked this pull request as ready for review January 14, 2025 16:13

danielhollas commented Jan 14, 2025

View reviewed changes

src/aiida/transports/transport.py Show resolved Hide resolved

danielhollas commented Jan 14, 2025

View reviewed changes

danielhollas requested a review from unkcpz January 14, 2025 16:47

danielhollas mentioned this pull request Jan 14, 2025

Add typing to aiida/common/{utils,extendeddicts}.py #6706

Draft

danielhollas force-pushed the comments-typing branch from 8d0fa45 to 9bbfb06 Compare January 15, 2025 17:13

danielhollas added 2 commits January 16, 2025 12:35

Run mypy on orm/{comments,computer}.py

f0ce39c

Run typing on orm/implementation/storage_backend.py

042e513

danielhollas force-pushed the comments-typing branch from 9bbfb06 to 042e513 Compare January 16, 2025 12:35

unkcpz reviewed Jan 16, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run typing on orm/{comments,computers}.py and orm/implementation/storage_backend.py #6704

Run typing on orm/{comments,computers}.py and orm/implementation/storage_backend.py #6704

danielhollas commented Jan 14, 2025 •

edited

Loading

codecov bot commented Jan 14, 2025 •

edited

Loading

danielhollas Jan 14, 2025

danielhollas Jan 14, 2025

unkcpz Jan 16, 2025

danielhollas Jan 14, 2025

unkcpz Jan 16, 2025

danielhollas Jan 14, 2025

danielhollas Jan 14, 2025

unkcpz Jan 16, 2025

unkcpz Jan 16, 2025

danielhollas Jan 16, 2025

unkcpz Jan 16, 2025

unkcpz commented Jan 14, 2025 •

edited

Loading

danielhollas commented Jan 14, 2025

danielhollas commented Jan 14, 2025 •

edited

Loading

unkcpz commented Jan 14, 2025

unkcpz commented Jan 14, 2025 •

edited

Loading

danielhollas commented Jan 14, 2025

unkcpz left a comment

Run typing on orm/{comments,computers}.py and orm/implementation/storage_backend.py #6704

Are you sure you want to change the base?

Run typing on orm/{comments,computers}.py and orm/implementation/storage_backend.py #6704

Conversation

danielhollas commented Jan 14, 2025 • edited Loading

codecov bot commented Jan 14, 2025 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

unkcpz commented Jan 14, 2025 • edited Loading

danielhollas commented Jan 14, 2025

danielhollas commented Jan 14, 2025 • edited Loading

unkcpz commented Jan 14, 2025

unkcpz commented Jan 14, 2025 • edited Loading

danielhollas commented Jan 14, 2025

unkcpz left a comment

Choose a reason for hiding this comment

danielhollas commented Jan 14, 2025 •

edited

Loading

codecov bot commented Jan 14, 2025 •

edited

Loading

unkcpz commented Jan 14, 2025 •

edited

Loading

danielhollas commented Jan 14, 2025 •

edited

Loading

unkcpz commented Jan 14, 2025 •

edited

Loading