Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LIU-413: Add Past Sessions to DataIslandManager Graph UI #292

Merged
merged 5 commits into from
Nov 5, 2024
Merged

Conversation

myxie
Copy link
Collaborator

@myxie myxie commented Oct 28, 2024

Note: This should not be merged until after #291 has been merged; make sure the branch this is targeting is changed to master when doing so.

JIRA Ticket

LIU-413

Type

  • Feature (addition)
  • Bug fix
  • Refactor (change)
  • Documentation

Problem/Issue

To progress towards being able to see past sessions and re-run them based on the Graph Configuration, we want to be able to provide an interface where users can see previous sessions that have run.

Solution

This PR aims to partially address this by updating the existing DataIslandManager web interface and REST APi, and adding the PastSessionManager.

This is an introduction of basic functionality to start off the work to managing past sessions. There is future work planned to introduce a sessions database, local/remote GitHub integration etc., which will expand on the PastSessionManager.

The work introduced in this PR:

  • Defines a 'valid' Past Session as one that has a *.graph file associated with it.
  • Use the PastSessionManager to searches the existing DALiuGE workspace directory for session folders, checks to make sure that each potential session folder has a *.graph file in it
  • Returns this via the CompositeManager REST API
  • Displays this in the DataIslandManager index page.
Example
image

Checklist

  • Unittests added
  • [ ] Documentation added
    • Reason for not adding documentation (remove this line if added)

Summary by Sourcery

Add functionality to manage and display past sessions in the web interface and enhance session management by supporting the dumping of physical graphs.

New Features:

  • Introduce functionality to load and display past sessions in the web interface, allowing users to view historical session data.

Enhancements:

  • Add support for dumping physical graphs to the workspace directory, enhancing session management and reproducibility.

…DIM.

- Create the new class
- Add basic table entries to see in the DIM.

Currently not possible to do anything other than see the past sessions.

This comment was marked as duplicate.

@myxie myxie changed the base branch from master to LIU-412 October 29, 2024 06:41
@coveralls
Copy link

coveralls commented Oct 31, 2024

Coverage Status

coverage: 79.734% (+0.06%) from 79.67%
when pulling bb69d85 on LIU-413
into 7163226 on master.

@myxie myxie marked this pull request as ready for review October 31, 2024 07:29
Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @myxie - I've reviewed your changes - here's some feedback:

Overall Comments:

  • Consider adding more unit tests for the PastSessionManager class to verify edge cases like invalid session directories and various file patterns. This will help ensure reliable behavior as the functionality expands.
Here's what I looked at during the review
  • 🟡 General issues: 2 issues found
  • 🟢 Security: all looks good
  • 🟡 Testing: 1 issue found
  • 🟢 Complexity: all looks good
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@@ -298,24 +298,95 @@
});
}

function fillDmTable(sessions, tbodyEl, sessionLink, DimSessionLink, cancelBtnSessionId, deleteBtnSessionId, hashCode) {

function loadPastSessions(serverUrl, tbodyEl, refreshBtn, selectedNode, delay) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: loadPastSessions duplicates significant code from loadSessions

Consider extracting common setup code into a shared helper function to reduce duplication

function loadSessionsBase(serverUrl, tbodyEl, refreshBtn, selectedNode, delay, isPast = false) {
    // Common setup code goes here
}

function loadPastSessions(serverUrl, tbodyEl, refreshBtn, selectedNode, delay) {
    return loadSessionsBase(serverUrl, tbodyEl, refreshBtn, selectedNode, delay, true);
}

function loadSessions(serverUrl, tbodyEl, refreshBtn, selectedNode, delay) {
    return loadSessionsBase(serverUrl, tbodyEl, refreshBtn, selectedNode, delay, false);
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a valid comment, but I think there is more risk in the refactor than duplicating the code, especially given our interest in moving way from a DALiuGE-based web interface.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your feedback, we will generate more comments like this in the future according to the following instructions:

- Include comments that identify code duplication and suggest refactoring to improve maintainability.
- Suggest extracting common code into helper functions when appropriate to reduce redundancy.
- Provide clear examples or code snippets to illustrate the suggested changes.

daliuge-engine/test/manager/test_dim.py Outdated Show resolved Hide resolved
Comment on lines 54 to 59
past_sessions = [
path for path in self._work_dir.iterdir()
if (path.is_dir()
and path.name not in excluded_sessions
and self._is_session_dir(path))
]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (code-quality): Inline variable that is immediately returned (inline-immediately-returned-variable)

list of past sessions.
"""

return any([expected_ext in f.name for f in session.iterdir()])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code-quality): Replace unneeded comprehension with generator (comprehension-to-generator)

Suggested change
return any([expected_ext in f.name for f in session.iterdir()])
return any(expected_ext in f.name for f in session.iterdir())

Comment on lines 558 to 563
pastSessions = []
for pastSession in self.dm.getPastSessionIds():
pastSessions.append(
{"sessionId": pastSession}
)
return pastSessions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (code-quality): We've found these issues:

Suggested change
pastSessions = []
for pastSession in self.dm.getPastSessionIds():
pastSessions.append(
{"sessionId": pastSession}
)
return pastSessions
return [
{"sessionId": pastSession}
for pastSession in self.dm.getPastSessionIds()
]

@myxie myxie changed the title (WIP) LIU-413 LIU-413 Oct 31, 2024
@myxie myxie changed the title LIU-413 LIU-413: Add Past Sessions to DataIslandManager Graph UI Oct 31, 2024
Copy link
Contributor

@awicenec awicenec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great start for the functionality we need to have in the future.

Copy link
Contributor

@awicenec awicenec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great start for the functionality we need to have in the future.

Base automatically changed from LIU-412 to master November 4, 2024 08:43
@myxie myxie merged commit 5195ed4 into master Nov 5, 2024
19 checks passed
@myxie myxie deleted the LIU-413 branch November 5, 2024 07:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants