-
-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Streamed Loading #56
Draft
Shinmera
wants to merge
3
commits into
master
Choose a base branch
from
loader-ng
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Streamed Loading #56
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Shinmera
force-pushed
the
master
branch
6 times, most recently
from
September 12, 2023 07:44
1be38da
to
afb0476
Compare
Shinmera
force-pushed
the
master
branch
3 times, most recently
from
October 12, 2023 09:19
4f867f5
to
3fbd53d
Compare
Shinmera
force-pushed
the
master
branch
3 times, most recently
from
November 13, 2023 12:12
86f03fa
to
391bc2f
Compare
Shinmera
force-pushed
the
master
branch
2 times, most recently
from
November 21, 2023 10:44
abcb908
to
b102af3
Compare
Shinmera
force-pushed
the
master
branch
9 times, most recently
from
December 5, 2023 13:58
816cb1a
to
4fe60d3
Compare
Shinmera
force-pushed
the
master
branch
2 times, most recently
from
December 11, 2023 10:31
f2e5fe6
to
1b6faad
Compare
Shinmera
force-pushed
the
master
branch
3 times, most recently
from
April 5, 2024 16:06
fc79f56
to
cf6f722
Compare
Shinmera
force-pushed
the
master
branch
4 times, most recently
from
August 19, 2024 20:57
91a40bf
to
c0fb2d2
Compare
Shinmera
force-pushed
the
master
branch
3 times, most recently
from
August 24, 2024 12:22
e6d8bfc
to
a8d853f
Compare
Shinmera
force-pushed
the
master
branch
2 times, most recently
from
November 8, 2024 22:29
339197b
to
1264753
Compare
Shinmera
force-pushed
the
master
branch
3 times, most recently
from
December 14, 2024 19:46
3a40ad5
to
3fe007a
Compare
Shinmera
force-pushed
the
master
branch
3 times, most recently
from
January 4, 2025 11:02
9e0be4b
to
c6bae42
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request implements a revision of the existing resources, assets, and loader system to add support for delayed or streamed loading.
Currently loading has to happen synchronously: a
staging-area
is populated with desired assets and resources. The area is sorted by dependencies between the resources and assets, and incrementally repopulated as new dependencies and resource artefacts emerge from loading an asset's data. This is fine and convenient, but it does not allow for data to be loaded in the background, forcing either synchronous loads during gameplay introducing lag stutters, or forcing long load screens to preload as much data as possible.This PR aims to separate loading into two phases, if desired:
allocated
minimally, only loading as little data into them as absolutely necessary. Other resources like framebuffers, shaders, etc. are loaded as before.In order to achieve this the following needs to be done:
deferrable-resource
is introduced. Resources of this type can be loaded minimally.allocate
when called on adeferrable-resource
will perform minimal allocation, rather than full allocation.load
when called on adeferrable-resource
will perform full allocation with data loading. The resource's data field must be properly populated before this function can be called. Alternatively, the standardupdate-buffer-data
orresize-buffer
may be used as well.deferrable-asset
is introduced. Assets of this type can be loaded minimally.allocate
when called on adeferrable-asset
will perform minimal loading, only producing as much metadata as necessary for standard operation, and producingdeferrable-resource
instances that are "close enough" to the real resource type once loaded fully.load
when called on adeferrable-asset
will perform full allocation with data loading. The data of resources it generates will be populated fully so that they may beload
ed as well.deferred-staging-area
is introduced. Whencommit
is called on such an area, the loader will perform a deferred load, and keep track of the assets and resources that have not been fully loaded yet.streamed-loader
is introduced. This loader keeps a background thread and derived context for use in background streamed loading. Whencommit
is called on adeferred-staging-area
with the keyword argument:stream T
, then it will automatically start loading in the remaining assets and resources in the background.