You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As of now it seems that the pre-processing steps related to the acquisition of the map URLs from the replay files is not performant.
Describe the solution you'd like
I guess that the mutexed sync.Map use within the function below is causing this issue.
It is adding each of the processed files right after it is read.
Instead of that I'd propose to add the entries to the mutexed maps per chunk or all of the maps from each of the chunks should be returned to the main thread and synchronized there.
...for_, replayFullFilepath:=rangechannelContents.ChunkOfFiles {
// Do not use the sync map here per file:processFileExtractMap(
progressBar,
replayFullFilepath,
urls,
processedReplaysSyncMap,
)
}
...funccreateMapExtractingGoroutines(
channelchanReplayMapProcessingChannelContents,
progressBar*progressbar.ProgressBar,
urls*sync.Map,
processedReplaysSyncMap*sync.Map,
wg*sync.WaitGroup,
)
...urls.Store(mapURL, mapHashAndExtension)
processedReplaysSyncMap.Store(
replayFilename,
persistent_data.FileInformationToCheck{
LastModified: fileInfo.ModTime().Unix(),
Size: fileInfo.Size(),
},
)
Additional context
The text was updated successfully, but these errors were encountered:
Is your proposal related to a problem?
As of now it seems that the pre-processing steps related to the acquisition of the map URLs from the replay files is not performant.
Describe the solution you'd like
I guess that the mutexed
sync.Map
use within the function below is causing this issue.It is adding each of the processed files right after it is read.
Instead of that I'd propose to add the entries to the mutexed maps per chunk or all of the maps from each of the chunks should be returned to the main thread and synchronized there.
Additional context
The text was updated successfully, but these errors were encountered: