Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transcode queuing concurrency is unbounded #144

Open
mojodna opened this issue Oct 18, 2018 · 0 comments
Open

Transcode queuing concurrency is unbounded #144

mojodna opened this issue Oct 18, 2018 · 0 comments
Labels
v1 wontfix v1 bugs that will not be prioritized for fix, in favor of v2

Comments

@mojodna
Copy link
Collaborator

mojodna commented Oct 18, 2018

This means that if enough images (URLs, in my case) are submitted, lots of transcoding requests will be submitted simultaneously, eventually resulting in ECONNRESET and ECONNREFUSED.

The number of concurrent promises should be limited here:

oam-api/routes/uploads.js

Lines 527 to 547 in ed8af6e

const transcoderPromisesAll =
Promise.all([uploadPromise, insertImagesAll]).then((results) => {
const sceneImageIds = results[1];
const transcoderPromises = upload.scenes
.reduce((accum, scene, sceneIndex) => {
const imageIds = sceneImageIds[sceneIndex];
const queuedImages = imageIds.map((imageId, imageIdIndex) => {
const key = [uploadId, sceneIndex, imageId].join('/');
const sourceUrl = scene.urls[imageIdIndex];
return transcoder.queueImage(sourceUrl, key,
`${config.apiEndpoint}/uploads/${uploadId}/${sceneIndex}/${imageId}`);
});
accum.push(...queuedImages);
return accum;
}, []);
return Promise.all(transcoderPromises);
});
return transcoderPromisesAll.then(() => {
return { upload: upload._id };
});

@cgiovando cgiovando added the v1 wontfix v1 bugs that will not be prioritized for fix, in favor of v2 label Apr 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
v1 wontfix v1 bugs that will not be prioritized for fix, in favor of v2
Projects
None yet
Development

No branches or pull requests

2 participants