Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: Eyevinn/shaka-packager-s3
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v0.2.2
Choose a base ref
...
head repository: Eyevinn/shaka-packager-s3
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref

Commits on Jul 2, 2024

  1. Verified

    This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    04fa504 View commit details
  2. 0.3.0

    birme committed Jul 2, 2024
    Copy the full SHA
    07e36b2 View commit details

Commits on Aug 30, 2024

  1. feat: add support for single segment file per stream

    * Support for packaging with a single segment file per stream and
    bytrange addressing
    * Support for packaging only dash or only hls
    * Support for specifying target segment duration
    
    Signed-off-by: Gustav Grusell <gustav.grusell@eyevinn.se>
    grusell committed Aug 30, 2024
    Copy the full SHA
    78d243f View commit details
  2. chore: npm audit fix

    Signed-off-by: Gustav Grusell <gustav.grusell@eyevinn.se>
    grusell committed Aug 30, 2024
    Copy the full SHA
    8b23231 View commit details
  3. refactor: make code a little bit nicer

    Signed-off-by: Gustav Grusell <gustav.grusell@eyevinn.se>
    grusell committed Aug 30, 2024
    Copy the full SHA
    4b8c115 View commit details

Commits on Sep 2, 2024

  1. chore: fix prettier warnings in README

    Signed-off-by: Gustav Grusell <gustav.grusell@eyevinn.se>
    grusell committed Sep 2, 2024
    Copy the full SHA
    06e6c1d View commit details
  2. docs: fix README

    Signed-off-by: Gustav Grusell <gustav.grusell@eyevinn.se>
    grusell committed Sep 2, 2024
    Copy the full SHA
    77a1ba2 View commit details
  3. Merge pull request #4 from Eyevinn/single-file-segment

    feat: add support for single segment file per stream
    grusell authored Sep 2, 2024
    Copy the full SHA
    6bf462a View commit details
  4. 0.4.0

    grusell committed Sep 2, 2024
    Copy the full SHA
    f6bfbfe View commit details

Commits on Sep 10, 2024

  1. feat: add support for setting name of manifest files through parameters

    Signed-off-by: Gustav Grusell <gustav.grusell@eyevinn.se>
    grusell committed Sep 10, 2024
    Copy the full SHA
    f4da9ab View commit details
  2. 0.5.0

    grusell committed Sep 10, 2024
    Copy the full SHA
    c0aed62 View commit details
  3. fix: export PackageFormatOptions

    Signed-off-by: Gustav Grusell <gustav.grusell@eyevinn.se>
    grusell committed Sep 10, 2024
    Copy the full SHA
    43ed59f View commit details
  4. 0.5.1

    grusell committed Sep 10, 2024
    Copy the full SHA
    61ee675 View commit details

Commits on Nov 9, 2024

  1. fix: cleanup downloaded source files before syncing to S3 (#7)

    * fix: do not upload source files to S3
    
    * fix: cleanup jobdir after upload
    
    * fix: force removal of staging dir
    birme authored Nov 9, 2024
    Copy the full SHA
    a985b79 View commit details
  2. 0.5.2

    birme committed Nov 9, 2024
    Copy the full SHA
    d718155 View commit details

Commits on Nov 17, 2024

  1. Copy the full SHA
    56139d4 View commit details
  2. 0.6.0

    birme committed Nov 17, 2024
    Copy the full SHA
    eac7d25 View commit details

Commits on Nov 23, 2024

  1. Copy the full SHA
    ae91f43 View commit details
  2. 0.6.1

    birme committed Nov 23, 2024
    Copy the full SHA
    654b9e8 View commit details

Commits on Dec 2, 2024

  1. fix: crash on error (#11)

    birme authored Dec 2, 2024
    Copy the full SHA
    88c743e View commit details
  2. 0.6.2

    birme committed Dec 2, 2024
    Copy the full SHA
    6909dd3 View commit details
Showing with 455 additions and 88 deletions.
  1. +22 −8 README.md
  2. +8 −8 package-lock.json
  3. +3 −2 package.json
  4. +52 −15 src/cli.ts
  5. +6 −1 src/index.ts
  6. +123 −11 src/packager.test.ts
  7. +217 −43 src/packager.ts
  8. +13 −0 src/util.test.ts
  9. +11 −0 src/util.ts
30 changes: 22 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -92,18 +92,32 @@ Run script locally

```
% node dist/cli.js -h
Usage: cli [options] <source> <dest>
Usage: cli [options]
Run shaka-packager with source on S3 or locally, and output to S3 or local
Examples:
$ shaka-packager-s3 -i a:1=audio.mp4 -i v:1=video.mp4 -s s3://source-bucket/folder -d s3://output-bucket/folder
$ shaka-packager-s3 -i a:1=audio.mp4 -i v:1=video.mp4 -s /path/to/source/folder -d /path/to/output/folder
$ shaka-packager-s3 -i a:2=audio.mp4 -i v:1=video.mp4 -s /path/to/source/folder -d /path/to/output/folder --segment-single-file --segment-single-file-name 'Container$KEY$.mp4' --segment-duration 3.84
Run shaka-packager with source on S3 and output to S3
Arguments:
source Source bucket URL (supported protocols: s3
dest Destination bucket URL (supported protocols: s3)
Options:
-i, --input [inputOptions...] Input options on the format: [a|v]:<key>=filename
--staging-dir <stagingDir> Staging directory (default: /tmp/data)
-h, --help display help for command
-s, --source-folder [sourceFolder] Source folder URL, ignored if input uses absolute path (supported protocols: s3, local file)
-i, --input [inputOptions...] Input options on the format: [a|v]:<key>=filename
--staging-dir [stagingDir] Staging directory (default: /tmp/data)
--shaka-executable [shakaExecutable] Path to shaka-packager executable, defaults to 'packager'. Can also be set with environment variable SHAKA_PACKAGER_EXECUTABLE.
--no-implicit-audio [noImplicitAudio] Do not include audio unless audio input specified
-d, --destination-folder <dest> Destination folder URL (supported protocols: s3, local file). Defaults to CWD.
--endpoint-url [s3EndpointUrl] S3 endpoint URL
--dash-only Package only DASH format
--hls-only Package only HLS format
--segment-single-file Use byte range addressing and a single segment file per stream
--segment-single-file-name [segmentSingleFileName] Template for single segment file name, $KEY$ will be replaced with stream key
--segment-duration [segmentDuration] Segment target duration
-h, --help display help for command
```

## Support
16 changes: 8 additions & 8 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@eyevinn/shaka-packager-s3",
"version": "0.2.2",
"version": "0.6.2",
"description": "Shaka packager with S3",
"main": "dist/index.js",
"bin": {
@@ -12,7 +12,8 @@
"pretty": "prettier --check --ignore-unknown .",
"typecheck": "tsc --noEmit -p tsconfig.json",
"test": "jest --passWithNoTests",
"postversion": "git push && git push --tags"
"postversion": "git push && git push --tags",
"start": "ts-node -T src/cli.ts"
},
"author": "Eyevinn Technology <work@eyevinn.se>",
"license": "MIT",
67 changes: 52 additions & 15 deletions src/cli.ts
Original file line number Diff line number Diff line change
@@ -27,6 +27,7 @@ cli
Examples:
$ shaka-packager-s3 -i a:1=audio.mp4 -i v:1=video.mp4 -s s3://source-bucket/folder -d s3://output-bucket/folder
$ shaka-packager-s3 -i a:1=audio.mp4 -i v:1=video.mp4 -s /path/to/source/folder -d /path/to/output/folder
$ shaka-packager-s3 -i a:2=audio.mp4 -i v:1=video.mp4 -s /path/to/source/folder -d /path/to/output/folder --segment-single-file --segment-single-file-name 'Container$KEY$.mp4' --segment-duration 3.84
`
)
.option(
@@ -53,28 +54,64 @@ cli
'-d, --destination-folder <dest>',
'Destination folder URL (supported protocols: s3, local file). Defaults to CWD.'
)
.option('--endpoint-url [s3EndpointUrl]', 'S3 endpoint URL')
.option('--dash-only', 'Package only DASH format')
.option('--hls-only', 'Package only HLS format')
.option(
'--segment-single-file',
'Use byte range addressing and a single segment file per stream'
)
.option(
'--segment-single-file-name [segmentSingleFileName]',
'Template for single segment file name, must contain $KEY$ which will be replaced with key of corresponding input'
)
.option('--segment-duration [segmentDuration]', 'Segment target duration')
.action(async (options) => {
try {
const inputOptions = parseInputOptions(options.input);
if (inputOptions) {
console.log('inputs', inputOptions);
console.log(
`dest: ${options.destinationFolder}, source: ${options.sourceFolder}`
);
await doPackage({
dest: options.destinationFolder || '.',
source: options.sourceFolder,
inputs: inputOptions,
stagingDir: options.stagingDir,
noImplicitAudio: options.noImplicitAudio,
shakaExecutable:
options.shakaExecutable || process.env.SHAKA_PACKAGER_EXECUTABLE
});
} else {
if (!inputOptions) {
console.error('Need at least one input!\n');
cli.help();
process.exit(1);
}
if (options.hlsOnly && options.dashOnly) {
console.error('Cannot disable both hls and dash\n');
cli.help();
process.exit(1);
}
if (
options.segmentSingleFileName &&
options.segmentSingleFileName.indexOf('$KEY$') === -1
) {
console.error(
'--segment-single-file-name argument must contain $KEY$\n'
);
cli.help();
process.exit(1);
}
console.log('inputs', inputOptions);
console.log(
`dest: ${options.destinationFolder}, source: ${options.sourceFolder}`
);
await doPackage({
dest: options.destinationFolder || '.',
s3EndpointUrl: options.s3EndpointUrl,
source: options.sourceFolder,
inputs: inputOptions,
stagingDir: options.stagingDir,
noImplicitAudio: options.noImplicitAudio,
packageFormatOptions: {
hlsOnly: options.hlsOnly,
dashOnly: options.dashOnly,
segmentSingleFile: options.segmentSingleFile,
segmentSingleFileTemplate: options.segmentSingleFileName,
segmentDuration: options.segmentDuration
? parseFloat(options.segmentDuration)
: undefined
},
shakaExecutable:
options.shakaExecutable || process.env.SHAKA_PACKAGER_EXECUTABLE
});
} catch (err) {
console.log((err as Error).message);
}
7 changes: 6 additions & 1 deletion src/index.ts
Original file line number Diff line number Diff line change
@@ -1 +1,6 @@
export { Input, doPackage, PackageOptions } from './packager';
export {
Input,
doPackage,
PackageOptions,
PackageFormatOptions
} from './packager';
134 changes: 123 additions & 11 deletions src/packager.test.ts
Original file line number Diff line number Diff line change
@@ -1,20 +1,57 @@
import { createShakaArgs, Input } from './packager';
import { createShakaArgs, doPackage, Input } from './packager';

const singleInputVideo = [
{
type: 'video',
filename: 'test.mp4',
key: '1'
} as Input
];

describe('Test doPackage', () => {
it('Both hlsOnly and dashOnly specified, throws error', async () => {
try {
await doPackage({
inputs: singleInputVideo,
dest: '.',
packageFormatOptions: {
hlsOnly: true,
dashOnly: true
}
});
fail('Should throw');
} catch (err) {
expect((err as Error).message).toBe('Cannot disable both hls and dash');
}
});

it('segmentSingleFileTemplate does not contain $KEY$, throws error', async () => {
try {
await doPackage({
inputs: singleInputVideo,
dest: '.',
packageFormatOptions: {
segmentSingleFileTemplate: 'Container.mp4'
}
});
fail('Should throw');
} catch (err) {
expect((err as Error).message).toBe(
'segmentSingleFileTemplate must contain $KEY$'
);
}
});
});

describe('Test create shaka args', () => {
const singleInputVideo = [
{
type: 'video',
filename: 'test.mp4',
key: '1'
} as Input
];
it('Should use first video file as audio source if noImplicitAudio not set', async () => {
const args = createShakaArgs(singleInputVideo, false);
expect(args).toEqual([
'in=test.mp4,stream=video,init_segment=video-1/init.mp4,segment_template=video-1/$Number$.m4s,playlist_name=video-1.m3u8',
'in=test.mp4,stream=audio,init_segment=audio/init.mp4,segment_template=audio/$Number$.m4s,playlist_name=audio.m3u8,hls_group_id=audio,hls_name=defaultaudio',
'in=test.mp4,stream=video,playlist_name=video-1.m3u8,init_segment=video-1/init.mp4,segment_template=video-1/$Number$.m4s',
'in=test.mp4,stream=audio,playlist_name=audio.m3u8,hls_group_id=audio,hls_name=defaultaudio,init_segment=audio/init.mp4,segment_template=audio/$Number$.m4s',
'--hls_master_playlist_output',
'index.m3u8',
'--generate_static_live_mpd',
'--mpd_output',
'manifest.mpd'
]);
@@ -23,11 +60,86 @@ describe('Test create shaka args', () => {
it('Should not use first video file as audio source if noImplicitAudio is true', async () => {
const args = createShakaArgs(singleInputVideo, true);
expect(args).toEqual([
'in=test.mp4,stream=video,init_segment=video-1/init.mp4,segment_template=video-1/$Number$.m4s,playlist_name=video-1.m3u8',
'in=test.mp4,stream=video,playlist_name=video-1.m3u8,init_segment=video-1/init.mp4,segment_template=video-1/$Number$.m4s',
'--hls_master_playlist_output',
'index.m3u8',
'--generate_static_live_mpd',
'--mpd_output',
'manifest.mpd'
]);
});

it('Should set --segement_duration option if segmentDuration is set', async () => {
const args = createShakaArgs(singleInputVideo, true, {
segmentDuration: 3.84
});
expect(args).toEqual([
'in=test.mp4,stream=video,playlist_name=video-1.m3u8,init_segment=video-1/init.mp4,segment_template=video-1/$Number$.m4s',
'--hls_master_playlist_output',
'index.m3u8',
'--generate_static_live_mpd',
'--mpd_output',
'manifest.mpd',
'--segment_duration',
'3.84'
]);
});

it('Should set correct output path for stream if single file segment specified', async () => {
const args = createShakaArgs(singleInputVideo, true, {
segmentSingleFile: true,
segmentSingleFileTemplate: 'Container-$KEY$.mp4'
});
expect(args).toEqual([
'in=test.mp4,stream=video,playlist_name=video-1.m3u8,out=Container-1.mp4',
'--hls_master_playlist_output',
'index.m3u8',
'--generate_static_live_mpd',
'--mpd_output',
'manifest.mpd'
]);
});

it('Should set correct output path for stream if single file segment specified with audio', async () => {
const args = createShakaArgs(
[
...singleInputVideo,
{
type: 'audio',
filename: 'audio.mp4',
key: '2'
}
],
true,
{
segmentSingleFile: true,
segmentSingleFileTemplate: 'Container-$KEY$.mp4'
}
);
expect(args).toEqual([
'in=test.mp4,stream=video,playlist_name=video-1.m3u8,out=Container-1.mp4',
'in=audio.mp4,stream=audio,playlist_name=audio.m3u8,hls_group_id=audio,hls_name=defaultaudio,out=Container-2.mp4',
'--hls_master_playlist_output',
'index.m3u8',
'--generate_static_live_mpd',
'--mpd_output',
'manifest.mpd'
]);
});

it('Should set name of master playlist and manifest as specified', async () => {
const args = createShakaArgs(singleInputVideo, false, {
hlsManifestName: 'myHlsMasterPlaylist.m3u8',
dashManifestName: 'myDashManifest.mpd'
});
expect(args).toEqual([
'in=test.mp4,stream=video,playlist_name=video-1.m3u8,init_segment=video-1/init.mp4,segment_template=video-1/$Number$.m4s',
'in=test.mp4,stream=audio,playlist_name=audio.m3u8,hls_group_id=audio,hls_name=defaultaudio,init_segment=audio/init.mp4,segment_template=audio/$Number$.m4s',
'--hls_master_playlist_output',
'myHlsMasterPlaylist.m3u8',
'--generate_static_live_mpd',
'--mpd_output',
'myDashManifest.mpd'
]);
});
});
Loading