Add options to yield multiple packages

[jpmckinney]

Follow-up to #83

Yielding a single package can lead to all memory being consumed.

We can do something similar to split-record-packages to yield a package once it reaches a given number of records. This applies to all commands that yields a single package.

[jpmckinney]

Yielding a single package can lead to all memory being consumed.

Fixed in #115.

We can do something similar to split-record-packages to yield a package once it reaches a given number of records. This applies to all commands that yields a single package.

--size option added to:

• package-records
• package-releases

Not yet added to:

• combine-record-packages: If there is demand, we might want to do combine-*-packages: Avoid buffering all inputs into memory (iterencode) [solved] #119 as well
• combine-release-packages: If there is demand, we might want to do combine-*-packages: Avoid buffering all inputs into memory (iterencode) [solved] #119 as well
• compile: See below

For compile, we can run:

cat files | ocdskit compile --package | ocdskit echo --root-path records.item | ocdskit package-records --size 1000

To avoid the echo step, we can change the --package option of the compile command to be an --into choice between 'package' and 'records', e.g.

cat files | ocdskit compile --into records | ocdskit package-records --size 1000

Since this would require a change to the API (to prevent incoherent options like --package --records and return_package=True, return_records=True), we'll postpone until there's demand.

[jpmckinney]

Opened follow-up issue #122, so closing.