Skip to content

Commit

Permalink
Merge pull request #66 from calgray/update-publish-tools
Browse files Browse the repository at this point in the history
Release 0.2.1
  • Loading branch information
calgray committed Jan 7, 2025
2 parents aed6df6 + 8b30618 commit ac6caaf
Show file tree
Hide file tree
Showing 10 changed files with 152 additions and 136 deletions.
4 changes: 1 addition & 3 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
name: Test and build

name: test
on: [push, workflow_call]

jobs:
qa:
runs-on: ubuntu-latest
Expand Down
File renamed without changes.
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,15 @@

## 0.2.1

## Added

* Added contributing document.

## Changed

* Updated release workflow process.
* Updated Readme
* Test dependencies audited.

## 0.2.0

Expand Down
97 changes: 97 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Contributing

Thank you for considering contributing to Athreading! We appreciate your efforts to help improve the project. Please follow these guidelines to ensure a smooth and productive contribution process.

Reading and following these guidelines will help us make the contribution process easy and effective for everyone involved. It also communicates that you agree to respect the time of the developers managing and developing these open source projects. In return, we will reciprocate that respect by addressing your issue, assessing changes, and helping you finalize your pull requests.

## Quicklinks

* [Getting Started](#getting-started)
* [Issues](#issues)
* [Pull Requests](#pull-requests)
* [Maintenance](#maintenance)
* [Testing](#testing)
* [Publishing](#publishing)

## Getting Started

Contributions are made to this repo via Issues and Pull Requests (PRs). A few general guidelines that cover both:

* Search for existing Issues and PRs before creating your own.
* Read this repository package license and licenses of any proposed dependencies to ensure compatibility.
* We work hard to makes sure issues are handled in a timely manner but, depending on the impact, it could take a while to investigate the root cause. A friendly ping in the comment thread to the submitter or a contributor can help draw attention if your issue is blocking.
* If you've never contributed to FOSS projects before, see this [the first timer's guide](https://auth0.com/blog/a-first-timers-guide-to-an-open-source-project/) for resources and tips on how to get started.

### Issues

Issues should be used to report problems with the library, request a new feature, or to discuss potential changes before a PR is created. When you create a new Issue, a template will be loaded that will guide you through collecting and providing the information we need to investigate.

If you find an Issue that addresses the problem you're having, please add your own reproduction information to the existing issue rather than creating a new one. Adding a [reaction](https://github.blog/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) can also help by indicating to our maintainers that a particular problem is affecting more than just the reporter.

### Pull Requests

PRs to our libraries are always welcome and can be a quick way to get your fix or improvement slated for the next release. In general, PRs should:

* Only fix/add the functionality in question **OR** address wide-spread whitespace/style issues, not both.
* Add unit or integration tests for fixed or changed functionality (if a test suite already exists).
* Address a single concern in the least number of changed lines as possible.
* Include documentation in the repo (**TODO** sphinx documentation).
* Be accompanied by a complete Pull Request template (loaded automatically when a PR is created).

For changes that address core functionality or would require breaking changes (e.g. a major release), it's best to open an Issue to discuss your proposal first. This is not required but can save time creating and reviewing changes.

In general, we follow the ["fork-and-pull" Git workflow](https://github.com/susam/gitpr)

1. Fork the repository to your own Github account
2. Clone the project to your machine
3. Create a branch locally with a succinct but descriptive name
4. Commit changes to the branch
5. Following any formatting and testing guidelines specific to this repo
6. Push changes to your fork
7. Open a PR in our repository and follow the PR template so that we can efficiently review the changes.

## Maintenance

This is a minimal Python library that uses [poetry](https://python-poetry.org) for packaging and dependency management. To assist automating the maintence process, this repository provides [pre-commit](https://pre-commit.com/) hooks (for [isort](https://pycqa.github.io/isort/), [Black](https://black.readthedocs.io/en/stable/), [Flake8](https://flake8.pycqa.org/en/latest/) and [mypy](https://mypy.readthedocs.io/en/stable/)) and automated tests using [pytest](https://pytest.org/) and [GitHub Actions](https://github.com/features/actions). Pre-commit hooks are automatically kept updated with a dedicated GitHub Action, this can be removed and replace with [pre-commit.ci](https://pre-commit.ci). It was developed by the [Imperial College Research Computing Service](https://www.imperial.ac.uk/admin-services/ict/self-service/research-support/rcs/).

### Testing

To modify, test and request changes to this repository:

1. [Download and install Poetry](https://python-poetry.org/docs/#installation) following the instructions for the target OS.
2. Clone `[email protected]:calgray/athreading.git` and change working directory
3. Set up the virtual environment:

```bash
poetry install
```

4. Activate the virtual environment (alternatively, ensure any python-related command is preceded by `poetry run`):

```bash
poetry shell
```

5. Install the git hooks:

```bash
pre-commit install
```

6. Run all checks and tests:

```bash
pre-commit run --all-files
pytest --benchmark-enable
```

### Publishing

The GitHub workflow includes an action to publish on release.

When preparing for a release, ensure:

* A suitable version increment complient with [Semantic Versioning 2.0.0](https://semver.org/) is made to `pyproject.toml` and `__init__.py`
* The Changelog is updated with the new version and contains a short developer facing summary of introduced changes
* A git tag is created in the same format as the pyproject.toml version (e.g. `0.1.0`)
* A Github release is created using the tag and with user facing release notes summary
44 changes: 2 additions & 42 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
[![Codecov](https://codecov.io/gh/calgray/athreading/branch/main/graph/badge.svg)](https://app.codecov.io/github/calgray/athreading)

[![Code style](https://img.shields.io/badge/code_style-black-000000.svg)](https://github.com/psf/black)
[![Checked with mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)
[![Pydocstyle](https://img.shields.io/badge/pydocstyle-enabled-AD4CD3)](http://www.pydocstyle.org/en/stable/)
[![Checked with mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://mypy-lang.org/)
[![Pydocstyle](https://img.shields.io/badge/flake8-enabled-blue.svg)](https://flake8.pycqa.org/en/latest/)

`athreading` is a Python library that allows you to run synchronous I/O functions asynchronously using `asyncio` via background threads. It provides decorators to adapt synchronous functions and generators, enabling them to operate without blocking the event loop.

Expand Down Expand Up @@ -122,46 +122,6 @@ asyncio.run(amain())

This example demonstrates how `@athreading.generate` transforms a synchronous generator into an asynchronous generator. The `asend` method sends values to control the generator's state dynamically, enabling interactive workflows while avoiding blocking the event loop.

## Maintenance

This is a minimal Python library that uses [poetry](https://python-poetry.org) for packaging and dependency management. It also provides [pre-commit](https://pre-commit.com/) hooks (for [isort](https://pycqa.github.io/isort/), [Black](https://black.readthedocs.io/en/stable/), [Flake8](https://flake8.pycqa.org/en/latest/) and [mypy](https://mypy.readthedocs.io/en/stable/)) and automated tests using [pytest](https://pytest.org/) and [GitHub Actions](https://github.com/features/actions). Pre-commit hooks are automatically kept updated with a dedicated GitHub Action, this can be removed and replace with [pre-commit.ci](https://pre-commit.ci) if using an public repo. It was developed by the [Imperial College Research Computing Service](https://www.imperial.ac.uk/admin-services/ict/self-service/research-support/rcs/).

### Testing

To modify, test and request changes to this repository:

1. [Download and install Poetry](https://python-poetry.org/docs/#installation) following the instructions for the target OS.
2. Clone `[email protected]:calgray/athreading.git` and make it the working directory.
3. Set up the virtual environment:

```bash
poetry install
```

4. Activate the virtual environment (alternatively, ensure any python-related command is preceded by `poetry run`):

```bash
poetry shell
```

5. Install the git hooks:

```bash
pre-commit install
```

6. Run the tests:

```bash
pytest
```

### Publishing

The GitHub workflow includes an action to publish on release.

To run this action, uncomment the commented portion of `publish.yml`, and modify the steps for the desired behaviour (ie. publishing a Docker image, publishing to PyPI, deploying documentation etc.)

## License

This project is licensed under the BSD-3-Clause License.
Expand Down
19 changes: 1 addition & 18 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ black = ">=23.1,<25.0"
flake8 = ">=6,<8"
flake8-docstrings = "^1.6.0"
flake8-pyproject = "^1.2.2"
aiostream = ">=0.4.5,<0.7.0"
threaded = "^4.2.0"
typing_extensions = { python = "3.13", version = ">4.12.0" }

Expand Down
21 changes: 11 additions & 10 deletions tests/performance/test_iterate_benchmark.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
import time
from collections.abc import Iterable

import aiostream
import pytest
import threaded

Expand All @@ -24,26 +23,28 @@ def square_iterate(iterator: Iterable[float], delay: float = 0.0):
yield square(item)


async def asquare_iterate_naive(iterator: Iterable[float], delay: float = 0.0):
for value in square_iterate(iterator, delay):
yield value
await asyncio.sleep(0.0)


async def asquare_list_naive(length: int):
return [v async for v in asquare_iterate_naive(range(length), 0.001)]


@athreading.iterate(executor=executor)
def asquare_iterate_athreading(values: Iterable[float], delay: float = 0.0):
yield from square_iterate(values, delay)


def asquare_iterate_aiostream(iterator: Iterable[float], delay: float = 0.0):
return aiostream.stream.iterate(square_iterate(iterator, delay)).stream()


async def asquare_list_athreading(length: int):
async with asquare_iterate_athreading(range(length), 0.001) as it:
return [v async for v in it]


async def asquare_list_aiostream(length: int):
return await aiostream.stream.list(asquare_iterate_aiostream(range(length), 0.001))


@pytest.mark.benchmark(group="iterate", disable_gc=True, warmup=False)
@pytest.mark.parametrize("impl", [asquare_list_athreading, asquare_list_aiostream])
@pytest.mark.parametrize("impl", [asquare_list_naive, asquare_list_athreading])
@pytest.mark.parametrize("stream_length", [100])
@pytest.mark.parametrize("num_streams", [1, 4, 16])
def test_iterate_benchmark(benchmark, impl, num_streams: int, stream_length: int):
Expand Down
71 changes: 14 additions & 57 deletions tests/unit/test_iterate.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,17 @@
import asyncio
import sys
import time
from concurrent.futures import ThreadPoolExecutor

import aiostream.stream as astream
import pytest

import athreading

if sys.version_info[:2] > (3, 9):
from contextlib import nullcontext
else:
from tests.compat import nullcontext

TEST_VALUES = [1, None, "", 2.0]

custom_executor = ThreadPoolExecutor()
Expand All @@ -19,6 +24,12 @@ def generator(delay=0.0, repeats=1):
yield item


async def agenerate_naive(delay=0.0, repeats=1):
for value in generator(delay, repeats):
yield value
await asyncio.sleep(0.0)


@athreading.iterate(executor=custom_executor)
def aiterate(delay=0.0, repeats=1):
yield from generator(delay, repeats)
Expand Down Expand Up @@ -54,7 +65,7 @@ def agenerate_simplest(delay=0.0, repeats=1):
@pytest.mark.parametrize(
"streamcontext",
[
lambda delay: astream.iterate(generator(delay)).stream(),
lambda delay: nullcontext(agenerate_naive(delay)),
lambda delay: athreading.iterate(generator)(delay),
lambda delay: aiterate(delay),
lambda delay: aiterate_simpler(delay),
Expand All @@ -65,7 +76,7 @@ def agenerate_simplest(delay=0.0, repeats=1):
lambda delay: agenerate_simplest(delay),
],
ids=[
"aiostream",
"naive",
"iterate",
"aiterate",
"aiterate_simpler",
Expand Down Expand Up @@ -121,57 +132,3 @@ async def process():
timeout=timeout,
)
await asyncio.wait_for(asyncio.get_running_loop().shutdown_default_executor(), 1.0)


def generate_infinite(delay=0.0):
item = 0
while True:
time.sleep(delay)
item += 1
yield item


@pytest.mark.parametrize("worker_delay", [0.0, 0.1])
@pytest.mark.parametrize("main_delay", [0.0, 0.1])
@pytest.mark.parametrize(
"streamcontext",
[
lambda delay: astream.iterate(generate_infinite(delay)).stream(),
lambda delay: athreading.iterate(generate_infinite)(delay),
lambda delay: athreading.generate(generate_infinite)(delay),
],
ids=["aiostream", "iterate", "generate"],
)
@pytest.mark.asyncio
async def test_iterate_cancel(streamcontext, worker_delay, main_delay):
output = []
async with streamcontext(worker_delay) as stream:
async for v in stream:
time.sleep(main_delay)
output.append(v)
break
assert output == [1]
await asyncio.wait_for(asyncio.get_running_loop().shutdown_default_executor(), 1.0)


@pytest.mark.parametrize("worker_delay", [0.0, 0.1])
@pytest.mark.parametrize("main_delay", [0.0, 0.1])
@pytest.mark.parametrize(
"streamcontext",
[
lambda delay: astream.iterate(generate_infinite(delay)).stream(),
lambda delay: athreading.iterate(generate_infinite)(delay),
lambda delay: athreading.generate(generate_infinite)(delay),
],
ids=["aiostream", "iterate", "generate"],
)
@pytest.mark.asyncio
async def test_iterate_exception(streamcontext, worker_delay, main_delay):
output = []
with pytest.raises(TypeError):
async with streamcontext(worker_delay, repeats="invalid") as stream:
async for v in stream:
time.sleep(main_delay)
output.append(v)
break
await asyncio.wait_for(asyncio.get_running_loop().shutdown_default_executor(), 1.0)
Loading

0 comments on commit ac6caaf

Please sign in to comment.