Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch python build to scikit-build-core #1816

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ jobs:
py_build_test:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
fail-fast: false

matrix:
os: [ubuntu-latest, windows-latest, macos-13, macos-latest]
python-version: ['3.7', '3.8', '3.9', '3.10', '3.11', '3.12']
Expand Down Expand Up @@ -146,8 +147,8 @@ jobs:
sudo apt-get install lcov
- name: Install python build dependencies
run: |
python -m pip install --upgrade pip setuptools wheel "flake8>=3.5" check-manifest
- name: Run check-manifest and lint check
python -m pip install --upgrade pip build "flake8>=3.5"
- name: Run lint check
run: make ci-prebuild
- name: Build and Install
run: |
Expand Down
4 changes: 4 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,10 @@ if(OTIO_PYTHON_INSTALL)
# can find them, rather as part of the C++ SDK package; so the variable
# OTIO_RESOLVED_CXX_DYLIB_INSTALL_DIR indicates where that is.
#
if (SKBUILD)
set(OTIO_PYTHON_INSTALL_DIR ${SKBUILD_PLATLIB_DIR})
endif()

if(OTIO_PYTHON_INSTALL_DIR STREQUAL "" AND CMAKE_INSTALL_PREFIX_INITIALIZED_TO_DEFAULT)
# neither install directory supplied from the command line
set(OTIO_RESOLVED_PYTHON_INSTALL_DIR "${Python_SITEARCH}")
Expand Down
31 changes: 0 additions & 31 deletions MANIFEST.in

This file was deleted.

14 changes: 2 additions & 12 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
.PHONY: coverage test test_first_fail clean autopep8 lint doc-html \
python-version wheel manifest lcov lcov-html lcov-reset
python-version wheel lcov lcov-html lcov-reset

# Special definition to handle Make from stripping newlines
define newline
Expand Down Expand Up @@ -28,7 +28,6 @@ LCOV_PROG := $(shell command -v lcov 2> /dev/null)
PYCODESTYLE_PROG := $(shell command -v pycodestyle 2> /dev/null)
PYFLAKES_PROG := $(shell command -v pyflakes 2> /dev/null)
FLAKE8_PROG := $(shell command -v flake8 2> /dev/null)
CHECK_MANIFEST_PROG := $(shell command -v check-manifest 2> /dev/null)
CLANG_FORMAT_PROG := $(shell command -v clang-format 2> /dev/null)
# AUTOPEP8_PROG := $(shell command -v autopep8 2> /dev/null)
TEST_ARGS=
Expand All @@ -54,7 +53,7 @@ test-core: python-version

# CI
###################################
ci-prebuild: manifest lint
ci-prebuild: lint
ci-postbuild: coverage
###################################

Expand Down Expand Up @@ -164,15 +163,6 @@ endif
$(eval DIRS += src/py-opentimelineio/opentimelineio-opentime-bindings)
$(eval FILES_TO_FORMAT = $(wildcard $(addsuffix /*.h, $(DIRS)) $(addsuffix /*.cpp, $(DIRS))))
$(shell clang-format -i -style=file $(FILES_TO_FORMAT))

manifest:
ifndef CHECK_MANIFEST_PROG
$(error $(newline)$(ccred)check-manifest is not available on $$PATH please see:$(newline)$(ccend)\
$(ccblue) https://github.com/mgedmin/check-manifest#quick-start$(newline)$(ccend)\
$(dev_deps_message))
endif
@check-manifest
@echo "check-manifest succeeded"


doc-model:
Expand Down
31 changes: 13 additions & 18 deletions docs/tutorials/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,30 +71,25 @@ with OpenTimelineIO due to spaces in the path.

The Python package is a mix of pure python and C++ code. Therefore, it is
recommended to use the python tooling (`python -m pip`) to develop both
the C++ binding and the pure python code. We use `setuptools` as our
python build backend, which means `pip` will call the `setup.py` in the root
of the directory to build both the pure python and the C++ bindings.
`setuptools` will take care of all the complexity of building a C++ Python
extension for you.

The first time `setup.py` is run, cmake scripts will be created, and the headers
and libraries will be installed where you specify. If the C++ or Python sources
are subsequently modified, running this command again will build and update everything
appropriately.

**Note** Any CMake arguments can be passed through `pip` by using the `CMAKE_ARGS`
the C++ binding and the pure python code. We use [scikit-build-core](https://scikit-build-core.readthedocs.io/)
as our python build backend, which means `pip` or [build](https://pypi.org/project/build/)
must be used to build the python package. `scikit-build-core` will take care
of automatically installing the required dependencies (cmake, ninja, etc).

By default, `scikit-build-core` will build the project in a temporary directory and will
not use caching. This can be changed by setting [`build-dir` in `pyproject.toml`](https://scikit-build-core.readthedocs.io/en/stable/configuration.html#other-options). Alternatively, you can also set `build-dir` via a command line
argument: `pip install --config-settings=build-dir=<path> .` or
`python -m build --config-settings=build-dir=<path> --wheel .`.

**Note** Any CMake arguments can be passed to `cmake` by using the `CMAKE_ARGS`
environment variable when building from source. *nix Example:

```bash
env CMAKE_ARGS="-DCMAKE_VAR=VALUE1 -DCMAKE_VAR_2=VALUE2" python -m pip install .
```

`python -m pip install .` adds some overhead that might be annoying or unwanted when
developing the python bindings. For that reason (and only that reason), if you want a faster
iteration process, you can use `setuptools` commands. For example you can use
`python setup.py build_ext` to only run the compilation step. Be aware that calling `setup.py`
directly is highly discouraged and should only be used when no of the other options
are viable. See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html.
Additionally, `scikit-build-core` supports [editable installs](https://scikit-build-core.readthedocs.io/en/stable/configuration.html#editable-installs).
You can use this to speed up your development.

To compile your own C++ file referencing the OTIO headers from your C++ build using gcc or clang, add the following -I flags:

Expand Down
95 changes: 90 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,10 +1,95 @@
[build-system]
requires = [
"setuptools",
"wheel",
"cmake>=3.12",
requires = ["scikit-build-core>=0.10.7"]
build-backend = "scikit_build_core.build"

[project]
name = "OpenTimelineIO"
version = "0.18.0.dev1"
description = "Editorial interchange format and API"
authors = [
{name = "Contributors to the OpenTimelineIO project", email = "[email protected]"}
]
license = {file = "LICENSE.txt"}
keywords = ["film", "tv", "editing", "editorial", "edit", "non-linear", "edl", "time"]
classifiers = [
"Development Status :: 4 - Beta",
"Topic :: Multimedia :: Graphics",
"Topic :: Multimedia :: Video",
"Topic :: Multimedia :: Video :: Display",
"Topic :: Multimedia :: Video :: Non-Linear Editor",
"Topic :: Software Development :: Libraries :: Python Modules",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",

"Operating System :: OS Independent",
"Natural Language :: English",
]

# Disallow 3.9.0 because of https://github.com/python/cpython/pull/22670
requires-python = ">=3.7, !=3.9.0"

dependencies = [
'importlib_metadata>=1.4; python_version < "3.8"'
]

[project.urls]
Documentation = "https://opentimelineio.readthedocs.io"
Homepage = "https://opentimeline.io"
Issues = "https://github.com/AcademySoftwareFoundation/OpenTimelineIO/issues"
Source = "https://github.com/AcademySoftwareFoundation/OpenTimelineIO"

[project.scripts]
otiocat = "opentimelineio.console.otiocat:main"
otioconvert = "opentimelineio.console.otioconvert:main"
otiopluginfo = "opentimelineio.console.otiopluginfo:main"
otiostat = "opentimelineio.console.otiostat:main"
otiotool = "opentimelineio.console.otiotool:main"
otioview = "opentimelineview.console:main"
otioautogen_serialized_schema_docs = "opentimelineio.console.autogen_serialized_datamodel:main"

[project.optional-dependencies]
dev = [
"check-manifest",
"flake8>=3.5",
"coverage>=4.5",
"urllib3>=1.24.3",
]
build-backend = "setuptools.build_meta"
view = [
'PySide2~=5.11; platform.machine=="x86_64"',
'PySide6~=6.2; platform.machine=="aarch64"',
]

[tool.scikit-build]
cmake.version = "CMakeLists.txt"

cmake.build-type = "Release"

logging.level = "DEBUG"
build.verbose = true

sdist.include = ["src/opentimelineio/adapters/builtin_adapters.plugin_manifest.json"]

wheel.packages = ["src/py-opentimelineio/opentimelineio", "src/opentimelineview"]
wheel.install-dir = "opentimelineio"

[[tool.scikit-build.generate]]
path = "opentimelineio/__init__.py"
template-path = "src/py-opentimelineio/opentimelineio/__init__.py"

[[tool.scikit-build.generate]]
path = "opentimelineview/__init__.py"
template-path = "src/opentimelineview/__init__.py"

[tool.scikit-build.cmake.define]
OTIO_PYTHON_INSTALL = "ON"
OTIO_CXX_INSTALL = "OFF"
OTIO_SHARED_LIBS = "OFF"
BUILD_TESTING = "OFF"
OTIO_INSTALL_PYTHON_MODULES = "OFF"

[tool.cibuildwheel.linux]
archs = ["x86_64", "aarch64"]
Expand Down
6 changes: 0 additions & 6 deletions setup.cfg
Original file line number Diff line number Diff line change
@@ -1,9 +1,3 @@
###############################################################################
# Python Distribution
###############################################################################
[metadata]
description_file = README.md

###############################################################################
# flake8
###############################################################################
Expand Down
Loading
Loading