Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Universal Reader and RetinaNet Normalization #80

Merged
merged 38 commits into from
Nov 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
0906855
Ensure images are converted to RGB format in UniversalReader methods
tomas-gajarsky Nov 3, 2024
f3a1170
Add tests for reading grayscale and RGBA images in UniversalReader
tomas-gajarsky Nov 9, 2024
a0e3128
Update Dockerfile to use Python 3.9.12 and Miniconda 4.9.2
tomas-gajarsky Nov 9, 2024
3474939
Update depedency conda lock
tomas-gajarsky Nov 9, 2024
44a81af
Fix mean normalization values in RetinaFace preprocessor configuration
tomas-gajarsky Nov 9, 2024
aa31044
Add integration test for preprocessor normalization order in detector
tomas-gajarsky Nov 9, 2024
6feff16
Add end-to-end test for face crop tensor properties in unifier
tomas-gajarsky Nov 9, 2024
3d2b3c0
Ensure PIL images are converted to RGB format in UniversalReader
tomas-gajarsky Nov 9, 2024
e50f175
Use torch.linalg.cross for vector cross product calculation in Lmk3DM…
tomas-gajarsky Nov 9, 2024
4f26000
Update tensor loading to include weights_only parameter in align.py a…
tomas-gajarsky Nov 9, 2024
4be52fc
Refactor UniversalReader to process numpy arrays and add tests for in…
tomas-gajarsky Nov 9, 2024
0a8f6ae
Convert numpy array to float tensor in UniversalReader
tomas-gajarsky Nov 9, 2024
2dc5096
Add research references and citation guidelines to README.md
tomas-gajarsky Nov 9, 2024
cdbddcc
Bump version to 0.5.1
tomas-gajarsky Nov 9, 2024
335ff83
Update CHANGELOG.md for version 0.5.1 release notes
tomas-gajarsky Nov 9, 2024
f73588d
Improve test code formatting and enhance readability in detector, rea…
tomas-gajarsky Nov 9, 2024
aecd78d
Update Miniconda version in Dockerfiles for development and testing
tomas-gajarsky Nov 9, 2024
8dd12de
Update docs
tomas-gajarsky Nov 9, 2024
cdc8a30
Remove platform specification from environment configuration files
tomas-gajarsky Nov 9, 2024
3e5d443
Update GitHub Actions workflow to use Miniconda setup and configure c…
tomas-gajarsky Nov 9, 2024
04a7c73
Update documentation to v0.5.1
tomas-gajarsky Nov 9, 2024
749290b
Add installation steps for libarchive20 and Mamba in GitHub Actions w…
tomas-gajarsky Nov 9, 2024
d39d818
Remove weights_only param from troch.load
tomas-gajarsky Nov 16, 2024
d0162d6
Update dependency lock
tomas-gajarsky Nov 16, 2024
ddda183
Install libarchive tools in conda pipeline
tomas-gajarsky Nov 16, 2024
8568742
Adjust docker pipeline to push versioned images to hub
tomas-gajarsky Nov 16, 2024
18929dc
Remove extra space from Dockerfile
tomas-gajarsky Nov 16, 2024
f17db3a
Update compose version and GPU device assignment
tomas-gajarsky Nov 16, 2024
32151c2
Update documentation
tomas-gajarsky Nov 16, 2024
bc53c14
Refactor conda workflow by removing libarchive-tools installation and…
tomas-gajarsky Nov 16, 2024
7eca885
Add support for Python versions 3.11, 3.12, and 3.13 in workflow
tomas-gajarsky Nov 16, 2024
02f44af
Remove Python 3.13 from workflow matrix
tomas-gajarsky Nov 16, 2024
640927c
Add support for Python versions 3.11 and 3.12 in setup.py
tomas-gajarsky Nov 16, 2024
7baf7a5
Add condarc file for classic solver configuration in conda workflow
tomas-gajarsky Nov 16, 2024
1bbc298
Update release date for version 0.5.1 in CHANGELOG.md
tomas-gajarsky Nov 17, 2024
cf35bbb
Update facetorch version to 0.5.1 in the example notebook
tomas-gajarsky Nov 17, 2024
e70dfd1
Update section title in README for clarity
tomas-gajarsky Nov 17, 2024
fb043b1
Refine README content for clarity and detail, enhancing descriptions …
tomas-gajarsky Nov 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 19 additions & 15 deletions .github/workflows/conda-env.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,22 @@ jobs:
conda:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.9
uses: actions/setup-python@v3
with:
python-version: 3.9
- name: Add conda to system path
run: |
# $CONDA is an environment variable pointing to the root of the miniconda directory
echo $CONDA/bin >> $GITHUB_PATH
- name: Install dependencies
run: |
conda env update --file environment.yml --name base
- name: Install facetorch from conda-forge
run: |
conda install -c conda-forge facetorch
- uses: actions/checkout@v3

- name: Create condarc file
run: |
echo "solver: classic" > condarc

- name: Set up Miniconda
uses: conda-incubator/setup-miniconda@v2
with:
python-version: 3.9
environment-file: environment.yml
channels: conda-forge, defaults
auto-activate-base: true
activate-environment: base
condarc-file: condarc

- name: Install facetorch from conda-forge
run: |
conda install -c conda-forge facetorch
20 changes: 16 additions & 4 deletions .github/workflows/docker-push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,25 +9,37 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set VERSION variable
run: echo "VERSION=$(cat ./version)" >> $GITHUB_ENV
- name: Docker compose build facetorch
run: docker compose build facetorch
- name: Tag image with version
run: docker tag tomasgajarsky/facetorch:latest tomasgajarsky/facetorch:${{ env.VERSION }}
- name: Login to Docker hub
env:
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
run: docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD docker.io
- name: Docker compose push facetorch
run: docker compose push facetorch
- name: Push images
run: |
docker push tomasgajarsky/facetorch:latest
docker push tomasgajarsky/facetorch:${{ env.VERSION }}
docker-push-gpu:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set VERSION variable
run: echo "VERSION=$(cat ./version)" >> $GITHUB_ENV
- name: Docker compose build facetorch-gpu
run: docker compose build facetorch-gpu-no-device
- name: Tag image with version
run: docker tag tomasgajarsky/facetorch-gpu:latest tomasgajarsky/facetorch-gpu:${{ env.VERSION }}
- name: Login to Docker hub
env:
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
run: docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD docker.io
- name: Docker compose push facetorch-gpu
run: docker compose push facetorch-gpu-no-device
- name: Push images
run: |
docker push tomasgajarsky/facetorch-gpu:latest
docker push tomasgajarsky/facetorch-gpu:${{ env.VERSION }}
2 changes: 1 addition & 1 deletion .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.8", "3.9", "3.10"]
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v3
Expand Down
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
# Change Log

## 0.5.1

Released on November 17, 2024.

### Changed

* UnversalReader to read PIL images as RGB
* UniversalReader to read numpy arrays to torch directly
* RetinaFace pre-normalization color space to RGB
* torch.cross torch.linalg.cross in 3D landmark drawer


## 0.5.0

Expand Down
55 changes: 43 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
[![PyPI - License](https://img.shields.io/pypi/l/facetorch)](https://raw.githubusercontent.com/tomas-gajarsky/facetorch/main/LICENSE)
<a href="https://github.com/psf/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>

[Hugging Face Space demo app 🤗 ](https://huggingface.co/spaces/tomas-gajarsky/facetorch-app)
[Hugging Face Space demo app 🤗](https://huggingface.co/spaces/tomas-gajarsky/facetorch-app)

[Google Colab notebook demo](https://colab.research.google.com/github/tomas-gajarsky/facetorch/blob/main/notebooks/facetorch_notebook_demo.ipynb)
<a target="_blank" href="https://colab.research.google.com/github/tomas-gajarsky/facetorch/blob/main/notebooks/facetorch_notebook_demo.ipynb">
Expand All @@ -17,16 +17,21 @@

[Docker Hub](https://hub.docker.com/repository/docker/tomasgajarsky/facetorch) [(GPU)](https://hub.docker.com/repository/docker/tomasgajarsky/facetorch-gpu)

Facetorch is a Python library that can detect faces and analyze facial features using deep neural networks. The goal is to gather open sourced face analysis models from the community, optimize them for performance using TorchScript and combine them to create a face analysis tool that one can:

1. configure using [Hydra](https://hydra.cc/docs/intro/) (OmegaConf)
2. reproduce with [conda-lock](https://github.com/conda-incubator/conda-lock) and [Docker](https://docs.docker.com/get-docker/)
3. accelerate on CPU and GPU with [TorchScript](https://pytorch.org/docs/stable/jit.html)
4. extend by uploading a model file to Google Drive and adding a config yaml file to the repository
**Facetorch** is a Python library designed for facial detection and analysis, leveraging the power of deep neural networks. Its primary aim is to curate open-source face analysis models from the community, optimize them for high performance using TorchScript, and integrate them into a versatile face analysis toolkit. The library offers the following key features:

1. **Customizable Configuration:** Easily configure your setup using [Hydra](https://hydra.cc/docs/intro/) and its powerful [OmegaConf](https://omegaconf.readthedocs.io/) capabilities.

2. **Reproducible Environments:** Ensure reproducibility with tools like [conda-lock](https://github.com/conda-incubator/conda-lock) for dependency management and [Docker](https://docs.docker.com/get-docker/) for containerization.

3. **Accelerated Performance:** Enjoy enhanced performance on both CPU and GPU with [TorchScript](https://pytorch.org/docs/stable/jit.html) optimization.

4. **Simple Extensibility:** Extend the library by uploading your model file to Google Drive and adding a corresponding configuration YAML file to the repository.

Facetorch provides an efficient, scalable, and user-friendly solution for facial analysis tasks, catering to developers and researchers looking for flexibility and performance.

Please use this library responsibly and with caution. Adhere to the [European Commission's Ethics Guidelines for Trustworthy AI](https://ec.europa.eu/futurium/en/ai-alliance-consultation.1.html) to ensure ethical and fair usage. Keep in mind that the models may have limitations and potential biases, so it is crucial to evaluate their outputs critically and consider their impact.

Please, use the library responsibly with caution and follow the
[ethics guidelines for Trustworthy AI from European Commission](https://ec.europa.eu/futurium/en/ai-alliance-consultation.1.html).
The models are not perfect and may be biased.

## Install
[PyPI](https://pypi.org/project/facetorch/)
Expand Down Expand Up @@ -301,10 +306,36 @@ GPU:
1. Run profiling of the example script: ```python -m cProfile -o profiling/example.prof scripts/example.py```
2. Open profiling file in the browser: ```snakeviz profiling/example.prof```

## Research Highlights Leveraging facetorch

### [Sharma et al. (2024)](https://aclanthology.org/2024.signlang-1.39.pdf)

Sharma, Paritosh, Camille Challant, and Michael Filhol. "Facial Expressions for Sign Language Synthesis using FACSHuman and AZee." *Proceedings of the LREC-COLING 2024 11th Workshop on the Representation and Processing of Sign Languages*, pp. 354–360, 2024.

### [Liang et al. (2023)](https://dl.acm.org/doi/abs/10.1145/3581783.3612854)

Liang, Cong, Jiahe Wang, Haofan Zhang, Bing Tang, Junshan Huang, Shangfei Wang, and Xiaoping Chen. "Unifarn: Unified transformer for facial reaction generation." *Proceedings of the 31st ACM International Conference on Multimedia*, pp. 9506–9510, 2023.

### [Gue et al. (2023)](https://research.monash.edu/en/publications/facial-expression-recognition-as-markers-of-depression)

Gue, Jia Xuan, Chun Yong Chong, and Mei Kuan Lim. "Facial Expression Recognition as markers of Depression." *2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)*, pp. 674–680, 2023.


## Acknowledgements
I want to thank the open source code community and the researchers who have published the models. This project would not be possible without their work.
I would like to thank the open-source community and the researchers who have shared their work and published models. This project would not have been possible without their contributions.


![](https://raw.githubusercontent.com/tomas-gajarsky/facetorch/main/data/facetorch-logo-64.png "facetorch logo")
## Citing

If you use facetorch in your work, please make sure to appropriately credit the original authors of the models it employs. Additionally, you may consider citing the facetorch library itself. Below is an example citation for facetorch:

Logo was generated using [DeepAI Text To Image API](https://deepai.org/machine-learning-model/text2img)
```
@misc{facetorch,
author = {Gajarsky, Tomas},
title = {Facetorch: A Python Library for Analyzing Faces Using PyTorch},
year = {2024},
publisher = {GitHub},
journal = {GitHub Repository},
howpublished = {\url{https://github.com/tomas-gajarsky/facetorch}}
}
```
Loading
Loading