Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Device Motion module #1217

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions packages/motion/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# @strudel/motion

This package adds device motion sensing functionality to strudel Patterns.

## Install

```sh
npm i @strudel/motion --save
```

## Setup SSL for Local Development
`DeviceMotionEvent` only work over HTTPS, so you'll need to set up SSL for local development.
install SSL plugin for Vite
`pnpm install -D @vitejs/plugin-basic-ssl`

add the basicSsl plugin to the defineConfig block in `strudel/website/astro.config.mjs`
```
vite: {
plugins: [basicSsl()],
server: {
host: '0.0.0.0', // Ensures it binds to all network interfaces
// https: {
// key: '../../key.pem', //
// cert: '../../cert.pem',
// },
},
},
```

generate SSL cert if its necessary

`openssl req -new -newkey rsa:2048 -days 365 -nodes -x509 -keyout key.pem -out cert.pem`

## Usage

| Motion | Long Names & Aliases | Description |
|----------------------------|-----------------------------------------------------------|------------------------------------------|
| Acceleration | accelerationX (accX), accelerationY (accY), accelerationZ (accZ) | X, Y, Z-axis acceleration values |
| Gravity | gravityX (gravX), gravityY (gravY), gravityZ (gravZ) | X, Y, Z-axis gravity values |
| Rotation | rotationAlpha (rotA, rotZ), rotationBeta (rotB, rotX), rotationGamma (rotG, rotY) | Rotation around alpha, beta, gamma axes and mapped to X, Y, Z |
| Orientation | orientationAlpha (oriA, oriZ), orientationBeta (oriB, oriX), orientationGamma (oriG, oriY) | Orientation alpha, beta, gamma values and mapped to X, Y, Z |
| Absolute Orientation | absoluteOrientationAlpha (absOriA, absOriZ), absoluteOrientationBeta (absOriB, absOriX), absoluteOrientationGamma (absOriG, absOriY) | Absolute orientation alpha, beta, gamma values and mapped to X, Y, Z |

## Example

```
enableMotion() //enable DeviceMotion

let tempo = 200

$_: accX.segment(16).gain().log()

$:n("[0 1 3 1 5 4]/4")
.scale("Bb:lydian")
.sometimesBy(0.5,sub(note(12)))
.lpf(gravityY.range(20,1000))
.lpq(gravityZ.range(1,30))
.lpenv(gravityX.range(2,2))
.gain(oriX.range(0.2,0.8))
.room(oriZ.range(0,0.5))
.attack(oriY.range(0,0.3))
.delay(rotG.range(0,1))
.decay(rotA.range(0,1))
.attack(rotB.range(0,0.1))
.sound("sawtooth").cpm(tempo)
```
82 changes: 82 additions & 0 deletions packages/motion/docs/devicemotion.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
import { MiniRepl } from '../../../website/src/docs/MiniRepl';
import { JsDoc } from '../../../website/src/docs/JsDoc';

# Device Motion

Devicemotion module allows you to use your mobile device's motion sensors (accelerometer, gyroscope, and orientation sensors) to control musical parameters in real-time. This creates opportunities for expressive, movement-based musical interactions.

## Basic Setup

First, you need to enable device motion sensing:

<MiniRepl client:idle tune={`enableMotion()`} />

This will prompt the user for permission to access device motion sensors.

## Available Motion Parameters

You can access different types of motion data:

| Motion | Long Names & Aliases | Description |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| Acceleration | accelerationX (accX), accelerationY (accY), accelerationZ (accZ) | Measures linear acceleration of the device, excluding gravity. Raw values are normalized from g-force. |
| Gravity | gravityX (gravX), gravityY (gravY), gravityZ (gravZ) | Indicates device's orientation relative to Earth's gravity. Raw values are normalized from ±9.81 m/s². |
| Rotation | rotationAlpha (rotA, rotZ), rotationBeta (rotB, rotX), rotationGamma (rotG, rotY) | Measures rotation rate around each axis. Raw values (±180°/s) are normalized. |
| Orientation | orientationAlpha (oriA, oriZ), orientationBeta (oriB, oriX), orientationGamma (oriG, oriY) | Relative orientation from its starting device position. Normalized from:<br/>- Alpha: 0° to 360°<br/>- Beta: -180° to 180°<br/>- Gamma: -90° to 90° |
| Absolute Orientation | absoluteOrientationAlpha (absOriA, absOriZ), absoluteOrientationBeta (absOriB, absOriX), absoluteOrientationGamma (absOriG, absOriY) | **Not available for iOS** <br/> Earth-referenced orientation using magnetometer. Same normalization as Orientation. |

Note:

- All motion values are normalized to a range of 0 to 1.
- Not all devices have the same sensors available
Check [DeviceMotionEvent API](https://developer.mozilla.org/en-US/docs/Web/API/DeviceMotionEvent) for browser compatibility
- Refer to [Oritentation and motion data explained](https://developer.mozilla.org/en-US/docs/Web/API/Device_orientation_events/Orientation_and_motion_data_explained) for more details

### Orientation vs Absolute Orientation

The key difference between regular orientation and absolute orientation is:

- Regular orientation (`oriX/Y/Z`) measures relative changes in device orientation from its starting position
- Absolute orientation (`absOriX/Y/Z`) measures orientation relative to Earth's magnetic field and gravity, providing consistent absolute values regardless of starting position

For example, if you rotate your device 90 degrees clockwise and then back:

- Regular orientation will show a change during rotation but return to initial values
- Absolute orientation will show the actual compass heading throughout

This makes absolute orientation particularly useful for creating direction-based musical interactions - for example, performers facing north could play one melody while those facing south play another, creating spatially-aware ensemble performances. Regular orientation, on the other hand, is better suited for detecting relative motion and gestures regardless of which direction the performer is facing.

## Basic Example

Here's a simple example that uses device motion to control a synthesizer:

<MiniRepl
client:idle
tune={`enableMotion()
// Create a simple melody
$:n("0 1 3 5").
.scale("C major")
// Use tilt (gravity) to control filter
.lpf(gravityY.range(200, 2000)) // tilt forward/back for filter cutoff
// Use rotation to control effects
.room(rotZ.range(0, 0.8)) // rotate device for reverb amount
.gain(oriX.range(0.2, 0.8)) // tilt left/right for volume
.sound("sawtooth")`}
/>

## Tips for Using Motion Controls

1. Use `.range(min, max)` to map sensor values to musically useful ranges
2. Consider using `.segment()` to smooth out rapid changes in sensor values

## Debugging

You can use `segment(16).log()` to see the raw values from any motion sensor:

```javascript
$_: accX.segment(16).log(); // logs acceleration values to the console
```

This is helpful when calibrating your ranges and understanding how your device responds to different movements.

Remember that device motion works best on mobile devices and may not be available on all desktop browsers. Always test your motion-controlled pieces on the target device type!
3 changes: 3 additions & 0 deletions packages/motion/index.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import './motion.mjs';

export * from './motion.mjs';
Loading