Structure through rhythm.
Small, distinct tactile patterns.
Research ↗Haptic communication / Wearable computing
Fermata
ABA shared signal.
An individual choice.
An individual choice.
A quiet cue.
A human reply.
Gesture recognition, tactile feedback
and a deliberate connection.
On the wrist
Stay with
the music.
Stay in touch.
A wrist movement sends a cue. A vibration asks for attention. A button leaves the reply to its wearer.
把“设备收到了”和“人回应了”分开设计。
2
paired wearables
3
intentional messages
6
motion channels
- My work
- Interaction, firmware & ML, recording tools, enclosure iteration, integration and testing.
- Collaboration
- Later carrier PCB supplied by a collaborator; reviewed and integrated.
- Current pair
- A v326 / B v328 · v322 recognizer. Working prototypes; recognition in use remains under evaluation.
01 Background → design decisions
Less vocabulary.
More intention.
Common Ground’s interest in nonverbal musical coordination carries forward into a direct exchange between two people.
Keep attention in the task.
movement → cue → reply
Deliberate requests and responses.
Research ↗The body changes the signal.
fit ↔ contact ↔ feeling
Evaluate electronics on the wrist.
Process ↗Design hypotheses to test. Earlier audience observation and room-scale lighting remain part of Common Ground, outside this wearable’s scope.
02 Interaction score
Send. Feel.
Choose to reply.
A packet ACK confirms delivery.
A person’s reply confirms intention.
Interactive illustration. It does not connect to a wearable.
Interaction scoreIllustrative sequence
Player A
Ready to send
A shared vocabulary
Choose a cue to beginPlayer B
Listening
Send a cue, then decide when Player B replies.
Attention
Wrist-turn gesture or attention button.
Wait
Downward-motion gesture.
Received
A deliberate button reply, distinct from the BLE acknowledgement.
Attention
Two pulses · blue
Wait
One pulse · amber
Received
One pulse · green
Receiver vocabulary. Pulse marks are schematic; feedback requires enabled output and a calibrated driver.
03 Electronics & signal paths
From a movement
to a sensation.
Sensing, communication and feedback share one small object.
The real constraint is their interaction: power, timing, antenna space and motor vibration.
A circuit you can read as a conversation.
Functional wiring / v328 sourceFunctional wiring, v328. Six LEDs run at brightness 12/255 on 3V3; long-term supply margin remains unverified.
6-axis IMU
LSM6DS3TR-C
104 Hz configuration
nRF52840
XIAO Sense
On-device inference + BLE
DRV2605L
I²C 0x5A
1040 LRA actuator
6 LEDs + 2 buttons
D1 light
D9 Attention / D10 Received
Component roles & interface notes +
| Part | Role / interface | Design or verification note |
|---|---|---|
| XIAO nRF52840 Sense | IMU + Bluetooth + control | Motion sensing on the body; firmware shares the sensing and feedback schedule. |
| DRV2605L + 1040 LRA | I²C / tactile output | Current hardware differs from the earlier DRI0056 proposal. Startup calibration and feedback masking matter. |
| Two buttons | D9 / D10, active-low input | Direct attention and human acknowledgement remain available alongside gestures. |
| Six addressable LEDs | D1 / 3V3 / GND | Brightness 12/255 in the source configuration; prototype setting, not a validated supply specification. |
| Battery + slide switch | Battery-path switching | Battery voltage in the UI is a reading, not a calibrated charge percentage. |
| Carrier + printed enclosure | Mechanical integration | Reserve openings, wires, antenna space and accessible controls before judging the fit. |
04 Electronic assembly
Inside the
wearable.
The carrier locates the controller and controls. The complete wearable also needs its haptic driver, light strip, actuator, battery and interconnects.
Controller + IMUImported module geometry
DRV2605L + LRADriver and actuator
6-pixel light stripIndependent light output
Battery + leadsPower and interconnects
Two levels of evidence. The original carrier comes from the collaborator’s CAD. Added modules follow photographs; battery and LRA dimensions were supplied by the author. Driver dimensions and LED spacing remain approximate. Leads are schematic.
Model provenance ↗The light has a place to sit
Support below.
Diffusion above.
The original bridge transfers the strip’s load to two ledges in the shell. A separate diffuser sits beneath the light window.
- WindowThe opening in the printed lid.
- Diffuser0.8 mm in the archived model.
- Six-pixel stripAligned to the window, supported below.
- Bridge → shell ledgesTwo feet locate the strip’s height.
Nominal mounting heights follow the archived CAD. The component envelopes still need a physical fit check.
The enclosure follows the assembly.
Component volume, cable access and repeatable closure belong to the same design.
EARLY CARRIER46 × 30 mm
→Earlier driver & pin map
COLLABORATIVE PCB≈ 43 × 33 mm
→Revise firmware & openings
ON-BODY ASSEMBLYContact matters
Wiring, restraint & fit checks
CAD studies document their own revisions. The final presentation records a snap-latch choice; later clearance variants are CAD/slicer checked, without retained physical assembly verification.
05 Embedded machine learning
Continuous motion.
Discrete intention.
Before classifying a gesture, the system must decide which part of the stream belongs to it.
- 01
Sense
6 channels
104 Hz configured - 02
Segment
Motion onset + quiet tail
Quality rejection - 03
Describe
128 resampled frames
58 features - 04
Infer
96 decision trees
3 class scores - 05
Gate
Score ≥ 0.65
Consume + cooldown
The 58-value description
42 / Six channels × seven statisticsMean, variation, extrema, absolute magnitude, crossings and differences.
7 / Window & stroke structureDuration, stroke counts and relative angles.
9 / Motion & stillnessAxis energy, magnitude and quiet-time features.
Random forest / structural illustration
Score bars illustrate the architecture, not a recorded prediction. A v326 / B v328 share the v322 model.
C++ / RfCore.hExact inference excerpt
unsigned until = tree + 32 < RfModel::TREES
? tree + 32 : RfModel::TREES;
for (; tree < until; tree++)
RfModel::tree(tree, features, probability);
if (tree < RfModel::TREES) return;Share the processor. Evaluate up to 32 trees per tick, giving sensing and BLE time to continue.
Python / aligned_windows.pyExact feature excerpt
grid = np.linspace(tt[0], tt[-1], FRAMES)
resampled = np.stack([
np.interp(grid, tt, vv[:,d]) for d in range(6)])
centered = resampled - resampled.mean(
1, keepdims=True)
centered[:3] /= 400.
duration = (tt[-1] - tt[0]) / 1e6
basic = features(
centered[None], np.asarray([duration]))[0]Match the representation. Resample to 128 time positions, center each channel and retain a fixed physical scale.
Offline training loop / explicit deployment
- CaptureLabel performed actions
- PartitionKeep whole sessions together
- FitWeight correlated windows
- ReplayReview misses & extra events
- ExportDeploy the selected C++ model
Training negatives misclassified as gestures receive extra weight in one fixed pass. Validation/test examples do not supply this pass. New recordings never retrain the bands automatically.
Physical feedback → recognition guard
The motor is also motion.
Continue saving raw samples. Exclude vibration-affected samples from recognition.
Raw log
Motorplayback
Recognitionexcluded200 msresume →
Independent event guard: consume the accepted movement + 800 ms cooldown.
Reject a broken windowGap > 30 msRaw clipping ±32000Haptic mask
06 Read the recording
See what the
model actually saw.
A recorded stream connects the movement, the annotation and the device’s decision. It also exposes pauses, feedback and incomplete attempts.
A mark is a human observation.
Group onset and cycle-end markers stay separate from detections.
Arrival time is not motion time.
Device time, host receipt time and browser input time are retained separately.
A detection is not a sensation.
Follow the event to the remote driver, then check what the wearer felt.
07 Recording & analysis system
A workbench for
two moving bodies.
A local Python service and browser panel connect live streams, action labels, transport events and reproducible session files.
Read both wrists together.
Two angular-velocity plots use the same vertical range and a 12-second view. Device state sits beside the signal.
Original UI pixels / A v326 + B v328 / historical capture
Label the action you performed.
Select the recording band and gesture. Human markers remain separate from the model’s detections.
EnterGroup onset
SpaceCycle 1 ends
SpaceCycle 2 ends
Capture protocol from the source. The selected group below retains two Space marks and no explicit onset mark.
Keep the packet’s identity intact.
Session, sequence and code link a request to its ACK. Output and human response require their own evidence.
Session 1957751726Sequence 8Code 1 · Attention
A
REQUESTCreatedA
TX_ATTEMPTAttempt 1A
ACK_DEVICE_DELIVERYPeer acceptedRendered ledger from the selected A USB record / A’s device clock. B playback and a human reply are absent from this file.
Follow one recorded group.
A v326 / v322 recognizer / USB desktop TestSelect an event to connect the motion, the operator’s mark and the device’s own record.
Gyro XGyro YGyro Z729 raw samples / 7 seconds
Cycle 1 marked
The operator marks a completed cycle. This records an observation; it does not trigger recognition or feedback.
Human annotation
cycle_end / cycle 1 latest sample: 745347656 μs source: Space key
One selected successful group from session 20261019_142008_test_cc691f. Times use A’s device clock and an excerpt origin of 743 s. Marker positions include human delay; they do not measure recognition latency.
Two links. Two different responsibilities.
Architecture / B v328WEARABLEASense · infer · feel
↔Partner BLEEVENT / ACK
WEARABLEBSense · infer · feel
A debug stream ↘↙ B debug stream
Local Python serviceBrowser panelIndependent debug subscriptions / USB or BLE capture
The partner link works without the computer. B routes notifications by subscription handle; connection order is not a reliable identity. Debug disconnection was checked without resetting the pair.
Capture protocol / one annotated two-cycle group
- RecordSelect A or B
Train / Test / Daily motion - EnterGroup onset
Prepare to move - SpaceCycle 1 ends
Keep the stream running - SpaceCycle 2 ends
Pause before next group - SaveClose files
Hash & export
Markers never trigger the motor or fake a detection. Human reaction and buffering delay remain in the record; annotations require review.
Capture files / availability varies by session version
imu.csvRaw stream + device eventssample_timing.jsonlDevice ticks ↔ host receiptgroup_onsets.jsonlStart of intended groupmarkers.jsonlCycle ends + undo historypair_events.jsonlPaired debug events / when availablesession.jsonRole, units, pose, quality countssha256.jsonSaved-file integrityPython / server.pyExplanatory pseudocode
now = time.monotonic()
raw.write(line + "\n")
timing.write(device_us, now)
if clock_reset or large_gap:
save_and_stop("clock discontinuity")
# Label the action, not the model output.
marker = {"type": "cycle_end",
"host_mono": now,
"last_sample": latest_sample}Stop on device-clock reset or a gap above one second. Preserve timestamps instead of pretending the clocks are synchronized.
Device μsMotion order & sampling intervals
Host monotonicReceipt order & pair-event correlation
Browser event timeWhen the operator pressed a marker
Analysis safeguards & known limits +
Two thresholds, two jobs. A gap above 30 ms invalidates a recognition window. A gap above 1 s or device-clock reset stops and saves the recording.
Separate provenance. A sessions and B sessions have separate directories; B recordings are not automatically added to the default training set.
Preserve missing evidence. Old firmware without output events leaves feedback unconfirmed. Cached battery values are marked as old readings.
Measure before claiming latency. Human marker delay and USB/BLE buffering are not corrected; these logs do not establish millisecond end-to-end latency.
08 Three engineering investigations
Trace the failure.
Check the change.
Each investigation connects a symptom to retained evidence, an implementation decision and a bounded result.
The model saw
a different window.
A classifier that worked on clean clips missed every target group in continuous replay.
Manual crop3.810 stwist score 0.866
Runtime window2.010 stwist score 0.438
One archived diagnostic group. A changed boundary produces a changed feature vector.
Evidence
0/8 groups matched. Three other twists produced no valid candidate window.
Change
Use the same causal window generator and 58-feature representation for training and runtime replay.
Checked outcome
7/8 matched, with three unassigned outputs. Window labels and model fitting changed together.
Historical development replay, not a blind test or proof that the classifier alone caused the improvement.
Keep the raw signal.
Guard the recognizer.
A motor and an IMU share one small enclosure. Playback must not masquerade as intentional motion.
Historical v316 replay1 = 1masked / unmasked accepted twists
120-second raw record12,513samples retained / 54 marked as masked
// Exact predicate, line-wrapped / Hardware.h
bool masked(uint32_t now) const {
return busy_ || !Fermata::elapsed(now, ended_, 200);
}Evidence
Later misses occurred after masking ended. Removing the flags produced an identical replay log.
Change
Keep every raw sample and its mask flag. Exclude playback and 200 ms recovery from recognition.
Checked outcome
The guard remained inspectable. This check did not show an accuracy gain or explain the later misses.
The controlled replay comparison is from the historical v316 DTW-era system. Current v322 logic retains the raw-recording/masking separation.
Received by the radio.
Not yet felt by a person.
Earlier Attention requests were acknowledged even when the receiver was not ready to vibrate.
11 October / v309ACK + NO_HAPTICThree requests acknowledged. Blue light was confirmed in two formal actions; flags reported unavailable haptics.
19 October / v3263 A + 4 B pressesSeven physical Received-button events matched packet, light and haptic evidence.
One retained button exchange / separate from the gesture recording
A
BUTTON_RECEIVED → REQUESTB
RX_ACCEPTED → FEEDBACK_HAPTIC_STARTEDA
Read the matched event ledger ↗ACK_DEVICE_DELIVERYEvidence
Delivery, motor playback and a felt cue require different checks.
Change
Retain calibration gates; correlate output logs by session / sequence / code. Keep the reply button separate from ACK.
Checked outcome
Across seven button tests, the author confirmed vibration and green light in both directions.
Different dates and firmware. These button checks do not prove every gesture cue was felt and are not a user study. Evidence and exact source excerpts ↗
Other integration checks
Radio reconnect loop
Clear stale RX / TX / CCCD handles. A 60.25 s follow-up had no drops; power and placement also changed.
Sensor bus held low
Seeed’s OUTPUT_H0H1 drive mode restored identification and approximately 104.33 Hz sampling in a short check.
Fit in CAD, fail in hand
Failed nut seating and a reversed button made fit, travel and reopening separate assembly checks.
09 Results & limits
An improvement.
With a tradeoff.
The v325 candidate improved the development replay, while validation matches fell. It was not the model deployed on the photographed pair.
Physical reply exchanges
Logged bidirectional button messages; author confirmed vibration and green light.
Recordings checked for parity
Python / C++ candidate replay consistency. This does not measure real-device latency.
Recognition on the body
Later worn tests still include misses and disconnection. Fit remains part of the sensing problem.
Planned demonstration / filming storyboard
One request.
A chosen reply.
Follow the four shots planned for the live demonstration. This is an interaction storyboard; footage has not yet been recorded.
Framing studyA’s wrist / close view
Shot 1 / 4
A asks for attention.
Keep A’s wrist and the shared activity in view. Film one deliberate two-cycle wrist-turn gesture.
Schematic framing / planned sequence
Physical prototype
Author photograph
The live take will keep the complete exchange in one continuous master shot. A filmed light or packet ACK alone will not stand in for the wearer’s tactile confirmation.
The study, kept open
Keep the signal clear.
Keep the response human.
Reading & precedents
Musical Haptic WearablesTurchet & Barthet / 2019TactonsBrewster & Brown / 2004MiMU mappingMovement, events and feedbackShape HapticsMaking a mechanism legibleCLUE BLE MIDI GloveConnecting components, code and useITP documentationProcess, systems and evidencePrecedents inform design and documentation choices. Their results are not presented as Fermata’s findings. Project evidence and asset provenance are indexed in the source notes.