Testing and Contributing

Running Tests

Steno's core logic is tested from the Swift package at StenoKit/.

Run all tests:

cd /path/to/steno
swift test --package-path StenoKit

Run a focused test by suite name:

swift test --package-path StenoKit --filter WhisperCompatibilityTests

Other useful suites in the package include RepairAwareCleanupTests, NoSpeechGateTests, RuleBasedCleanupEngineAccuracyTests, HistoryStoreTests, WhisperCLITranscriptionEngineRichOutputTests, LaunchAtLoginMutationPolicyTests, and OverlayHitTestingTests. Browse StenoKit/Tests/StenoKitTests/ for the full list.

Smoke and Release Benchmarks

Run the smoke benchmark to check the evaluation setup:

# Smoke fixture (preflight, not release evidence)
scripts/run-smoke-benchmark.sh

The scripts require Python 3. Full release evaluation also needs the prepared LibriSpeech subset described in docs/release/release-eval.md in the selected app checkout. It expects librispeech-test-clean-0000.wav, 0006, 0007, 0008, 0010, 0024, and 0026 with that same prefix and .wav extension, matched to the embedded reference text. An arbitrary LibriSpeech directory is not equivalent.

To evaluate a model on your hardware:

STENO_WHISPER_CLI=/absolute/path/to/whisper-cli \
STENO_WHISPER_MODEL=/absolute/path/to/ggml-large-v3-turbo.bin \
STENO_VAD_MODEL=/absolute/path/to/ggml-silero-v6.2.0.bin \
STENO_LIBRISPEECH_ROOT=/absolute/path/to/librispeech_test_clean \
scripts/run-release-eval.sh

Smoke fixtures check the evaluation setup. Use the full corpus for release evaluation and record the source version, model, runtime, hardware, and limitations. Older compatibility results apply only to the versions and combinations they tested.

Release checks cover CI, security, packaging, signing, and notarization. Also test recording, text insertion, media controls, accessibility, and the minimum supported macOS version in the app. See the CI/CD Pipeline overview for the automated stages, release approvals, artifacts, and recovery procedures. The repository’s release checklist records the required app acceptance checks.

Before pushing app or package changes, run scripts/check.sh from the app repository. It regenerates the project with the pinned XcodeGen, confirms every hosted test file is in it, runs the package tests, builds the app, and runs the hosted tests. Use scripts/check.sh --clean after a shared struct or enum changes shape. It does not run the native runtime suites, the benchmark, or packaging; CI runs those.

To run the app checks individually after generating the project:

xcodegen generate
xcodebuild test -project Steno.xcodeproj -scheme Steno -destination 'platform=macOS' CODE_SIGNING_ALLOWED=NO
xcodebuild build -project Steno.xcodeproj -scheme Steno -destination 'platform=macOS' CODE_SIGNING_ALLOWED=NO

Coding Standards

Follow the app contributor guide:

• Swift 6 strict concurrency
• protocol-first design for core services
• no force unwraps in production logic
• no singleton shortcuts for mutable core state
• accessibility labels on interactive UI controls
• reduce-motion-aware animation behavior

For UI consistency in the app target, use StenoDesign tokens rather than hardcoded visual values.

Pull Request Checklist

Before opening a PR:

• code builds cleanly with xcodegen generate followed by an Xcode build
• relevant swift test --package-path StenoKit suites pass
• new logic has tests in StenoKit/Tests/StenoKitTests/
• permission-sensitive changes are manually validated on macOS
• project.yml updates are followed by xcodegen generate
• commit messages are clear and scoped

Contribution Paths

Areas to contribute:

• insertion reliability for more app classes
• local cleanup quality (repair phrases, literal preservation, prompt contamination, no-speech gating)
• compatibility-matrix entries for additional validated hardware/model rows
• benchmark and release-eval tooling in StenoBenchmarkCore
• docs and troubleshooting improvements

Start with a small, reproducible issue and include validation steps in your PR description.