Skip to main content
SoftsasiAI for Engineers
Sign in

§ masterplan.md

Synesthesia Sound Lab

An unconventional audio experiment where users draw shapes that turn into generative music and color bursts.

All ideas
Unconventional

01

App overview and objectives

A browser instrument that turns sound into visual art in real time: hum, play or upload audio and watch it become a composition you can save, tweak and print. It should feel like an instrument, not a settings panel.

02

The problem worth solving

Music visualisers are decorative and disposable. Making the output ownable — saveable, printable, shareable — turns a toy into a creative tool people return to.

03

Target audience

  • Primary — Musicians and generative-art hobbyists looking for cover art and visuals.
  • Secondary — Teachers demonstrating frequency, amplitude and harmony to students.

04

Roles and permissions

  • Visitor — read-only public pages, can sign up.
  • Member — owns their own records, cannot see other users' data.

05

Core features

  • Microphone and file input with live frequency analysis
  • Several mapping presets (frequency to colour, amplitude to form, tempo to motion)
  • Parameter panel for palette, density, symmetry and decay
  • Freeze and export the current frame as high-resolution PNG or SVG
  • Save compositions to a gallery with their parameters
  • Share link that replays the composition from the saved audio fingerprint

Deliberately later

  • Video export of a full track
  • MIDI device input
  • Print-on-demand posters

06

Technical stack

React shell, Web Audio API for analysis, Canvas or WebGL for drawing, all client-side; a small backend only for saved galleries.

Why: Audio processing must be local for latency and privacy — nothing should be uploaded to visualise a hum.

Alternatives: Server-side rendering for higher-quality output (better prints, slow and costly) — best kept as a paid export option.

07

Conceptual data model

Composition — owner, title, parameters, thumbnail, source type, created_at

Preset — name, mapping rules, default palette, author

ExportJob — composition, format, resolution, status

User — account, saved compositions, plan

08

Integrations

  • File storage for exports
  • Optional payments for high-resolution or print
  • Analytics on preset popularity

09

UI design principles

  • Break the grid on purpose: overlapping layers, oversized type, one surprising interaction per screen.
  • Motion carries meaning — elements should morph between states instead of hard-cutting.
  • Keep exactly one experimental idea per page so the interface stays usable, not chaotic.

10

Security considerations

  • Microphone access requested only on explicit user action, with a visible recording indicator.
  • Audio never leaves the device unless the user explicitly saves a recording.
  • Signed URLs for exported files.

11

Development phases

Phase 1 — Prove the core

  • — Microphone and file input with live frequency analysis
  • — Several mapping presets (frequency to colour, amplitude to form, tempo to motion)
  • — Parameter panel for palette, density, symmetry and decay
  • — Static content and design system in place
  • — Basic analytics

Phase 2 — Make it real

  • — Freeze and export the current frame as high-resolution PNG or SVG
  • — Save compositions to a gallery with their parameters
  • — Share link that replays the composition from the saved audio fingerprint
  • — Accounts, sign-in and password reset
  • — Empty, loading and error states everywhere

Phase 3 — Polish and launch

  • — Performance, accessibility and SEO pass
  • — Legal pages, contact route and 404 handling
  • — Wire up: File storage for exports
  • — Wire up: Optional payments for high-resolution or print

Phase 4 — Grow

  • — Video export of a full track
  • — MIDI device input
  • — Print-on-demand posters

12

Challenges and solutions

Risk — Frame rate collapse on low-end phones

Solution — Adaptive quality — reduce particle count and resolution automatically when frames drop.

Risk — Output looks samey

Solution — Ship several genuinely different mapping algorithms rather than one with sliders.

Risk — Browser autoplay and mic permission friction

Solution — A clear one-tap onboarding that explains why access is needed.

13

Future expansion

  • Live VJ mode for events
  • Collaborative jam sessions
  • Plugin for streaming software

14

Page list (13 pages)

  1. 01 PUBLIC Lab — The instrument: input, live canvas, parameter panel.
  2. 02 PUBLIC Gallery — Saved and community compositions.
  3. 03 PUBLIC How It Works — The audio-to-visual mapping explained simply.
  4. 04 PUBLIC Save — Naming, export format and resolution choices.
  5. 05 PUBLIC Share — Public playback page for one composition.
  6. 06 AUTH Sign up — Create an account with email or a social provider.
  7. 07 AUTH Log in — Return to the account, with error and lockout states.
  8. 08 AUTH Reset password — Request a reset link and set a new password.
  9. 09 APP (signed in) Account settings — Profile, email, password, language and delete account.
  10. 10 PUBLIC Contact — Contact form plus real address, phone and email.
  11. 11 LEGAL & SYSTEM Privacy policy — What data is collected, why, and how to remove it.
  12. 12 LEGAL & SYSTEM Terms of service — Rules of use, liability and account termination.
  13. 13 LEGAL & SYSTEM 404 not found — Friendly dead end with search and links back.

15

Page map

PUBLIC           AUTH               APP (signed in)      LEGAL & SYSTEM
───────────────  ─────────────────  ───────────────────  ───────────────────
├─ Lab           ├─ Sign up         └─ Account settings  ├─ Privacy policy
├─ Gallery       ├─ Log in                               ├─ Terms of service
├─ How It Works  └─ Reset password                       └─ 404 not found
├─ Save
├─ Share
└─ Contact

key flows:
  Lab ──▶ Save
  Save ──▶ Gallery
  Gallery ──▶ Share
  How It Works ──▶ Lab