I'm a music researcher, software engineer, and producer. I made Story Jam.
This doc is intended to inform you of not just the product, but the centuries of work that have led up to its implementation.
> Are you tired of the barriers in traditional music composition? Story Jam is here to break them down.
> Designed for anyone with creative ideas — from poets to film directors — our tool offers a new way to create and edit chord progressions, powered by cutting-edge music theory.
The UI is inspired by https://neal.fun.
Neal has fun, simple, addictive, and intuitive designs and I hope Story Jam has that too.
*Who Story Jam is for: Storytellers*
Story Jam makes music composition accessible and meaningful to anybody, with or without musical training.
It is designed for those who crave musical control but struggle with traditional composition methods. This includes film directors, slam poets, and self-taught musicians.
Story Jam is not music production software.
Do not expect fancy sounds or synthesizers. It's purely a composition tool, designed to spark your creative process.
*Try it out!*
The demo is free on the homepage, no login required!
This is an MVP, so it has an "introductory" feature set. Feature requests welcome; help me build the product you want.
This service is built on a novel new music theory I have developed called Monic Theory.
Monic Theory is a rigorous proof for music. Not "Western music": music.
Monic Theory describes the tonal space of any conventional music on earth (except noise music. For that just use `Math.random()`).
It describes the static and transient function of chords, instantaneously and differentially over time. This model enables empirical measurement of chords and the relationship between chords.
(hint: It is nothing you have seen in Xenharmonic Alliance. This is a new approach I have been developing over the past 10 years.)
Therefore, Monic Theory enables us to describe (or "predict" if you will) a chord progression to invoke a certain feeling.
*Music Composition*
Three people who helped set up the environment for Monic Theory are composers Paul Hindemith and Harry Partch , and music theorist Heinrich Schenker.
These folks independently contributed new ideas to music composition and analysis. All of these people lived without access to rapid computation. This is critical for the Partch case, as he computed many tables of frequencies by hand to support his compositional technique.
Partch recognized the human-math-music relation in "Genesis of a Music." He includes in this text some samples of his hand-computed tables of frequency values of overtones and (importantly) undertones which support the basis is technique.
Partch's techniques were so far-fetched that he had to construct new instruments to perform his scores. Similarly, I had to build a digital synthesizer to render the output of Monic Theory. (See: https://github.com/ckmahoney/raudio).
*About me*
I was a working composer and violinist from 2007 until 2017, and I have been a software engineer for the past 7.5 years.
*Livecoding*
I was a volunteer organizer for Livecode.NYC, an NYC livecode community; and am the volunteer creator of Data Dancers, Atlanta's livecode community. I am passionate about algorithmic art and have provided about a dozen workshops over four years on the topic.
The request is valid; you just need the right tools for the job.
Story Jam lets you design chord progressions without needing to know about music theory, instead offering intuitive terms like "lightness", "darkness", "drifting" and "roaming". They mean about what you think they mean.
The first 80s song I heard was a literal copy of Phil Collins. But there are no emotions attached to it (for me), and the lyrics are random. It’s more like supermarket background music IMHO, not something I would pay for, especially when we have centuries of music to discover already, why make fake stuff like that?
Edit: I have just heard the funniest most ridiculous metal song ever without a touch of metal inside. Breathe of Death, it’s like a bad joke.
If thats the future of anything, I’m going back to plain C (code) when I retire and I’ll never approach the internet ever again.
In my opinion training on all music is no more theft than Taylor Swift listening to the radio growing up (as long as we don't regurgitate existing songs which would be bad and useless anyway). I think an alternative legal interpretation where all of humanity's musical knowledge and history are controlled by three megacorporations (UMG/Sony/Warner) would be kinda depressing. If the above is true we might as well shutdown OpenAI and delete all LLM weights while we're at it, losing massive value to humanity.
It’s intellectual property laundering. A company selling a button that launders the blood sweat and tears of generations of artists is not the same as a person being inspired and dedicating themselves to mastery.
Humans create value. AI consumes and commoditizes that value, stealing it from the people and selling it back to their customers.
It’s unethical and will be detrimental in the long run. All profit should be distributed to all artists in the training set.
It won't be detriment to consumers who ultimately decide the value. If I could AI gen a better tasting cocacola for cheaper that would be beneficial to consumers and coke wouldn't deserve a cut. Get gud, as they say.
> In my opinion training on all music is no more theft than Taylor Swift listening to the radio growing up (as long as we don't regurgitate existing songs which would be bad and useless anyway).
I beg of you, speak to some real life musicians. A human composing or improvising is not choosing notes based on a set of probabilities derived from all the music they’ve heard in their life.
> I think an alternative legal interpretation where all of humanity's musical knowledge and history are controlled by three megacorporations (UMG/Sony/Warner) would be kinda depressing.
Your impoverished worldview of music as an artistic endeavor is depressing. Humanity’s musical knowledge extends far beyond the big 3.
> If the above is true we might as well shutdown OpenAI and delete all LLM weights while we're at it
Now we’re talking.
> losing massive value to humanity.
Nothing of value would be lost. In fact it would refund massive value to humanity that was stolen by generative AI.
The difference being that a musician being influenced by other musicians still has to work to develop the skills necessary to distill those influences into a final product, and colors that output with their own subjective experiences and taste. This feels like a conveniently naive interpretation to justify stealing artists' work and using it to create derivative generative slop. The final line in your comment is pretty telling of how seriously you take this issue (which is near-universally decried by artists) -- some other massive company is doing a bad thing, so why shouldn't I?
edit: I have to add how disingenuous I find calling out corporations owning "all of humanity's musical knowledge and history" as if generative AI music trained on unlicensed work from artists is somehow a moral good. At least the contracts artists make with these corporations are consensual and have the potential to yield the artist some benefit which is more than you can say for these gen-AI music apps.
I don't see how the amount of work that went into it changes the core fact that all art is influenced by that which came before, and we don't call that stealing (unless you truly believe that "all art is theft").
My point re: LLMs wasn't meant to exclusively be a "they're doing it" one, the hope was to give an example of something many people would agree is super useful and valuable (I work much faster and learned so much more in college thanks to LLMs) that would be impossible in the proposed strict interpretation of copyright.
edit responding to your edit:
Re: moral good: I think that bringing the sum of human musical knowledge to anybody who cares to try for free is a moral good. Music production software costs >$200 and studios cost thousands and majoring in music costs hundreds of thousands, but we can make getting started so much easier.
Is it really consent for those artists signing to labels when only three companies have total control of all music consumption and production for the mass market? To be clear, artists absolutely have a right to benefit from reproduction of their recordings. I just don't think anyone should have rights to the knowledge built into those creations since in most cases it wasn't theirs to begin with (if their right to this knowledge were affirmed, every new song someone creates could hypothetically have a konga line of lawyer teams clamoring for "their cut" of that chord progression/instrument sample/effect/lyrical theme/style).
I think we intuitively allow for artists to derive and interpolate from their influences because of a baseline understanding that A) it is impossible to create art without influence and B) that there is an inherent value in a human creating art and expressing themselves. How that relates to someone using unlicensed music from actual humans to train an AI model in order to profit off of the collective work of thousands of actual human artists, I have no idea.
edit:
> I think that bringing the sum of human musical knowledge to anybody who cares to try for free is a moral good
Generative AI music isn't in any way accomplishing this goal. A free Spotify account with ads accomplishes this goal -- being able to generate a passable tune using a mish-mash of existing human works isn't bringing musical knowledge to the masses, it's just enabling end users to entertain themselves and you to profit from that.
> Is it really consent for those artists signing to labels
Yes? Ignoring the fact that there are independent labels outside the ownership of the Big Three you mention, artists enter into contracts with labels consensually because of the benefits the label can offer them. You train your model on these artists' output without their consent, credit or notification, profit off of it and offer nothing in return to the artists.
A) Agreed! B) So I guess the argument here is that this doesn't apply to AI music. I think that if someone really pours their soul into the lyrics of a song and regenerates/experiments with prompts until it's just right, and maybe even contributes a melody or starting point that's still a human creating art and expressing themselves. It's definitely not as difficult as creating a song from scratch, but I've been told similar arguments were made regarding whether photography was art when that became a thing.
btw, if the user of the AI doesn't do any of the above then I think the US copyright office says it can't be copyrighted in the first place (so no profiting for them anyway).
> if the user of the AI doesn't do any of the above then I think the US copyright office says it can't be copyrighted in the first place (so no profiting for them anyway).
Am I understanding right that the point here is that while you are able to get away with using copyrighted material to turn a profit, your end users cannot, so no worries?
1. Anthropomorphizing the kind of “influence” and “learning” these tools are doing, which is quite unrelated to the human process
2. Underrepresenting the massive differences in scale when comparing the human process of learning vs. the massive data centers training the AI models
3. Ignoring that this isn’t just about influence, it’s about the fact that the models would not exist at all, if not for the work of the artists it was trained on
> Is it really consent for those artists signing to labels when only three companies have total control of all music consumption and production for the mass market?
This premise is false. I have made plenty of money busking on the street, for example. Or selling audio recordings at shows.
> {o be clear, artists absolutely have a right to benefit from reproduction of their recordings.
This is correct. Artists benefit when you pay them for the right to reproduce. When you don't (like what you are doing), you get sued. Here's a YouTube video covering 9 examples:
> I have made plenty of money busking on the street
That's why I specified mass market. However, given a choice between literally being on the street and working with a record label I'd probably choose the label, though I don't know about others.
> pay them for the right to reproduce
My point is learning patterns/styles does not equate to reproducing their recordings. If someone wants to listen to "Hey Jude" they cannot do so with our model, they must go to Spotify. There are cases where models from our competitors were trained for too long on too small a dataset and were able to recite songs, but that's a bug they admit is wrong and are fighting against, not a feature.
> in most cases it wasn't theirs to begin with
In most cases they did not invent the chord progression they're using or instruments they're playing or style they're using or even the lyrical themes they're singing. All are based on what came before and the musicians that come after them are able to use any new knowledge they contribute freely. It's all a fork of a fork of a fork of a fork, and if everyone along the line decided they were entitled to a cut we'd have disaster.
Law should be considered to be artificial rules optimized for the collective good of society.
What's the worst that can happen if we allow unregulated AI training on existing music? Musician as a job won't exist anymore lest for the greatest artists. But it makes creating music much more accessible to billions of people. Are they good music? Let the market decide. And people still make music because the creative process is enjoyable.
The animus towards AI generated music deeply stems from job security. I work in software and I see it is more likely that AI can be eventually able to replace software devs. I may lose my job if that happens. But I don't care. Find another career. Humanity needs to progress instead of stagnating for the sake of a few interest groups.
I don't work as a musician so it's nothing to do with job security -- I think that using artists' output without their consent in order to train a soulless AI model for some tech middleman to profit from is repugnant, and the cheap rhetoric about democratizing music and "bringing music to the masses!" adds insult to injury. I can guarantee if OP's intellectual property was violated in this project, like somebody ripping off their model or trademark, they'd be suing, but they conveniently handwave away mass scale IP theft when it comes to musicians.
I’m skeptical about how much value AI art is going to really contribute to humanity but as a lifelong opponent of copyright I have to roll my eyes when I see people arguing against it on behalf of real artists, all of whom are thieves in the best case and imitators in the worst.
Yeah every musician has a story of writing a new song, bringing it to the band, and they say "oh, this sounds just like [song]." It's almost impossible to make something truly novel.
But beyond the originality !== novelty discussion, I'm not sure how we've come to equate 'creativity' (and the rights to retaining it) to a sort of fingerprint encoding one's work. As if a band, artist or creator should stick to a certain brand once invented, and we can sufficiently capture that brand in dense legalese or increasingly, stylistic prompts.
How many of today's artists just 'riffing' off existing motifs will remain, if the end result of their creative endeavours will be absorbed into generative tools in some manner? What's the incentive for indies to distribute digitally, beyond the guarantee their works will provide the (auditory) fingerprints for the next content generation system?
I have written and performed many songs over many bands. At no point did anybody compare my work to any other artist's work, because it is genuinely unique.
Megacorporations owning copyrights to the majority of IPs(music, games, etc.) is a capitalism/monopoly problem. How does getting rid of copyright and allowing your company to profit off other peoples work in any way solve that issue?
no one can actually explain the value OpenAI adds to humanity. What massive loss? What have we gained from this entity other than another billionaire riding a hype cycle?
These high-quality music models require pirating many, many terabytes of music. Torrents are the main way to do it, but they likely scraped sites like Bandcamp, Soundcloud and YouTube.
AI music is a weird business model. They hope that there's enough money peddling music slop after paying off the labels (and maybe eventually the independent music platforms) whose music you stole. Meanwhile, not even Spotify can figure out how to be reliably profitable, serving music people want to hear.
When I use an FFT to view the spectrogram on YouTube music videos, it is very obvious that YouTube applies a lowpass filter at 16kHz on all videos (true since 2023 at least).
While this does retain the majority of useful information, it explains why the youtube version of your song feels just a little more 'lifeless' than the high quality version you have elsewhere.
The original recording contains high frequency detail that got lost. Your human body uses that high frequency detail to orient itself in space with respect to sound sources (like reverb, reflections, or ambient sounds).
It is interesting from a data storage point of view because this could result in massive savings. Consider audio is recorded at 44.1khz or 48kHz but is actually stored at 32kHz. They have effectively saved 25% in audio file storage at marginal customer experience.
> it explains why the youtube version of your song feels just a little more 'lifeless' than the high quality version you have elsewhere
Having hearing sensitivity over 16 kHz is unusual. If you're under 15 years old and kept your ears pristine by not listening to loud noises, you might be able to hear it. Older people are out of luck.
Moreover, even if you can hear above 16 kHz in loud pure tones, there is so little content in real audio/music above 16 kHz that it makes no practical difference.
> massive savings ... effectively saved 25%
Not really. Going from a 48 kHz sampling rate to 32 kHz is indeed 2/3× the size for uncompressed PCM audio. But for lossily compressed audio? Not remotely the same. Even in old codecs like MP3, high frequency bands have heavy quantizers applied and use far fewer bits per hertz than low frequency bands. Analogously, look at how JPEG and MPEG have huge quantizers for high spatial frequencies (i.e. small details) and small quantizers for broad, large visual features.
It is true that your ears do not perceive the higher frequencies in the same way (through pitch detection). However, if you put on a headset and apply only frequencies above 16kHz, you will distinctly notice a change in the pressure in your headset's ear cups.
Good point about the savings. I was using uncompressed format as the reference, but it is indeed unlikely that YouTube serves out lossless audio.
I also should have used the word "delivery" instead of data storage. Those are two separate problems: where the original asset is stored (and how, if they don't store raw originals), and also how the asset is delivered over the web.
> However, if you put on a headset and apply only frequencies above 16kHz, you will distinctly notice a change in the pressure in your headset's ear cups.
If you put something above 16 kHz at full scale and/or if you play it extremely loud then maybe. With typical music content at typical volumes, I doubt it.
> When I use an FFT to view the spectrogram on YouTube music videos, it is very obvious that YouTube applies a lowpass filter at 16kHz on all videos (true since 2023 at least).
Maybe with a browser that doesn't support Opus and gets AAC instead (Safari?). With Firefox or Chromium on Linux I get up to 20 kHz, which by design is the upper limit in Opus codec.
First time I have ever heard someone say my Minimoog, OB8, Prophet and modular synths weren't synthesis.
ADSR is subtractive even if you ignore the filter.
The (ideal) square wave contains the odd-integer harmonic frequencies, where the (ideal) sawtooth has all harmonic frequencies.
I think starting in the digital world may make this less clear?
You are subtracting overtones from a non-sinusoidal set, the sound synthesis in subtractive synths is the more like choosing digits to construct a representable number.
Additive synths are actually far more restricted...remember that the set of computable numbers is not quite as small as the cantor set, but is getting there.
Hold up, I'm going to send a email to every synth company that sells synths with filters and explain to them that they aren't selling synthesizers but transformers. I'm positive that it will be received well!
So I presume your complaint is that by synthesis you mean taking two things, smashing them together, and producing a new thing. In which case, sure, subtractive synthesis isn't synthesis unless:
- Two oscillators undergoing detune, sync, ring or amplitude modulation, or fm prior to getting fed into the filter?
- An LFO combined with an oscillator?
- An envelope (controlling the filter or amplifier) combined with an oscillator?
Perhaps these things might be considered combinations? I agree this is weak. You can blame the RCA Mark I and II for calling subtractive synthesizers "synthesizers".
Yes, but all synthesis types are transformation unless you are just replaying/outputting a waveform in one way or another without manipulation so that is really an all encompassing way of describing all synthesis methods.
Subtractive synthesis has a particular meaning in common use whether it’s right or wrong.
Somewhat true … maybe it’s really a hybrid, subtractive usually includes generating the initial sound to subtract from though (the oscillator) and even basic subtractive synths often have capability there (different waveforms, octaves, PWM, etc)
Pretty much everything in audio processing is a filter, whether it's called a filter or not, but that's overly reductive. Synthesis is just creating audio from parts.
That's not exactly true. In digital signal processing, delays and filters are effectively one in the same. This is because you implement digital filters using digital delays. For example, the simplest low-pass filter is just a summation of the current and previous sample: y(n) = x(n) + x(n-1).
“Filters are delays” doesn’t imply “delays are filters”. In particular, the type of effect known as a delay (e.g. a delay guitar pedal) isn’t a filter, certainly not in the musical sense, which is the relevant sense here.
> “Filters are delays” doesn’t imply “delays are filters”.
Purely logically, no, but that's not really the practical sense we're talking about. And by introducing a digital delay, you do induce a filter on the sound. So if you have a delay, then you have a filter.
> In particular, the type of effect known as a delay (e.g. a delay guitar pedal) isn’t a filter, certainly not in the musical sense, which is the relevant sense here.
I think it's best to reconsider my original comment. I was arguing that it isn't useful to suddenly rename subtractive synthesis as a "transformation", because you could just rename all of synthesis "filtering". But that's not useful, and synthesis just means building up sound from parts. I.e., it's best to work at a given level of the parts and think about things like waveforms, envelopes, LFOs, pulses, triggers, delays, reverbs, etc., most of the time.
You're getting downvoted for some reason but this is a perfectly fine way to think about subtractive synthesis. (From a compositional perspective anyhow.)
but also, like, it only runs in certain conditions it sounds like here.
Can this line of code be always running? What if the app launches itself as a background process?
For a feature I just learned about so many questions, but it's Apple in their store so hedges betting it isn't as dangerous as it sounds. But perks your ears up
By design, apps run in background on iOS only as the system permits. Mostly they don't, except for a limited time in response to notifications, and in some other such sharply circumscribed situations.
I strongly doubt an app asserting this flag would pass review without giving the user a way to opt freely back out in the app's settings.
That goes double for the flag being entitlement-gated, one social reason for which requirement would be that it structurally signals a need for closer review. (I don't have an inside source at Apple, but my experience of their review process prompts the inference.)
The intent is to offer a user the option to strengthen a precommitment, not to offer an app dev the option to permanently infest a device. You can play games in app review, but only up to a point. Trying something like that seems like a good way to get deservedly permabanned.
Granted, I still wouldn't use the app, because I don't need it. But if I did, I'd feel pretty OK about enabling that feature. I might expect it to annoy me enough and aid me so little I'd turn it off again, but I'd see no need whatsoever to worry about being stuck with it.
What I'm wondering is, why would any app need to actually prevent uninstallation (and thus violate a fundamental right of the user to control their device), when it seems what such app makers would want is to prevent closing the app / escaping to the home screen.
How many people actually decide to uninstall an alarm app as it's going off?
Presumably enough to make it worth Apple's while to implement the option under an umbrella entitlement over tools that may help support perhaps flagging self-discipline. Presumably too, it was worth the app dev's while to go through the effort and process of implementing the assertion. I have some mild doubts about the fundamental efficacy of such an approach, but its motivation seems clear enough.
For any more than that you're asking quite the wrong person, I'm afraid. For one, I don't intuitively understand the use case; the way this works for me is that I set my phone to play reveille when I need to wake up, and when it does, I do. That makes convenient use of some early training which is all well and good for me personally, but it leaves me little able to speak to the concerns of those for whom nothing so simple can serve.
I'm a music researcher, software engineer, and producer. I made Story Jam.
This doc is intended to inform you of not just the product, but the centuries of work that have led up to its implementation.
> Are you tired of the barriers in traditional music composition? Story Jam is here to break them down.
> Designed for anyone with creative ideas — from poets to film directors — our tool offers a new way to create and edit chord progressions, powered by cutting-edge music theory.
*The product*
https://storyjam.tenpens.ink
v1.0.0
The UI is inspired by https://neal.fun. Neal has fun, simple, addictive, and intuitive designs and I hope Story Jam has that too.
*Who Story Jam is for: Storytellers*
Story Jam makes music composition accessible and meaningful to anybody, with or without musical training.
It is designed for those who crave musical control but struggle with traditional composition methods. This includes film directors, slam poets, and self-taught musicians.
Story Jam is not music production software. Do not expect fancy sounds or synthesizers. It's purely a composition tool, designed to spark your creative process.
*Try it out!*
The demo is free on the homepage, no login required!
This is an MVP, so it has an "introductory" feature set. Feature requests welcome; help me build the product you want.
*How it was made*
This service is built with:
- Monic Theory
- Yew (Rust) frontend
- TypeScript/Express webserver
- Postgres database
- Hosting by https://render.com
- Stripe API
- SendGrid (email) API
- Dygma Defy keyboard
- Love
*The chord progression suggestion logic*
This service is built on a novel new music theory I have developed called Monic Theory.
Monic Theory is a rigorous proof for music. Not "Western music": music.
Monic Theory describes the tonal space of any conventional music on earth (except noise music. For that just use `Math.random()`). It describes the static and transient function of chords, instantaneously and differentially over time. This model enables empirical measurement of chords and the relationship between chords.
(hint: It is nothing you have seen in Xenharmonic Alliance. This is a new approach I have been developing over the past 10 years.)
Therefore, Monic Theory enables us to describe (or "predict" if you will) a chord progression to invoke a certain feeling.
*Music Composition*
Three people who helped set up the environment for Monic Theory are composers Paul Hindemith and Harry Partch , and music theorist Heinrich Schenker.
These folks independently contributed new ideas to music composition and analysis. All of these people lived without access to rapid computation. This is critical for the Partch case, as he computed many tables of frequencies by hand to support his compositional technique.
Partch recognized the human-math-music relation in "Genesis of a Music." He includes in this text some samples of his hand-computed tables of frequency values of overtones and (importantly) undertones which support the basis is technique.
Partch's techniques were so far-fetched that he had to construct new instruments to perform his scores. Similarly, I had to build a digital synthesizer to render the output of Monic Theory. (See: https://github.com/ckmahoney/raudio).
*About me*
I was a working composer and violinist from 2007 until 2017, and I have been a software engineer for the past 7.5 years.
*Livecoding*
I was a volunteer organizer for Livecode.NYC, an NYC livecode community; and am the volunteer creator of Data Dancers, Atlanta's livecode community. I am passionate about algorithmic art and have provided about a dozen workshops over four years on the topic.
https://ckmahoney.github.io/portfolio/posts/livecoding/
thank you for reading.
May the flow of Spices be with you :) naltroc