Get Weird with Audio on the Web

Captions provided by White Coat Captioning (https://whitecoatcaptioning.com/). Communication Access Realtime Translation (CART) is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings.

JASON: Hello, everyone, and welcome to another episode of "Learn with Jason". Today on the show we have Ken Wheeler. Ken, how you doing today?

KEN WHEELER: I'm phenomenal. How are you?

JASON: Doing pretty well. I'm super excited to have you on the show. We've talked about this for a while. I'm excited to make it happen here. For those of us who aren't familiar with your work, do you want to maybe give us a little background?

KEN WHEELER: Yeah. So I'm Ken Wheeler. I made a bunch of dumb ass open source back in the day. Haven't recently and sold out and build trading dashboards now and occasionally do music demos.

JASON: I have definitely been following your Twitter for the music content lately. It's so much fun to watch you make beats which is something I pretend that I can do, but I definitely can't. It's like I pretend that maybe someday I'll be doing that stuff.

KEN WHEELER: I don't think I have tweeted about tech in some time.

JASON: Yeah. Tech Twitter is a whole lifestyle. A stress ball.

KEN WHEELER: It's a thing. It certainly is a thing.

JASON: But, yeah. So I'm pretty excited about this though. The tech that you're not tweeting about, you have done a bunch of stuff that has really made a splash in the open source community. You were formidable for a while. So you did work on spectacle, Erkel, your first one was what was it? Slick slider?

KEN WHEELER: That web pack dashboard. I created all of those. The good folks at Formidable took it from dumpster fire to product.

JASON: That's a great part about having a team, right? You can just whip something out there and then people

KEN WHEELER: I'm very much an ideas guy. I'll go and build something. They'll see it like, wow, this is kind of cool. They'll look under the hood like, Jesus fuckin' Christ. All right. Urkel wasn't initially that bad. I did a nice tight scripting at first. Yeah, some of those were a little on the cowboy side, at least initially.

JASON: Yeah. Definitely. I feel that. I'm super excited today. We're going to kind of combine all of this because so actually, I have a question. You talk about yourself as a producer and you've made, like, full albums, right? You have got music on spotify, you're putting out actual tracks. But when you were younger you did a lot more music. You did a ton of music.

KEN WHEELER: Yeah, like as a job.

JASON: Yeah. Do you want to talk about that a little bit? I have never heard the story.

KEN WHEELER: No, I mean when I was in high school, when we were children it was like bands and everything like that. You know, we would play in hard core bands like New York hard core, that kind of stuff. And then I got really into rap music. Before any rap production or anything, I thought turntables were the coolest shit there was. I would see people scratching like that's the coolest thing you can possibly do. So I mowed lawns and got turntables. It was really funny because back then there were hip hop records available. I just didn't know where to get them. So it was me spinning like my parents' classic rock music. So everything was very Rick Ruben. Like "walk this way" drum beat like
( Beat boxing ) So then the first beats that I made, it was wild. There was no fruity loops or logic or Ableton or any of this shit. So you have a mixer, right? You have two turntables and a mixer and one channel for each turntable, right? So I took one of those channels and I would record one turntable into a tape recorder. Remember a tape recorder?

JASON: Yeah.

KEN WHEELER: I had two whatever the Walkman equivalent was of a cassette tape.

JASON: Yeah.

KEN WHEELER: I would record into that. I would take the tape, switch it to the second channel, play that while recording on the initial one and doing the thing on the turntable. That's how I did multi track recording initially.

JASON: Wow.

KEN WHEELER: You would have the
( Beatboxing ) . Then come in and just layer them and shit. It was crazy. It was so dope. You know how shitty tape sounds and shitty cables. When you layer it, by the end it's like it sounded crazy. This shitty distorted but it was dope because it was low fi and raw. So I made beats, produced my friends. We would get all high and make songs about people. In high school you have a fight with somebody. This kid is a fuckin' nerd. Play it at parties and everybody's like, oh! So that was cool. Then, you know, eventually ended up trying people are like, these beats are pretty good. You should do something more than, you know, make fun of your rivals at parties. I was like, yeah, fair enough. So I went and met with labels and did this thing. There was this New York City beat battle, right? Everybody goes and plays beats. So there was a beat battle and I won it two years in a row. Did a bunch of stuff. Then I would sign and I was making stuff. Then drugs ruined everything I'm kidding. I think napster ruined everything. It changed the industry model. But after it stopped being the big money deal

JASON: Mm hmm.

KEN WHEELER: I went and I was a recording engineer in house for the guy who sang "one more time" by daft punk.

JASON: Yeah.

KEN WHEELER: I worked for him for two or three years. That was fun. Sitting there like cutting up samples and making house music and recording shit. You know, producing little acts. That was a lot of fun. From there, I don't know. Shit got wild. That actually was drugs. You know, from there I kind of got out of it professionally. When you do something professionally it stops being as fun, if that makes sense.

JASON: I feel that for sure.

KEN WHEELER: You know what I mean? So I tried to get out of it. I was like a boiler room stock broker. I worked for a match making service. I was working in construction. One day I lied my way into a job making flash websites for restaurants and the rest is history. But ever since then, I have always recreationally made beats because it was fun.

JASON: That's super cool.

KEN WHEELER: That's the story.

JASON: Yeah. That's a great story. It's funny. There's a lot of parallels to kind of how I got into tech as well. I started out as a touring musician. Kind of failed my way up that chain and got into web design because it was the way to keep the band going because we weren't a good band. We did not get signed.
But today we're going to do kind of like a merging of the worlds. We're going to use some open source. We're going to use some music and make a drum machine.

KEN WHEELER: Yeah.

JASON: Which I think is going to be super fun. If you don't mind, let's get some code going.

KEN WHEELER: Okay.

JASON: So I'm going to switch over into this other view where we can see a screen. We'll do a quick shoutout here. Follow Ken on Twitter if you don't already. Also just a quick shoutout to our sponsors. We have live captioning going by White Coat Captioning, so thank you so much for being here today, it helps make the show more accessible to more people and we appreciate that. That's made possible by sponsorships from netlify, FAUNA, SANITY, and Auth0, all of whom are working to help make the show more available to more people. Thank you very much. With that, we're going to jump over to this screen here where we are able to start coding some stuff. I'm ready.

KEN WHEELER: Okay. What kind of drum machine do you want to make?

JASON: I mean, I feel like it would be probably like a simple sequencer would be potentially the easiest thing. I don't know.

KEN WHEELER: Yeah. We have 90 some odd minutes so we could do a sequencer. Let's do that. Do you want me to show you some initial web audio before we got into that?

JASON: Yeah, absolutely.

KEN WHEELER: It will help me justify the use of some of the tools I'm about to show you.

JASON: Absolutely, in that case, let's do it.

KEN WHEELER: Say we're hanging out in react right here. Let's do a start stop.

JASON: Where are you at now? Let me reload. I might have lost the live. Sign in what? Why'd you kick me out?

KEN WHEELER: I threw you out. Sorry, boss.

JASON: All right. I think I'm back in. Are you still in this collab session. It says we're live. I don't see you in it.

KEN WHEELER: Pass me a link, papi. Wait, is this it now? Do you see me typing anything?

JASON: Let me check. No.

KEN WHEELER: No? All right. Send a link over.

JASON: I just dropped one, again, in your Twitter.

KEN WHEELER: Boom shakalaka. There we are.

JASON: Here we go.

KEN WHEELER: All right. So that's not a constant. I like using let. It's going to yell at me. So let's do a state thing here just to control playing a sound or not playing a sound. So we're going to say it's not playing initially here. Let's just do a button.

JASON: Okay.

KEN WHEELER: All right. And then playing the old switch a roo. Am I right?

JASON: Indeed.

KEN WHEELER: Too much fun. I'm going to get myself a little more screen real estate here. Now we're talking business.

JASON: You want this to say "play," right?

KEN WHEELER: Yeah. Thank you, sir. You could do onclick, right? So descriptive. Let's make sure this works. This should give us the old play stop a rooni.

JASON: Look at it go.

KEN WHEELER: We actually want to play music, right?

JASON: Yes.

KEN WHEELER: So we want to create an audio context. Up here we can do context equals new audio context. "Audiot." All right. So we're going to do a use effect so that we can do gangster things when the playing state changes.

JASON: Right.

KEN WHEELER: I'm about to after all this scaffolding I can get into why this is such a pain in the ass. So first of all I don't think that audio will play in your browser without actual user interaction which is truly meaningful shenanigans. However, we do have a button, so that will work.

JASON: Mm hmm.

KEN WHEELER: Now, you would think right? Like you would just be able to do you know what sustained release is?

JASON: Yeah, yeah.

KEN WHEELER: So it's more granular than that. The web audio API, you start with an oscillator.

JASON: Okay.

KEN WHEELER: Are you familiar with oscillators?

JASON: Yeah, but we should explain them for people. Attack, sustain, release, that's like the way a sound starts, right? The attack is how quickly it goes from zero to on, sustain is how long it continues for and release is how long it takes to go from on to off. So if it has a slow attack it would fade up. And then the sustain is like if you turn sustain all the way up it plays forever. Turn it down, it's staccato. Oscillator, I'll let you explain those. I don't understand them as well.

KEN WHEELER: Yeah. So an oscillator is different waves, right? Imagine some shit that just spins.

JASON: Mm hmm.

KEN WHEELER: It's waveforms, but not like you would think if you're looking at, like, win amp or something like that. It's like a sine or a cosine or, you know, repeating

JASON: Yeah. I'm going to pull up

KEN WHEELER: Do a picture. A repeating graphing calculator set up.

JASON: Something like this?

KEN WHEELER: Precisely. Yep. What the oscillator does is generates the wave there. When you're seeing it get smaller in the middle that's a release or a decay rather, right? You know, the higher up it goes in either direction is the louder it's going. That's called your gain. The smaller it's getting, the quieter it's getting. That's an oscillator. This right here, this particular wave is a sine wave which is literally math.sine, right? Other waves are a little bit more algorithmic so to say. Square wave or something. You have a thing called period time. What period time is is the time it takes to do a full oscillation. Can you pull up an oscillator gif?

JASON: Let's see what we can find here.

KEN WHEELER: There should be a circle one. If you've seen a circle spinning and it

JASON: Oh, yeah, yeah. That's sort of it.

KEN WHEELER: You know what I'm talking about though, right?

JASON: I do. I'm looking to see here we go.

KEN WHEELER: Perfect. Yeah. The full circumnavigation of the circle is your period time, right? So that's the full loop of that wave, right? This doesn't just apply to music. It applies to electricity, any signal, all kinds of shit. But you would sit there and say, like, you know, for a square wave, right, which you might hear in UK grime or something like that for a bass.

JASON: Mm hmm.

KEN WHEELER: You're saying go down to the bottom and then hop back up to the top at half period time. When you make it halfway through the circle, flip the switch, right? So everything is not like this sine or logarithm or whatever. Some of them are a little bit more yeah, there we go.

JASON: This is a great post from Josh Comeau about how this works. It will be in the show notes if you want to really dig into how it works.

KEN WHEELER: Right. So an oscillator creates these, right? And we can create an oscillator.

JASON: So I'm back in the code now.

KEN WHEELER: Yeah. So if we go in here do you see me? I'm still in. All right. So I can say let's put osc up here, right? If playing, put that in our hooks dependencies. If it's playing, what we're going to want to do is use our audio context to create an oscillator.

JASON: Whoa.

KEN WHEELER: This creates your oscillator object, right?

JASON: Okay.

KEN WHEELER: So the next thing you would need to do is do a frequency. Let's just say 440. Center A or whatever.

JASON: Mm hmm.

KEN WHEELER: What you want to do is connect it. So everything is like this node based connection graph thing in web audio. So you want to basically if you want to connect it to the speakers you say context.destination.

JASON: Okay.

KEN WHEELER: Then you can say osc.start and that will start it playing.

JASON: So if I click the button this should start making a sound?

KEN WHEELER: Let me put in the destructor first or it will get annoying. So osc.stop. This should play if you click it. ( Tone )

KEN WHEELER: Click stop and see if it stops.

JASON: It stops.

KEN WHEELER: Yeah.

JASON: That's it. We're done. We made music, y'all.

KEN WHEELER: We made music. If you want to blow everyone's eardrums out you can change the waveform type. It's defaulting to sine. I could say osc.type. Sawtooth. Don't do it for long. It's terrible. ( Tone )

JASON: Oh, that's a lot.

KEN WHEELER: Square wave is cool if you listen to drum and bassy stuff. You hear a lot of square waves. ( Tone )

JASON: Oops.

KEN WHEELER: That's terrible. Oh, my ears. I dropped it down. This should be more esthetic if you click it now. ( Bassy tone )

KEN WHEELER: Okay. Now we're talking. That's a little dub steppy. If you want to modify the volume on this, you could in theory create a gain, right? Gain is like a volume bad boy. It's not used yet. I'm like, what the fuck is your problem? Create gain. So here's where this gets interesting, right? I can connect my oscillator to the gain.

JASON: Yeah.

KEN WHEELER: So I believe I can just do that and then gain.connect to the destination which creates the signal chain, right?

JASON: Okay.

KEN WHEELER: Here I can say that the gain.gain believe it or not.

JASON: Of course.

KEN WHEELER: Value equals 0.5, right? This will be usually any time you do anything it's at 0.5. That's a safe volume. Now try it and see how you feel about the volume now.

JASON: It should be quieter now, right?

KEN WHEELER: It should. ( Bass tone )

JASON: Much quieter.

KEN WHEELER: It's not blowing everybody's eardrums out.

JASON: Chat will be happy about that.

KEN WHEELER: Oh, yeah. It still is quite loud. I may attempt something to make an oscillator point, but it may not work. Are you familiar with LFO?

JASON: I'm passingly familiar with LFO. I have seen it and clicked buttons but I don't know what they do.

KEN WHEELER: So you can use an oscillator to not just make noise, but to oscillate properties of these things.

JASON: Uh huh.

KEN WHEELER: That's how you make, like, dubstep kind of shit. I'll do let LFO, let LFOgain here, right?

JASON: Okay.

KEN WHEELER: So I can say lfo equals context.createoscillator. LFO gain equals context.creategain. Leave that here and say lfo.frequency.value is, man, I don't know. Two or something. Something like that. I don't know the exact thing right now. But down here we have this gain that has this value here. I can say LFO gain.gain.value equals 0.5. LFO.connect so we're going to connect that to this gain and then LFO gain this might not work. For argument's sake, if it does, it will be sweet. So I'm going to just see if that
( Bass tone )

KEN WHEELER: Yeah, I can't connect directly on that. It didn't like that at all. Not running?

JASON: Let's see.

KEN WHEELER: Yeah. Well, I tried. ( Bass tone )

KEN WHEELER: Did that just go a little bit wow wow?

JASON: It did. You're hearing that, right? It's doing a little bump.

KEN WHEELER: Yeah, a little bit. Well, that's the long and short of it. But as you can see just to make a dubsteppy type thing, it's an unbelievable pain in the ass here.

JASON: Yeah, this is a lot.

KEN WHEELER: It really is a lot. So we're going to use something that makes this suck a bit less. You know, this is very I mean, it's not bare metal, but for all intents and purposes. So we're going to use a thing called tone JS.

JASON: Okay.

KEN WHEELER: Do you want to pull up the site to show the folks at home?

JASON: Tone JS. This one here?

KEN WHEELER: Yeah. So this is my jam. Most of the demos you see me use use this. What it is is it makes all this much easier. I could do, like, playing samples and we would be probably out of time in here. You know, they take care of so much for you. They do the attack, decay, sustain, release stuff. If you want to make one with multiple parts you have to create oscillators, start them at the same time and mix them together and shit. It's absolutely devastating.

JASON: They just make it happen for you?

KEN WHEELER: They do. It gives you helpers. One specific helper is the scheduling. Any time you want to do anything over time you definitely have a bad time if you don't have yeah, tone.JS there.

JASON: Am I looking right at this one?

KEN WHEELER: Yep, that one right there.

JASON: Okay.

KEN WHEELER: Look at that.

JASON: We got it.

KEN WHEELER: Look at us. We're good to go. What I'm going to do real quick is if we're going to build a drum machine, we need drums.

JASON: Okay.

KEN WHEELER: That goes without saying. I'm going to get some basic drums to work with and drop them in there. You can't hear my sound, can you?

JASON: No.

KEN WHEELER: I'm going to drop this into there. I'm going to give a kick and clap. Maybe a high hat. Can you see it in the public folder there? Look at that. The first thing we want to do is load the samples in. Well, no, the first thing is import tone. I have a weird little type error there, but I digress. Are you seeing that?

JASON: Tone.Synth. Maybe I'm supposed to do capital.

KEN WHEELER: Perhaps.

JASON: That would be ridiculous if that did that.

KEN WHEELER: Is it like a named one?

JASON: Let me look at an actual example here. Music oscillator. Show me the code. All right. We're going to look at demos. Let's see what version this is on. 14.7.39.

KEN WHEELER: I don't know if they changed it. I'm going to put our tone on a version I know will work.

JASON: Can we get it on GitHub?

KEN WHEELER: I changed it there.

JASON: It's no longer exploding. Good.

KEN WHEELER: Yeah. Got to love javascript. Am I right? Jesus Christmas. Okay. So we have our stuff in the public folder which makes it accessible. So we're going to create what's called a sampler. Right?

JASON: Okay.

KEN WHEELER: I'm going to create a sampler for the kick real quick. Const quick equals new tone

JASON: Got it?

KEN WHEELER: Tone.sampler and you give it a note. Let's do C zero.

JASON: That's like when you say C zero, that's like the middle C on a piano, right?

KEN WHEELER: That's like the first C on a piano. But the reason you do C zero, at least for me, is C zero is usually the first pad in a bank on a drum machine.

JASON: Mm.

KEN WHEELER: So that's what that's all about. Let's make this button play that.

JASON: Okay.

KEN WHEELER: Right?

JASON: Is that in the use effect?

KEN WHEELER: It's probably not going to be playing or set playing, but I'll leave that there. We can do that. Let's just make it just actually play for a minute.

JASON: Okay.

KEN WHEELER: All right. What is this thing's problem?

JASON: It doesn't like that we're

KEN WHEELER: Oh, give me a break. Waah, waah. Kick, right?

JASON: Cool.

KEN WHEELER: So we have the handle click here. If I remember correctly, I should be able to say kick, trigger attack and C0. That should do it. If it doesn't, I'm a silly goose.

JASON: It's not playing. But it's also not throwing errors. Let me reload, just in case. Do we need to do, like

KEN WHEELER: I need to route it to the master track.

JASON: I also think we need to get the path to that wav file. I think right now or is it just

KEN WHEELER: That should work.

JASON: Okay. ( Kick tone )

JASON: There it is.

KEN WHEELER: Uh oh. Uh oh.

JASON: Right? Now we get to learn how arrhythmic I am.

KEN WHEELER: Did you want to play house music or something for a minute? ( Thumping )

JASON: Not working out for me.

KEN WHEELER: Let me hear you say way oh, way oh You're old enough to remember that, right?

JASON: I am.

KEN WHEELER: You've been to a school dance before.

JASON: I have. Now that I'm an honorary house D.J. at this point, the chat's playing along. Here we go.

KEN WHEELER: Let me hear you say way oh

JASON: So now we have the guts of this, right? We're able to play a sample in the browser and this could be any wav file, right? We can grab whatever and it would play.

KEN WHEELER: Yeah.

JASON: So if this was a snippet out of a song or somebody saying a word, whatever. All those things will work.

KEN WHEELER: Why don't you set up clap and hat and try those in the handle click.

JASON: All right.

KEN WHEELER: Just keep that bad boy in there.

JASON: Clap. And then we'll switch this out to be clap. And just so it's clear. ( Clap, clap, clap )

KEN WHEELER: Uh oh.

JASON: This is great. We're in good shape here. The last one what was it? Snare?

KEN WHEELER: Hat. Like a high hat. Like tsss.

JASON: Okay. ( Hat )

KEN WHEELER: Wow.

JASON: Really, we could copy these over and say handle clap and handle hat and then we'll change this one to be handle hat. So now if we handle hat, handle clap. What don't you like? Oh, they're not

KEN WHEELER: Nate's right. We could put those samples in the same samples and switch the notes.

JASON: Oh, that's a good should we do that?

KEN WHEELER: Probably.

JASON: Let's do that instead. So which ones would you typically use?

KEN WHEELER: You could just do D0 and E0.

JASON: So D0 will be a clap and E0 will be our hat.

KEN WHEELER: We could use sharps but who feels like writing a hashtag all the time?

JASON: Then we can do drums. And this will instead be drums trigger, I think hat was E0.

KEN WHEELER: That's right.

JASON: Clap is D0. Then we'll do one more for the kick.

KEN WHEELER: None other than C0.

JASON: Excellent. Okay. And then if we drop this thing down and we can handle the kick.

KEN WHEELER: Fragment has a react prefix on it.

JASON: Whoops.

KEN WHEELER: Wow.

JASON: Okay, so now
( Sick beat )

JASON: Right? This is the very basics of a drum machine.

KEN WHEELER: That's right.

JASON: If we wanted to be real courageous we could set up more samples, get on stage and tap on our iPads for this.

KEN WHEELER: You can literally tap on your iPad for this.

Holy buckets, did that just work?

JASON: Indeed, it did work. I'm pumped. I'm not going to lie. This is way more approachable than I expected, especially after the initial set up of plain web audio browser APIs.

KEN WHEELER: Yeah. I have built music apps in O camel where you are literally making the drum sounds out of math, generating waves and shit. There's no crate oscillator there. You're sending floats and array buffers to the sound card and creating the oscillator with math functions. And then you're doing digital signal processing on them. That shit is rowdy. The next step of rowdy is your web audio, but 90% of the things I build, I will use tone. As easy and approachable as this is, it's still very powerful. You can build crazy things with it and we're going to build crazier stuff with it. When you see what it takes to make the most rudimentary sound, you appreciate this abstraction so much.

JASON: Absolutely.

KEN WHEELER: A lot of unnecessary things that you would need to do the granularity. This takes you away and lets you be productive to make an app. You can prototype shit really quickly if you want to make a sound board or something like that. You could do that.

JASON: Which all of us need. I expect you to do one of those immediately. The expo repo is on the learnwithjason.dev. It's the first video on the site. So we've got like
( Hat )

KEN WHEELER: Yep.

JASON: I can't click fast enough to do a trap beat, but we're getting to the point where we've got the building blocks in place. Do you still want to do sequencing or something else?

KEN WHEELER: I'll do sequencing. Sequencing is fun.

JASON: All right.

KEN WHEELER: I'll show more of the tone API which is kind of fun. You can make actual music with the sequencer.

JASON: Let's do it.

KEN WHEELER: So I guess we're going to want to bring the playing button back and get rid of these. We want to be able to start and stop it.

JASON: Okay.

KEN WHEELER: We can call that trigger attack, but we're going to do it a little bit differently when it's party time.

JASON: Then do we want to bring back all the you state stuff?

KEN WHEELER: Yeah. We can get rid of that. Come back here and we can keep this all right here. Drum stays. We have play. We have stop. All right. So we have this right here. So the first thing that we want to do is probably generate a bunch of steps, right?

JASON: Mm hmm.

KEN WHEELER: So we have our button there. Let's make a little thing here, doesn't have to be pretty. Let's call this don't have to be different files either. I don't give a hot God damn. I haven't done a style and react in so long. Does it really mean that? You know, just for shits and giggles. Border, solid gray or something.

JASON: Okay.

KEN WHEELER: Let's say, you know, padding ten or something like that.

JASON: This is the problem. Once you start writing CSS you can't stop.

KEN WHEELER: Just can't stop. Oh, a little off, right? Yeah. So here's the thing. We're going to need to track steps. When you have a step sequencer there are varying resolutions. Usually we go with 16.

JASON: Mm hmm.

KEN WHEELER: Let's create a function, right? Let's say first of all, let's say const instruments. We'll create an array of stuff, right? So we'll name these. We'll say kick, clap, hat. Right?

JASON: Okay.

KEN WHEELER: Maybe instruments is the wrong thing. Let's call it tracks. Naming and caching. Right? So these are our tracks, right? And then we want steps and our steps is going to be an array of arrays. An array of objects or arrays. Let's say const steps equals tracks.map, right? Then we're going to have our T. Let's return an object where name is T.

JASON: Okay.

KEN WHEELER: And steps are let's make a method called generate steps.

JASON: Okay.

KEN WHEELER: So up here we could say function generate steps and then AVAR? Am I out of my mind? And then we'll say and then for each one of these we'll say steps.push zero. Say zero is off and one is on.

JASON: Okay. I understand what's happening now. Basically what we're saying is this is like for those of you who aren't familiar with a sequencer what we're going to do is put up, you know, like a loop. So that loop is going to be broken up into steps. In this case it will be 16 steps.

KEN WHEELER: Do you want to show a picture of a step sequencer?

JASON: That's a good idea. I was going to verbally describe it. Step sequencer. So here's a complicated one. Basically for each track you have like this is so small. Whatever. This is fine.

KEN WHEELER: Not to me.

JASON: Okay. All right. Let's zoom in here. You can see we've got a bass drum, snare. For each of these, this is a 32 step sequencer.

KEN WHEELER: Looks to be.

JASON: Yeah. So we've got 32 steps and if the thing is lit up, when it hits that step it plays that sound. So you're defining your loop and it's going to count this you know, if it's a 32 step you're going to end up with a 1 Y and a 2 Y and a 3 Y. If it was four step it would be one, two, three, four. So instead of just a four step beat that's like
( Beatboxing ) You can add eighth notes, 16th notes to get to things like you can do the more trap sound well, I guess for trap we need 32nd, 64th notes. Whatever.

KEN WHEELER: Yeah.

JASON: We're going with 16 to keep it palatable. It would be a lot.

KEN WHEELER: Just to have fun. It becomes a lot of steps. The last time I made one of these I did the steps in Canvas. It was so many. I had a 32 one and managing it was a little rowdy. But here we are. So here we have generate steps. Let's take a look at what this looks like. You have a console there?

JASON: Yeah. Console is here. There we go. We've got our kick and this is an array of just zero, it looks like.

KEN WHEELER: Have I fucked something up here?

JASON: Steps.push. Return why isn't that working?

KEN WHEELER: In theory, we should have, you know, like 16 some odd steps in each one of those, right?

JASON: Yeah. Let me refresh this. Maybe it's just hold on. Push does push mutate or do we have to

KEN WHEELER: It shouldn't. Oh, God. Jesus, Mary, and Joseph.

JASON: There we go. All right. Here we are. We got it.

KEN WHEELER: Oh, man. Real bunch of pros here. Okay. Yeah. So we can have let steps, set steps, right? React.usestate. Let's call that initial steps so we don't have a collision. Well, no. God damn it. Okay. I think that should work, right?

JASON: Yeah.

KEN WHEELER: Fair enough. So we'll create these rascals, right? Let's render out our steps. This is going to be the most annoying part.

JASON: Okay.

KEN WHEELER: Steps.map thank you, sir. This pair programming is fun, man.

JASON: I love it. This is one of the things I love most about the show is getting to sit and build something with somebody way smarter than me.

KEN WHEELER: We like to have fun.

JASON: So there's so many uses of the word steps, it's just killing me here.

KEN WHEELER: Here, I guess let's just return another div I guess for each track.

JASON: We're going to need a row, right? We'll have to show the name and then boxes for each of the steps.

KEN WHEELER: Yeah. I'm going to make it all divs because I don't care right now as far as that's concerned. Our first one is going to be step.name.

JASON: Okay.

KEN WHEELER: Wow. Would you look at that?

JASON: Look at it go.

KEN WHEELER: Just look at it. And then here we can do step.steps yeah, you hook me up with some style while I'm doing this. Step steps.

JASON: I think that ought to make it work.

KEN WHEELER: That should be fresh.

JASON: Step steps.

KEN WHEELER: Step steps Let's say background is S, zero and gray or light blue.

JASON: Okay.

KEN WHEELER: All right? We should probably put a size on that, huh? So let's say

JASON: 15 by 15 or something, 10 by 10.

KEN WHEELER: 10, width 5, margin 2 or some shit like that.

JASON: That works. Getting there.

KEN WHEELER: On the name let's give that a style equals width 100 or something. Wowee.

JASON: There we go.

KEN WHEELER: Yeah. Let's do width 15, height 20. I like that a lot better.

JASON: It's looking good. That's the one I want at center.

KEN WHEELER: Now we're talking business here. Wowee. Okay. There was a naming discrepancy here. Tracks. I like that better.

JASON: Yep. That's right. Okay. So we got our tracks. And then so now it looks like what we need is we need to make it so that when I click one of these, are these already clickable? Oh, you did the cursor.

KEN WHEELER: I did the cursor pointer on them.

JASON: So we need to change its value to one in the array and that would light it up.

KEN WHEELER: Yep. So here we can say update step, right? So what we probably want here is index.

JASON: Do we want the step index?

KEN WHEELER: We do.

JASON: We need them both.

KEN WHEELER: So here we can do index and step index, right?

JASON: Okay.

KEN WHEELER: Yeah. Let's make a little function for that. Tracks. So work. New tracks, track index, step index is equal to oh, this is going to get gross.

JASON: We just need that to be the old tracks, right?

KEN WHEELER: Map of undefined? Oh, yeah. We never returned it.

JASON: Okay.

KEN WHEELER: Ken and Jason learn react.

JASON: No! Okay. Tracks.map didn't work because

KEN WHEELER: Um

JASON: That definitely should have worked.

KEN WHEELER: No, I'm a dumb ass. I did an object structure on an array. Java script expertise over here today.

JASON: There we go. Getting a key warning.

KEN WHEELER: We should put keys on these bad boys.

JASON: I'll do the tracks. Step. There we go. Okay. So okay.

KEN WHEELER: Whoa, what's going on here?

JASON: It doesn't like that's just drums. Whatever. Okay. So it's doing something, but not let's log this thing and see what we get. Make a change. It logs. And we clicked kick. Oh, wait. I know what we did. We've got to do steps and then the step index.

KEN WHEELER: Man. Got to do it here, too.

JASON: Oh, yep. That should have given us what are we missing here?

KEN WHEELER: We're missing something. Tracks

JASON: Let me refresh this straight up. There we go. Wait, it was working.

KEN WHEELER: Wow.

JASON: So it got the first one and then it lost its ability to get new ones for some reason.

KEN WHEELER: So I think tracks is already in the scope. All right. Let's do that. There might be some silliness taking place here.

JASON: Old tracks. Then I'm going to refresh it again.

KEN WHEELER: Still no.

JASON: Still no. So let's do hat 1, so that should mean hat 1 got updated and it didn't. It takes the first one and dumps whatever else. Let me refresh and make sure it works on any yeah. It takes the first one and dumps on us. And that happens because why?

KEN WHEELER: Let me see something here for a second. See if that makes any difference. What is actually being updated here? When I click it a second time, it logs twice. So it's turning it on and off.

JASON: Oh. Interesting. Why is that happening? The second click is logging twice.

KEN WHEELER: That's right.

JASON: So what up, chat? What did we do wrong? Hooked on hooks, that's us. So we got our click. This div does terminate. So it's not like a wrapper triggering another click handler or something.

KEN WHEELER: No.

JASON: Let's see. This shouldn't affect anything.

KEN WHEELER: Yeah.

JASON: My favorite old okay. It's not firing the effect which makes sense because we didn't tell it to. But it is double firing this little buddy. Do we need this doesn't need to be an effect, does it?

KEN WHEELER: Maybe a use call back?

JASON: Hmm. Oh, this needs to be a function. No, it doesn't.

KEN WHEELER: 40 and 16.

JASON: Needs to be a function. Oh, yeah. I understand we're not okay, that. No! Still doing it.

KEN WHEELER: Are you shitting me?

JASON: Unnecessary dependency tracks?

KEN WHEELER: Let's try to do it right off of tracks.

JASON: Yeah. Okay.

KEN WHEELER: Rather than the callback.

JASON: Okay.

KEN WHEELER: Let's put tracks in there now.

JASON: Let's reload this thing. There we go. Oh, do you know what it was? We were mutating tracks and I'll bet that was triggering the the thing.

KEN WHEELER: Yeah, whatever. I hate react hooks.

JASON: Hey, we got a sequencer. Look at it go.

KEN WHEELER: How much fun is that?

JASON: So this is cool. If we want to do a standard four on the floor we'd stack it up like that. And then we can do one of these deals. Then we can just light this thing up. If we get that plan that's going to be a barn burner of a track.

KEN WHEELER: Yeah. ( Beatboxing ) . Do you want to actually make it?

JASON: Yeah. Let's do it.

KEN WHEELER: We're going to use the scheduler here. Just looking at the docs for one second. Okay. So we're going to do an effect up here for our scheduling. I'm going to say react, use effect. All right. And then tracks. How much fun are we having there? What we are going to do is use tone transport, right? We're going to say tone, transport, schedule repeat.

JASON: Okay.

KEN WHEELER: So what this gives us is you're scheduling a loop of sorts. It's going to be 16th notes. So this is going to let us do a little schedule there. And then we probably want to maintain a step index here. So let's say let's just do that in a ref. I don't use a lot of state these days. A lot of the time it just causes performance problems. It's a use ref which is akin to putting it on this of a class.

JASON: Right.

KEN WHEELER: We're at zero now. I can say on each one of these, step index.current is equal to step index.current. Then we're going to say greater than 15, I guess.

JASON: Mm hmm.

KEN WHEELER: Zero. So that would reset it back if we made it to the 16th step.

JASON: Got it.

KEN WHEELER: Step index.current plus one. That gives us our current step index there. Yeah. So here, now we can say tracks for each track and this is where we would actually schedule the play, right?

JASON: Got it.

KEN WHEELER: Essentially what's going on here is this loop here, the schedule repeat happens on every 16th note. So for every box this is going to loop. So we are incrementing the step index on each one of those across there. You can't see my cursor.

JASON: So on each index we start at zero and it goes all the way up. Once we get here it's going to drop back down.

KEN WHEELER: Yeah.

JASON: And we end up at zero again. So that's how we create the loop.

KEN WHEELER: Mm hmm. Yeah. So in here we have this transport where the loop exists. For each of the tracks like how we played them before we can say if they're on then trigger them, right?

JASON: Mm hmm.

KEN WHEELER: So I can say let step equal track.steps step index.current. Like that. And then we say if step equals one what we probably want is our index here. We're going to do it a little hard code y.

JASON: Okay. Yeah. Because that way we don't have to map otherwise we need some way to track which note is which index is which instrument. So this way

KEN WHEELER: Yeah. Let's just do that. This is the right way to do it. All right. So const trackindex and then kick, clap, hat. So it would be C0, D0, E0. So then down here if it is one, right? We could say drums.triggerattack and we are going to say track index and then our index of the track

JASON: Perfect.

KEN WHEELER: Yeah. Then that, in theory should work. Otherwise it's just a no op. So

JASON: This is, like, running now, right?

KEN WHEELER: No. This is not running yet. What we need to do is we need to actually do start and stop on it. So if it is playing we would call it tone.transport.start.

JASON: Okay.

KEN WHEELER: Else tone.transport.stop and then also if we're going to redo these we should probably call them on transport.cancel, I think it is.

JASON: Okay.

KEN WHEELER: We'll certainly see. And then we want to set a BPM.

JASON: Is that the time or

KEN WHEELER: No, no.

JASON: Okay.

KEN WHEELER: So time is you would use to start at a certain time if you wanted to offset this by a certain time.

JASON: Got you.

KEN WHEELER: I don't think we're going to do that here, at least not yet. This gives you the actual time. If you wanted to do a triplet, you would use this time and then do addition on it. Does that make sense?

JASON: Yeah, yeah.

KEN WHEELER: Cool. So let's set the BPM real quick and try this fuckin' thing.

JASON: Okay.

KEN WHEELER: So tone.transport.BPM.value. I think it is. We have to check the docs on it. Yeah, that's it. Let's say 90. Let's see if that works.

JASON: All right. Let's do a four on the floor here. ( Beats )

KEN WHEELER: Oh, my God. We are the shit. Oh, my God. ( Claps )

JASON: We have a little bit of lag in the loops.

KEN WHEELER: Yeah. I'm going to take this step down one. It is 0 to 15, not 0 to 16. ( Beats continue )

( High hat )

JASON: Yeah.

KEN WHEELER: Doo, doo, doo

JASON: This is a fully functional sequencer. We can extend it, build on it.

KEN WHEELER: Yeah, that works.

JASON: That's pretty awesome. I feel like this is one of those things that if you were to tell me, a lot of web developer, hey, go build a step sequencer, my immediate thought would be I'm going to implode on this. There are too many moving parts. I don't know enough about playing sounds. I don't know all the stuff. What I love about this is we're barely a hundred lines of code in here and we have actually build a fully functional sequencer. Anything else we add here is just extending this. It sounds like there is way deeper down this rabbit hole we can go. But at a rough level this is it. We can deliver this and we have actually fulfilled the brief, right? I have done it before. You actually track the time and use request animation frame, stuff like that. It is a lot of code, especially if you're going to sit here and load up the sounds. You have to create a buffer and you need to load it. Using tone for this is like using react suspends for image loading. It's one of those things that's gnarly as shit. So much of the complexity is taken away by tone.

JASON: Yeah.

KEN WHEELER: What we're writing here is the bare minimum that it would take to get it to work, right? And then if you really wanted to do it and be a champion about it, sure, I'd probably create a reducer for the steps or something. If I felt like going through and again, I've done it before. Each one of these had a piano roll on it. You do it in Canvas because suddenly you have if you have a step sequencer, right, and it's a piano roll, right? You can have 200 things on the screen, right?

JASON: Yeah.

KEN WHEELER: If you have it seven or eight times over, performance starts to degrade.

JASON: Sure.

KEN WHEELER: That many HTML elements getting updated is rough versus a drawing paradigm. But as far as the actual music of it, this really works fine.

JASON: Yeah. This is super cool. We've got 15 minutes left. So we can call it up here and say, holy crap, job well done. Or if you want to show any other parts you potentially mentioned doing an oscillator in here?

KEN WHEELER: Yeah. Do you want to do an oscillator here?

JASON: Why not? Maybe we can just add an overlaying tone.

KEN WHEELER: There is no reason you couldn't. Let me look at the tone docs for one moment.

JASON: Okay.

KEN WHEELER: I'm going to look at an instrument.

JASON: I'm going to play a track here. ( MUSIC )

KEN WHEELER: So in theory this should work, this synth right here.

JASON: All right.

KEN WHEELER: Let's add another track and call it synth.

JASON: F0 or something?

KEN WHEELER: No, we're not going to do that.

JASON: Fine.

KEN WHEELER: No, no. This works a little bit differently. This is not a drum thing.

JASON: Sure, sure.

KEN WHEELER: This is a totally different one. We're not triggering a note here. We can trigger several notes here. Right?

JASON: I lost track of where you are.

KEN WHEELER: Up top here.

JASON: Got you.

KEN WHEELER: Each one of these are different playable things. You have drums and then you have synth. So the track index only maps the drums to their trigger index inside of the sampler. For the synth, I might hard code a chord that would play. So we could have a C minor chord or something. We have synth here for a track. If we came down to our schedule repeat, I could say if index is equal to four three, this is where we'll handle the synth, okay?

JASON: Okay.

KEN WHEELER: Rather than triggering an attack right there, I could say so drum will just trigger the sample, right?

JASON: Mm hmm.

KEN WHEELER: If you do that on an oscillator, the shit will just play. It sounds crazy.

JASON: Sure.

KEN WHEELER: Here I could say synth.trigger attack release, I believe it is. And then I want to say that it's this, right? Say we want to do C3 we're going to do D sharp.

JASON: I love that you're pulling a chord from memory here.

KEN WHEELER: Yeah, if this is there then it should play a chord when this goes right here.

JASON: Okay.

KEN WHEELER: Let's bump that up just one before we do it.

JASON: Now try that with your beats.

KEN WHEELER: Okay.

JASON: Let's just get a basic beat. ( Beat )

( Multiple beats )

KEN WHEELER: New intro. Do you want to see something cool?

JASON: I did so many weird things.

KEN WHEELER: Keep it going. Watch this. We're programming and we are the masters of our own destiny, right?

JASON: Right.

KEN WHEELER: I can say if index is less than seven then I can do conditional chords here. Look at that right there. I'm going to do step up to let's see. Right? What we probably want is A sharp 3, then a D4 and then that should be fine.

JASON: Okay.

KEN WHEELER: Do a couple of those synths real quick. Let's see how that goes. ( MUSIC ) Greater than sixth, let's say. Oh, oh, wrong one. I'm using the track index, not the step index.

JASON: Oh, okay.

KEN WHEELER: I was like, what in the what up, peanut? ( MUSIC )

JASON: I love it.

KEN WHEELER: Now we're cooking with gas.

JASON: This is amazing. ( MUSIC )

KEN WHEELER: Let's bump up that BPM.

JASON: All right, all right.

KEN WHEELER: Boom. Hit play.

JASON: 120. ( MUSIC )

KEN WHEELER: Ready? Uh oh. Little chop and screw. Chop and screw. Java script, it won't be in synch. ( MUSIC )

JASON: Oh, man. This is so much fun. But just looking at this and looking at how many different customizations we were able to make, how quickly you were able to say I want this faster, slower, throw in different chords. The amount of games I can see people being able to play with this where you could do you know, like an assistive you basically taught me how to play a synth without me having to do a synth. You just made sure no matter what button I pushed it would sound good. That's really what makes this stuff cool. If you want to teach somebody to do this stuff, you don't just throw them in a room with a piano and say, make great music. You make things work sort of and then they dig deeper. I think that really does open up a lot of doors for play, creativity and for dipping your toes into this stuff without having to be like a trained musician.

KEN WHEELER: One cool thing about this is my entire music career has essentially centered around manipulating different machines to make music that I want. Right?

JASON: Mm hmm.

KEN WHEELER: If you're Jimi Hendrix you take a guitar and face the pick ups toward the amp. I'm switching out cassette tapes. Or a lot of time people in step sequencers like fruity loops. If they wanted, like, trap high hats they would go half time so they had the right resolution to do that. The nice thing about this is we are the masters of our destiny as far as what we want to make and how we want to make it. So I made a drum machine called the trap lord 9000 for a talk one time and I added a simple feature I have never seen. Yeah, the hype beat things. So I made a thing where rather than go half time with this, right? You could just shift click it and it would become a triplet.

JASON: Oh, that's cool.

KEN WHEELER: Like if you wanted a high hat trill like you would do for a trap, rather than having it basically trick the sequencer into letting you operate at that resolution you could go like now shift click one of those. See that step blinking? If you hit play you'll see what I mean.

JASON: All right. Let's go. ( Beats )

JASON: That's so cool.

KEN WHEELER: Do you know what I mean?

JASON: That's so cool.

KEN WHEELER: Some UX like that, I know that I need. ( Bass )

KEN WHEELER: Oh, shit.

JASON: Oh, man. This is cool. This is super fun, right? You should all go play with this.

KEN WHEELER: Try the buttons on the bottom. Do the hey. Do the yeah, for sure. You've got to hold it. ( Air horn )

( MUSIC )

JASON: Oh. That's super cool. This is so much fun. Is the talk where you build that up somewhere that I can send to people?

KEN WHEELER: Yeah, I believe. I believe that was react Berlin. I think so.

JASON: The new hotness?

KEN WHEELER: Yeah, I'm pretty sure I did that there.

JASON: Cool. All right. I'm going to drop it in the notes for people who want to check it out.

KEN WHEELER: Go to the end and make sure.

JASON: Okay. Oh, yeah. This looks right. Okay, cool. So if you want to see something else built with kind of the same spirit, looks like significantly more polished than what we threw together today, that will be a good next step. With that, I think we're pretty much out of time. Ken, if people want to do more are there resources or places to check out next?

KEN WHEELER: Yeah. You could definitely read Josh's article. There's also like an Ableton thing that teaches synthesis. From there yeah, the Ableton tutorial. Learning synth. Phenomenal resource.

JASON: Awesome.

KEN WHEELER: There you can understand it. It's very difficult to string the code together if you don't understand what you're building. Understanding what exactly how you make the music, then when you go to code it you'll be like, I have to do this. It's not going to give you the answers but it will give you the questions to ask.

JASON: What's great is it turns into the same sort of thing as programming, right? Like I have an idea for an app. Because I know how to program I know the questions to ask to get me to the finished product.

KEN WHEELER: Precisely.

JASON: If you understand how the music is made you may not know how to make the music, but you know how to get there.

KEN WHEELER: Right. Say I want the wubba wubba on a square base. Turns out you're using an LFO to tweak a low pass filter, what have you.

JASON: I love it. I also just realized you're the first person to come on the show who can actually drop a soundcloud link. So check out Ken's soundcloud.

JASON: The latest shit is so dope. You have to play it.

KEN WHEELER: This one? Banana clips?

JASON: Oh, here.

KEN WHEELER: LA monster remix. I found this unreleased Kanye song. ( MUSIC )

JASON: That's cool. All right. I have to stop that. I don't want to get a DMC A takedown when I publish this. Check it out. So this has been great. I'm super excited to have had you on the show. This is super fun. Thank you so much for hanging out with us. Chat, stay tuned. We're going to raid. Add the "Learn with Jason" calendar. One more shoutout to our sponsors. White Coat Captioning, FAUNA, netlify, Auth0 making it all possible. Thank you so much. See you next time.