Unify Data Sources with GraphQL

Captions provided by White Coat Captioning (https://whitecoatcaptioning.com/). Communication Access Realtime Translation (CART) is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings.

JASON LENGSTORF: Hello, everyone. And welcome to another episode of Learn With Jason. Today on the show, we've got Jamie Barton. Jamie, how you doing?

JAMIE BARTON: I'm doing very good. How are you?

JASON LENGSTORF: I'm doing great. I'm superexcited to have you on the show. Before we jump into what we're doing today, let's talk a little bit about you. Want to give us a little bit of a background?

JAMIE BARTON: Yeah, of course. So, I've been working in DevRel for the last 67 years, I think. I crossed lots of different domains, Ecommerce, mostly working with GraphQL over the last four years. I was working, um, as a Ruby On Rails Dev. I was building stuff with Flash and Action Script. So I've been around a while. These days, you'll find me working with all the cool tools. [Laughter].

JASON LENGSTORF: Very cool. Yeah. So that's� that's funny, my� except for the Rails, I think you and I had the same career trajectory. I was doing a lot of Flash stuff, back in the day.
Well, cool. We wanted to talk today because you've got a lot of experience and insight into something that I think is becoming a more and more common pain point for devs, which is, how do you take the unbundled services that make up today's professional landscape? And by that, I mean, everything now is a service. You buy the bestinclass API for content, for Ecommerce, for, you know, data, whatever it is. And then you have these excellent tools, but when you go to build a web appealing with it, you have, like, 15 APIs you have to deal with. So you've doing a lot of juggling. It's running through multiple APIs and so you end up building a lot of� at least in my experience, a helper function that called two endpoints so I could make my internal function return the object that I needed.
That space is, I think, like, this is the blessing and the curse of unbundling everything is, you great these great tools that do one thing really, really well and you get the downside of having a million API endpoints to hit.
So, I feel like you've been working on this space and you're specifically in this space right now. How are developers solving this problem for themselves?

JAMIE BARTON: I think you're totally right. We have an abundance of different SDKs, APIs that we all plug in, like you say, for these bestofbreed tools� I don't know if that's the term we're still using these days. We use these bestofbreedtools to use everything. For a while, we created these helper files and API routes and serverless functions when we wanted to run serverside stuff and we had APIs on top of APIs and create the typed API routes.
And I think� in amongst all of that, you'll know this more than anyone I guess, when Gatsby first came out, you could talk to one API to fetch data. Yes, a problem existed for mutating, you had to create all of these service endpoints. But for query and data, that was quite a nice experience, for a while, actually. And now, as more things are moving back to the server, we're now again installing SDKs and we might be able to run code on the server with server components, but we have the problem of installing dozens and dozens of node modules to use all the tools we want to use. They all have APIs, they have data APIs and the one thing we're focused on is trying to bring that stuff to the edge faster and users aren't having to pay for it. I still think today, people are creating these endpoints, but what a lot of people are doing is creating these subgraphs and a supergraph on top of it or a federate service, whatever you want to call it�

JASON LENGSTORF: You just introduced a handful of new vocabulary words there. I want to take a second and think about what that means. GraphQL, in general, the idea of GraphQL is that you have sort of one endpoint and then within that endpoint, you've got� like� a grouping. So, you would hit the GraphQL endpoint and say, I want to get Account or End User and you nest the fields you want from them.
When you say a subgraph or a supergraph, what does that mean in the context of GraphQL?

JAMIE BARTON: The best way to understand this, for anyone, it's more of an architecture thing. These different microservices, you know, all these kind of dedicated services that do one thing, you can think of those as subgraphs. So, there could be a user service, there could be a friend service and then you have all of these different services that you call out to and those kind of create the subgraphs and then the supergraph, or, you know, whatever you want to call it, is a gateway to all of these services so you might have a GraphQL API that has all of these things, but when you call the different things, they go to different services beneath that. Yeah, so I think that's the best way to describe it is you can have all those functions you have, you can go through a another layer. You can put stuff at that layer, so, things like cache or whatever, so you don't go to the services beneath it. But, yeah, let's not get on a tangent. [Laughter].

JASON LENGSTORF: This is such an interesting space� they're dumping the recycling out, outside, so it's glass bottle being thrown around�

JAMIE BARTON: Throwing out all those different microservices and introducing one. [Laughter].

JASON LENGSTORF: This is a really familiar problem to me. One of the ways I got into open source was I was at IBM and we had severaldozen teams that had to speak to each other so we did it through microservices. Each team had their APIs and they were exposing those APIs sometimes with an API open definitions, sometimes without. Sometimes there was documentation, sometimes there wasn't. And there wasn't a communication channel. People didn't know whether the APIs worked, if they were maintained. The teams didn't know which endpoints they exposed were still being used or which pieces of data in those endpoints were actually necessary. So we� we looked at it as� this was before concepts like a subgraph or a federation. GraphQL was pretty brandspanking new. If we got each of these services to write a GraphQL schema that defined their APIs, you know, the benefits to me, and my team, were that we'd one place that we can go and look and it's selfdocumenting because if you write a GraphQL API, the schema is the source of truth. You get documentation out of that.
The other thing that we really liked was that the teams that were exposing these APIs only had to notify the central layer. They didn't have to tell the other 30 teams what changed. There were all these benefits and these upsides. Granted, like you mentioned, the complexity doesn't go away. Complexity� you don't, like, remove complexity, you manage it. So you try to find the right layer to move it to, that makes if feel as manageable as possible, and, you know, there were a lot of� I mean, this is� I did this back in 2016, I think. So, a lot has changed since then. There are a lot of advancements in the space and a lot has improved. And that's actually why I'm really excited for this because I haven't been in this space for five or six years. Oh, my god. [Laughter]. So, let's talk a little bit about� about, like, how you're solving this problem, specifically, at Grafbase. So, to set the stage, like, I'll come in, if I'm a standard company, and I'm going to have internal APIs. So, I've got my core service, maybe that's a Node microservice or Ruby, that's maintained by one team. I've got my frontend team, but they're running memberships through Stripe or analytics, maybe we're a dev tool service and we're reading through GitHub. All those have to play together nicely in our appealing. So, if we're coming to Grafbase, how do we solve this problem?

JAMIE BARTON: I want to unpack that a little bit, as well, because the thing that is our secret sauce, actually, is something that people don't realize is possible with what you just said. What I'm saying is, if you've got all these different data sources, some of them can belong in one region and these can be scattered all over the world, where your data is. And a lot of services allow you to pay to, you know, replicate that to multiple regions. But you have this kind of growing number of multiregions you have to pay for and then that's efficient because you make one request and it's going to make hops all around the world to get back to you and that's fundamentally, our secret sauce, our bread and butter, what we're solving. We can take whatever data source you connect and bring it to the edge and we're building on the Cloudflare network so we can do that with 300 data centers across the world so that's fundamentally, I think, the message that I would like the company to start projecting, is that forget about just unifying data sources and GraphQL, but your data can be somewhere, in one location that you pay for, but you can bring to the edge with another service in front of it and then you get all these other bells and whistles on top of that with the service.
So, with Grafbase� in a moment, we'll go to the website and we'll sign up for an account and deploy something really quickly from our templates but then we're going to dive into the code, I think. How we can bring in API or a REST API and transform that to GraphQL and then there's other things like, if you've ever deployed� I'm sure, I think I'm� I'm sure you have created a GraphQL server and you create a resolve. We're ultimately trying to be this onestop shop for deploying GraphQL APIs at the edge. But the beauty of that is, you can connect all of your data sources. Yeah, I think that answers your question, but I wanted to talk about that edge thing because I think that is really, really important.

JASON LENGSTORF: Yeah, especially as we're� it feels like it wasn't that long ago, when services would kind of explicitly be U.S.only or Europeonly or whatever. And so they weren't really thinking about, what does it mean if I'm in� the data's in San Francisco and the customer's in Australia, the speed of light means that that request is going to take an extra, what, 200 milliseconds? It's not fast to send a request all the way around the world. And when you're doing a lot of requests, if all of your data is in U.S. East 2, folks in Australia are going to have a hard time with your appealing. It's going to feel slow to them all the time. So finding a way to globally distribute in a way that doesn't have an exploding cost or doesn't have exploding complexity is very tempting for most businesses today, where when we launch, we're launching globally, we're not thinking about geography.

JAMIE BARTON: People, including myself, make the assumption, if I create an API route at the edge, then I solve all my problems because it's at the edge, around the world. If your data is in U.S. East and you're in Singapore, it goes to the closest edge but it still has to make that hop over to the U.S. And back to the edge, back to you. It could have been quicker. So, what we are saying is bring all your data to the closest edge, essentially.

JASON LENGSTORF: When you say that, it makes me think about, like, so� that's an API endpoint with a single data source, right. So, when you hit the edge, and then you've got that round trip to U.S. East 2 or whatever, that's before you consider that once you hit U.S. East 2, the data there is going to make a request to something else that, maybe if you're lucky, is in U.S. East 2 and is fast or maybe it is in Singapore. Initially connected to that local host that's, you know, 500meters away from your house, or whatever Cloudflare's density is these days. [Laughter]. The complexity of trying to get this right, when you have sufficiently� I mean, any kind of complexity in your application, it just gets out of hand so fast because you're not just thinking about one hop now. You're thinking about every hop in the chain and so what you're promising here, with Grafbase, is that you've taken that and you've decided, we're going to manage that for you so that when you interact with Grafbase, if you hit our edge point, everything is going to happen here. If you're hitting that Cloudflare edge, that is only change that's going to be made.

JAMIE BARTON: Small caveat, we do this through edge caching. We will� and we also� with that mechanism, if you request something, it's not in the cache, it's not in the edge, it's not distributed, we'll go to the origin. We revalidate so you'll get the data fast, but it might be outofdate by a millisecond so the next request will get it. Thanks to GraphQL's Stream and Defer that's coming, if you did get that stale response, you can stream the update to the page and no one would really know. So that's what we're working on. That was just a small caveat. We're working with data APIs so that's how we're able to keep it fast.

JASON LENGSTORF: Nice. I want to see this in action. So, why don't I take this over into the pair programming view here...and, all right. So, first and foremost, we're talking to Jamie today, so make sure you get on the social network.
This episode, like every episode is being livecaptioned. We've got Vanessa from White Coat Captioning. A real human is really pushing buttons, which is wonderful. That's made possible through Netlify and Vets Who Code, both kicking in to make this show more accessible. Make sure you go and check them out.
And I think that's all I know. Let me go to the Grafbase site. We're going to be looking at that.
And that's all I know. So, if I want to try this out, what should I do first?

JAMIE BARTON: I think the first thing is to sign in. Um, so, there should be a "get started." You can sign in with GitHub.

JASON LENGSTORF: I'm in.

JAMIE BARTON: Cool. So, um, what we'll do, now, I guess� this is using� what you see here is a few templates we have for our database offering. This is here if you want it. It's basically� it's almost a store, with a few extra things, like uniqueness, length validation. We'll get started with this so we can deploy something and then we can dive into the more advanced stuff. Maybe pick any one of those, chat room, blog, whichever.

JASON LENGSTORF: Let's do a blog.

JAMIE BARTON: Cool.

JASON LENGSTORF: So I'm going to connect my GitHub account...

JAMIE BARTON: And this is the� the way we got started here is actually with the web interface so you can also do all of this through the CLI, as well, which is cool.

JASON LENGSTORF: Okay. I'm going to add an account. I'll add it to Learn With Jason, if I can find it. There it is. I'm in too many GitHub repos. [Laughter]. All right...

JAMIE BARTON: Cool.

JASON LENGSTORF: And, let's see. I'm in Oregon so that seems to make sense. I assume I don't need any environment variables.

JAMIE BARTON: Not at the moment. Click "deploy." It should create it on GitHub. We've got an endpoints and that's our GraphQL endpoint you would use when you go on to build a frontend. You can use this URL.
So, this is just the dashboard right now. As you start to do more things with your API, more things will� you know, will happen here. But� let's talk about maybe some of the concepts that's going on here. We got project. If you move into deployments now, here, you can get a list, like you would if you're with Netlify. And you deploy something, all of your lists of deployments would happen here and all of your branches. And then with Pathfinder, that will launch with your project� maybe it needs to be refreshed or something. That should be deployed anyways. Once it's done, that'll be there. And then Schema, you'll be able to see the schema once it's deployed. It's obviously still deploying as we speak.
The great thing about the analytics, as well, it's builtin. So when you are making requests, you kind of get full view of that. So, there, we've got an "inprogress" deployment. This normally only takes a few seconds, so, yeah, it will assign domains and certificates. Once that's done, you'll get a URL and you're free to use it. If you go over to GitHub now, to that� that URL, you can� yeah, click that. What you'll see is, you have a project here, and you have Grafbase and you have "config." It's using a TypeScript config. Now, for anyone that's familiar with GraphQL, you can use GraphQL SDL to do all of this. These days, we're all using TypeScript so we have an SDK. Anything we change in there, if you were to create a branch now, we would take the configuration and deploy an immutable URL, that contains the changes and when you're happy and you've merged the pull request, those changes go to prod. Same experience, I think, that all of the frontend things do with the preview deployments. We wanted to take that experience in, as well.

JASON LENGSTORF: That is� I mean, that's great. It feels very common to� I guess� how I'm used to working with the web, which is always nice. So, we've got a successful deployment.

JAMIE BARTON: Yeah, so�

JASON LENGSTORF: And, look at this...

JAMIE BARTON: Yeah. So, this is what we call Pathfinder. It's similar experience to GraphQL or GraphQL Playground. This is something that is not just specific to Grafbase. Next week, we are launching this UI as a desktop appealing so you can use any GraphQL appealing and make requests with, as well. And inside of there, you can make GraphQL queries and whatever. So, I think what we could do� that's a single query. You could go to the Postcollection query and you could� you could create that query.

JASON LENGSTORF: Okay. So, let's go with� let's see...edges, node. We'll get the title.

JAMIE BARTON: So, you'll need to, with this, before we run it, is pass some arguments so if you go into the arguments and click "first," we require no more than 100. That returns the data. That's a very GraphQL experience, right. Query response. These queries, you'll see, on the right pane, at the very bottom, that you get the history. If you open the history...you should see there that you get a response. And then if you run that again, you'll see that response time decreases because it's then cached. Imagine that with all of your different data sources. We do that. So, this is using our own database right now. But, yeah...where should we look now? So, the� this is a query, obviously, GraphQL does mutations to mutate stuff. We build all of those, as well, for the� all of the different things to create and many as well. So, those are all generated automatically.
Um, but, yeah, this is very, very standard stuff. Very basic stuff, I guess, if anyone's used any of the providers that do all of this. The real fun is with the connecting stuff. Here, you can type in anything inside of there, I think, for the input. Just hit "return." Yeah.

JASON LENGSTORF: You can do something like...

JAMIE BARTON: Yeah.

JASON LENGSTORF: And then it wants more things.

JAMIE BARTON: It probably wants a lot more, like an author and stuff. We could type all of that out.

JASON LENGSTORF: Oh, wait. What does it want? Oh, we need a subfield so I would get back an ID. So, if I run this "field of slug is required."

JAMIE BARTON: It's going to ask for all the required fields. Still requires something, I think.

JASON LENGSTORF: Or, we're going to get in to a spot where I probably don't know where to go. Yeah, this is going to take longer than we probably want to spend. [Laughter]. But, like, the fact that it's just coaching me through this and saying, "hey, you all have to send in a user." I haven't read a document. I know that GraphQL works like this so I can basically fumble my way to working queries, which is� I don't think something that is� I don't know how you would do the same thing with a REST API. This is maybe the biggest argument I've made in favor of GraphQL. If you've got a team and the team isn't great at docs or communication and it's got to be crossfunctional, GraphQL is a way to basically say, the code in courses communication. Because there's no way to make a GraphQL API that somebody else can't use through trial and error.

JAMIE BARTON: Yeah. Yeah, yeah, yeah. That's� when we get into the local version, you'll also see you can see the full schema and there's a little bit more to that, as well. So, yeah, what you see here, you can see everything in there. That's one of the things I think a lot of people enjoy with GraphQL was the selfdocumenting feature. It was just built in. So you could create documentation automatically using all of this� all of this fun stuff. So, yeah, like you said, you don't need to go to the docs and when you make a mistake, it often tells you. That has its concerns, as well, so we are working on ways to disable that because, you know, say someone wants to do something bad and you want to disable this kind of thing, obviously, we want to give people the tools to do that. So, yeah, we're working on that, as well, to make that possible.
But I think� yeah, we could now take� if you want to clone that repo that it created, on GitHub, to your local machine, we can�

JASON LENGSTORF: Set it to private. I'm going to make it public so people can see it. If I want to do that, I do it where? Okay. So, now, we've got...a repo that everybody can go look at, if they want to. And I'm going to clone it and VS Code is not open yet, so let me open that...and then...and, do a GitHub repo clone, was Learn With Jason Blog...okay. And, there we go. Okay. So, now we've got our setup.

JAMIE BARTON: Yeah, we've got everything there. So if you open up the terminal now� we might need to run NPM Install. You can run NPX Grafbase Dev...and if you're happy with that, yep, everything we've seen online, all running locally. You didn't need to do anything.

JASON LENGSTORF: Ah, cool. Where did this go? It's here. Where on earth� oh, it opened in the wrong window. Um, okay. I need...to make...yikes.
[Laughter]. Had this very helpful thing, that is not being helpful at all. It's so great when it works and then when it rebels, it rebels so hard.

JAMIE BARTON: I was like, wow, those are, just, like, assigning themselves positions, that's cool.

JASON LENGSTORF: They're supposed to do what I say, but they do what they want. [Laughter]. I'm in Local Host and I've got the same thing with Pathfinder and if I want to hit the endpoint, which there is no benefit of me doing now, it right here.

JAMIE BARTON: The endpoint with the Pathfinder, it'll call out to the GraphQL API. For the Local Host, you see the root here, the UI experience. In the future, we want to bring more of that production dashboard feeling into the local experience so everything you do in prod, or in your dashboard, you can see locally, as well.

JASON LENGSTORF: Nice.

JAMIE BARTON: And with what you've got here� we don't want to waste our time doing that query again. That is now fetching from a local database that we've created for you, using SQLite. So�

JASON LENGSTORF: Oh, that's cool.

JAMIE BARTON: You're not having to do any data requests, which is cool.

JASON LENGSTORF: And that shows in the speed is half of� because, you know, it's all local. Very cool.

JAMIE BARTON: Yeah. So that is basically getting started. You can also run NPX Grafbase Init and it'll ask you what kind of project you want to create. I wanted to go through the web experience because that's ultimately where these end up. Now what I think we could do is look to add a connector for a data source�

JASON LENGSTORF: Let's do it.

JAMIE BARTON: So, maybe we could begin with a GraphQL API, um, we could stitch that in. So�

JASON LENGSTORF: Sure.

JAMIE BARTON: � I think we could go to the docs, if you wanted to, if that's easier. Or we could just write it in code, whichever you prefer.

JASON LENGSTORF: We can just write it in code. Do you have an GraphQL API in mind or do we need to stand one up?

JAMIE BARTON: Very topofmind is the Trevor API, it's very old.

JASON LENGSTORF: What is it?

JAMIE BARTON: Country.trevorblade.com.

JASON LENGSTORF: Okay.

JAMIE BARTON: Grab that URL.

JASON LENGSTORF: Got it. And then I'm coming over here.

JAMIE BARTON: So what I would suggest now, actually, where we've got Comment, Post and User, get rid of all that. Now if you want to� instead of the import, Import Connector...then we can create a new const that it called "countries" and that can be equal to "connector." And then TypeScript and do�. And it'll show GraphQL. Parenthesis, we can give a name for the API, this is custom. So this is what will be used for the namespace. If you are using multiple Country APIs you might want to use different names. We could go as far as adding environment variables, you could deploy those if you wanted to. Yeah, I think that's all we should need for this, I think. You can move outside of that const definition and do g.datasource, and then I think in parenthesis, we can pass Countries...yeah. And then, as a second option, inside of those parenthesis, you could add an object with some options. We don't need to do that, but you could disable the namespace, if you wanted to, in there. So, that's really cool when you want to add the caching on top of any API and not have to change your queries.

JASON LENGSTORF: Yeah. Actually, I was going to say, like, I don't understand what you mean so I'm excited to learn more about that.

JAMIE BARTON: Yeah, yeah. So, we'll do that. So, save� save that. That should be running, still. Stitch this API into Countries, as a namespace...

JASON LENGSTORF: And, let's just get "name" and run it.

JAMIE BARTON: Cool. Cool. So�

JASON LENGSTORF: And there's a list of countries.

JAMIE BARTON: Magic. Awesome.

JASON LENGSTORF: That's pretty handy that that just worked.

JAMIE BARTON: Obviously, that API doesn't have any auth around it. If it did, you could specify environment variables. If your appealing had a tokengenerated, you could pass it through to the origin, as well, which is quite cool. You'll notice, here, that the Countries is the name of the namespace that you specified, so that's wrapped everything that's kind of stitched in, inside of that, but if you go back to the code and you add to the data source line, you add in Object after Countries, you can specify namespace as false and then that will remove that prefix. There you go. So, you got Countries.

JASON LENGSTORF: Now it's broken. And I need to change this to, like, that. And then we get the same thing. Okay. Yeah. All right. So� so, basically, if we are 100% certain that there will be no collisions in the queries� or mutations inside of two APIs, we can drop the namespace and then they just kind of merge together?

JAMIE BARTON: Yeah, yeah. Definitely. So, if you open up that History pane again, I'm not sure if the caching stuff is just run locally. If you run that a few times, you should see the runtime. It's going straight to the origin. In production� we'll push this to production later on� you'll get that reduced time. When there's no collisions, if you just wanted to use Grafbase to speed up GraphQL API that you already have and bring it to the edge, you can disable a namespace and your developer or whoever's implementing the API doesn't have to change the query. It just works. You change the endpoint and we work as a proxy.

JASON LENGSTORF: Ooohh. So, in that sense� I hadn't really thought about it like that. I can use Grafbase without using Grafbase. I can just build my API in whatever I want, proxy it through Grafbase and I get edge caching for free?

JAMIE BARTON: Yes.

JASON LENGSTORF: That's a pretty dangcool onramp.

JAMIE BARTON: Having to be able to disable the namespace, using it without using it is pretty cool. It just works. I think what we could do, actually, I'll show you some of the caching config, for anyone that's interested. At the Config Default Export, we can then add Cache. And then that's an object and we can pass in an array of rules and then we can pass� so, you can configure multiple rules for different types. So, here, we'll pass Types, and then that's an array of strings so we'll just� we'll cache everything, so we'll call it Query, with a capital Q. And that's the GraphQL root type query. Here, we're going to cache everything and then add in max age, as well, whatever you want that to be. 30, 60, whatever. Yeah.
So that's going to cache everything for that period of time. And then, once that time's up, it's gone out the cache and then it'll go back to the origin and update the cache.

JASON LENGSTORF: I'll make this shorter here.

JAMIE BARTON: That high is probably okay because those countries are not ever going to change.

JASON LENGSTORF: That's true. At least not within a few days a week.

JAMIE BARTON: It's not going to change, right. So, imagine that was a product API, products rarely changes the data. With these rules, you can be really specific and say "cache a GraphQL type" and you can then specify what field is the cache key, so a product ID's the key. So if anything changes at all in the cache, with any query you've made, if those two things are part of the key, we'll bus that for you. So it's quite cool, when you think about that, I guess.

JASON LENGSTORF: Yeah, that's really cool. Thinking about how I would use this, one� like, the old joke, the two hard things in computer programming, cache is one of them. [Laughter]. When you're dealing with caching, one of the things that I see go wrong the most is that if you're caching at the API route level, you have two options. You let data be outofdate through the cache duration. Or, you invalidate everything on every data change. And, both of those solutions are fine, in context, but not great. And, so then� then people start to get clever. They'll start to do subcaching of whatever thing it is based on query parameters and somebody finds a way to break their cache and they can't get out of this loop and production goes down, right. And so, I� I find that when it comes to caching my� my instinct is, like, go as coarse as possible. My product is going to have all the details, inventory. If I can say my cache key is the inventory level and the price. If we fix a typo in the description, that's okay, that can age out. If I change the price, that needs to happen right now and if those are the only two fields I care about, that's a type of precise caching that I understand and that I think anybody can really understand, that I� and I assume you all have done the work to make sure this isn't one of the things where I, through my own unwitting behavior, can put myself through a death caching spiral.

JAMIE BARTON: There are multiple APIs to resolve anything that could be outofdate. There are perfect cases where you're not bothered when the description is updated because it can be cached for a day. We have APIs where you can actually remove the cache, you can purge the cache. But if you� let's say you have a GraphQL API, let's take this Countries one, for example. It's only got queries, right. But imagine you had mutations to create a country or a continents or whatever, and you created a new country that belonged to something else, we can detect, through that mutation, the ID and the type, and we can then invalidate anything in the cache with those� with that. And you can configure that as much or as little as you like, so you can say, um, you know, with what we call Mutation Invalidation, you can pass in different properties to say� in different scenarios. So you might not want that default behavior. You might want to cache it on a different field instead of ID. You can do that, as well. We've got some reasonable defaults. If you want to kind of turn any levers or knobs or whatever, all the tools are there to do that, which is cool.

JASON LENGSTORF: So, a silly, little programming aside, I didn't go to school for computer science, right. And so every once in a while, we talk about concepts that require very computer sciencey, like caching. As you were talking about this, the only computer science term I know about caching, pinged in my brain� and I was like, you got to work out a way to say Merkle Tree. I don't actually� I don't need to say that. I thought it was worth mentioning, if you don't know computer science words but you get excited when you get a chance to say them. [Laughter]. I also have no idea if Merkle Tree has anything to do with what we're talking about now. It is one of four computer science words I know.
Anyways, so, I mean� I mean, this is supercool. The fact that with 21 lines of code here, we have pulled in an external API. We have proxied it and added caching, which is already, like, holy crap. And, the fact that we can then say, like, specifically how long we want it to cache for and with another couple lines of code, which I think, if I just autocomplete my way, is this the� is this the invalidation key, or something, where I can say what the fields are? And so I would just add the name or something?

JAMIE BARTON: Yeah. There's a certain way to write this out, I don't use that often. Might be an object or just a string, but, yeah. And also, I just noticed there, we announced this� we're announcing this tomorrow, actually, is Scopes is another property in there. So, imagine  this Countries APIs returns a data that is accessible to anybody. Imagine you are caching data specific to you. Say you login� wherever you buy things from and you want to view your orders, but you want to query for your own orders and cache that. If you cache the query to Orders, it's going to be someone else's orders, probably. With Scopes, you can say "scope it to an API key." Or a subclaim in the JWT or a random header, which is cool.

JASON LENGSTORF: That's dope! So, like� that's fascinating because that means that things that I've always just assumed you don't cache� you never bother caching a userspecific query because the risk of accidentally showing somebody else's data is just too hi. [Laughter]. Like, whatever, just don't bother. It's fine, they can wait an extra 100 milliseconds. But what you're saying is here, if I set this header to be their specific authorization token, then for the duration, it will be cached?

JAMIE BARTON: Yeah, so I think� it's probably a Claims that would use there, with a JWT claim, we could say "sub" or "ID" or "user ID" and then we will decode that key and use that.

JASON LENGSTORF: That's so cool. That's incredibly cool. Okay. So, we don't have a user to actually test this with so we can� we can let that sit. I mean, that's� that is a thing that I have just� it's the sort of problem that when I run into it, I say, "I'm not dealing with that" and I move on. Now there's a button you can push that makes it happen, that's pretty dang compelling.

JAMIE BARTON: And that behavior, again, is default. If you want to change any of that, you go and configure that. We've got defaults in place. You don't have to think about this. It works in a way that you probably wouldn't expect it to. We are fundamentally really big on DX so that's why we love this stuff, I guess. [Laughter]. Yeah, I think what else would be cool is how we could create a GraphQL API from Stripe, Stripe having an open API spec. If you want to Google the URL, I think it's there, we can convert it to GraphQL.

JASON LENGSTORF: It's this one, right?

JAMIE BARTON: Yeah. View the code, maybe. Open API folder...spec 3.JSON. Yeah. If you click� yeah, and then just get that URL.

JASON LENGSTORF: Okay.

JAMIE BARTON: I think we can use that. I don't know if it's JSON or YML. We'll find out. Back in the code, we could enable Name spaces for that other query, just so we don't break too much.

JASON LENGSTORF: Okay.

JAMIE BARTON: You could remove the option here. And then that nests it again.

JASON LENGSTORF: Stripe, I guess. And we'll say "connector" and I saw open API in here earlier. Say, "Stripe."

JAMIE BARTON: Yeah.

JASON LENGSTORF: Like that?

JAMIE BARTON: Yeah. That's all we need.

JASON LENGSTORF: Okay. That's all we need. Okay. So then I'm going to set this up with Stripe...and we're going to go look at our API and see what happened...

JAMIE BARTON: So, it should be in there now, it's working on it.

JASON LENGSTORF: It is. It is restoring previous states. Holy crap, here we go.

JAMIE BARTON: Yep, so, this is taking that specification. It doesn't look too nice right now, but I'll show you how we can fix that. It's taking the Open API spec and it's transformed all of those queries in there. If you click Mutations, you'll see the mutations are created, as well. Create Stripe checkout tokens is a popular thing we do, that will be in there somewhere. You have an API for Countries and Stripe in one API. Now imagine that Countries wasn't the Countries API but it was Shopify, you can integrate Stripe. Maybe you weren't using Shopify, maybe you were building your on commerce solution, the Countries API that you added could be a CMS, we could have products in our CMS and then Stripe is our payment and cart and then we build our own platform by unifying this� this stuff, which is cool.
So� yeah. So, these queries don't look too nice. We've got, like, Post� I can't remember what was on the queries, but they don't look that great so we've got some kind of customizations that you can do, as well.

JASON LENGSTORF: Okay.

JAMIE BARTON: Inside of the code. So, as another option, we could add Transforms...and then we'll just need to� if you do Schema� do an arrow function with Schema being the argument...I think we could do Schema.� and it should give us some help. Query Naming. And then, in parenthesis, we can provide� I think there's only one option, a string option?

JASON LENGSTORF: Okay.

JAMIE BARTON: Oh, no, there's two. Operation ID. And then now, hopefully...it's a huge API, so it looks a little bit different now. It's using a different property. So, it doesn't look too gross.

JASON LENGSTORF: Well, they're at least grouped now, which is much more useful. But the queries look great.

JAMIE BARTON: Yeah, yeah. There's ways with the mutation to do that, as well. If you go back to the code, there's way to remove stuff, as well. So you can do schema.exclude and then you could pass in there. Let's say, for example, you wanted to exclude every query, you could do that. Or you wanted to just keep a certain subset of queries, you could do that, as well, which is pretty cool.

JASON LENGSTORF: Okay.

JAMIE BARTON: Yeah, it's� Stripe's a huge API, but you might only use one part of it so what's the point of created all those queries every single time?

JASON LENGSTORF: Right. That's supercool.

JAMIE BARTON: Yeah. So, if you did make a query to Stripe here, it's not going to do anything� I don't want to waste your time going to get a Stripe key order there. So, if anyone wanted to, they could pass headers inside of this Pathfinder and you can add a custom header and then you can configure that so that passes along to Stripe, as well. This could be Stripe API Key and you put in the value and then that sends it on over to Stripe. If you wanted to store your API keys, you can create a�.env file locally and when you deploy it, you add that key, as well. And then it would act� if you made a query to Grafbase then, it would just work, it would just let you in because you've got that key.

JASON LENGSTORF: Got it. But to do that, we would have to do something like� like� are we doing it like this? How do you do the� the replacement for it?

JAMIE BARTON: Yeah, I'll show you. So, like, ignore that value. Leave that one asis for now. In the second box, you could just add any header you wanted. Authorization. And then, yeah, whatever key you wanted, A, B, C. And then if you turn that one on, there, now if we go back to the configuration...and say, after transforms. And the same kind of signatures, headers. We can create another function. Sorry, similar to the transforms with the�

JASON LENGSTORF: Oh, oh, I apologize. So, we go with� is it headers, like this?

JAMIE BARTON: Yeah.

JASON LENGSTORF: Okay.

JAMIE BARTON: And then, I think you can do headers.set. I think that's the� yeah. And then here�

JASON LENGSTORF: Oh, great. Here, we could do� got it. Then we would do the process.m.

JAMIE BARTON: You can. We made a helper for this so it's type safe. You can do .env and pass a value that lives inside of your variables file. We handle that if you go on to create that, that would give you type safety there.

JASON LENGSTORF: Cool. Got it. So if I do this, right� so, this is going to fail because there's no auth header set. Does that override this or anything? Injection fail because I don't have my thing set.

JAMIE BARTON: Where we've got g.env.authheader, if we have the authorization token saved in an ENV file, we want to forward your authorization file. If you pass an object instead, and then call Forward...you can then specify the name of the custom header that you called Authorization, as well. So, the first authorization is what Stripe expects if you're forwarding your header. So that should work, I think.

JASON LENGSTORF: Got it. Okay.

JAMIE BARTON: It'll work, but it'll tell you "access denied." [Laughter].

JASON LENGSTORF: That's cool. Okay. So, like, this is� I mean, this is the kind of stuff that� that� I think it just� it's very initially confusing when you start working on these things because, you know, you've got APIs within APIs and then it kinds of turns into this sort of, like, nesting doll of� of, like, okay, how do I get everything to go all the way through. So it's nice to see that these are kind of figured out as a firstclass service. And it's also nice to know that if I wanted to, I can pull those tokens in and manage it, here, at the schema level so that nobody using the API� like, if it's an internal API, they need an API key to call Grafbase, but they don't need to have the individual API keys for all the services that that unlocks. That's a huge benefit because I don't know about y'all, but every company I've worked at with more than 15 people has this horrifying culture of a secret DM in a plain text. [Laughter]. It's terrifying. [Laughter]. So, this is nice, that you can sort of get rid of the need of some of that by managing these elsewhere.

JAMIE BARTON: Yeah. Yeah. For sure. I think one other thing� we've not got time to go into this because I want to show you something else that is cool. But with that auth inside of Pathfinder, you saw that X key. If you pass the key to Grafbase, it gives you access to everything, but what about if you want to use the authorization header to use a token from something like Clerk, you can configure with Grafbase, your auth provider� I'll show you, very quickly, what it looks like in the code. If you go to the very top and we just import Auth and we create, just below that line� or wherever you like� a new const, and just call it Clerk. And then we have different authorization options here. JWT is very popular. If you call Open API Connect, then we can give a name to this� just parameter, sorry. We can give an issuer URL and any provider that supports Open API Connect will give you a URL that is for the issuer and what we can do is, we can inspect authorization headers that come into the Grafbase API and get that header URL and use Open API Connect and use that issuer to get the� I'm guessing, the secret or the URL for the actual JWT to decode it correctly. We can decode that on the server and make sure you're good and that's because�

JASON LENGSTORF: That's supercool!

JAMIE BARTON: You say, hey, I'm using Grafbase, this is what they're going to sign it with. When you make a request, we'll call Clerk, authorize it and let you through and then there's certain things you can do to cache that, as well, so you can say "don't go to Clerk for every request, go every once in a while." With that auth option, you can then say, "make my Stripe queries require Clerk Off, but my countries, make them public, I don't need an API key."

JASON LENGSTORF: Gotcha. That's superhandy. I love the sort of move toward making this stuff so configurable because auth is one of those things, auth opens up so many fun things to build on the web but it's such a pain. Having to roll your own auth solutions, it takes the fun out of building things. Knowing it's kind of baked into a lot of these services means that as somebody who wants to experiment and play, if I can grab Clerk and then I can say, all right, I want people to log in and I'm going to have a set of admins and regular access and public access, I can do that through a couple fields in the Clerk UI and then I can just grab it here and make sure that controls access to my API. Like, that's so much more pleasant than ways that I've had to roll this in the past.

JAMIE BARTON: Yeah. Yeah. Definitely. And this kind of� we try and make everything work together, so, with the Clerk token, you could specify the Clerk user ID is part of the scope. You could also add groupbased authentication, as well, authorization. So, any admin can access the Stripe appealing, but only�

JASON LENGSTORF: Is there a good� like, we don't have time to cover this, but is there a good place that somebody could go if they wanted to learn more about that, like, a doc or a tutorial?

JAMIE BARTON: Grafbase.com/guides. But there's stuff in here. There is documentation, as well, /docs. Log in with NextAuth. You can do that, as well. So, because of these standards, you've got the Open ID Connect, we conform to that and everything is able to work in a unified way, which is cool.

Very slick. You had said that you wanted to make sure that we got through something else so I want to make sure that I don't derail us too hard here. What is the next thing you wanted to show?

JAMIE BARTON: We could go on and on and on with adding different data sources. So far, we've connected a GraphQL API, and any Open API support two or three specification, you can create. There is MongoDB, what that allows you to do is create a connector to MongoDB and create models inside of this config so you could create a user model with a name and we'll handle everything needed to query from that or mutate to that table.

JASON LENGSTORF: Nice!

JAMIE BARTON: And unique fields, have unique queries. So, we'll do all of that automatically. It's working with Mongo right now.
Let's do the Countries API, we don't want to get that Stripe API key. If we go back to Pathfinder and if we� on the lefthand� on the left side...if you click the second icon down, under the logo. The third, whichever one you want to count from. [Laughter]. And then we go to...Countries Query. And then we'll take one of the things here. Countries Country, yeah. If you want to copy the name of that.

JASON LENGSTORF: Countries Country?

JAMIE BARTON: Yeah. That's the type name. Now back inside of the config now, if we do� just below� yeah, just below wherever. It can go wherever, to be honest. If you do "g.extend," and then inside of parenthesis, pass the type name. If we pass an argument�

JASON LENGSTORF: Object, got it.

JAMIE BARTON: And then inside of here� I'm doing all of this from memory. [Laughter]. I've done so many demos with this. [Laughter]. I think if you press Space, you'll get a bunch of options we can do. Maybe not?

JASON LENGSTORF: Am I in the� it looks like what it wants is a schema� it just gets a record, right?

JAMIE BARTON: Weird. Weird.

JASON LENGSTORF: So the definition it wants is query input. So, do we� are we adding a field here?

JAMIE BARTON: We're adding a field, yes. Yes. Then you should be able� there we go. You figured it out. [Laughter].

JASON LENGSTORF: And then, for this, we can probably just do something, um, every country is my favorite country.

JAMIE BARTON: Yeah, so, this is just a file name. So, if you just remove that function�

JASON LENGSTORF: Remove the function.

JAMIE BARTON: And then call the file whatever you like, "food," "Jason."

JASON LENGSTORF: Oh, oh, I gotchu. Do I have to put an extension on it?

JAMIE BARTON: It wants Returns.

JASON LENGSTORF: Returns is missing. I need to do a G String.

JAMIE BARTON: Or a g.boolean. These are the different GraphQL scalers that these will generate. So now we've saved that inside of there. Inside of the Grafbase�

JASON LENGSTORF: Now we create�

JAMIE BARTON: Create� resolve as folder inside of Grafbase and call it "favorite.ts." Then, we can just export a default function and you could just do "true." It doesn't really matter. But what you can do, inside of those arguments, is fetch data from the parent or pass arguments, get the contacts for the server."

JASON LENGSTORF: Do you have a way to� we'll just� yeah, I'm going to name this. Do you have a way to get types out so it autocompletes?

JAMIE BARTON: That is comes in a few weeks' time. You have to install a GraphQL generator to do that. This will be there, in a few weeks. That's what we're working on now.

JASON LENGSTORF: Nice. I've created this revolver. Theoreticallyspeaking then, when I come out here, I'm going to reload the page, I'm going to go to Countries. We've got our Countries. We go into Country. I'm going to get the name and whether or not it's my favorite. And, what are you mad about? Oh, you need the� oh, Countries. So, we'll get the name, favorite and we get� do we� we can just run this.

JAMIE BARTON: Cool.

JASON LENGSTORF: Woohoo! And this is the kind of stuff that I think is so impressive about being� like, working with GraphQL is� is nice for these reasons and, like, people have a lot of feelings about GraphQL, due partially to projects like Gatsby, which did a lot of work to popularize GraphQL and used GraphQL for a lot of things it shouldn't have been used for. People have developed opinions like, oh, GraphQL is too much work. This kind of stuff, I think, is so magical when you get to work with these APIs. Because what you're basically saying here, I know as I'm working with my thirdparty APIs, the reason we're using the thirdparty API is because it add value. We are using Stripe to empower something else, which means I need to be able to extend and modify and use that data and so if we're not doing it here, which would make sense, is the data� the data layer is the source of truth for data, we're writing middleware and they are gnarly, calling your fetch APIs, now you don't have a single source of truth, you have the upstream, raw data source of truth and then you have the midstream, like, middleware source of truth and when somebody runs into a code, if they didn't write that code, they have to debug in three different places. One of the things I've always loved about GraphQL, you can modify the data in place and it feels to the end developer, who's actually consuming the API, like it's all one, solid thing.

JAMIE BARTON: Yeah. Yeah. Yeah. I love it. I love it, as well, for that reason and I think two things here. I think, one, you can extend� like you just said, you can extend any API with your custom logic and you've probably done this, where you get your results from an API, then you do�.map. Then you get the fields you want, then you return a function, then you might have a callback and you do that in multiple places and it's silly. So, here, you can do that resolver function. So it's not making any HTTP requests. You get it deployed to the edge and that function becomes part of the service, so if it's calling a thirdparty API, it will do it.
One example, if people do go to the guides, is how you can extend Contentful. One thing I do in the guide, actually� I search Contentful or extending any GraphQL APIs. Yeah, it's the third one. So, in this example, I have a Contentful model for Property that has a name, image location, then I extend the location and I fetch from the query, the latitude and longitude, and then I make a request to the Weather API to get the current weather. Imagine you had a website with all these properties and you wanted to show the current weather. You can make one request to get this data. Yeah. I think it's cool because for a while, I think we went away from this and kind of created all these different endpoints to get data. I like GraphQL because it can put everything back in one place, in a single source of truth.

JASON LENGSTORF: And again, it's got to be the right tool for the job and what you're describing here is, like, if you've got a website and that website is displaying data. So, what you've mentioned here, you've got properties and you want to show current weather or restaurants and you want to show the current wait time or how long it would take you from your current location, those are things that lots and lots of people are going to load. If your website is doing well� which we hope it is� then your big issue is going to be, how do you make this stuff fast because these are complex queries. If you've got� what we end up with is not just that we have to call multiple APIs but we end up with a waterfall of data queries where you have to request the property information, then you have to parse it, then request weather information, then you have to parse it and return that to the frontend. You've got three� three to four legs of this trip, to get data from the user's browser, up to all these different points and back down with the data they need and if you don't have a way to, like, efficiently combine and cache that� especially when it doesn't change. If people are looking at Spain and they want to see the properties in Spain, the list doesn't change, minutetominute so you can get pretty aggressive about caching that maybe you want the whether to update hourly because it looks like you're showing the current temperature. Every hour, there is one, small query and every query is lightningfast. This is when you start looking at the right tool for the job.
I'm� you can do it however you want, this is a really nice way to solve this problem.

JAMIE BARTON: Yeah, yeah. Definitely. I enjoy using it to bring in other data sources and extend it without having to build my own server. You create a TypeScript file there, that can do custom logic, you can fetch data from the root, you can access environment variables, you can do whatever you need to do inside of there. So, yeah, I like it. [Laughter].

JASON LENGSTORF: Yeah. I think this� it's just a really nice� it's a nice workflow, especially when you're in a team where different people control different parts of the stack and if you've got, like, different owners for different APIs and then you've got somebody who's the back of the frontend owner, which is what I ended up being at IBM, then GraphQL's such a good solution for that because it allows the back of the frontend person to create a unified way of dealing with this instead of having to help each team maintain their own set of custom middleware.
The other thing that we didn't talk about is, like, GraphQL, because you only request perfield, like, can I see things, like, usageperfield as opposed to perquery to say, hey, we've got this really expensive, custom field and nobody uses that?
[Laughter].

JAMIE BARTON: The next thing of that is to be able to look at the field, like, tracing. I think, you know, some people might be familiar with tracing. Tracing that down to the field level. It might be that no one implemented the APIs that we built, is fetching the weather field so we can just get rid of that. But I think with caching, the concept is while we validate, we support. Before when you mentioned there could be one, slow query every hour, actually, there couldn't be, it would be fast, it would just be updated in the background. So, it's pretty cool that, you know, as� on the web, we've moved that along and things like GraphQL's able to bake that in and it's not GraphQL� there's something else anymore. You can use whatever you want.

JASON LENGSTORF: You can even do something like� I know that a lot of people have feelings about GraphQL mutations and sometimes I share those feelings. Why don't you use GraphQL for your query layer. Granted, you do tradeoff if you're not using mutations, you're not going to get the instance cache. But if that doesn't matter to you, then you can get this really nice query layer with good caching, basically for free, by stitching in your REST APIs. So you can kind of build and you can build the right solution for your team, based on everybody's skill level, who was to work with it, what their existing preferences are. I like the way that the space has matured to, instead of saying, the only way to get benefits of GraphQL is to only use GraphQL. You use it where it does make sense. It should be making your experience of building the application you maintain better.

JAMIE BARTON: Yeah. Yeah. 100% agree. And I� as much as I create content around GraphQL and work at a GraphQL company and have done for many years, I� I agree, GraphQL is not the best solution for some things. But what I think it really shines at is this ability to unify all of these different sources and when you put it in the context of caching, or performance, like the Stripe example, what a pain to convert. Because of things like specifications, like the GraphQL Spec, you can convert it to REST if you wanted to. We try to kind of not ignore, but we don't talk about GraphQL so much because it's just an implementation detail. It's an API, at the end of the day, at the edge.

JASON LENGSTORF: Got it.

JAMIE BARTON: I try not to get too hung up on GraphQL too much. It's just a fetch request at the end of the day. [Laughter].

JASON LENGSTORF: Well, cool. So, I think we probably don't have time to do any additional code. So let's do this, for anybody who wants to go deeper here, what should next steps be for somebody who wants to explore? We've linked to the guides. I'm going to drop another link to your social media here. Where else should somebody go?

JAMIE BARTON: Yeah. So, there is Grafbase.com, the docs are on there, as well. And that will teach you everything you need to know to get going with Grafbase and I have my own weekly videos, that is at graphql.wts. I show you things and teach you things. If anyone is knew to GraphQL, there is stuff on there, as well.

JASON LENGSTORF: All right. Well, let's� let's do this then. Let's give one more shoutout to our captioner, Vanessa. That's made possible through the support of our sponsors, Netlify and Vets Who Code kicking in to make this show more accessible to more people.
Make sure you go and look at the schedule. It's a little sparse right now because I've been lazy about my admin work. There's lots and lots coming up. You can get on the newsletter, you can subscribe on YouTube. You can go wherever you want to go.
Jamie, thank you so much for spending some time with us today. This was an absolute blast. Do you have any parting words for folks?

JAMIE BARTON: Thank you very much. And, yeah, I� if anyone can tell me how to get a Twitch username, get logged into my Twitch account, please let me know because I'm locked out. [Laughter].

JASON LENGSTORF: All right, y'all. We're going to call this one done. We're going to find someone to raid. Thank you so much for hanging out with us and we will catch you next time.