Why Should Web Developers Care About Logging?

Captions provided by White Coat Captioning (https://whitecoatcaptioning.com/). Communication Access Realtime Translation (CART) is provided in order to facilitate communication accessibility and may not be a totally verbatim record of the proceedings.

JASON: Hello, everyone, and welcome to another episode of Learn With Jason. Today on the show we are talking to Mat Wilk. Mat, how are you doing today?

MAT: I'm great. Thanks for having me on. It's a pleasure.

JASON: Give me just a second while I'm working through some challenges with my overlays here. Well, that's tomorrow's, what is going on? Why is (laughing). I can't win with this thing. Okay, so, something is going real weird with my scenes right now, and they are actively rebelling, and I don't know why. Now it's showing tomorrow. Cool, very cool. Great. Loving this for me and my scenes. Everybody bear with me for a half second while I debug my own silly code. I know what I'm doing. I'm a competent developer. I can write code. How's this going to go for us? Let me switch this over. We did it! Hello, everyone, welcome to another episode of Learn With Jason. Today on the show we have, as you can see on the name tag, Mat Wilk. How are you doing today?

MAT: I'm great. Thanks for having me on.

JASON: Okay. Off to an exciting start. It is so great to have you on the show today. I'm really excited, because we are continuing this special series of Observability Week. This is day two of three this week, and then we've got a bonus episode four coming a little bit later in March, where we're going to be talking about observability, why it matters, how it helps developers, and I think specifically to today, why web developers should be thinking about it. Because I think, you know, a big experience for a lot of web devs is that logging observability tracing, that's a thing that other developers do. And web developers are like, no, I console log, right. So, I think we're going to talk a little bit about that today. But before we get into that, let's talk a little about you. For folks that aren't familiar, can you give us a bit of a background on you and your work?

MAT: Yeah, sure. Just a little note on your previous sentence. I think sometimes developers think that, you know, logging is necessary evil. Like also error capturing and error throwing. But I think it's changing in the industry. Now it's more accepted to do it simultaneously when you create a PR. So, this is something that's moving in the right direction. But answering your question, so I've been a web developer for around 16 years or so. I mainly work with big companies. I didn't start this way, obviously. I remember during my studies I was, you know, building some websites for colleagues of mine. Not really colleagues, friends. And I got hooked. I actually started electronics and computer science, not really programming.

JASON: Sure.

MAT: But I said to myself, okay, since I'm building websites, why not just go that direction, which I did. And after working a few years in the industry, I moved to London. And I've been in London for the last ten years almost. And I had the privilege to work for big companies, which I really appreciate, and I learned a ton of useful information. So, in the context of observability, you know, it wasn't such a thing as observability when I started programming. And I think progressing and along with methodologies, like other best practices, and I think it's becoming more embedded into the industry nowadays.

JASON: Yeah. So, I think from my perspective, and I think I'm coming from a very, you know, I think I would be I would have originally described myself as like a front of the front end developer. I work primarily in the design side, the visual side, and over time I've sort of moved toward the back end of the stack a little bit more. But for me, and I think a lot of folks who would identify similarly, when we start thinking if you say observability to me, I kind of know what you mean, but I think a lot of logging is when I hit a problem, I console log. Own that console log, I'll try to figure out what that is and go fix it. And I know there are more tools out there, because I've heard of things like if you're running a backend server or your database or whatever it is, you have things like tracing and open telemetry and these really robust logging systems that give you a lot of visibility into what's going on. But I've always sort of felt like maybe that's not for me. Maybe that doesn't work on the web. But I don't think that's the case. I feel that's maybe a thing that I believed a long time ago that's not true anymore. So, how does my concept of console logging fit into the broader world of what observability enables in the modern web and app and backend space?

MAT: Yeah, that's a brilliant question. Logging as a concept is like an umbrella name for a broader aspect of different things. In the industry when we talk about logging or observability, we usually have a few different concepts in mind. And we can distinguish between different types of items. And these types we call them MELT. It's a shortcut for metrics, events, logs, and traces. There are subtle differences between all of them, and we can go through what they are. For example, a metric is usually a numeric measurement of an application or a system. Typically, it's like reported on a regular schedule. What's also interesting about different types of telemetric data is that depending on the platform that you use it with, you can, for example, roll the data up. So, whenever you send the log data, whenever you send the data to a platform, which is typically as we defined numeric data, after some time you don't want to use as much memory of that platform, so you roll the data up. So, for example, a difference between metrics and logs is that typically if we log an event to a platform, we never roll it out, because logs, by definition, is a message about a system or an activity in the system, and usually the shape of the log is, let's say, an object with attributes, but they don't necessarily have to be numeric. So, you can't really roll them up. So it means that logs have different shape, and usually it's preferable to keep all the logs in the platform or in the system or in the file, whatever you're using the logs for, to keep them. Or maybe in the

JASON: Okay.

MAT: And they also serve a different purpose. So, you know, metric measures a specific telemetric data. For example, CPU. We have the CPU usage, so this is typically a metric we want to log with metrics.

JASON: Right, right.

MAT: Metrics applies to different telemetry data. It's also an umbrella. So, when we talk about metrics, it doesn't have to be specifically metric that we just defined. It can also be a log, it can be an event, or it can be a trace. So, let me just quickly go through what an event is and what the trace is.

JASON: Okay.

MAT: So, event, a record of something that happened in your application. And events are typically reported when something happens. So, for example, let's say that user presses a button in your application, or maybe a file is uploaded, or another example of it would be, for example, let's imagine that you have a vending machine and you are provider of vending machines and you want to know how the vending machines are performing. So, when someone buys something, you consent an event to a telemetric platform with specific attributes that, for example, a bar, fizzy drink, at that time, for that price, from this vending machine. Et cetera, et cetera.

JASON: Got you. Okay, go ahead.

MAT: So, the last concept of MELT, which is trace, is a path request has gone through in the system. Usually, it applies to the distributed systems. You know, nowadays we have microservices, and we want to know, for example, whether a request went through our front end to the back end site of the application, and then maybe to a third party system.

JASON: Okay. So, to repeat that quickly, the acronym here is MELT, short for metrics, events, logs, and tracing. So, a metric is some kind of quantitative data. So, if I'm measuring page load times, or I'm measuring the amount of JavaScript loaded on a given page load or whatever that is, something I can measure and chart over time, that's a metric. An event would be if I'm trying to gauge what people are doing in my app or what actions I'm taking. Somebody signs up, somebody logs in, somebody activates a new feature. That would be events. Logging would be kind of anything I want to track, what I would traditionally put in a console log, and that's sort of unstructured, it can be whatever. That's my qualitative data. And then tracing would be if I'm using my API deployed to serverless functions, my front end deployed on Netlify, whatever, and I have a database over here and maybe some other services over there. Tracing would be when somebody clicks this button, hits this API, and that goes to the database, and that comes back. So I can see kind of how things move through my collected systems together. Is that a good overview?

MAT: That's a very good understanding of it. Typically, traces are related to HTTP requests when we talk about web development. So, basically, whatever is related not whatever is related, but from a technical perspective, how it works is usually you add a header to an HTTP request and send this header via different systems that you're maintaining. And thanks to that header, for example an ID, you're able to record that to a database or to a telemetric platform. And while you're able to recognize what systems this HTTP request goes through. And one more point about logs. Logs are typically used to recognize problems in the system. Not necessarily, but we'll see later when we go through our exercise that you can have different type of logs. For example, debug logs, error logs, info logs.

JASON: Right, right, right. Yeah, so, it sort of gives you a more wholistic picture of what's happening within your app, which is, I think, maybe this is the part where I start to see how it does connect the dots for web developers, because a lot of times when somebody would say something like tracing to me, or event logging, or things like that, it sort of I just kind of glazed over. Oh, well, somebody else handles that. The DevOps team will put that together. And it didn't really occur to me that a lot of the things that I would think of as somebody else's responsibility are things that I'm already doing. So, a good example of this is like events. I mean, I'm constantly well, when I was working at companies, I was constantly asked by the product team, hey, can you put these event logs in, we want amplitude events to see when somebody converts, when somebody activates this feature. That's a piece of observability that I wasn't really considering to be a part of that process. I was always trying to measure the core web vitals, the Lighthouse scores, things like that of people using my sites. I'd try to use a tool that would give me real user measurements and not synthetic tests like I ran a Lighthouse test on my own machine and it seemed fast, right. So, I'm already doing a lot of these things that would be considered observability, but doing each of them in a very unmanaged way. My performance metrics are, I don't know, over here in a pile. I don't keep them at all, I just kind of look at what the scores were for the last 24 hours and never look at them again. I don't have any trends over time, I don't have a way to say, oh, our core web vitals in Australia are garbage. We need to fix that. I just know that, hey, on average, it's okay. It's pretty fine. I don't see outliers or anything like that. The same with the events. It goes into amplitude, or it goes into a tool similar to that, where I throw it into a hole and maybe product uses it, maybe they don't, but I don't get any information back that helps me understand how people are using my apps. Then there's other pieces I hadn't really considered how useful they would be, like tracing, where I know for the systems I've worked on that my form submits to a serverless function, which calls out to this third party API, which calls back to the serverless function, gives me data on app. But nobody else on the team knows how I built that thing. So, having tracing where I can see this is the components in the system, this is how they are all connected together, so I can make more informed decisions about how I'm going to debug something, how I'm going to make choices about how we grow the system in the future. Those are all pieces of information I have wanted and in certain cases have used in I would say less than optimal ways, because it's all been very efficient ad hoc kind of stuff. So, what you're telling me is that if we're intentional about this as web developers, we can put all of these things together under one umbrella that we call observability and use it in a way where I have visibility into my application. So, can you talk about how does that work? If I want to pull all of these pieces of data together, which sound useful, and I want to be able to look at them, how do I do that?

MAT: I guess there are different ways of doing that. Typically, what's been happening in the industry in the recent years is that you would be provided to a so called agent, a small program that would be deployed either to your browser application or maybe to your server window application. And this would collect telemetry data. This is still a standard, really, but there is a trend of open telemetry, which is a new standard, new kid on the block, let's say. Because you don't really want to be vendor specific. You don't want to be locked into a specific vendor.

JASON: Right.

MAT: So, a lot of teams nowadays start experimenting with open telemetry. But it's like having an agent installed on a machine or in the browser, it's still a standard nowadays.

JASON: Okay, yeah. Well, cool. So, this is, you know, in this video, Observability Week is all kind of brought to us by New Relic. So, New Relic, the intention of New Relic is to give us the ability to sort of put all of the things we care about across the entire MELT spectrum into one dashboard to give us a little more visibility. Something that theoretically if anybody still worked in an office, we could have one big TV and show the New Relic dashboard and be able to identify quickly, hey, something is going wrong with the website.

MAT: Yep, that's absolutely right. And what I would add to it is that you can also monitor different parts of your application using different metrics. But I guess as we go through the exercise, we can talk about it in more detail.

JASON: Sure, sure. Well, and maybe that's as good a time as any to make the transition over and start working on a little bit of code so we can show how this works instead of just talking about. So, let me switch into pair programming mode here. And I'm going to have to fix my scenes again real quick. Everybody, please hold while I do that. Here we go. Hey! All right. Fixed it on the first try that time at least. All right, before we get started, and let me pull this off so it's not as confusing for everybody. Let's talk a little bit about our show. We've got live captions on this show, like we do on every show. Thank you to Ashly from White Coat Captioning, who is here taking down all these words. Observability Week is a special partnership with our sponsor New Relic. So, thank you New Relic for making this possible. And thanks to Netlify, Nx, and Pluralsight, who are also pitching in to make this show more accessible, more fun, give us a little more bandwidth to do more creative things. It means a lot to me. We are talking today to Mat Wilk. You are at webdev.engineer, which is a great you recall, by the way. Let me throw that in here for everybody who wants to follow along. And we are talking today about how to do observability, specifically using New Relic. So, I'm going to throw that in the chat, as well. Okay, so, Mat, at this point I have sort of reached the end of what I know to do here. So, why don't we opened in the wrong window. Come on, don't be like that. Come over here. All right. So, this is me on the web. Just a web dev looking at an expert asking for a little guidance. What should I do first? (Laughing).

MAT: All right. So, I'm guessing you have access to GitHub.

JASON: I do, indeed.

MAT: That link that I sent...

JASON: Yes, let's get that first link here. Here's the repo we're going to be working from today. This is a create t3 app app, which we haven't really used T3 App on Learn With Jason. We have used the component pieces, TypeScript, Tailwind, tRPC. That's our starting point. Actually

MAT: Main branch, yep, that's the one.

JASON: So, I can copy this here. So, if I come out to my terminal, and I'm going to do it right here. Launch repo.

MAT: So, while you're doing that, this is a Gitly app with the standard T3 components. As you mentioned, which is the Next.js, TypeScript, tRPC. As I was preparing this demo, I was learning tRPC while I was preparing it, and interesting I was watching your videos, as well.

JASON: Yeah, there's a great episode with Alex Kat who did tRPC, the creator of tRPC. So, if we pull that up, here is an episode if you want to dig a little bit deeper into what tRPC is and how it works.

MAT: Sorry, so, there was one more episode that really helped me. The tanstack. Because I had issues with rendering, I wanted to up animations, and I realized that, you know, you talked about a specific thing, a property you can to pass the query, which keeps the previous result from the HTTP call. And this really helped me, you know, work on the animation and make it work. Because otherwise the whole thing would re render and I wouldn't be able to get results the way I wanted them to be.

JASON: Yeah, yeah, this episode, this is one of my favorite episodes that we've done. Dominic was such a great teacher. The TanStack Query stuff is just very nice to use. That's a good one to check out. Welcome to 425 Show and raiders, thank you for coming by, really excited. The launch window I'm using is a browser called Arc. If you're interested in checking this one out, it's called Arc, I love it, I use it all the time. So, I've cloned this repo, and I have opened it up in VS Code. So, I'm looking at the repo here.

MAT: Yeah, probably a few things worth mentioning is it's a typical directory structure as you get it with the Create T3 App. There's nothing special about it. I left everything on default pretty much. I added a few subpages, so there is a blog page, there is the main page, obviously. And there is a post page. And there's also an about page. Mainly we're going to be working with the blog page and the post page.

JASON: Okay.

MAT: So, maybe we can just go through it and see how it looks like, so we could just do install.

JASON: Yeah, I ran the NPM install already. So, I'm going to start it you have. I'm going to make the assumption this is using NPM run dev. I'm missing environment variables. Okay, so, I need

MAT: You're missing there is the .example. So, if you just rename it. That should fix it. Missing we're not really using it.

JASON: It's saying it needs this Discord client ID. Do I need

MAT: Yes, yes, yes. Since it's a default T3 stack, it means the main page actually wants you to use the out to log.

JASON: Oh, so it wants to use got it, got it. Need to make sure it's defined. Okay, here is our local host, and it's now doing the thing. And because we're not using the next auth, we can ignore this stuff. So, we have our homepage here and looking specifically at our blog posts. So, here's a list of our blog posts. Those show up

MAT: Yeah, you can scroll down and also press log more. So, it's going to log more posts. And what we're using under the hood is the JSON placeholder API. What happens is that the front end is sending a request to the back end of our application, and then the back end is sending a request to JSON placeholder API. And JSON placeholder API, for people that don't know, is just an API that you can test your things with.

JASON: Yeah, you just give it an object shape and it comes back with Lorem Ipsum and all of those objects, right?

MAT: And you can also press on one of the blog posts, and it's going to redirect you to the posts page with the ID of the post.

JASON: Yeah, this is great. This is kind of exactly what I would expect a blog to function like. I've got a slash blog that shows me previews, when I go in, each blog has its own specific URL, in this case under slash posts, the ID of the post, and we have a homepage to tell everybody where we are and what we're doing. Yeah, this is great. So, what should we do next?

MAT: So, we want to integrate the New Relic browser agent with the application. So, what the browser agent is for, it helps us pull all the telemetry data, and we'll talk about what exactly telemetry data is from the front end perspective, but it pulls all the data from the front of the application and sends it to the platform. So, now we're going to be focusing on making sure that we're able to get the New Relic browser agent integrated with the app and send the data to the platform, to New Relic platform.

JASON: So, the browser agent is something that I can install manually in a generic way, but you also have specific agents for specific frameworks, right. So, the T3 app is using Next.js. So, there's like a Next.js agent, right?

MAT: So, it's a good question. So, what happens is that we have the browser agent, which cares only about the front end pages. So, we don't really need to worry about that back end or the Next.js or Node.js. And we're going to use the node agent, as well. Which will help us integrate the back end themes of the app, because Next.js is a metaframework, which uses Node.js and express under the hood. I hope it makes sense.

JASON: Yeah, no, I think and this is one of the things that can be a little tricky if you're not familiar with how a meta framework like next works. It's executing in multiple contexts. So, part runs in Node, part runs in the browser, and you kind of have to know which is which so you can instrument that properly. If we want to get data, we need to instrument both places. Okay, so, if I want to do the browser agent, where should I start?

MAT: Okay, so, there are two different ways of integrating the browser agent. One is a copy/paste method, which is a bit menial, so we'll skip that, but it's worth mentioning it's possible. So, copy/paste the browser agent script and put it into the head of your application. And then it will, you know, run a script as you visit a page. And send the telemetry data to your application. But instead we want to go for a more convenient method, which is an NPM package. And it's a New Relic NPM package. So, this

JASON: Which one is it?

MAT: This package... yeah, it's called New Relic NPM. So, basically, if you just NPM install New Relic...

JASON: This one here?

MAT: Yeah, this is the one. This is the one. New Relic node agent. And what node agent is able to do is also able to return a browser agent for us. It has a specific metric to return the script. So, instead of copy/pasting from the New Relic UI, we can use a method from the node agent to return the browser agent. Yes, I think I

JASON: Got you, okay. So, we have that set up. And it's now installed, so I'm going to hide that terminal. If I want to use it

MAT: Yeah, we need one more thing. Actually, we need a couple more things, but let's go through them step by step. So, the other thing that we're going to need is the either a New Relic configuration file, or you can so, when you install the New Relic NPM package, if you want it to work properly with your subscription to the platform actually, it can be as well. You need to know that API key. And you need to provide the API key to the New Relic node agent. So, I'm just

JASON: That's this file here, or I need to actually run this here?

MAT: I can't see it clearly.

JASON: Put it a little bit bigger.

MAT: Yeah, let's go with this one actually.

JASON: So, I'm going to run this and we'll copy it to newrelic.js.

MAT: I think we're going to have TypeScript complaining. So, if you change the file name, it's going to

JASON: CJS?

MAT: Yeah.

JASON: TypeScript, leave this alone, got it.

MAT: So... there are two things that we need to do here. We don't need to worry about the configuration. It's the different configuration to use. The only two properties that we need to worry about is the application name and the license key.

JASON: Okay. So, for my application I assume now is when I go to the New Relic dashboard. So, I'm going to log in. All right. Active session limit reached... what?

MAT: If you kill the earlier session.

JASON: Okay. Here we go, I'm in.

MAT: Yeah, that's perfect. So, go to the bottom left corner, if you press yes, this button. Then API keys.

JASON: Okay.

MAT: You can copy/paste. I'm seeing you've already created some API keys. For example if you copy the second from the top API key, then copy key. And copy to the license.

JASON: This one I want to do off screen, right?

MAT: Preferably, yes.

JASON: Actually, hold on. Before I do that, let's set the application name. Can this be anything I want, or do I need to define one?

MAT: Can be anything you want.

JASON: Okay, we're going to say T3 app Observability Week. Then I'm going to replace this with my license key. I assume I don't commit this to GitHub, that's how we make that work? So, I've saved that and it's now closed so that we don't leak those details.

MAT: Probably it's a good time to mention sorry for interrupting. Probably it's a good time to mention that you don't have to use the New Relic configuration file. You can use environment variables, as well. So, you don't have to worry about the configuration file.

JASON: Got you, okay. I'm going to come down to the bottom and include my NewRelic.cjs so that doesn't get committed to GitHub, so that will get ignored now. Now I think the next step is I had to actually execute this, is that right?

MAT: Let me think if there's anything else we need to do. We're integrating the browser agent, right? Okay. So, we need to pull the New Relic browser agent script and upload it into the head of the application. So, in the pages, document the file, which is not a standard file which is included, but when you create T3 stack, this file is not included in the build.

JASON: Yeah, this is sort of like an advanced Next.js use case, because by default, they sort of put this in one way.

MAT: Exactly. So, I created this file for us so we don't have to write it from scratch. So, what we want to do here, we want to first of all import the New Relic package.

JASON: Okay. Default like this?

MAT: That's the default, yeah. And what we're doing here, we're getting the New Relic node agent. So, what's interesting about this document file is it's run only on the back end side. And usually you use it when you want, for example, change the HTML or the body of your entire application. So, for example, when you want to include some scripts like New Relic browser agent script, then it's a perfect place to do it. But, actually, there is also a New Relic types package, which we can also install, so we don't have to TS ignore.

JASON: Okay. So, is that NPM install...

MAT: At New Relic/types if I remember correctly. Let me check. Sorry, @type/newrelic.

JASON: Okay.

MAT: Okay, amazing. And now...

JASON: There we go.

MAT: So, now we want to get the New Relic browser agent script from the package. And to do that, we need to call a method, which is called get browser timing header. So, basically, we need to do it inside let me check. So, we need to do it inside the document. On line 10? Yeah, line 10. Let's do it on line 10.

JASON: Okay.

MAT: Then there is one more trick which we need to do here, and this is a property. And this property... the method takes an object as an argument, and the property name is has to remove script wrapper. And this we want to set to true. So, what this does is it removes the script wrapper. Basically, when you call this method, normally, you would get an URL browser agent wrapped in a script tag. Since we don't want that here, we're just getting rid of the script wrapper, and the browser agent is just a script. Okay.

JASON: Got ya.

MAT: And the next thing we want to do is the Next.js has a concept of script. So, if we import script, from Next.js script

JASON: Next script or

MAT: Next script, next script. Next script.

JASON: Okay. Wait, is there a different one? I think this is the next boot strapping stuff.

MAT: Yeah, exactly. There is a if you put your cursor on line 4, I think yes, line 4. And import script starting from capital letter.

JASON: Got it. That's going to go in the head you said?

MAT: Yeah, that's going to go in the head. This is a self containing tag.

JASON: Okay. So, we want to use a property so, it requires a property which is called ID. And you can call it new relic browser agent or whatever. Then the other thing is the dangerously set in HTML.

JASON: Almost got there. Come on. You were right there. That's the browser agent?

MAT: It has to be an object. So, inside of the object that we already have, we need another object with a property of HTML, yeah.

JASON: Underscore or single underscore?

MAT: Double underscore. Also the browser agent has to be so, because we return props from the get props method

JASON: Do I need to include this? I need to include that here?

MAT: Yeah, yeah.

JASON: Why are you mad?

MAT: And we'll get rid of the TypeScript error in an S.E.C.

JASON: Okay. This needs to be props. So, I'm pulling props in from here? I've not used a class style

MAT: I would do props.browseragent. We need to get rid of the TypeScript error. So, we need to define a type at the top of the file, which is document props, let's say. So, if you go open the online 10, if you open the triangle brackets I guess you can call them. Sorry, not my document, the extend document.

JASON: This one, this one.

MAT: Yeah, document props. Then the problem is, as well

JASON: In the promise, document initial properties.

MAT: After the document initial props, yes, document props. And I hope it's going to make our TypeScript type errors go away. At least it should.

JASON: It's saying it still has an any value, and I feel like we can potentially just kind of tell it to go away. Oh, it's still mad. How dare you. Okay. I can also just turn off TypeScript.

MAT: That's one of the options. So, we can run it without any issues, because only when revealed it would fail, but not during that. So, we can limit that, to be honest.

JASON: Okay. So, that's in there. That works. Okay, so

MAT: So, this script, the next script tag is an interesting concept, because you can pass also a strategy property to it. So, if we also the other one. The one that we passed the browser agents to.

JASON: I got it.

MAT: We can use different strategies, and we want to use before interactive strategy. So, it means that our script is going to be loaded before the page is becoming interactive. So, as soon as possible pretty much is what you were saying to Next.js.

JASON: Got ya, got ya, okay.

MAT: I think that should be pretty much it. We have the key, we have the browser agent. We put in the script tag. In the head. So, I think we can try it out.

JASON: Okay, let's try it out. So, I'm going to start the server again. NPM run dev. And we've already got this one running in here.

MAT: Are we getting any errors in the console? Just want to make sure everything is sorry, not ID.

JASON: Sorry, say that again.

MAT: Sorry. Actually, we can check it here whether that browser agent is included. So, if you right click on the page

JASON: If I right click on the page.

MAT: Yes, view page source. We should be able to see the browser agent script.

JASON: Turn on the line route. That might make it easier. Here is our script.

MAT: Yeah, this is big blob of text, is the browser agent script.

JASON: Got it. Okay.

MAT: So, it's good. Then we also can check it in the network tab.

JASON: Yeah, looks like my browser by default is blocking New Relic scripts, which is not ideal. How about we just turn this off? Thank you. Okay. So, that's better. We've got things are doing what we expect now. So, we've got all those pieces in here, we are loading the New Relic agent. Here it is. And I think that's the thing, right. Can I go and look at this now?

MAT: Yes, please. Let's see.

JASON: Okay, so, I go into those weren't visible. Okay, so, I'm going to come back here, and where would I look to actually see this?

MAT: Yeah. So, the browser on the left hand side.

JASON: Here's the one I just created.

MAT: Yep.

JASON: Great.

MAT: And we should see some data coming in.

JASON: Yeah, it's saying that our cumulative layout shift needs a little work. We've got user time on the site. That's cool. Page loads. So, this is good. This is useful information, right. And if I kind of collapse this down, you get slightly more compact dashboards, which is really nice. Yeah, so, immediately, with just that one change, we already start to get some good data, where we can see user centric page load times, we've got I don't know what front end versus back end means. Can you...

MAT: Yes, front end versus back end is the time that is spent on the front end side of things and the back end side of things in relation to HTTP requests.

JASON: Got you, got you. Okay. Then we've got our load and route change. Perfect. Yeah, this is looking good. So, if I come back out here and click around a little bit, we'll click into one of these posts, we'll go to the home page, we'll go back to the post, we'll load more, do another quick page load. And then if we come back out here, we should have even more data showing up, which is what we wanted. So, we can kind of see really quickly that this data starts to become useful. We can see what's going on, we can see how long the page is taking to load, and all those details that we care about, which is exactly what I want. And I like that I didn't have to think very hard. Put that browser agent in, get that insight for free, which is really nice.

MAT: Yeah, this is like a curated view of the most important telemetry data that a frontend developer would care about. So, we have the error right, which for now is at zero, which is right, which doesn't happen that often. We have the, you know, web vitals, which are crucial if it comes to web development. Basically, what this page focuses on is that user centric experience. So, you want to make sure by looking at this page that your users are happy with the performance of your web page. So, that's why we have all of this data surfaced here in this curated view.

JASON: Got ya.

MAT: We can navigate to any of those charts and dig deeper. For example, if you click on any of them, we can immediately see that, for example, the cumulative layout shift is not performing as expected. And I know why.

JASON: On the blog page. But it's fine on the post page, for example.

MAT: Yeah.

JASON: This is also really useful, because, I mean, let's be honest, y'all. How many of you are actually checking every page? You check your homepage. You maybe check your most critical landing pages, but you never check an individual blog post or something like that. So, being able to see, oh, we've got this one page that takes forever to load, and we don't you know, how are you going to identify that unless you actually navigate to that page. And for the vast majority of us, the sites we work on, we don't navigate around to a lot of the subpages all that often. You work on the page you're working on and kind of move to the next thing. So, having something alert you about individual pages is really helpful.

MAT: And if we had users from different parts of the globe, which you can see below, we could also see a split between, for example, how the page performs in this country, how it performs in the other country. Another amazing thing about it is developers are usually biased towards their own tests, so to speak. So, whenever you create a test, you can imagine what a user would do. That is not usually the case, right. Users can be unpredictable. So, they can be hiding away that a developer never imagined. And this view can also highlight some issues with certain pages or certain behaviors of the user.

JASON: Got ya, yeah. You can see all sorts of stuff, like I'm not going to check every browser on every device type on every operating system. It's just not going to happen. So, being able to see quickly, holy crap, everybody using Safari is getting errors, hey, maybe we should look into that.

MAT: Hopefully, we don't have to worry about Explorer anymore.

JASON: Yeah, we got the browser agent installed, and this gives us visibility into the client side. But a lot of what happens on Next is going to be server side. So, we have API services here, I assume serverless functions, because I saw an API folder. There's also just the building part of Next itself, which we might care about. So, how do we go further, get more of this put together?

MAT: Okay, so, now I'm going to surprise you, because we actually have already instrumented the backend application. There is one catch, though, which we'll talk about in a sec, but if you go on the APM, left hand side, you see APM and services, we should have the service already there.

JASON: Interesting, okay. How did this work?

MAT: Yeah, we got it already, because we installed the New Relic node package, and it picks up the configuration that we also included in the application, and it sends all the node telemetry data to the platform. Because we use the New Relic NPM package and configure it with the configuration file, which said to the package, here is the configuration file, here's the node application, and whatever the application is doing to the platform.

JASON: The package is just smart enough to know here's a file, so whenever I'm actually kind of at a loss to how this works. How does it function?

MAT: Let me explain. The New Relic package we installed is an NPM package that is able to monitor Node.js applications and also has a few integrations out of the box. And one of these integrations is for express, Netlify, I think there is also integration for Netlify. Actually, if we go to the node modules and the New Relic package. And I think it would be integrations or something like that.

JASON: Aggregators, collectors.

MAT: Instrumentation?

JASON: Here we go. Here's a big ole list. We've got AWS. See what else is in here. Express, Fastify. Here's

MAT: Redis. There's a few packages out of the box

JASON: Cool, very cool. It just knows to use these.

MAT: Yeah, yeah, yeah. And also something that we're hopefully going to have time to talk about, it has also a logging frameworks integration. But it's not fully functional yet. So, if you go back to the New Relic UI, and go to distributed tracing here on the left hand side. We can see there is a small icon, if you could just in the middle of the panel, there is a small arrow to the left. If you could just close that. Yeah, this one. Thank you. So, what we're seeing here is that now the node agent thinks that all the requests have the same path. So, it's not able to recognize whether we're asking for the blog page or whether we're asking for the post page, and whether there is any tRPC instrumentation or maybe this is GraphQL or maybe a standard HTTP. So, now what we need to include is the Next package. So, there's the New Relic Next package that we want to include.

JASON: Yes. Okay. So, let me drop a link to that in the chat and we'll open up. Let's see, this one here. This is the package. And I'm going to go here, open up the kernel again, stop this and say NPM install New Relic Next. Okay.

MAT: Yep. And one more thing that we need to do is modify our dev script, dev and start scripts if we're using it in the build, in the production mode. And we need to use a node option.

JASON: Okay. So, if you just type underscore options, like all capital letters, because this is like an environment variable. So, we are telling node to run our application with a specific option. And then open single port. r and name of the package, which is @newrelicnext. Close quote. And that's it. Let's restart the server. And probably we need to navigate.

JASON: Let's go back out to our site here. We're going to go into a blog post. We'll go home. We'll come back. Let's navigate a little bit more. Click into the blog again. Maybe one of these. Get over here. Okay, so, we navigated around a little bit. I'm going to come back over here. And it will take a second for this to show up, right?

MAT: Yeah, it's going to take a second, and the reason why is because distributed tracing is sampled. So, you know, it doesn't collect every request, because in most cases it would be an overkill. It usually takes some time for the logs to sorry, for the traces to show up.

JASON: Okay. So what we just did is something that I think is maybe like brand new to a lot of devs, which is we didn't install anything into our app. We're just telling node that when it starts to pull the New Relic Next package into memory, which allows it to then automatically modify whatever it needs to modify to send that data.

MAT: Yes. So, what the package does under the hood, it wraps the Next.js handlers with specific methods that are exposed by the New Relic node package to collect the data.

JASON: Gotcha. Cool. So, should we do something else and come back to this, or do you want to look at you set one up. Look, we did a little bit of where it's a cooking show and you make up a cake, put it in the oven, and magically there was a cake that was cooked in the oven, because we knew some of these things would take some time. So, I can switch over to the other service to show how this works, or keep going once we got all the pieces in place.

MAT: Yeah, I guess we could show the distributed tracing in the other service.

JASON: Go to the finished one, right?

MAT: Yep.

JASON: Okay, so, here is our distributed tracing. And how do we how do you interpret this?

MAT: Yeah, so, normally, because this is a dev environment, right, so we have a lot of calls that are related to this specific dev environment. So, you can see that Next.js something or maybe Webpack or whatever. But normally you would have the standard tRPC. So, what we can do to simplify it, we can type tRPC. Sorry, not this one. The one below. There is the box just below the graphs.

JASON: Oh, this one, got it. Okay.

MAT: Then we filter out all the tRPC related calls. As you can see

JASON: Got it.

MAT: There is the get posts by ID. And this shows us, okay, yeah.

JASON: Let me go back and I'll go when you tell me.

MAT: No, that's fine. You can navigate straightaway. So, what we can see here is that it groups our calls into traces. And traces are combined of spawns, so called. And a spawn represents a piece of work that an application does. So, for example, if you now press on any of those, we can see that, first of all, here on the left hand side where you were hovering over is our browser app.

JASON: Yeah. So, check this out. It's a little small, but you can see it says browser application here. And then it shows the call, and this is our services, which is the API.

MAT: Exactly, yep.

JASON: This is really this is handy, because we can see this is where the touchpoints were in our application that did work. And being able to see them sort of put together like this is really handy, because, you know, otherwise you just have to know that.

MAT: And what else you can see here, if you press on first of all, you can expand all. There is this expand all on the left hand side.

JASON: I don't know if there is anything to expand in this one.

MAT: Press on any of those, because we have two rows. So, one row is showing us the backend call, and then row above it shows us the frontend. What I did on purpose is created different errors. The way the application works is that there is like a 5% or 10% chance of an error.

JASON: Okay.

MAT: And we what you did here, you selected a trace with an error. So, from what I'm seeing here, this, normally if it was a fully functioning call, it would also show us another call to the JSON placeholder API. But apparently this one didn't even reach the API. So, it went from the backend from the front end to the backend, and there was an error, 500 error. But if you press on.

JASON: Here's one that actually failed. I'm going to expand all. So, we've got our frontend code and here's the error that we got. And then we've got the API code.

MAT: Then the call to the JSON placeholder API, which you can see the parameters of in the description. And it also shows us whether there was an error or not.

JASON: Uh huh. And then this one here, if I can oh, there's uninstrumented time. So, there's kind of like the in between stuff.

MAT: Yeah, but the reason why it's uninstrumented is because, basically, I'm kind of created this app with a lot of errors. And, you know, these errors can sometimes overlap. So, that's why, you know, these traces might come over to the platform as uninstrumented, because there were so many errors on the way.

JASON: Okay. Right. Because if there's an error, there's nothing to if it fails before it can even report, it's like, okay, something went wrong and we don't know what it was.

MAT: Yeah.

JASON: I got you. Again, this is cool, because we can see, too, here's one that didn't succeed, but that got further than the last one, where it shows us where the data went, and ends up at this third party system. So we can kind of see, hey, there was a failure over here somewhere. And I guess you're kind of just simulating a failure here. Which then we can find that in the trace and see, yeah, we got a 500 error.

MAT: Yep.

JASON: Useful information.

MAT: Yeah. Also, you can imagine it doesn't have to be a third party API. It can also be a database. So, you could instrument New Relic with a database and see the calls from your backend to the database, and you could also see a performance of the database, what's going on in the database side of things. Et cetera, et cetera. Maybe just try refreshing the whole page here.

JASON: Might be that I didn't do enough to get sampled, too, because I only did a few clicks.

MAT: Yeah, that's possible. It will restart the app?

JASON: We did, yeah. We saw it in action, we know it will happen eventually. Sampling is a good thing, because you don't want to be sending literally every transaction. That would be murderous on a system that had more than even a few hundred users probably, but yeah, this is great. What else should we be looking any other instrumentation we want to do, or are we fully set now?

MAT: There are a few interesting things that it can do. So, we talked about metrics. Which we instrumented, that we see the metrics in the consult. So, throughput, error rate, all of these are metrics. So, we left with logs and events. So, I think that logs might be more interesting. So, let's go ahead and instrument the logs. So, here you're in the instrument already.

JASON: Am I?

MAT: I think so.

JASON: How do I check? This is all logs. All logs, yes, that's a good point.

MAT: So, if we go to APM&services. And click on the app that we are working on. Then logs.

JASON: Okay. Now we've got here and we have no logs found. So, we want to get these logs here.

MAT: As I mentioned earlier, New Relic node package can kind of out of the box instrument with logging frameworks. New Relic supports framework and Winston framework. These are quite popular logging frameworks. So, let's install Pino.

JASON: Like that?

MAT: That's exactly the package that I want installed.

JASON: Okay.

MAT: It's pretty straightforward to use with New Relic. Normally, when you log data, there are a few considerations you want to make. For example, where you want to log the data, whether you want to log the data via an API, whether you want to put it into a file, then you need to worry about how to send this file to somewhere, right. There are a lot of different things you need to consider if you were to log without a platform. Platform like New Relic, you can simply send logs to the platform and you don't have to think about all of these things. Because you already have an endpoint. The instrumentation is already done. Let's go ahead and make a file. I think there should be a utils directly. Okay, here we want to upload the Pino framework. And then we want to export const. Logger equals Pino. That's pretty much it. Pino is a function. Yeah, it's going to run like that, as well. There are parameters we can pass through, but we don't have to. Now what we want to do is log an event. Let's go to a server. Let me double check. Routers and posts. These two procedures are the ones we've seen in distributed tracing. There's get posts by ID and get all posts. Also what you can see on line 21 is that there is a 5% chance of showing an error. It's just an error. So, this is what I did. This is why we've seen these errors in the other instrumentation. So what we want to do here is import the logger that we just created. It was utils logger.

JASON: Okay, so, we've got our logger.

MAT: Yes. And now let's say that we want to log that we, for example, got to the GitHub posts page. So, in the just before or maybe after the call to fetch, you can type logger.info. And a message.

JASON: Okay. We've got our logger.

MAT: Yeah. And maybe you could also include the limit and page, so that would have additional information. So, there are two ways of doing that. So, framework is smart enough, and we can provide, for example, an object as a first parameter with those properties.

JASON: Okay, I just throw in limit, page, and that's it?

MAT: That's it. And now let's I don't think we need to restart anything. The dev environment should restart.

JASON: I don't have it running. Let me start it up again.

MAT: Yep.

JASON: Okay, it's off to the races. I am going back to my site here, and this should be the one that loads all posts, right. So, I'm going to navigate a couple more times.

MAT: Or you could press log a couple of more times.

JASON: Yeah, good call. So, I'll go to my console here. Let's load more. Load more again. Okay.

MAT: If you go back to your ID, you should see the logs in your ID actually.

JASON: Here they are, okay.

MAT: So, this is what Pino produces for a developer. At the same time, the New Relic NPM package collects all the logs and sends it to New Relic. So, if we switch now

JASON: Now we have logs.

MAT: Yep. And if you press on any of those, we could also see the properties that you send, which is the page and the limit property.

JASON: Here's our limit. And here's our page. This is slick. This is really nice.

MAT: And also there is one more feature, which I really like. Is that when you have a distributed trace, the log is automatically correlated with the trace for you. So, if we go to the distributed tracing, and hopefully you're going to have more traces now

JASON: Looks like it's done it for us now, good.

MAT: Yeah.

JASON: Get any of the tRPC stuff? Still hasn't sampled any of the tRPC.

MAT: That's interesting.

JASON: But it is picking things up, which is good. It just hasn't yet hit that particular route. So, eventually, it would get those. Then that would correlate it, right? Just time checking, we have about three minutes left here.

MAT: Okay. So, let's go to the other project, which is already instrumented. So, we just quickly see how this works together. So, I guess just go to one of the tRPC end points.

JASON: Okay, tRPC, I'm going to get into the get all posts. We've got...

MAT: Yeah, select any of those.

JASON: Okay.

MAT: Actually, this one is not automatically correlated with the log, but let's try a different one with an error. So, the one that has an error. So, at the top you can see another tab, which is called logs. And what it does is it automatically correlates the log with the HTTP request. And also

JASON: That is super helpful, because what I always want to do is I want to do like a console.error if something goes wrong. But the problem is if I'm shipping this out into the world, my users have to know how to open the console, they have to know how to find that error, they need to be willing to take the time to send it through to me. But if I've got this distributed trace here, and the error that comes through is connected to that, I can go find my errors and just go look. I don't have to rely on my users to even know that an error occurred for me to get info that lets me debug it. So this to me feels like where it starts to become this is why. This is why you do this work, this is why you instrument things. If you're building an app and need to have visibility into is my app good, or does it only run on my machine. Am I getting errors you know, I get emails sometimes from people, hey, your website broke. What happened? Then they try to explain, I can't replicate the error. You kind of end in a shrug, well, let's see if it happens again I guess. This eliminates that problem. If somebody reports an error, I go into New Relic, look for the failures, and I can see what happened. Oh, this particular time of day, there's this weird overlap where my dates are wrong, or whatever the heck is going on with my scenes today that was causing me to see yesterday's and then tomorrow's stuff. Who knows why that was going on? But where we landed is, you know, I needed to I need to be able to solve that problem. And this would give me visibility as opposed to what I've been doing in the past, which is educated guesses.

MAT: Yeah, exactly. The correlation is beneficial, because you don't have to see through all the logs and search for the dates, search for the time, and maybe correlate it somehow magically, which is virtually impossible, right. How you correlate a log with a call that happened or maybe didn't happen. And here on a single pane of glass.

JASON: Yeah, yeah. And that is really nice. So, there's a question in the chat, this is like Sentry. We've had Sentry on the show before, and Sentry does give you errors. Is there anything that's like notably different that you're aware of? We don't need to get into a straight up shop for shop comparison, but are there things in addition to the error reporting from Sentry that make New Relic worth investing in?

MAT: I'm not really familiar with Sentry to be honest.

JASON: That's a question for another time. So, with our last like one minute here, is there anything else that you wanted to look at on this fully instrumented site before we wrap it up?

MAT: Maybe if we go to error inbox, this is something that is worth mentioning. Yeah, just stay in the NPM.

JASON: Stay in the NPM, okay. And I'll look at my finished app, and inbox. Here?

MAT: Yeah, here you also have errors grouped, you can see how they happened, what was the error exactly. This is also super useful. This can be also correlated with logs or traces. So, this is a feature that I also wanted to highlight, which I personally use, as well, which might be useful to others.

JASON: Got it. Yeah, that's very helpful. And it looks like, you know, if it's something you're expecting and you want to clear it off, you can ignore it. You can mark it as resolve if you ship to fix, and that way it will show up again if you didn't fix it, or disappear otherwise, which is also very helpful. And that, I think, is a great stopping point. This has been excellent. Where else should people go if they want to learn more? I'm going to drop a couple links. You sent me a blog post about how to monitor Next sites on New Relic. So, I'm going to share this in the chat for everyone. Anyone else you want to shout to?

MAT: I guess New Relic documentation is a good resource.

JASON: Great. Let me throw in the link to the docs page here. We'll throw this up, as well. There it is. And then any others?

MAT: If someone wants to reach me personally, they can do so with visiting my web page or maybe a GitHub profile.

JASON: Let's throw here's Mat's website one more time, webdev.engineer. An excellent domain name. We've also got here's your GitHub profile. So, if you've got questions, you want to find the demo.

MAT: Oh, the blog post that you linked to, it also has a repo. And if anyone has an issue with anything, with the instrumentation of Next.js application or maybe T3 application, then you can just write an issue, and I'm going to respond. I'm the maintainer of the repo.

JASON: That's in this blog post here?

MAT: Yep.

JASON: Great. So, this is linked in the show notes if you want to get in here. And you can send questions directly to Mat via GitHub issues. With that, this has been an absolute pleasure. Thank you so much for taking some time with us, Mat. This episode, like every episode, has been live captioned. We have had Ashly from the White Coat Captioning team here with us all day. Thank you very much, Ashly. This episode and all of Observability Week is being brought to you special from New Relic. So, thank you New Relic for putting this together. We've got Netlify, Nx, and Pluralsight also kicking in as sponsors to help me cover the live captioning and a bunch of other things that keep the show running smoothly. And we have a whole lot of good stuff coming up. We've got Daniel Kim coming in to do day three of Observability Week. We're going to talk about location based Next.js middleware and something really cool called synthetics, which we'll talk a bit more about tomorrow. Then on Thursday, this isn't related to Observability Week, but we have Nat Alison coming in. We're going to do generative art. It's going to be an absolute blast. Make sure you like, subscribe, follow, whatever it is you want to do to make sure you don't miss an episode. With that, I think we're done. Mat, thank you so much for spending time with us today, really appreciate it. Chat, as always, thank you for taking time out of your day to learn a little bit with me. See you all next time.

MAT: Been an absolute pleasure. Thanks a lot for having me.

JASON: Thanks, Mat, bye.

MAT: Bye.