Interface is everything, and everything is an interface

Metalab designs interfaces for top brands around the world, helping them design, build, and ship their products.

Connect with Wesley on Twitter and LinkedIn.

Congrats to Populist badge winner SiddAjmera, who won the badge for their answer to Angular FormControl check if required.

TRANSCRIPT

Ryan Donovan: Enable efficient AI at the edge. With machine learning at the edge, advanced HMI, low power, and built-in security, Infineon PSOC Edge Microcontroller Family is designed to power smarter, more intuitive user experiences, and next generation devices. Get started at infineon.com/psocedge. That’s I-N-F-I-N-E-O-N dot C-O-M forward slash P-S-O-C-E-D-G-E.

[Intro Music]

Ryan Donovan: Hello everyone, and welcome to the Stack Overflow podcast, a place to talk all things software and technology. I am your host, Ryan Donovan. Today, we are talking about interfaces. We talk a lot about the mechanics, the backends, the AI, but today we’re talking about front end and how it all works. My guest is Wesley Yu, head of Engineering at Metalab. Welcome to the show, Wesley.

Wesley Yu: Hi. Thanks for having me, Ryan.

Ryan Donovan: Of course. So, top of the show, we’d like to get to know our guests. How did you get into software and technology?

Wesley Yu: Oh boy. Yeah. This was not a straight path for me. From a very young age, I knew that I was a builder, that I liked to kind of make things, but it took me, I think, longer than anyone else that I know to find the medium that I wanna to build things in. So, coding. Yeah. I don’t know. I’m not a programmer by training. I actually went to school for photography and then for media studies, so that’s writing about movies and video games. Very fun thing to do, but not a super employable skill.

Ryan Donovan: That’s true. Yeah.

Wesley Yu: I dunno. I started in photography. I liked doing wedding shooting events, kind of a technical endeavor, but also a creative endeavor. But ultimately, I dunno, it was hard to kind of run a business separating the art from the business side of it. Somehow I got into AM news radio production, so kind of finding interesting stories and guests for AM news talk shows. So, I learned a ton from this job. I think translating kind of complex ideas into interesting stories and problems that people could get invested in. And then, for whatever reason, I moved into content marketing. So, for a tech startup, combined some visual design skills from my photo days with communication from my radio days. And I really kind of knew nothing about tech at that time. I was, you know, looking for a change to get out of media and journalism, and I found myself at this company that was enrolled in something called Y Combinator, and I had no idea what that was. So I went down to Sunnyvale, California, with them, and then I was just exposed to this ambition and energy of tech in Silicon Valley in like 2013. At this point, you know, I wasn’t a programmer, but I was surrounded by programmers. They’re all working so hard to build the future, and it was intoxicating for me. I never really met so many people that were so passionate about their work, and I was like, ‘that’s cool. It must be nice to love your work so much.’ But you know, I’m not a software engineer. I tried to learn programming, and I failed. So, womp-womp, I’ll keep on doing content marketing. Yeah, it wasn’t until my partner, she was also looking for what she wanted to do in life, and she started this coding bootcamp. One day she phoned me, and she said, ‘hey, I think you should quit your job and do this coding bootcamp thing. It’s really fun. I think you could get it.’ So, on a whim, I quit my job, learned Ruby on Rails, and really the rest is history. Yeah, I was very lucky to get my start at Metalab, and that’s how I got into tech.

Ryan Donovan: And now, obviously, you’re head of engineering at Metalab. It’s an interesting path because you started with the art and presenting things to a person, right? And now you’re doing the tech behind presenting an interface to a person. So, what does Metalab do?

Wesley Yu: What does Metalab do? I think, you know, in short, we design and build digital products for very ambitious companies. So, I think what we do best is to find the product DNA of very early-stage companies that we hope later become household names. So, we’ve worked on Slack, and Coinbase, and Uber, but we also have another side of the house. We work with leaders to evolve their products. You know, think the Googles, Amazons, and Metas out there. But essentially, we come in, think about a product, try to build the DNA of that product, and ship kind of an MPV1, V2.

Ryan Donovan: For a lot of consumer products, especially, the key to that takeoff is having a good, simple interface. People talk about the cell phone revolution, like there were cell phones, and then there was the iPhone, and everybody had to have one. What is it about a good interface to technology that makes the technology understandable?

Wesley Yu: Yeah, that’s interesting. I think that’s changed a lot over the last 20 years. I mean, you mentioned the cell phone, the iPhone coming out in 2007 – that was a big boom. That kind of changed people’s expectations with how they interact with technology. I think there was a productivity app boom in the 2010s that set new expectations. There was of course crypto, and now there’s AI. So, with mobile, you kind of built this expectation with consumers that there are adaptive screen sizes, that you have gestural inputs like scrolling, and pinching, and swiping, and all that replacing point and click. The interface became this thing that directly manipulates, you can kind of drag things around, expand things, long-press on things, and then of course you also get access to like location, cameras and sensors, that unlock new possibilities. So, I think many expectations have changed just from the mobile device itself. Not to mention, even just the ability to notify people. You know, mobile phones are in your pocket. So, now you start to build habit loops, notifications, into applications that draw you back into the device and demand your attention in some ways that I feel are less inviting these days and more invasive. But those are kind of the concepts that have built some foundation in interface.

Ryan Donovan: It’s almost like with the mobile phone, it’s not only you having an interface into the device, the device has a little bit of interface into you.

Wesley Yu: Yeah, yeah, exactly. Certainly, with like social media and this, that, and the other.

Ryan Donovan: And you mentioned crypto and AI, which you typically don’t think of the sort of interface with those technologies, right?

Wesley Yu: You know, the crypto Web3 boom can be seen as a little contentious, but I think it also set new expectations for users in terms of user interface. Crypto and Web3 was very much leaning on interfaces, deriving verifiability and trust on the screen. You know, before crypto and Web3, we would rely on institutions to drive trust. If you did something you could, you know, call customer support and get that thing kind of reversed, but you can’t do that on the blockchain. There’s no customer support that you can call to reverse that. And so, there’s this kind of balancing act between information density that you can use to verify things that are happening on the screen and making sure that they’re right, and simple user interfaces. So, I think it has kind of set some expectations in terms of the density of information that people expect and the concept that they need to learn.

Ryan Donovan: Like you said, we’ve had all these. Changes in the possibilities of what an interface could be, but it’s also required a lot of technical innovation for the things like gestural and the accelerometer-based interface stuff. With these technologies, they’ve enabled different ways of interacting, right? With gestural swipes and stuff on the mobile phones, or the accelerometer-based movement of the phone. Do you think that these have required different technological paradigms to implement, to have interface with the backend, or are they sort of extensions of the mouse being a relative movement in space?

Wesley Yu: I think they are kind of extensions of existing paradigms. I think outside of kind of gestural interactions, accelerometer, using those types of things, I think one kind of better example is of technology changing interfaces or tech trends changing Interfaces is with this rise of productivity applications. We now develop this expectation that software is real-time and collaborative, that if you think Google Docs, or Figma, or Notion, or Miro, these interfaces now have multiple cursors, and this idea of presence. And with that comes things like history, and versioning, inline comments. I think this change, this kind of trend in technology productivity, has far more of an impact in the technical implementations of applications, because now more things are stable, more things are real-time.

Ryan Donovan: With having like a lot more states and a lot more sources of state to sort of track, what’s the technological programmatic thing that you have to sort of adapt to handle that?

Wesley Yu: There’s lots of different strategies to resolve changes, but essentially, you kind of would wanna track individual changes and be able to merge and come back to a state that makes sense for everyone. Depending on the application, there are different strategies to do this, but that is definitely a technical challenge, depending on the interface you’re working with.

Ryan Donovan: So, I know managing state is a huge point of research and contention in front ends. Do you have recommendations on it? Are there sort of best practices?

Wesley Yu: Yeah, certainly. I think there are blind corners that our team has just seen around multiple times as we’re building something new. There’s kind of a big rise of serverless, I think, in the last 10 years or something like this. And serverless, stateless systems, they kind of disappear as you don’t need them. And a designer can throw a little green bubble that says someone is online, and it just totally throws out that architecture. And so that’s something that we consider a lot is like, ‘okay, is there some potential that this thing is going to be stateful in the future?’ And planning for that eventuality because we just kind of simply know that when you’re interacting with tools, it’s not you getting a task done by yourself anymore. The expectation is that you are getting a task done in collaboration with others, and in some cases those others are humans; in today’s world, sometimes that other is some sort of AI system.

Ryan Donovan: You know, everyone is talking about AI today. We had an article a while ago that the AI isn’t the app, it’s the interface, it’s the UI to other applications’ information. And I wonder if you see AI that way, or you see it as another intermediary app?

Wesley Yu: When we think about products that integrate AI, we see them as kind of two buckets. One is AI that enables businesses that couldn’t exist without AI, and then there is AI that is used as an accelerator or another modality to accomplish tasks that you could otherwise do manually. So, I’ll give you some examples here: so, for an AI that enables a business that could not exist without AI, one of the products that I’ve worked on in the past is an application that taught kids how to read. So, you know, almost 40% of fourth graders in the United States are at a level of reading that is below where they should be. And then once a student falls behind, it’s really, really hard to catch up. And so, we worked on this application to really kind of scale and automate the interactions between an expert reading teacher and some sort of student. And we did this by generating custom story books based on the children’s reading level, like maybe they’re learning CBC, which is constant vowel consonant. These are words like cat and rat, and fats. Maybe they’re learning digraphs, which is like two letters making a single sound. We would include all the words that they knew and the words they didn’t, the words that they were learning, and we kept track of all these kind of specific concepts and patterns to generate custom storybooks. And this was both with text LLMs and also with generative image generation to kind of create these illustrations. And then there was the actual reading experience that we built, as well. This was like a speech-to-text model that would evaluate phonemic accuracy of individual sounds within words. So, if you think about ‘the,’ it’s ‘thu-uh’. And we need to break speech down to that level of detail. This application has AI all throughout it, but it doesn’t feel like it’s tacked on. It’s actually very core to the experience, and I think somewhat invisible. And without AI, I wouldn’t be able to complete the same task. Products where AI is an accelerator, like another modality, an example of a product that we worked on is a product for policymakers, lobbyists, and journalists that are reporting on policy. So, this company we’re working with, you know, they’ve got this wealth of policy data, government documents, meeting minutes, articles, these huge directories of people, roles, and organizations. Literally millions of indexed documents that is by and large, kind of unstructured text. So, even without AI, this data is extremely valuable. But policymakers, lobbyists, journalists would sift through this information to find what they’re looking for. And then with the help of LLMs, it gives us this opportunity to structure data in a way that is far easier to query. And that’s the modality I think that we’re kind of talking about with AI that is additive, that’s an accelerator. It allowed these people, these users, to ask questions like, ‘what is the top health policy headlines of today?’ I think somewhat kind of an easy question for LLMs to answer, but then you kind of get into more complex questions like, ‘what bills are Coca-Cola lobbying for in 2025, and how much have they spent?’ That’s kind of a little bit more complex for an LLM, and then you get kind of a little bit crazier than that, like, ‘which lobbyists used to work for the Department of Energy, and then which bills are they lobbying for now? That’s an impossible question for LLMs to answer with just kind of regular reg, and so, this type of AI integration is different. You know, you can reasonably take this away from the product, and people can still derive value, but AI certainly makes it easier.

Ryan Donovan: Yeah, it is an interface into that data, right? Although you know, there is an argument to be made that a lot of applications are just interfaces into data, right?

Wesley Yu: We’re all basically just cred-monkeys, you know, reading and writing from databases all day long.

Ryan Donovan: So, when you’re designing these interfaces for problems, what are the sort of common things that you run into that you’re like, ‘everybody should be doing this.’ Or is it always custom?

Wesley Yu: I don’t know if there is a rule right now. I think we’re still really early, but something I like to lean on is when we interact with AI systems, LLMs, coding agents, yada, yada, we often like to think about them as a pair-programmer, like another kind of entity that you’re interacting with. And I think we can pull a lot of the paradigms from productivity apps back into the systems that interact with AI. And I think specifically coding IDs have perfect infrastructure in order to do this. And then, I think many applications don’t quite have that type of infrastructure yet. So, when I say, you know, YIDs, coding IDs, have this perfect infrastructure, it’s like we have version control, we have diffs, we have commits, and all this stuff allows you to interact with the LLM in a way that is very visible and very well understood by the developer. I don’t think that we built those concepts yet for people interacting with consumer applications or these types of tools, but productivity is the closest there. You know, you have inline comments, you have cursors that are moving around, and things like this.

Ryan Donovan: It’s interesting. Yeah, the developers are already tracking changes. They’re already doing this stuff that is helpful for the LLM to understand context. You mentioned Uber, or something – are there paradigms like that for a sort of consumer marketplace like that, where understanding the changes that are happening– what are the sort of track changes in other applications that people aren’t thinking of?

Wesley Yu: One thing that I think about a lot is, you know, as we spend more and more time interacting with AI, we start to build this shared memory with them, that’s kind of between me and OpenAI, ChatGPT– and in psychology, this kind of shared memory is called transactive memory. It’s the idea that in close relationships or in teams, we split up knowledge. You remember some things, I remember some things, and we have this kind of shared system of memory. And it’s efficient. It’s kind of emotionally deep because it also shapes how we feel connected, and then I think, you know, just to get dark for a second, when someone passes away, we don’t lose just them; we also lose part of our memory, and they call this like grief of memory or social memory loss, and we feel that absence when it goes away. And so, that is something that we need to make visual. That is something that we need to put in an interface so that it’s not lost when the LLM shuts down, because obviously they’re not a real person, they don’t have real memories. We need to store that information somewhere. And so, we think a lot about how to build that context, both on the backend, sometimes using knowledge base or knowledge graph like Neo4j. We start to build out how we understand this person so that as a new system boots up, they can query their shared memory, so to speak. But we also think a lot about context and context compression, these types of attributes, I think, people don’t always think about, like, how much of the context window am I using? Consumer doesn’t really think about that; I think a developer does think about this type, or what model am I using? What specific model? I think these types of things, these types of concepts, are not publicly understood.

Ryan Donovan: Yeah. It’s interesting implementing a sort of shared transactional memory like that, and you talk about context. At one point, does that memory become too much? Do you run into things like context Rot? The needle in the haystack problem.

Wesley Yu: Yeah, absolutely. Managing context is a whole beast onto itself, ’cause as the context window builds bigger and bigger, the drop off of whether or not an LLM can retrieve that information doesn’t go to zero. But it becomes very, very difficult for an LLM to find something that’s in the middle. Right now, we rely on systems like RAG, so, vector search of unstructured text. A big thing that is happening right now in the last year is graph reg. So, instead of structuring queries against a vector database, we actually get the LLM to write a Sparkle query for Neo4j, and that is more like natural language. I feel like they do actually generate these queries quite well because you’re working with known entities and edges, which are relationships, and so on, and so forth. And so, I think that’s a great way to start building context that doesn’t live within the context window that can be retrieved at the time of inference.

Ryan Donovan: Mm. It’s not just a constant context; it’s sort of a ‘just in time’ context. So, tell me a little bit more about the knowledge graph aspect of it, accessing that, interacting with that, because it sort of sounds like you’re almost building a semantic understanding underneath the hood.

Wesley Yu: When we’re working in this context, we think a lot about ontology. Like, what are the different things that exist out there? And when we’re thinking about retrieval augmented generation, which is, I think, how we retrieve up-to-date information, there are those two phases. One is gathering the information, retrieving the information, and then there is the actual generation of the text. And if one is bad, you know, the LLM can’t make that better. Both sides need to operate efficiently in order for you to get results. So, at the highest level, it’s about building this knowledge graph, having an understanding of the concepts, the nodes, and edges within the system in a way that both the LLM and the consumer share. And so, that’s really important, and that’s based on the use case that you’re working with. You know, in this example of the policy application that we’re working on, those are articles, and people, and votes, and bills – these types of things. But in a different setting, they may be different nodes, different edges.

Ryan Donovan: You talked about the iPhone being the sort of interface-killer app for mobile phones, and I feel like ChatGPT was the sort of LLM-killer app. Are there ways to improve the LLM interface outside of the sort of text call it and response that a chatbot does?

Wesley Yu: An interface that I’m not 100% sold on, but one that I actually like to use quite a bit is a voice interface. In my work at Metalab, we’ll often need to learn a new industry very, very fast, and that kind of involves watching YouTube videos and reading articles and things like this. I often have my phone with voice open, and I ask it just the stupidest questions about things, and just having that kind of peer programming partner, so to speak – someone that I can ask questions that I have, someone I can ask, ‘look up documentation of this thing while I’m engaging with content that may be above my level.’ I think that interface, that voice interface, is really important, but then I also– I always ask ChatGPT to summarize what we talked about, and put it somewhere, and that process is so manual. I gotta copy and paste and put it somewhere else. That needs to be solved. I’ve been talking to you, we’ve built some shared memory together, how do I get that out of there and into my head in a way that I can understand?

Ryan Donovan: Some of the criticisms I’ve heard about LLMs is that it does reduce that friction, that there is no friction of getting it out of the space there and you don’t engage with it. You don’t process all the articles. Do you think that is a valid one? Are there useful frictions to build into an interface?

Wesley Yu: I think this reduction in friction is actually good. I think that the way that we should approach this is that AI should invite curiosity and accelerate learning, and the hope is that an LLM one day kind of becomes the perfect tutor. You know, if you can imagine the only impediment to your learning was yourself, not the curriculum, not the access to the information. The next thing that you needed to learn wasn’t too hard, it wasn’t too easy, and it’s presented in the exact way that you need to understand it. I think that that is frictionless, and that we should really strive for that. Because I think if you think about what a world would look like, if the limits of human intelligence look like that, that would be kind of awesome in my mind. You know, the comparison that I like to make in this regard is with marathon runners. Everyone is a marathon runner these days. My wife, my sister, my brother, they’re all capable of running marathons. Not me. But this was unthinkable 50 years ago. You know, the New York City Marathon started with 127 runners. Half of them didn’t finish the race. Today, the New York City Marathon has 50,000 runners. Now, apply that to intelligence, and that’s the future that I hope AI will bring.

Ryan Donovan: That is a promising future, but I think one of the frictions I’ve seen most complained about is the actual friction in learning – that it’s so easy to get to the product of the learning, right? You get to the essay, you get to the finished test, and you don’t struggle in absorbing the material. You’ve designed learning interfaces. Are there ways to mandate that the user engages with the material in a way that facilitates learning?

Wesley Yu: I think in the context of learning, that is certainly a consideration that we take is that we don’t want these systems to give you the answer, because that robs you of the experience of learning. However, I think that’s kind of not the tack that many companies are taking. They want to automate further, make people more efficient, even maybe if it is the detriment of truly understanding these systems. And to some extent, I think that’s okay. I don’t know how to program in binary, but I don’t see that as an impediment to me building systems for consumers.

Ryan Donovan: We’ve seen a few very future-looking demos of AI creating interfaces on the fly. Do you think that this is something that is going to happen? It is useful to happen, should we want spontaneous interface generated by AI?

Wesley Yu: I still think that people know best how other people wanna solve problems and how we should structure problems that make it easy for people to solve. You know, imagine you’re a travel arranger for a reality TV show, like Teen Moms or something like that, which is a real user that we are building for. You need to arrange flights for this huge cast and crew that are flying to all different locations, different times, they’re location scouts, a big crew, makeup, blah, blah, blah. You know, if you’ve ever booked flights in your life, you know this is a complicated task. You’re juggling variables that no human brain can hold, and humans know best how to externalize memory. You know, who is booked, who is pending. What action needs to be done next? Humans have a really good sense of how to black box systems so that you don’t need to understand the internal workings or things, you can kind of hide irrelevant information and only present the information that you need to understand. This kind of progressive disclosure, changing information density, so on, and so forth. I think that right now, humans are best to design that, but in the future, maybe we can teach an LLM to do this. But I think that this type of problem is challenging to verify, and so therefore challenging to teach an LLM to do correctly. I think an LLM can certainly verify whether or not a function was written correctly, maybe even if a feature was kind of done correctly, but to verify whether an application meets the needs of a consumer, that’s extremely hard to verify. It needs to be verified with the market, and I don’t think that LLMs are gonna get there anytime soon.

Ryan Donovan: It is that time of the show where we shout out somebody who came onto Stack Overflow, dropped some knowledge, shared some curiosity, and earned themselves a badge. Today we’re shouting out a populous badge winner – somebody who dropped an answer that was so good, it outscored the accepted answer. So, congrats to SiddAjmera for answering ‘Angular FormControl check is required.’ If you’re curious about that, we’ll have it in the show notes. I am Ryan Donovan. I host the podcast, edit the blog here at Stack Overflow. If you have questions, concerns, topics to cover, et cetera, et cetera, you can reach out to me at podcast@stackoverflow.com. And if you wanna reach out to me directly, you can find me on LinkedIn.

Wesley Yu: Cool. Hey, I’m Wesley Yu. I lead engineering at Metalab. If you wanna find me, you can find me on LinkedIn or on Twitter.

Ryan Donovan: How can they find out more about Metalab?

Wesley Yu: Just head straight to Metalab.com.

Ryan Donovan: Thank you for listening, everyone, and we’ll talk to you next time.