Untitled

Discuss this episode in the Muse community

Follow @MuseAppHQ on Twitter

---

Show notes

00:00:00 - Speaker 1: One of the luxuries of industrial research is that you’re not bound to the traditional rigor and neutrality required of academic research or just science in general. We’re allowed to have an opinion. We had a number of people who are reviewing the essay comment, what’s what the feelings, take this feeling section out, it’s not defensible, and I felt like it needed to be addressed, because to me, that’s the most important part.

00:00:30 - Speaker 2: Hello and welcome to Meta Muse. Muse is a tool for deep work on iPad and Mac. This podcast isn’t about Muse product, it’s about the small team and the big ideas behind it. I’m Adam Wiggins, joined by our guests today, James Lindenbaum. Hey there. And Shimon Kjeski. Hello, both from Ink and Switch. And James, you and I have been colleagues and friends for a pretty long time now, so I happen to know that similar to Muse team member Yula, you are a huge cocktail nerd. Any experiments in that area these days?

00:01:04 - Speaker 1: Oh, there’s always ongoing experiments. Yeah, I recently decided to move to keging cocktails when you have a bunch of guests coming over. I often will batch up a cocktail. So it’s faster to serve, and you can also be really persnickety about, you know, micro adjustments to amounts and things like that and really dial in the recipe, and I decided to move to kegging the cocktails on low pressure nitrogen, so they could be cold and pre-diluted and ready to drink. It’s basically front loading the work so that I don’t have to do much. I can actually hang out with my guests, but there’s always interesting things you learn when you start changing things around like that.

00:01:40 - Speaker 2: And I do feel like the cocktail preparation is part of the experience of being a host or something like that. I guess if you have a lot of people there and then you’re doing nothing but being heads down in your bar, then that’s not really being a very good host, but there also is something to the, yeah, the prep tool, I guess.

00:01:59 - Speaker 1: Well, as you well know, I’m a bit of a perfectionist. I enjoy having the time to really like try to perfect a cocktail, really dial it in. And one of the ones I made recently, I stole from this really awesome bar in San Francisco called Kona Street Market, and there’s this drink called the Banana stand. It’s an Arrested Development reference.

00:02:18 - Speaker 2: It’s the first thing that popped into my mind, always money in the banana stand.

00:02:23 - Speaker 1: And it really blew my mind when I had it, and then I’ve talked to the guys there about it, and then I’ve been just like gradually trying to recreate it on my own and get it dialed in. But I think we’re there, I think we’re close enough to perfection. We’re certainly close enough that you would have made me ship it at this point.

00:02:37 - Speaker 2: It’s a little inside joke there for the listeners, James and I have a long time, let’s call it productive tension, usually productive of, I like to ship stuff, and he likes to make it perfect, and hopefully somewhere in the middle of that is sort of an ideal place to be. And we’d love to hear a little bit about both your backgrounds, maybe Shimon, you can start us out.

00:02:57 - Speaker 3: Yeah, sure. So, for the last 2 years or so, I’ve been principal investigator at GSwitch and occasionally doing consulting research projects. Before that, my background is I’ve been running a small R&D studio and we’ve been basically working on unusual interface problems, things like designing and building the Spola acting engine for European Space Agency or some kind of like interface for exploring machine learning for molecular synthesis. And before that, my background was in actually creative coding. I was doing work on museum art pieces, doing for interactive art, data visualization, stuff like that. What I do also, other than work, I make a little bit of experimental music and various computing projects for fun.

00:03:40 - Speaker 2: Yeah, I feel that the music experiments that you do and performances sometimes, right? Also bleeds into your, yeah, creative coding, artistic interfaces, a little bit.

And I think I personally take a lot of inspiration from the prosumer world of like audio gear and the interfaces there that are sort of designed to create art but also intended to be pragmatic, right? You’re doing a performance or something like that and those knobs. You gotta be able to grip it in the right way or whatever, so I feel like you often bring your music world, electronic music world stuff, and you work in I can switch, and I always enjoy that personally, but I’m also a person with a little bit of an electronic music background, so maybe that’s why it appeals to me.

00:04:21 - Speaker 3: Yeah, the word there is definitely a little bit different and interesting, so I know if there are any UI designers listening to this, I encourage you to just browse Pinterest for music stuff. There’s a lot of interesting differences there.

00:04:33 - Speaker 2: And James, you and I have worked together for a very long time, including perhaps most notably in co-founding Hiroku, and we also created the In Code Switch Research Lab together with some other great folks, but maybe you can fill in a little more of the story there.

00:04:48 - Speaker 1: Sure, yeah, well, I have always been, as I like to say, constitutionally unemployable, so I started a number of things over the years, but yeah, most notably was probably Hiroku with you and our other co-founder O Ryan.

After that, I found that there were a lot of people coming to me, founders of developer facing companies who, you know, wanted help, and I ended up advising and sitting on boards and whatnot, and eventually starting what is now a venture capital firm called Heavy Bit, which specializes in, you know, developer facing infrastructure kind of stuff. And so I’m still there, I spent a lot of time there, though I’ve kind of worked my way from being a founding full-time partner there to being a more part time. Yeah, and then you and I co-founded the lab and can switch, which is where I spend, let’s say more of my time these days. I’d like to spend all of my time there, but, you know, there’s so many things to do.

00:05:39 - Speaker 2: Well, let’s be honest, when it comes to paying the bills, investing in developer tools companies is probably a better gig than weird research.

00:05:47 - Speaker 1: Yeah, I mean, you know, the path to money is more clear, that’s certainly true. It’s still enjoyable though, there’s so much innovation happening on that front, it’s still intellectually interesting, and there’s a lot of fun stuff happening there, but it certainly feels a lot closer in than the weird stuff we’re doing at the lab.

00:06:04 - Speaker 2: And you both recently published an essay on your latest research project called Ink Base. Of course, I’ll like that in the show notes, but can you give us an overview for folks who haven’t read the essay yet?

00:06:14 - Speaker 1: Sure, yeah. So, in the lab, we have a research track that is all about programmable ink, sort of a combination of doing stuff on tablets, thinking about digital ink, and thinking about end user programming.

And this project Inkbase, it was basically the 5th, depending on how you count the 5th project in that track of 8 that we are now that we’re currently working on project number 8.

Yeah, in that track, and we’re publishing this essay a little bit out of order because we wanted to take the time and this one to sort of lay a little bit more of the groundwork, sort of define what the problem is, what we’re trying to do. So we spent a little bit more time writing it than some of the write-ups of projects that came subsequently, like Crosscut and Untangle. But yeah, we’re happy to have this out there and have people, you know, start to grok what it is that we’re doing here with this weird program of link stuff.

00:07:05 - Speaker 2: Yeah, so this whole track of end user programming is certainly one that, I mean, you know, that concept is something that even fed into Hiroku.

It was something that was a kind of founding idea that we knew we wanted to bring into the lab.

The three of us worked on an essay titled End user Programming that kind of touched on a history of that field and some light experiments, some of the first work you had done with the Lapshaman.

But part of what I like about this ink-based project specifically is this is taking the idea of sketches and trying to kind of take what do we like about spreadsheets and the rough computation that you can do there kind of on the fly interacting with the document in a way to take advantage of the dynamic medium, but not something that’s writing an app per se.

Certainly has much in common with, for example, potluck.

We had Maxson. Jeffreon just recently, but they were very focused on OK, classic plain text and the searches, etc. and I feel like this is almost a complete other take on that tablets, stylus, sketching, kind of very loose and informal, but informal and programming are not things that we normally think of as being combinable and indeed it is that for me, at least observing this kind of track of research from the outside in this specific project, it is that tension between the formality of programming.

And the systems thinking and so forth that we want from our computational tools and the looseness, sketchiness, I’m just figuring it out, I’m not sure yet, messiness that is part of thinking tools, tools for thought.

So I think it’s a very evocative idea to start with and then the resulting project itself, which you can see some videos of in the essay also is only further teases that.

00:08:49 - Speaker 1: Yeah, that’s one of the reasons we like working with digital ink. I mean there’s a number of reasons, but one of them is that it sort of forces you into this sort of sketching, informal, loose, fast and loose kind of mindset, and It just underscores how not fast and loose most of our programming capabilities are.

And so it kind of forces us to think about, you know, what are the right affordances, how could we design a system that would let you stay in that sort of frame of mind but still get some of the benefits of a dynamic medium. This sort of weird analogy that was sort of the prompt for the Inkbase project was, you know, we look at spreadsheets and I personally am a huge fan of spreadsheets, as I think many of us are, and I think about the analogy, you know, if you think about spreadsheets as we have them today, as they compare to their analog predecessors, you know, a giant pad of paper and a slide rule or a calculator, it’s not just that modern spreadsheets let you do what you would have done with the analog version a little bit faster, it’s that they actually let you have thoughts you wouldn’t have had. Using the old version, because you can see this dynamic model and you can get intuitive understanding of how it works. You can play what if scenarios, it sparks new ideas. And so, we think, OK, you’ve got the traditional sort of actual spreadsheet, you know, the paper analog version, that is to modern spreadsheets as sketching in a notebook is to Question mark, right? Like, what is the thing that goes in that box? And I don’t feel like we know what it is or have really seen it. And so, Inkbase was sort of a little bit of a study or an experiment around that prompts. Like, what would that thing look like? What would it feel like? What would it be like to be able to work in that spreadsheet like way, but with ink.

00:10:28 - Speaker 2: Now I think a question that might be in the audience’s mind is how this prompt that you just described relates to visual programming, which visual programming is something where, yeah, maybe it’s more accessible to the average person or requires less programmer brain, less symbolic manipulation. Do you see it as related to that world of things or is it its own beast?

00:10:51 - Speaker 3: So I have a little rant.

00:10:53 - Speaker 2: I forgive you that.

00:10:54 - Speaker 3: Yeah, that might be too early in the podcast for a rant, but I don’t think a lot of these projects that people mention as visual program are really visual in the sense that we talk about or we think about, uh, things like, like not to taxonomize the whole field, but there’s things like projection editors, maybe scratch comes to mind where you have blocks of code that just snap together or maybe things like Max MSP for musicians, which is Basically nodes and wires interface. So, these kinds of interfaces.

Still require thinking in this very like abstract symbolic way. You just manipulate the code, not in a text buffer, but on a screen, like moving it around in dimensions. What we think about in this thread is more about visual programming as a way of working with like actual embodied objects that you can see on the screen and interact with. So you’re not thinking symbolically, but concretely about the domain, the problem at hand, and this is kind of the thread we’ve been following. So, In a sense, both are visual, you could argue that code in a text box is also visual, but the meaning of visual is kind of different, the way we think about this and the way these sorts of projects think about it.

00:12:05 - Speaker 2: Right, when you think of one of those nodes and wires, kind of visual programming languages or something like Scratch, which I think is a great product for kids to learn to program, it is really about sort of taking a conventional program and making it not text, not pure text, but something that’s a little more gooey, point and clickable. There’s a lot of value to that and there’s many domains where that makes sense, but that does seem like almost a different realm from I have the sketch and I want to bring it to life using the dynamic medium of computation.

00:12:36 - Speaker 1: Yeah, there’s a really important distinction between programming with ink versus programmable ink, right? So, you know, what we’re not trying to do is help you write programs with the use of ink. What we’re trying to do is just use ink, you know, for the properties that ink has, digital ink, but also allow it to be dynamic in the ways that you would expect from, you know, the dynamic digital medium, and so, The programming is not the end, it’s the means to have ink that does more interesting things than just, you know, sit statically on the page.

You know, a lot of times we have these amazing devices and we have this pen and all this computational power in the iPad, let’s say, but a lot of the iPad apps that make use of that basically just let you paint pixels on the screen with the pen. And it’s just, there could be so much more there.

And that’s a big part of, you know, what we’re thinking about with digital ink in general at the lab, and then with the programmability in particular, you know, what if that ink could respond the way that a spreadsheet does reactively to other things on the canvas, or, you know, what is the nature of digital ink even at all, right? I think that’s a question that we’re still kind of asking ourselves and doing studies around, you know, what if it worked more like string that you could pull around the page, or what if it worked more like paper clips, right? Like a little wire thing. Where you, you drew it, but then it wants to retain its shape. So when you pull on it or bend on it, it tries to retain some of its shape so it preserves a little bit more of the intention or of the movements of the person who made that mark, right? A fun prompt that I like is thinking about digital ink as a byproduct of someone moving their hands. It’s more the moving of their hands that’s interesting and the ink is a byproduct. And if you think about it that way, then you start thinking about totally different kinds of affordances and ways to treat ink. So there’s a whole realm there that we’re kind of thinking about that sort of intersects in this Venn diagram with sort of end user programming and dynamic behavior.

00:14:27 - Speaker 2: I think you already teed up our topic there, which obviously is programmable ink, and I always like to start with definitions. I think we’ve gotten into it a little bit, but yeah, you’ve mentioned digital ink, and I assume here maybe the first thing that comes to mind there is I scribble on an iPad or potentially another tablet and stylus, and yeah, some ink like marks appear on my screen, and that’s it. Is that basically what you mean by digital ink or is there more nuance to it than that?

00:14:56 - Speaker 1: Yeah, I mean, I think that working out that definition is sort of part of the work that we’re doing that, you know, I expect to take a while, so I, I don’t have a crisp answer on that. When I say digital ink, yes, I’m mostly thinking of, you know, the things that appear on the screen when you move a, you know, pen or stylus around on an iPad or a a remarkable or, uh, you know, some kind of device like that. But I think there is this interesting question of, aside from those pixels, you know, what is it really that’s there? What is ink in general and certainly what is the digital version? I think those are really interesting questions.

00:15:29 - Speaker 2: And then, how would you define programmable link?

00:15:33 - Speaker 1: Well, you know, again, I think, or maybe taking a very shallow stab at it with this ink-based essay, but, you know, when we think about programmable objects or dynamic objects, we often think about objects that they have some behavior, you know, they respond in some way, or they change in some way, or they’re able to be changed by some part of the system.

And so that’s mostly what we’re thinking about when we think about programmable ink or dynamic ink is just something that is aware of its environment and can be manipulated either by itself or by something else, you know, on the canvas or in that environment, or even by the user.

I mean, a lot of ink that you make in these tablet programs are It’s sort of dead, ink, it’s difficult then to pick that ink up and move it around, you know, some will allow you to have selections or drag some things, but even basic things like scaling or deforming ink in a way that’s smart, like a classic example is you draw an arrow from one thing to another in your notebook, and then you wanna move. One of those items around, the arrow doesn’t follow. You then have to manually move the arrow yourself and reposition it and rotate it and scale it. And when you scale that arrow, it scales proportionally, sort of naively, and it no longer looks like an arrow, right? Or it no longer points in the right direction or the arrowhead, you know, isn’t facing the right way or whatever. And so, That’s a very difficult problem to solve from a sort of computer science perspective, from just like a human doing stuff on a screen perspective, it seems crazy that that doesn’t work correctly. And so, we try to look for places where there is that strong sort of dichotomy, where it seems like we’ve just gotten used to something being a certain way, but it actually seems kind of terrible from just a human experience perspective, especially when you get in the context of tools for thought, where I think The tools have a really huge impact on the thoughts that we have and the work that we do, and the output of that work. And so these small differences or small bits of friction, or small biases that the tools create actually are really important.

00:17:31 - Speaker 3: I think it’s hard not to mention McLuhan at this point, right? Like, first we make the tools and then they make us. I think we really believe in this feedback loop of how the things you use impact what you can do. And this is like a hope of being better at sketching, being able to sketch dynamic models.

00:17:48 - Speaker 2: Yeah, maybe one way to think about the, it’s called the current implementation of digital link, of which Muse, you know, counts in with us, but I think it’s something we’ve thought about a bit more than others, probably because I think it’s pretty clear if you’re procreate, you know, you’re a pure art tool and it’s just like you want nice brushes, so you can create a beautiful picture. And maybe if you’re a diagramming tool, it’s really clear that it’s important that your arrows and boxes connect to vertices on the thing, and you don’t want to just like draw a rough circle around something or underline something.

But one of my favorite small examples that I certainly hope we’ll implement at some point is the idea if you highlight something with a highlighter, and then you go in and like, type in that text, as the highlight kind of stretch out the way that it would if you had selected the text and, you know, right click, select highlight, something like that. And that does get into the realm of, OK, how much do you want it to like automatically detect what you’re doing and make inferences about it. But if we do think of today’s digital link as mostly being just a direct transliteration of well I could draw on a sketchbook before, now I’ve got an iPad or a remarkable or a Android tablet and that more or less looks like a sketchbook and I can load an app that gives me a blank page that probably looks like a sketchbook and I can draw things and yeah, maybe I have a few more tools for manipulating. I can erase, I can undo, I can select and move things, I can duplicate, that’s nice, but it’s a pretty direct thing, maybe in the same way that the first word processors were essentially just, hey, what if we had a typewriter on this computer thing? And it was only later on that we started to get into much richer things like say hypertext, where you could never, yeah, that concept doesn’t make sense in a tight document, but once you’re in the virtual realm and the dynamic medium of the computer, now you can do more, but you start with that transliteration, and then over time you figure out how it can grow into something that goes beyond its kind of analog roots.

00:19:39 - Speaker 1: Yeah, I mean, I think there’s a lot of shared roots obviously between Muse and this track of work, you know, they have kind of shared roots philosophically and some of the ideas of the lab, and I think one of those big areas is when we talk about tools for thought, I think there are a lot of questions around what is thought work, right? What is thinking, what is thought work, what is the product of thought work. A lot of our current tools today don’t really respect the work product of thought work. You know, just a basic example that I’m always ranting on is browser tabs. One thing that a lot of people spend a lot of time doing these days in the course of thought work or knowledge work is basically curating their own sort of research path, right? Whether you’re researching, you know, a new toaster to buy or doing real serious academic research or whatever. A lot of times it starts in a browser and you click a bunch of links and open a zillion tabs, and then you go through those tabs and you decide, you know, each of these tabs, is it interesting or not, is it? relevant or not, you close some, you leave some open. Maybe you open some additional tabs from links that you see in there.

And that is actually work that you’re doing. And then you end up with this sort of browser window full of tabs or multiple windows full of tabs, and that’s your work product. You just spend a bunch of time and used your brain to produce that work product, and you may want to come back to that or do something with it, or pause or whatever, but most of our tooling. With the exception of some interesting, you know, newer experiments, most of our tooling does not respect the placement of those windows or the locations of those tabs or the fact that they’re open or not as your actual work product. And so, it’s treated as ephemeral, it gets lost, it’s very difficult to manage.

When we make notes in a notebook, most sort of personal note taking apps treat your notes as like this very important sacred thing that they shouldn’t lose.

But to me, they have the same value, the same amount of work has gone into them as, you know, these browser tabs that are open, for example.

And so, I think when we start thinking about what is thought work, what is work product, we start to get into these questions of what would tools look like that are more respectful of these different stages of thinking. And to me, you know, rumination, I always thought was an interesting word that was used around a lot of the muse idea, you know, there’s this point where you’ve piled all your stuff, at least for me. I think that there is a point where I’ve kind of gathered all my stuff, laid it all out on the floor, and I just want to stare at it for a while, and just like, move it around with my hands. And that’s a really important step that most people can relate to, but it’s not really supported directly as a first class thing by most tools. That are available today digitally.

And I think similarly sketching, not the art version of sketching, like you might do with Procreate, but sketching, as in thinking with your hand, you know, sketching by putting marks on a page, whether it’s words or drawings or doodles, or whatever, that is also, in my opinion, a very under-supported activity. That is a really critical activity. And the earlier you are in the process of having an idea, the more fragile that idea is, and the more sensitive to your tooling. Those ideas are.

So, you know, imagine trying to do that thinking type of sketching in a tool like Adobe Illustrator. It’s basically impossible because the interface is designed for this high precision, high fidelity outcome, and so, you just wanna like, stick a box in the corner and keep going, but in order to make that box, you’ve got to select the right tool, you’ve got to decide which kind of box, you gotta decide what kind of corner radius you want. Does it have a drop shadow? By the time you’re done with all of that, You’ve lost, at least for me, maybe this is ADD brain, but I’ve completely lost the idea by then, you know, ideas are like these little wisps that you’re trying to capture quickly before they, you know, dissipate. And so the nature of that tooling is really important.

00:23:09 - Speaker 3: Yeah, I’d like maybe to add two things to this, both kind of tangential. Like one thing that we started talking about, or maybe being able to like vocalize lately at the lab is this idea of the same way we have like napkin math or back of the envelope like mathematics, just figuring out orders of magnitude or something or whatever. And interesting parallel is back of the envelope computation, like how do you make these little interactive things with the same approach of like roughly just hand waving at the thing. And another idea bookmark maybe here is I lost it.

00:23:41 - Speaker 1: So you’re just proving my point.

00:23:45 - Speaker 2: Well, actually one thing I’d love to hear from you, Shimon, is, you know, there’s been a number of projects in this track. thinking base is certainly a very notable one, but you have become pretty accomplished.

Obviously you were instrumental in creating the tool, but you also are probably the most accomplished user of these various research tools, you know, prototypes in the world, and indeed you’ve given some good talks including recently. Strange loop where you kind of give live demos or maybe they’re videos, I’m not sure, but in any case, it shows your depthness with these different tools. So I guess I have to just ask like, what does it feel like? What does it feel like to have programmable link at least in this early stage?

00:24:26 - Speaker 3: The phenomenology of to use.

It’s kind of maybe hard to describe, right, like. It definitely feels distinctively different to how you approach doing things on your computer.

So like, for example, like one thing that comes to mind is an idea we play around with crosscut, like a different paper in the same thread, where we basically create like a little drum machine, just about connecting a couple of dynamic objects together.

We have one that moves left to right. We connect the line that says vertical, a box that finds things inside that box and that controls a drum rhythm.

So working this way is completely different to how like, Create my own drum machine if I wanted to, which I did a couple of times, which often means I have to turn on my Max MSP or like Python or whatever, figure out what is the correct like MIDI signal to send somewhere, basically switch to this logical thinking, Oh, I will have like 16 steps, so there’s an array that I need to care about now. And this array has values in it and whatever and start thinking about very symbolically.

I’m trying to solve versus kind of the tinkering pre-college like approach of the other things we’re doing.

And there’s a lot of parallels like that to me where I have a very strong maybe programmer brain because I’ve been doing it for a while where The way you approach solving problems in these tools is totally different. Like, you explicitly tried not to have these things that feel wrong in programming.

Like one example often comes to my mind is this spooky action at a distance, where you say, oh, there’s this database over there. It has like an abstract ID. I’m gonna grab that entity. And bound it, whatever, like grab a property from it, and so on. Where in inkbase, you say, oh, this thing the left, make it red. And that feels like very concrete. You can see results of your actions immediately and you also think it is very humane spatial way where like something to the left is much more obvious that ID with like UI ID that has 64 characters or whatever, right? Like there is a different way you use your brain and think about things and that really left a strong impression on me.

00:26:27 - Speaker 2: I think that is somewhat how research works, right? If you were starting with like a really burning pain point to solve in the kind of classic sense, you’d really be starting a commercial product.

Whereas research, I think is, at least for me, is driven by a sense of how things can be different.

You see the capabilities of the computational medium, particularly maybe emerging new technologies like tablet, you know, low stylus latency as one example that I think was an inspiration for us. And you think about how you would like computers and our computing tools to be, and you see what the potential is with either the way technology is today or the way it’s evolving, and you can kind of extrapolate out and think of some end state. You you’re not really starting with a specific use case, you’re starting with a vision of how things could be, so necessarily use cases do get a little bit kind of backed in there. To me that seems natural.

00:27:21 - Speaker 1: I also think we often have, at least for me, I have a lot of use cases that I want to have a tool like this for, but you need to have a pretty full fledged version of this tool in order to actually carry out that use case and experience it, and we’re quite a long ways from being there, I think.

And so, with some of these projects, we’re trying to see how one of these use cases could be implemented.

In some of them, it’s really more about the feeling of the tool, and Inkbase is one of those projects. We explicitly said with Ibase. OK, we’re gonna kick the can down the road on, like, what is the right programming model and what is the right interface for doing this programming and all that, and we just want to get to a place where we have dynamic ink that we have programmed, that’s on the screen that we can interact with, just so we can get sort of a little glimpse, a little vignette of that and see what it feels like. And I found the results to be very compelling, but again, these feelings are very sort of subtle and nuanced, and I think important for when you put them in context of the way that the tools you use bias the results, I think the nuanced differences and feelings of tools are really important, but they’re also kind of hard to describe, and we’re kind of grasping at that in this essay, one of the things we’ve started to talk about in the lab when we think about the design of different tools, is sort of the quote unquote natural grain of the tool. And we give a little definition in this essay, we think about how tools, most tools, physical hand tools as well as digital tools, have some sort of natural grain. A way or set of ways that the tool can be used that are easy, fluid, efficient, sort of encourages you to use the tool that way, and then there are ways to use tools that are against that grain. And not that you can’t use them for those things, but they just don’t work super well. Think, you know, using a machete as a screwdriver instead of as a machete, or think using Microsoft Excel to do artwork, you can absolutely do that. It’s just kind of against the grain of the tool, and so a lot of times we ask, what kinds of use, what kinds of feelings do we want to be with the grain? Like, what direction do we want this grain to go? And for example, With sketching and sketchy ideas, and early thoughts, we want the grain to be very much encouraging you to keep thinking and not get distracted with, you know, high fidelity thoughts, you know, is this thing pointing to the other thing? Is my square, you know, a perfect enough square, whatever. Another thing in that bucket in terms of trying to develop a sense of what things feel like we’ve started to talk about, I think this is one of Simon’s originally, is working with the material. Sort of a phrase we talk about, which sort of evokes this. More physical thing you might experience, like, like in art, if you’re working with, let’s say, clay or some medium charcoal, it has a very specific kind of feel, and certain things that it wants you to do and certain things that are difficult to do, and just having your hands on those materials kind of shape the outcome. You kind of just let the material in some ways guide where you’re going. And I think that we found that this ink base has a little bit more of that working with the material feeling, which is what we were going for, where you’re actually It’s sort of like, we talk a lot in the sort of end user programming world about direct manipulation. You know, where you’re working on something directly versus indirectly from some program on the side or whatever. This is sort of a flavor of that, but even more direct, where you’re not only directly touching the thing that you’re trying to manipulate, but you’re actually being influenced by how it feels. And ink, I think, is one of those things where when you interact with it, the way you move it around, the way you create it, you’re having something change color or change shape while you’re drawing on it, and not after you finish your stroke, but live while you’re doing it as a very specific working with the material kind of feel. And it’s hard to put my finger on exactly what that means, or it’s hard for me to describe exactly how your results would be different with that feeling versus another, but it is quite distinct, and it’s something that I personally am drawn to, and it’s something that I like about the real world that I feel is missing in a lot of our digital tools. And so I think part of this is a quest to obtain some of that in the digital realm.

00:31:24 - Speaker 2: I support your quest.

The feelings or obviously you used that word a lot, how does it feel or what feeling does it create or what things does it encourage in that direction.

You had an interesting aside in the essay titled Research and Feelings, which I’ll just read the first sentence of here. It says, perhaps controversially.

At the lab, we believe, seeing what it actually feels like to play with an imagined system is itself a valuable research result and goes on to talk a bit more about that.

And actually I think that is an interesting, I don’t know, meta learning or something like that site contrarian insight of the lab and something we were able to do because we’re somewhat unique, not quite academic, not Quite industry position, and I, I think it was Martin Klepp and I feel like was the first one that called out when he first started working with us, which is he said, basically the fact that we are not constrained by the conventional definition of rigor, which is, OK, we user tested this with 10 people, and with this P-value of whatever they were able to complete the task in 2 seconds less. Which is a very good reason science focuses on those kinds of very concrete, measurable, rigorous findings, but I think that misses something huge in the computing tool space.

00:32:40 - Speaker 1: I do think that’s a really interesting aspect of what we do at the lab, and the lab is engaged in what we call industrial research, which, you know, has been talked about a bit before on this podcast, and To me, one of the luxuries of industrial research is that you’re not bound to the traditional sort of rigor and neutrality required of basic or academic research or just science in general. You know, we’re allowed to have an opinion, we’re allowed to chase intuitions, and I, in fact, put that into the essay as sort of a defense or an explanation because We had a number of people who were reviewing the essay for me comment on, you know, what’s what the feelings, like, take this feeling section out, it’s not defensible. And I felt like it needed to be addressed, but I wasn’t going to cut it because to me it’s, that’s the most important part. So I kind of wanted to add a little side note, defending having a research result that says something about feelings. But, you know, I do think that that’s a schism that we often see where in sort of academic circles. Oftentimes it’s about novelty, and if an idea has been described before, then working on it further is not interesting. It’s not an interesting result, and we disagree with that. We think actually taking some idea for how something might feel different or work differently and actually building it and seeing if that is in fact true, is actually valuable. And sometimes we do that and we are compelled by the result and it directs, you know, further research, and sometimes it’s a disaster, and that’s surprising. It doesn’t work and we learn things about why it doesn’t work, and sometimes the results are sort of meh. You know, it’s, we have these great expectations about how this thing could feel different and then we make a thing and then it’s like, yeah, it’s kind of a hassle and it’s really not that much better. And I think all that’s really interesting fodder for understanding the nature of this problem and where we’re headed. I also think conversely, on these sort of more pragmatic sort of startup engineering, product oriented side of the spectrum, people build things all the time, but they often don’t stop to think for long enough about, you know, why they’re building them, or what the nature of those things are, or what the most important aspects of those things should be. You know, it’s sort of in the quest for ever closer to to use. To research, doing what the users are asking for, you kind of start to get away from these first principle kind of based approaches. So at the lab, we’re trying to strike this balance between this sort of overly pragmatic staring at your feet, just doing the next step kind of thing, and this overly, you know, impractical sort of ivory tower pontificating without actually seeing what the results are. We’re trying to be somewhere in the middle. And I think our research hopefully reflects that, and I think our talking about feelings is sort of part of that quest.

00:35:30 - Speaker 3: So if I can expand on this just a little bit, maybe from a different perspective, I think it might be on the metal. I think maybe a common critique of HCI as a field is that you’ll get what you measure kind. So if your focus is on making things that are measurable, there’s a bunch of research that you just won’t do because you, you can’t really measure it.

Like I think that’s why a lot of people think that’s the problem with focusing on measuring mouse click speed or like how quickly you can get to a task done or whatever. Which prohibits this very like exploratory programming system kind of style of research, which is very hard to measure because what do you even measure and against what else which has its own problems.

The Second thing that comes to mind is we keep research logs as we work at the lab on the mental level, and a lot of things in these projects start with like this specific approach just feels correct or feels right.

For example, one of the recent ones in the project we’re on right now were about like measuring angles of things and someone made this example of just making like a little thing that you snap to the angle and that turns into a number that just felt good. That’s why we’re pursuing this further and I think this following feelings at some level is, is kind of correct of like leads to very interesting places, places that academia would go to basically because, yeah, again, how do you measure that?

00:36:49 - Speaker 1: Yeah, I mean, maybe back to this sort of the right tool for the job idea. Most of the things that we’re trying to do with these tools are certainly things that you can do with other tools, right? You can make a sketch a number of ways, you can think a number of ways, you can do calculations a number of ways. It’s more about You know, sort of having the right context, having the right tool, bringing the tool to the problem rather than the problem to the tool, those kinds of things. You know, maybe a concrete example, just yesterday, I was trying to get the square footage of a small space, and I made a sketch of the shape of the space, and then I went around with my Little laser, you know, measure and measured the lengths of all the walls. And I did this as a sketch because it’s very difficult to walk around a room with a computer and then put those numbers in and describe which wall those numbers are associated with. It’s much easier to just write those numbers on a sketch of the shape of the room. And it doesn’t feel right to sit down and open up a graphics program and try to make a perfect version of that room before measuring it. It really feels like a thing that wants to be a sketch on a napkin or whatever. However, once you have done that, I then need to break that shape up into a bunch of squares and calculate the area, and that starts to feel like a spreadsheet problem. Because once I do that, I often, when you measure things in the real world, they often don’t entirely add up. You know, the two segments of the wall on the left side don’t add up to exactly the wall on the right side, and you kind of need to figure out if you made a serious measuring error or if it’s just, you know, the world isn’t perfect.

And so, you need to do this math, but then you need to check the math against the other side, and you may need to make some adjustments. It feels very spreadsheet like and that you want the machine to sort of help you check your math versus Pulling out a calculator and doing these things like 30 times. And so now I’m suddenly sitting here with this thing that should be a sketch with some numbers on it, wanting to do a little bit of math, which I want the sort of power of the digital medium for, but my options are basically, I have to set the sketch down and look at it while I open up a spreadsheet and do it in a spreadsheet, sort of disembodied from the sketch where I can’t associate this set of numbers being multiplied with this area on the diagram. Or I’ve got to do it with a calculator, and I don’t get the power of the spreadsheet, and it’s hard to check my math. This isn’t an important problem.

It’s not like a thing I can’t solve. It’s not a thing that people don’t solve every single day. You can open up a sketchup, and then you can put all the numbers in there, and it’ll tell you the area, for example, but it just doesn’t feel like the right tool for this job. It feels like you should be able to sketch this thing out on your iPad, walking around with it, and then You know, do that math, and like, draw the boxes on there yourself, and then, you know, do the multiplication and see what it adds up to, and maybe jiggle the drawing a little bit. That’s when you realize that, you know, they don’t add up. And I think having tools like that, it’s hard to imagine exactly what we would do with those things, but I find personally that I have uses, little use cases like that every day. That I would reach for this tool if I had it. And then you start thinking about situated software and the way spreadsheets, one of the things I think is interesting about spreadsheets is that often a piece of software evolves out of that process. So you do that once and then you throw it away, it gets lost in your, you know, Google Drive or whatever, but then you go to do it again, and perhaps you want to, you know, rework some of that logic. An example from my life is batching cocktails, which we we talked about a bit earlier. You know, often when you want to scale up a cocktail, certain things don’t scale linearly and you start doing a bunch of math and you need to convert units, and it’s easier to weigh things than measure volumes when you’re doing large amounts and Blah blah blah. So, you end up building a little spreadsheet to do this thing and you throw it away after you make your cocktail, but then maybe you go to do this again a week later, and it saves you time to open that thing up and duplicate it, and then adjust it for a different cocktail. And then after you’ve done that 2 or 3 times, you might start to say, you know what, this seems to be a thing I keep doing over and over again. Why don’t invest a little bit of time cleaning the spreadsheet up, making it a little bit clearer, making it More, you know, sort of input output driven, so I can just paste in the ingredients and have everything turn out the right way. Maybe I’m going to invest in adding a table that does unit conversions from ounces to, you know, milliliters to weights or whatever, and you eventually end up with this little piece of situated software. And this is a true story from my life. I have this thing, and then a friend sees it, and then they’re like, Oh man, I have the same problem all the time. Will you send me your spread? Sheet so I can start using it. And now we’ve basically made a piece of software. And I think that’s a really important sort of flow that this is something we call gradual enrichment in the lab for lack of a better name, we’re still grasping at what the right name is for this, but this idea that you start out loose and sketchy like you would on a napkin and you end up with a piece of software, and at no point did you sit down to write a piece of software. You just keep incrementally adding little bits of behavior over time. And only as the payoff is obvious. You’re only investing little bits at a time when you’re gonna get an immediate sort of payoff for that investment. And we would like to see that same gradual scale up with sketching, staying right in place where you make that sketch, like having to stop and change tools to throw away the sketch, move to your laptop, open up Illustrator o