Untitled

Discuss this episode in the Muse community

Follow @MuseAppHQ on Twitter

---

Show notes

00:00:00 - Speaker 1: With spatial computing, there’s a level of trust that the user is placing in you as a developer that most software developers have not had to handle. On a phone, if the app crashes or freezes, it’s annoying, but it’s not going to make you sick. It’s not going to viscerally affect the central nervous system. Whereas in the case of any immersive software, it will. You’re going to directly put their brain in a state that is uncomfortable or even harmful.

00:00:33 - Speaker 2: Hello and welcome to Meta Muse. Muse is a tool for deep work on iPad and Mac, but this podcast isn’t about Muse product, it’s about the small team, the big ideas behind it. I’m Adam Wiggins here with my colleague Mark McGranaghan.

00:00:46 - Speaker 1: Hey.

00:00:47 - Speaker 2: Joined today by our guest Eliochenberg of SoftSpace.

00:00:51 - Speaker 1: Hey, Adam, hey Mark.

00:00:53 - Speaker 2: And Elio, I understand that you’ve been doing a little bit of breath work recently.

00:00:58 - Speaker 1: Yeah, so I was just sharing with you some of my learnings on the importance of breathing, which I feel like a lot of people maybe have figured out before, you know, way before I came across this topic, but I started trying some Wim Hof breathing before some of my like cafe work sessions, which is equal parts actually very invigorating and effective. I find it helps me focus and also makes me feel like a complete weirdo sitting in public, like staring out the window and breathing really intensely. So I recommend it to people who are looking for ways to, you know, quickly get in the zone and focus when they maybe are a bit distracted. And if you have any tips, you know, on different resources, I’m very open. I’m very curious about this.

00:01:39 - Speaker 3: What does this breathing technique entail? What are we signing up for here?

00:01:42 - Speaker 1: So, I mean, Wim Hof breathing specifically is this cycle of very intense breath in, breath out. There’s nothing too technically complicated about it, it’s more just about sticking to a certain rhythm and at the end of, I think like 20 or 30 breaths, you hold your breath for about a minute. There’s a very helpful Spotify podcast episode that’s like 5 minutes long, that just guides you through it. And so there’s all this drumming and, you know, Wim Hof is kind of like they’re motivating you through the whole thing. So I find that after I do this breath work, I am indeed able to just like really get in the zone and whether it’s for writing or cracking some other like tough cognitive problem, I’m definitely more focused afterward than without doing this.

00:02:30 - Speaker 2: It feels a little bit adjacent to meditation somehow, but I also know you breath work, I don’t know about the specific one, but just the topic generally, I’ve known people in the psychedelic community that basically say you can get unbelievable altered states. One example here you’re giving here is like, yeah, greater focus or something like that, and you wouldn’t believe it because yeah, breathing is so fundamental, it’s literally automatic and What is there to it? It seems so simple. There’s some incredible potential there to affect ourselves. I never dabbled myself, but I’m certainly curious.

00:03:04 - Speaker 1: Yeah, I mean, so one discipline I came across this holotropic breathing, I believe it’s called, which is you can breathe yourself into a very altered state that’s akin to chemically altered psychedelic states.

00:03:17 - Speaker 2: Have to give that a whirl. And tell us about first what SoftSpace is and then love to hear about your journey and how you got there.

00:03:25 - Speaker 1: Sure, so I am the founder of a software company called SoftSpace, and we’re building a product called SoftSpace.

Which is a spatial canvas for thinking. So it is a 3D augmented reality app that lets you organize and make sense of the ideas, the images, the websites, PDFs that you are working with in your creative projects or in your personal or professional projects.

And the way we frame the value proposition is that Soft space shows you the true shape of your ideas, and there’s a lot of research that has been done over the years into the immense, almost like superpowers that we have around spatial memory, spatial reasoning, and up until very, very recently. which we’re going to talk about in this episode, until very recently, we didn’t have the technology to really tap into those innate abilities.

And so the best that we had was like a larger display, a computer display for, you know, showing you more windows at the same time, but that’s only scratching the surface when it comes to the brain’s ability to make sense of and to remember and to think about objects in space, which we have evolved over millions of years to do very, very well. And so I started building this company in 2017, way before, you know, the current crop of hardware, standalone headsets was really even on the horizon with this kind of, I guess, expectation and faith that eventually the technology would catch up to this idea, and I think that it’s starting to, and that feels really good.

00:05:06 - Speaker 2: And my first introduction to your product was we met in a cafe in Berlin last year and you handed me the, I guess this would have been the, at the time, the latest version of the Oculus, which I think has been, or in the last 10 years has really been on the forefront of this, and, you know, it has this element where I can still kind of see the environment, so I’m not just completely zoned out in a public space, but I’m also seeing essentially notes and other ideas floating in space and indeed I can interact with them and Yeah, the how viable is it relative to the Hollywood version of virtual reality that we have been seeing for ages is a huge question and for sure an app developer like yourself that chooses to not only pick a particular platform, but the technology in general, you’re making a bet that the amount of time you’re going to be working on it will overlap with the eventual viability of it for your particular use case or your particular market.

00:06:01 - Speaker 1: Correct, yeah, and I mean, I would say one of our investors said it’s still early, but it’s no longer too early, and I think that’s getting more and more true all the time.

I mean, even with, of course, the very big news of Apple finally entering this space, I think we’re still a little ways out from really mainstream adoption of computers they wear over your eyes, but if It were ever going to happen, this is the path that I think the industry, you know, needs to take to get there. And I think one of my personal motivations for continuing to work on SoftSpace is to offer a vision for what our augmented reality spatial computing future could look like that I think we want to want, right? So, I think up until very recently, the overwhelming popular imagination when it came to VR, for example, was at best like a little bit goofy and at worst kind of dystopian and not something you would necessarily want the next generation of humans on the planet to be living and working in because it felt very disconnected, it felt very escapist perhaps, and I think that this technology is So much more than what we’ve been able to imagine up until this point. Like we’ve been able to imagine a lot with essentially nothing, right? And fictional depictions of, you know, the metaverse or fictional depictions of very futuristic holographic UIs, but those have really only been fictional and now we’re finally seeing.

The reality of it, and I think that there are many possible paths technology can take, and the underlying power of it has nothing to do with the computers or the chips or the lenses.

The underlying power of this is the fact that the human brain and body are inherently spatial, right? We are spatial organisms. And so whatever positive outcomes or whatever negative outcomes come from this technology will be rooted in that reality. And so I’m both optimistic and also now that the reality is finally here, you know, we see Apple making a big move for it. I’m a little bit trepidatious about sort of where this could all go. I mean, we’ve seen with other technologies that people had very optimistic visions for, right, turned out maybe not completely positively. So I think this is at least has that risk, if not a greater risk because of how it works if it is.

00:08:32 - Speaker 2: Yeah, and we’ll definitely get on to all the present and future here, but can you tell us a little bit about your background? What would lead you to, you know, that moment in 2017? What you said, this is what I want to be doing.

00:08:44 - Speaker 1: Yeah, absolutely. So, I was in architecture school and I was halfway through my second year, and I took a summer job at a design and art studio here in Berlin called Studio Oliver Liasson.

They had just bought the Oculus DK2, the Development kit 2 VR headset.

It made quite a splash. A lot of people who are excited about technology had gotten their hands on one. I really wanted to check one out. The studio got one thinking it would be like any other piece of consumer tech, you could boot it up and try stuff out, but it really was a development kit. There was nothing that you could do with it if you didn’t code something up yourself. And so luckily I got a job as the research resident, poking around with this thing, trying to figure out both how it could be used as a medium for artworks, as well as a tool for the production of artworks that maybe weren’t digital or virtual of themselves, but would benefit from some sort of like virtual visualization or some other tooling around that.

00:09:48 - Speaker 2: I mean, architecture is certainly a place where use cases spring to mind very readily. Let’s walk a client through kind of a design that we made, you know, in some CAD tool or let’s do some design work there. So presumably those are the sorts of things you were exploring.

00:10:04 - Speaker 1: Yes, and I would say much more than that as well because this studio is very much an art studio first and foremost, and one with a history of being interested in the body, the human body, how we relate to ourselves and to others and what different spaces and different spatial effects like lights, acoustics, atmospheric effects can do to our sense of ourselves and others.

And so this is actually Maybe where the most exciting promises of virtual reality at the time, it was only VR virtual reality came in because you could create effects that would be physically either very difficult or impossible to do. So one of my favorite demos that we built was this non-Euclidean, sort of like castle that you walked through. So it was back in the era of like really long cables that connected you to a PC. We had the PC in the middle of an open area. The user would put on the goggles at one edge of the open area and walk in a circle. And as they walked, they would walk through doors, and around each door was a new room with an artwork in the center, and as they walked, at some point, you know, they would realize, wait, I should be back where I started, but I’m not. I’m actually somewhere else. I’ve actually entered yet another larger room that shouldn’t physically be able to have fit into this floor plan. These were the kinds of experiments that we were doing, and during this period of experimentation, um, I came to two formative realizations. So the first was that the physical building that the studio was in, it had about 110 people at the time, and it was in this old beer brewery in the middle of Berlin. The physical studio itself was an incredibly important part of the creative and production process. We walked around and there are models everywhere, images pinned up on boards, books, there’s like libraries all over the place, half finished sort of sketches laying around at people’s desks, and this physical space was in and of itself a framework on which the creative process hung. And that was something incredible to see, and also, you know, this is quite a successful studio, and I felt that having that space was a major asset for the studio to be able do its work.

And the second realization, as I was working with VR was that many of the same qualities of that physical space actually don’t have to be physical in and of themselves. So the images that you had pinned up, the notes that you had laying around, these were actually at the end of the day, just media for holding information, right, for conveying information, and you could do something very similar with a purely virtual environment, you know, you can’t completely recreate it, but Not everybody has access to a giant beer brewery or even a very large room, right, to lay out all of their thoughts and their ideas. Maybe this technology could democratize access to space for thinking, space for doing your best work.

And once that idea kind of sparked in my mind I couldn’t stop thinking about it and sort of stereotypical, like I was like laying awake at night dreaming, you know, oh, if you could also make this multi-user, then you can like meet with people from anywhere in the world. And so at some point I thought, OK, this has been great, but I need to go see if I can build this thing, and I didn’t really know what I was doing at the time. But apparently I was starting a tech startup, a software startup, so we got a bit of funding. I was very lucky that we had a wonderful investor Boost VC make a bet on us, and they flew us out to San Francisco and we learned, you know, like, what’s a product, what’s a market, and we’re still around, we’re still around chugging away.

00:13:45 - Speaker 2: of that story where it’s the serendipity which you know often is a big part of any kind of creative spark, but here both that they were, yeah, you had this opportunity to work with this cutting edge technology for a different purpose, obviously, they wanted to create art or explore the spatial environments that they were working on and then you also through that exact same opportunity had access to information in a space. And then making that kind of leap of, can we make information in a virtual space.

00:14:18 - Speaker 1: Very interesting, right? And, you know, so I was in architecture school at the time, I ended up dropping out to keep running this idea, but because of my background in architecture, and because also of the fact that the tech at the time was only VR, you know, everything that the user was seeing had to be digitally rendered.

SoftSpace started with a much heavier focus on the design of the virtual environment, because I believed then, I still believe now that the environment is a critical factor. And getting you into a certain kind of headspace, letting you think through certain problems that you just need the right kind of environment to do.

But over the years of working on the various versions of SoftSpace, of course, we also then started doing a lot more design and development work around information architecture and user interface design. And by now, when we have finally the possibility of pass through augmented reality, There’s almost no sort of virtual environment design anymore. I’m not directly thinking about what the digital environment of our app should look like, although I have some ideas about what the ideal space you should be in, maybe when you’re trying to get focused on some work, but we’re now grappling much more directly with problems around. Yeah, information architecture, the right primitives that the user should be working with to help the user work directly with their ideas, with the information that they’re trying to make sense of, and the right UI paradigm and language to express these elements in.

00:15:57 - Speaker 2: And maybe we can briefly define by virtual reality, you’re referring to something that is 100% immersive, you have no awareness of your surroundings, and then I don’t know, it’s augmented reality and mixed reality kind of the same.

Two words for the same thing, but at least as I understand it, it’s something where there’s some combination of you still see the world around you, but you have these additional things from the digital things sort of superimposed, you might say, and I know there’s even different technologies on that which include actual pass through goggles or it’s projected on your retina or something versus you’re still looking.

Scres, we have external facing cameras that kind of bring the reality into or bring what you would see if you were looking in that direction into the space that you’re in. So interesting, I hadn’t even thought about how the mixed reality or augmented reality actually greatly reduces the amount of, I guess just stuff that you need to be rendering or think about or design, which is maybe a good feature.

00:16:55 - Speaker 1: Correct, yeah, I think by this point, my sense is that VR is pretty clearly defined. I think most people would give you a pretty coherent, similar definition of VR. I think between augmented reality, mixed reality, extended reality, I think the definitions there are, you know, you’ll have as many different definitions as people you ask. I would say that within that spectrum of taking something that is virtual and then also showing you the physical space you’re in, there’s also a spectrum of that virtual information being aware. Of your physical environment. So I guess some people would say true augmented reality has to engage very thoroughly with your physical environment.

00:17:41 - Speaker 2: So you would have a file, some representation of a file, there’s a version where it just floats in the air and some basically random place and there’s another version where I can kind of detect that my desk is here, so it sort of puts it on my desk. In the right orientation.

00:17:55 - Speaker 1: Yeah, I mean, there are merits and demerits of how much the virtual system can be aware of or should be aware of your physical environment, but I guess, you know, it’s in the term augmented reality that some AR purists would say it’s not augmented reality if the virtual is not literally adding to your physical environment.

00:18:17 - Speaker 2: So the mixed reality is a little more neutral in a way. It could be somehow adding or interacting with the environment you’re in, but it could just be you just have like a heads up display overlaid on top of what you’re saying, correct.

00:18:29 - Speaker 1: Yeah. So there’s a term that encapsulates all of these different categories, which I’m a personal fan of spatial computing.

And spatial computing, as far as I know, as a really concrete concept was coined by Scott Greenwald at the MIT Media Lab in 1995, and he was talking about digital systems, computer systems that maintained and used references to physical objects in physical space, or parts of the user in physical space.

It was very broad, but over the years and very, very Recently, I think it’s been taken up by some members, some participants in the XR ecosystem to mean this sort of very general idea of a computer or computing system that engages with The fact that you are a human being in space, and very directly. And I like this because it places the emphasis not on the technical capabilities of a system, or on the specific UI design decisions that the developers might have made, but it really sort of focuses attention on the underlying material of what we’re designing with, which is Three dimensional space. I mean some people would say 4D space time, but it’s the idea that you can place things, you can work with information that has this intrinsic quality to it, of like being somewhere specific relative to the human being, and that this poses both great opportunities and new and, you know, previously unencountered challenges.

00:20:13 - Speaker 2: Well, you teed up our topic today, which is spatial computing, but certainly encompasses. I like the perspective of VR and AR as means to an end. They are a way of accomplishing the goal of making computing more spatial, whether we bring it into our space or whether we make it just access the spatial capabilities of our minds. I think starting with the human centered or starting with the benefit or starting with the user’s mental model is a better way to talk about really any technology here.

00:20:41 - Speaker 1: I agree, and I think that that’s maybe an angle to this technology that has been under communicated, and I hope the community of developers and the big players and small players that we find a way back to that foundation for any successful product or industry, right? Like, what is the actual value of this? Beyond the novelty, beyond the technical wizardry, beyond, even I would say the hedonic qualities, like maybe it is just really nice, right, to have this massive surround screen that you can watch, you know, your NFL games on. But beyond those, why do we need this? What will this unlock? What does this add to our lives and to our work that We would be poorer for if we didn’t have it as opposed to, oh, if it wasn’t this, we’d be still playing games on our phones instead, and it would be all kind of a wash.

00:21:41 - Speaker 2: So what are some of your answers to that in terms of what you’re trying to bake into your product or influences you’ve had from academia or other thinkers who have been pondering this topic.

00:21:53 - Speaker 1: Yeah, I spoke earlier about the fact that our brains and our bodies have these spatial superpowers that are not fully or even really well used by existing. 2D user interfaces, displays, input systems, etc.

A very telling quantitative metric is that from the original 1984 Macintosh to the, I’m using an older model computer, but the 2020 iMac Pro, and by now Apple’s latest and greatest are much faster than the iMac Pro, but the computing power increased by 10 million times, by a factor of 10 million.

If you count, you know, the CPU, the GPU and the display area increased by a factor of 10.

And it’s still a rectangle, right, that you click around on with a mouse.

And now there’s nothing inherently wrong with that.

I mean, clearly the iMac Pro was a very successful product and help you do a lot of amazing things that the original Macintosh, you know, you wouldn’t be able to imagine using that to do.

But, you know, you have to wonder what this massive discrepancy in capabilities precludes.

And I think now that we see at least 2, and hopefully soon more of the large tech players.

Looking at that question seriously and proposing answers to it, I think we’ll start to see what computers might have been able to help us do all along, or already have the computing power to help us do all along, right, but simply didn’t have the display technologies to make that possible.

Very concretely, I know that training, any sort of scenario where human users need to be learning something that’s very experiential. These are use cases that are already very valuable, so pilot training, a physical simulator, apparently these are like in short supply and they’re very expensive to run and take, you know, months to book, and a lot of these are being replaced now with VR systems and that makes a lot of sense to me.

There are pilots running with VR surgery or VR surgery planning use cases. So these very high value, very sort of intrinsically spatial use cases where, you know, we had all the computing power necessary to do these things before, and now we have the display technology as well.

What I am personally motivated by in building soft space. Is the belief that there’s tremendous value to working with 2D information in a 3D environment.

And I think that a lot of the 3D use cases are in architecture, with manufacturing, with surgery, you know, A, there are people who are far more knowledgeable about those specific domains than myself, who can work on those problems, and B, I think those problems are very well served because there’s such an obvious connection between, you know, a 3D display and the 3D model or something.

What I think is relatively under explored, but has the potential to impact a lot more people directly. is giving people a way to work better with information that’s intrinsically two dimensional or best represented two dimensionally, but in a spatial context and If you look at Apple’s marketing materials and the imagination that they’re offering for what spatial computing looks like, this is actually their Vision, right? There’s like maybe 1 3D model in all of their hours of marketing material. Most of the time they’re showing you documents, they’re showing you photos, they’re showing you app windows or web browsers, but in this 3D context. And so I would like to think that the design minds at Apple are pursuing a very similar thesis that there is tremendous value in letting people work with 2D information, which has the advantage of being portable to all the other devices that, you know, we already have. You can print 2D information out on a piece of paper and mark it up, so it’s a lot more flexible and a lot more universal, but there’s a lot of value in letting you work with that in a 3D context, and that is essentially what SoftSpace is.

00:26:20 - Speaker 2: Yeah, well, we’ll certainly come to talking about the Vision Pro.

I’m sure folks are curious to hear your take on that, but yeah, since we’re sort of talking about use cases here, it’s often the case for any new technology that you figure out something new and impressive you can do with computing or some other technology first and then you sort of figure out how that can be used and often we’re surprised by The use cases that end up coming out, you know, I don’t know that the people that invented TCP IP predicted e-commerce, for example, but often that has to be discovered once the technology exists and is in the hands of a lot of developers and end users.

And I do think that’s one where to me it feels like VR and AR has been pretty impressive for quite a while.

You mentioned using the Oculus dev kit. I think I tried it first around 2013. A friend of mine had it and yeah, you know, very much long cable connected to a PC, you know, pretty limited, but it had a little, you know, demo of someone riding down a roller coaster and it basically became a party trick for him to essentially put this on people who had never experienced it before and everyone else would stand around and watch them react to that. So that was fun.

But it doesn’t become a thing that’s deeply integrated to your life.

And certainly my dabblings in the past, which are not as extensive as yours, is that games and immersive experiences, maybe like sort of interactive movies or something like that, are kind of a good place to start, partially because of the immersiveness of the environment, partially because I don’t know, games are always a good place to start.

Indeed, if I was to try to name a killer application off the top of my head for VR, probably Beatsaber is the first thing that comes to mind.

Then you go from there to, yeah, of course those either domain verticals like surgery, training or pilot training or architecture design or walking a client through a space or something.

But then there’s this whole world of like collaboration, right? We’re going to a remote first world, we want to have meetings, we miss our whiteboards, we miss the body language side of it, and then you have just productivity software and that’s something where that feels like it’s gotten the least attention.

And maybe that’s because when you think of productivity software, a word processor, a spreadsheet, a video editor, a design tool, coding, yeah, it’s very much about those 2D rectangles. I’m not even sure if 2D rectangles are the perfect or most pure form of representation of that. It’s just something, yeah, starting from paper and scrolls and then books and then up to computer monitors and And even phones, obviously, writing also is a big part of all of that, that’s the format we’ve always used.

So then you can bring that to. This 3D environment, but in the end it just happens to be a rectangle that’s sort of like floating or you can make bigger or you’re sort of mapping the same two dimensional window metaphor into that environment, but it sounds like you think that one way to kind of interpret that like, well, if you’re going to bring productivity software into some kind of spatial computing environment, OK, let’s just make it a floating 2D window and one interpretation of that was like, well, that’s really kind of Inspired in the sense that it’s just a very direct mapping, but it sounds like you think actually there’s more promise to it than that, that there’s a reason why so many of these past iterations of our information technologies tend to revolve around writing and kind of one dimensional or two dimensional squares or rectangles of some kind, and there’s value to bringing that to a virtual spatial computing environment.

00:29:54 - Speaker 1: Yeah, I do. And I would distinguish between a 2D UI paradigm, like a window or a grid for that matter, and information content that is inherently 2D or is best represented in two dimensions like text or images or a PDF page.

So, One of the big shifts that I’ve made in my own thinking about how to design for spatial computing happened when I Came across Rome research, and at the same time I started using Notion myself.

I never actually got into Rome so much, but I read a lot about the thinking behind the design of Rome, and in both these cases, Rome and Notion, these are block-based note taking tools or productivity apps.

The conceptual and technical and, you know, UI primitive is the block of content, the block of information. And this paradigm in both these cases works within one app, so the app has control over what its UI elements are, and it’s decided that OK, it’s gonna be a block of text or block of an image, but there are others who have been doing work into Speculating about what an entire computing environment or entire operating system that revolved around these what are currently would be considered subunits of computing information, what an entire operating system that worked this way might be like, what advantages it would have over our current paradigms.

And once I kind of really wrap my mind around what block was, I essentially shifted my own development model toward working with blocks, because Blocks to me, map so much better to the underlying material of thoughts and of creativity than, you know, a Word doc or an Excel spreadsheet do.

And so for me, one of the promises of spatial computing is to give you more Powerful ways of displaying information that is kind of around a block in size, displaying the relationships between those items, because for Rome, a big part of its appeal to a certain kind of user was the ability to represent explicitly the links between the blocks, right? So back linking and being able to explicitly construct arguments, drawing from pieces of evidence or pieces of information that are elsewhere in your database in your notebook. And on a 2D display, there’s just all these limitations around like how much more other information you can show, how you represent these links in an infinite spatial canvas or an infinite 3D spatial canvas, you have many more options.

At the same time, you know, that sounds great and it sounds powerful, and why don’t we all already work in this like a beautiful mind kind of memory palace. Well, there are also real constraints on our ability to process that much visual information, and you do pretty quickly hit a point where it’s overwhelming, you know, there are times when you do prefer to just have one piece of text in front of you that you’re focused on, they’re thinking about, and to have a few other relevant or Supporting materials close by at hand, but not to have everything you’ve ever thought about, you know, everything, every topic, visible at once to you. And so, a lot of the design work and research that we’ve done has been around trying to probe the edges and map the landscape of not only what’s technically possible, but what from a human user point of view is desirable, at which moments.

You know, it’s a lot of fun, it’s very exciting, and sometimes I’m like, should we be doing this? You know, shouldn’t some large tech company with billions of dollars be doing this research? I hope they are, but, you know, we may very well be one of a few group of people who are doing this research because these questions couldn’t be asked even a few years ago. There was no hardware platform for which these questions even mattered. And so now that we do have the hardware foundation. To start answering these questions, and now we need to develop software for which having good answers to these questions, you know, is important, then now we’re doing the work and trying to map out that territory.

00:34:26 - Speaker 2: And I’m glad you are, but I still think it is a niche and a niche, right? The kind of interest in not just productivity software, but specifically thinking, idea oriented tools on this new platform.

I think the big companies are thinking about the hardware, the operating system, the much more kind of mainstream.

Can I exactly watch something or shop or do other kinds of things that are more common operations, and I think you mentioned this in the beginning. that you see it as something that is potentially very widely distributed in the same way that like note taking is widely distributed or email is widely distributed, but I think that’s quite a number of steps down the road.

So it sort of makes sense to me that maybe only smaller players are interested in this right at the moment.

And you mentioned the coming across Roman notion after you had started this company and already working in this space, so it’s quite interesting because you now mentioned two things. One is the VR to AR VR to, yeah, some kind of pass through, I can see part of my environment and how that changed your application. And then yeah, tools for thought appearing presumably, I don’t know, made you feel like more like you had a home or a community of people that were thinking about the same thing, even though obviously, as far as I know, you’re one of the few who’s thinking about this specific kind of environment and hardware platform, but in terms of like how do we use computers for thinking and ideas specifically, suddenly now there’s a thing happening there.

00:35:52 - Speaker 1: Absolutely, I was thrilled to discover the tools for thought community that it existed, mostly on Twitter, so, you know, you can tap into it from wherever, because, I mean, people who are really into, you know, their personal knowledge management into these tools, it’s never going to be a vast majority of the population or of the user base, but I think that these people are maybe very Impactful, you know, they might be working in fields like investment or in tech, or running product teams, where the decisions they make and the knowledge they have access to or can make sense of reverberates beyond just their personal life and work into, you know, organizations that they’re a part of, into the markets that they are selling to. And so there’s leverage there, you know, to make an impact and It’s also a larger, you know, market or a larger group of people than I would have thought before I came across the tools for thoughts ecosystem. It was certainly large enough to support at least a few pretty successful venture backed software companies, and there was a path, you know, you can see a path, for example, for notion, to go from more of an enthusiast user base to a larger, broader, maybe more enterprise focused markets. Once they got the primitives right, or once they sort of better understood who would be the power users and who would benefit from the power users’ work, but who didn’t, you know, themselves need to be sort of like crafting the notion uh wiki for eight hours a day themselves. So, I think that, yeah, me coming across that community and then also that community being very open and very excited for some of the demos that we’re showing with these sort of like force directed 3D force directed graphs of linked concepts. We got a really good response from that community as well, and that was a really important source of feedback, and an important source of just engagements to motivate us to keep going and also to provide really good signals and like, OK, which features might matter more, which use cases might matter more and which not. Of course, the thing that’s happened since Tools for Thought summer was AI and specifically large language models. AI has upended everything about everything, but it’s, you know, definitely upended our working assumptions about what knowledge work was, what the tools would be, what the roles would be, what the objectives of knowledge work would be, and I think everyone building. Software in this space, you know, we all have to have our own theory of change around what impact AI is gonna have and how our projects will stay relevant in a drastically transformed future. One of those changes is that, so maybe tools for thought will become unnecessary in the future because we won’t be thinking for ourselves anymore, right? We’ll just have this sort of all knowing AI oracle that will be able to pull out the right answer, the best answer, you know, at the moment that we need it’s, and the answer will be fed to us through our super thin Apple Vision Pro 10, you know, glasses. That’s one version of the future. Another might be that humans do stay in the loop because, you know, there are still experiences and values and judgments that we make that you can never by definition replace with an automated system, and that there is still value in having better tools for thinking, for having better processes for making sense of new information that’s coming in. And that AI can lower the barriers to using those tools because, you know, maintaining a sort of up to-date Rome notebook is, you know, at least a halftime job, and not many people have the bandwidth to be doing that, but maybe if some of those friction points and some of those barriers could be lowered, then we could have tools that you could on their own be Making a lot of the connections that previously had to be done manually, but still, you were the one sort of gardening this knowledge garden. You were the one shaping it and deciding what’s important, what’s not important, and drawing from it, you were the one harvesting its fruits and using them in your day to day life or work.

00:40:23 - Speaker 2: For sure, a lot of, yeah, productivity systems, note taking systems, settle cast and GTD, etc.

They do attract folks who maybe get just satisfaction from the investing in those systems, the transcribing of the notes, the capturing of them, the gardening of them, the finding the connections between them, and many people certainly get huge value from that, me included, and I think that long predates the current tools for thought summer, as you said, you know, I think of something like the Steven Johnson wrote, very prolific author. wrote some time back about using Devonthink, which is super old school app that you know you type in a bunch of notes and it has like a little very rudimentary algorithm for finding connections between them and how that helps him have new ideas and get value from that.

But yeah, he is someone who is willing to take that time and invest in a system, and I feel like the vast majority of people just find that way too tedious, but maybe there’s some element of These advancements in large language models can help us with the tedious parts where you can still get the benefit of the end result.

While you’re not just fully outsourcing the decision making or the sense making or the judgment calls or the aesthetic calls to the computer, you’re getting it to fill in some of the more tedious parts that not everyone has patience for, but in the end, you’re still the one that, you know, is making the calls.

00:41:50 - Speaker 1: So, there are so many interesting threads in this conversation that we’ve had so far, and I think there are also many interesting ways in which these threads unexpectedly overlap and connect back to each other.

So earlier you had talked about some of the earliest use cases for VR that you had experienced as a party trick for gaming, you know. Actually one of my favorite is fitness. I personally do not use VR for fitness, but I’m very impressed by the apps and by the stories of people who have found a way to achieve previously very, you know, difficult goals, fitness goals through virtual reality and through some of these fitness apps like Supernatural. And I really like this model for how spatial computing can fit into Our lives and work, or actually any technology for that matter, can fit into our lives and work, that it’s this really time boxed and place boxed use case, you know when you begin and you know when you end, but then, even when you’re not using this app, you are enjoying the benefits of having that practice of having that in your life, you know, in this particular case you’re feeling physically healthier. And, you know, you’re able to hit these goals that you had, but maybe had difficulty achieving in other ways, like going to the gym or going for a run, and that’s very much a model I would like to adopt for our own product, whatever we build, you know, the idea that we make something that makes you, let’s say, smarter, or makes you more creative, or makes you talk more. Coherently, you know, about ideas that are important to you, even when you’re not in the headset, even when you, you know, you step out and you’re just grabbing a coffee with a friend or you’re going for a hike, that somehow we find a way to tap into the parts of your brain that remember complex information that makes sense of it in a way that your laptop screen doesn’t, and that therefore makes you like a more interesting conversation partner even when nobody has any gadgets on them, right? I mean, they’re definitely sort of, it’s almost like an aesthetic preference of mine, that like, I would like the future we live in to still have room for unauugmented and unmediated, you know, human to human interactions. There’s another future where we just all have these like tiny AI like earpieces, and they’re telling us what to say and what to think all the time. Sure, but I prefer a world where our technology is helping us to achieve goals that we have. For ourselves, you know, whether it’s mental health or physical health, or creativity, or productivity, or just being an interesting conversation partner, but then can also get out of the way, right? They do the work and then we step away like a little bit closer to the ideal versions of ourselves, but we’re not dependent on a continuous subscription to like, you know, the software product to stay that way. So that’s tied back to VR Fitness. Another interesting tie in here is that there has been some research recently that suggests our brains use or creatively misuse spatial navigation, neural circuitry to keep track of concepts and memories. And this I found fascinating because, you know, I’d always kind of thought of this. The idea of like conceptual space as a helpful metaphor, as a useful sort of metaphor because we can’t like, otherwise visualize, you know, what it means for this idea to be close to this one but far from that one. But it seems like there is some evidence that this is actually what’s happening, you know, in our brains, and If that is the case, so a lot of this research actually came out of interpretability research in AI like computer scientists trying to understand what’s going on inside a large language model, what is a latent space, you know, like, what makes one word closer to another word in this like, super high dimensional space. And then realizing that there are actually some mappings back to how human brains work and how human language works and how human beings express ideas through language, etc. So I’m not a neuroscientist or computer scientist, so this could all well be just my sort of fanciful misinterpretation of all this. But, you know, if indeed there is some concrete underlying mechanism that ties space and ideas together, then I would say that’s an even stronger argument to Investigates what a spatial user interface displays for working with information could be and how that could help us to come up with designs that are better synthesize the underlying sort of requirements of the user, or come up with theories that better synthesize the different pieces of evidence that we’re trying to fit together. Etc. So, it could be that this is not only a metaphorical connection between, you know, a semantic space and like mapping out ideas on the big wall and the actual ideas themselves, there could literally be a real phenomenon going on here. There are papers that point to evidence that this is what’s going on.

00:47:08 - Speaker 2: And you’ve got a couple of links here you’ve shared with us that outline some of these explorations and discoveries, so I’ll put those in the show notes and listeners can follow through and read those to make their own judgment.

Yeah, well, so far I like that we haven’t talked too much about the technology and really focused on the user and the big ideas here and your unique take on this.

But with that said, now let’s talk about the hardware and the technology and you know, I was interested to go read about the history of it. I found an interesting link I’ll put in the show notes, but going back to even the 60. and 70s people strapping these ridiculous contraptions to their head and trying to figure out head tracking and all this kind of stuff. I feel like there was some kind of maybe awareness of OK, the hardware with the miniaturization has happened with mobile computing and internet and all this sort of thing that lots of big companies and lots of investment dollars went into Many platforms, most of which have not panned out, but nevertheless have produced some very impressive things.

We already talked about that early Oculus demo or kind of dev kit that we both had access to. One that to me was a really, I don’t know, wow moment was the Google Glass concept video from, yeah, I think it was around that same time, 2012, 2013, something like that. And yeah, I remember people that I knew, not even in the technology world saw that and just were floored and just said, you know, this is amazing, this is something I want to have. Now, of course, the reality didn’t live up to what was in this concept. Video, Microsoft’s got the HoloLens. Magic Leap is one that, yeah, it was the secretive project and billions of dollars of investment were going into it. I think they did develop some genuinely impressive hardware, but in the end, yeah, too early, couldn’t get there, couldn’t get the two-sided market of developers and and users, too expensive, too weird, that sort of thing. And then obviously you’re choosing to build on Oculus, which is now owned by Meta and has been through many iterations here. So what’s your take on the kind of currently available hardware? What made you choose this platform that you’re on now and how do you see the good enough is a weird thing. To talk about because there’s so many different aspects head tracking and input mechanisms and that sort of thing, but I think it also depends a lot on the application. It’s clearly been good enough for certain kinds of