Untitled

Discuss this episode in the Muse community

Follow @MuseAppHQ on Twitter

---

Show notes

00:00:00 - Speaker 1: The iPad is the perfect device of being able to immerse yourself and just being able to explore versus the Mac is all about getting things done and about speed and efficiency. If we embrace that, we naturally end up with apps that are quite different.

00:00:24 - Speaker 2: Hello and welcome to Meta Muse. Muse is a tool for thought on iPad and Mac, but this podcast isn’t about Muse the product. It’s about the company and the small team behind it. I’m Adam Wiggins here today with my colleagues Julia Rogats.

00:00:38 - Speaker 3: Hi, Adam, nice to be back.

00:00:40 - Speaker 2: And Leonard Sursky. Hi. And Yuli, I know you often spend winters traveling in sunnier places, and you recently returned from Panama. How was the experience of working remotely and during sort of travel holiday activities this time around?

00:00:56 - Speaker 3: Yeah, it was fantastic as always. And in this case, on my winter trip, I actually got to indulge even more in the traveling part since I switched, I think about half a year ago to working only 4 days a week. So I had 3 days off at a time. Sometimes I took an extra day here and there, so I had a lot of time to travel around.

Discover the country and then yeah, spend a few days a week working on news actually often interweaving this with long distance travel.

I’m often stuck in, you know, 6 to 8 hour cross country bus rides and those actually ended up being perfect opportunities to just have a deep focus day and kill a lot of time doing that.

00:01:39 - Speaker 2: We could usually tell when you reconnected to the internet because a whole bunch of commit messages and like pull request, things would kind of stream into the Slack channel simultaneously.

00:01:51 - Speaker 3: Yeah, that’s right. I do have the advantage that a lot of the stuff that I’ve been working on actually I can work on offline, so we were kind of in a phase where We didn’t have to do a lot of design decision discussions. It was more fixing bugs, implementing little features, so I would just make sure that I had at least, you know, 10 of those small things queued up for the trip and then would just work through those while offline, then connect back online and yeah, have a nice new update for everyone that was waiting for it.

00:02:20 - Speaker 1: Yeah, and I’m quite impressed by how you’re able to do that. Julia, both combining sort of vacation and work. Like whenever I’ve tried to do that, I both got nothing done and had a bad time, basically. And so I think it’s especially impressive with the launch we are working on.

00:02:36 - Speaker 2: Well, it certainly seems like a skill you’ve developed Julia, which is the ability to be really focused when you need it and then switch out of that and go fully into OK.

Now in this interesting place, I want to explore have adventures be fully present in my environment for a day. A few days, whatever it is, and when you have that work block, whether it’s on the bus or just days you set aside sitting in the hotel or whatever. So I think a lot of that is, at least it seems to me like a skill you’ve built up over quite a lot of years because this is just the lifestyle that you want to lead and so you’ve spent the time to create your mental discipline around that.

00:03:14 - Speaker 3: Yeah, it surely does require some discipline, I would say, but it’s, yeah, it’s something that I’ve cultivated over the years and being able to. Shut out work when it’s not time to work and really enjoy life is something that’s really important to me and then that’s actually where I draw the energy to then be going back to the computer and be productive.

00:03:35 - Speaker 2: So our topic today is the design and implementation of our Mac app, that’s Muse for Mac, and those following our story now we’ve had the Muse 2.0 product has been in beta for a little while. We’re coming up on a launch, and one of the key features, probably the most notable feature for users and customers, is the Mac app.

And I thought it would be really great to get both of you on here to talk about this while it’s still fresh in your minds, because I think this really is, while other folks on the team maybe have been deeper on things like the sync engine, for example, you two have been really the mind melded dynamic. The duo that is making the Mac app come to life and I’m more of a user, an avid user of it, but I also get to follow along all your design discussions in the Slack channel. I find it just really fascinating and I hope we can dive into a bunch of those things today.

Yeah. And maybe we could start with kind of an overall design approach. So obviously Muse 1. X was an iPad only app and we have a lot of unique concepts in there, this open canvas, the nested boards, the different types of media cards and Then when we’re thinking, OK, we want to add this additional platform because we know the desktop is such an important place for doing work, but we don’t necessarily just want to do what Mark would call the transliteration just porting it straight across without a lot of thought because the desktop is not only a different. Set of hardware, but it’s actually I think you’re in a different almost mindset when you’re sitting at your desk at your keyboard in this focused posture. You probably have a different almost approach to your work than you do when you’re leaning back, for example, on a sofa or reading chair with your iPad and pencil. So I’d be curious to start off with kind of what’s the overall design approach? How do we think about bringing new to this new platform?

00:05:27 - Speaker 1: Yeah, I think it’s all about the balance.

So on one hand, we have this iPad app already. Um, we have news on the iPad, and we have a lot of users for it and we sort of have established a certain design there. We have made a lot of design decisions and now we are bringing that to a completely new platform that, you know, in some ways is connected to the Mac, but it still has its own set of sort of rules and conventions and also its own set of users that have different expectations than iPad users. And so we are trying to balance building really this native Mac app from the ground up, like what, what would it look like if Muse on iPad didn’t exist and now we are building Muse just for the Mac. With sort of the existing iPad app and trying to make it into one coherent model.

00:06:15 - Speaker 2: And I think that’s a really unique piece of our approach. We’ve talked a bit before, perhaps maybe on the native apps podcast episode, where we said that many and most apps kind of tend to have a home in one place like a platform like they originally.

Instagram was a phone app and then maybe there’s a web version, but it’s kind of a companion or an add-on or just an additional thing or similarly, maybe have a lot of Sass tools, notion might be a good example.

It’s really native to the web, it feels most natural and It’s the baseline to be in a web browser on a desktop computer, and so when they make an iOS app, it feels like a little bit of a bolt on, it maybe doesn’t follow a lot of the platform conventions you expect. It’s often lagging behind, you know, the features that are in kind of the main platform, and I think the The idea of something where we want to, like you said, bring it to this new platform, design it as if it was a new app there while also sharing a lot of primitives and concepts and of course actually the data because it syncs between them. So actually it has to share all that exact data set. I think that’s an unusual thing.

00:07:23 - Speaker 3: And I think also kind of sharing some of the core values of news. So one of the things that we had in mind designing the iPad app is that we wanted it to feel really fast and fluid.

And one of the things we did to achieve that is to have this quite sophisticated gesture system where you can use all of your fingers, you can move cards around, you flick them off the screen to delete them, and it is a bit of a learning curve there, but if you really do learn this design language, then you are able to work with a tool really quickly and efficiently.

And obviously we couldn’t just bring that 1 to 1 to the Mac because you can’t use all your 10 fingers on the Mac, at least not currently. But we still want users to be able to work quickly and efficiently with the app. So thinking about how to bring the app to a new platform, keeping this value of making it feel very fast and fluid, but using other tools such as really good cursor support, keyboard shortcuts. I’m sure we’re gonna talk about that in more detail, but yeah, those were some of the thoughts for sure.

00:08:25 - Speaker 1: And in a way, it’s taking something that’s traditionally seen as a weakness of native apps, like you have to build a new app for every platform, you know, that’s terrible. Let’s do a web app and you have one app, it runs on everything. But I think we’re sort of trying to take that and turn it into a strength by saying, OK, we have a chance to build an app for each platform and, you know, it can be different and we can leverage the sort of system strengths for each. And I think if we get it right like that can be a really big advantage for native apps and sort of what it needs to. You can compete with web apps.

00:08:58 - Speaker 2: Yeah, that is a lot of the appeal. It obviously is not a user benefit other than I suppose, being more universal or being available on more platforms, but it is a benefit to the company to have less code to maintain or just a smaller engineering team or less work to do to keep everything in sync in the sense of features, you know, if you have a native Android app and a native iOS app versus if they share a code base with I don’t know, React Native or something like that.

So notably here, you know, when we come to the technical side, we are indeed sharing a code base between the two with only a 5 person team and only 3 of those are actually engineers, you know, that’d be pretty tough to do 2 full complete apps with the team that size, but we are getting leverage from the shared code base, right?

00:09:42 - Speaker 3: Yeah, we’re definitely there, and I would say I’m even quite surprised by how much code we were able to use or reuse across both platforms.

You know, the very first time we flipped that switch, so maybe for some context, the app is built in Mac Catalyst, which is a framework that Apple released a few years ago.

That lets iOS apps be ported to the Mac and basically just clicking a checkbox, making it run, and then see what it’s like, and news was actually quite usable from the very first build, but then I guess it’s the classic 80/20 rule of, you know, most of it works really well, but to get to that really polished state that I hope people feel the Mac gap is in.

You’ll actually spend a long, long time in our case, several months polishing it and improving it, but most of the basic logic in the app is just the same between both apps and notably also the sync layer that we were working on in parallel that lets users use their data on both devices.

That’s all just shared across all platforms, and that means a lot less code to maintain, fewer bugs to fix.

If we want to introduce a new feature, we can do it on both platforms simultaneously.

Yeah, less testing to do. So it’s really a great advantage. And there’s, of course, a few downsides like you end up sprinkling your coat quite a bit with if I’m on the Mac, have this component look like this. If I’m on the iPad, should look a little bit differently, but it’s really quite manageable, at least at the moment.

00:11:16 - Speaker 1: And maybe Muse is even in a bit of a unique position there and that we have this what we call the Muse canvas, which is basically on the iPad, it’s most of the app, right? We don’t even have any visible UI Chrome by default, but you have the canvas where you place all your content and we don’t really want to mess with that at all.

Like we don’t want to move your content around or change how it looks on different platforms. So I think that was one of the first things we were able to more or less set in stone for bringing news to the Mac that, OK, the canvas is going to look the same. We just have to make sure it works and adapts to the system conventions and like that’s already 90% of the app logic, right? And so then we have all this UI Chrome around it where on the Mac, I think we actually have a bit more since, you know, you have the menu bar and you have sort of a bit more of an established system UI that every app needs to provide.

00:12:09 - Speaker 3: But at least as you said, these interface elements, the menu bar are actually perceived by the user of belonging more to the operating system around it, so not necessarily Chrome off the app itself. And on the Mac we were even able to remove some of the Chrome that we have on the iOS app where on the iPad, when you tap on the canvas and the action bar pops up that lets you do all kinds of actions. We just moved all of that into the menu bar or into keyboard shortcuts. So in a way, the Mac app is even more focused on your content now and doesn’t have any buttons that get in the way.

00:12:46 - Speaker 1: And for me, that was actually a really nice positive difference from designing for the iPad. That a lot of the time when on the iPad, we would have to build our own interaction or our own interface element for the user to see. On the Mac, there’s already an established standard for that. There’s maybe even already like a view that Apple built that you can plug into. And yeah, there were just a lot of times where basically we could have something on the Mac without having to rebuild it from the iPad just because Apple already provides that.

00:13:18 - Speaker 3: Yeah, definitely rediscovered my love for right click contexts menus. Yeah.

00:13:23 - Speaker 2: Same, yeah, I do a lot with the right click stuff on the Mac. Yeah, it’s interesting because you know we have such a, I don’t know if you call it quite an internal culture, but just a basically a pattern of needing to always reinvent everything in some way as we’re building for the iPad because there is so much less precedent for productivity software there.

And it was almost kind of relaxing to realize that, yeah, the Mac has such a rich and long history of great productivity software, and there’s obviously specific documentation like the Apple Hig, but there’s also just a lot of great apps that you can look at and basically say, oh, folks have already figured this out and it works great, we don’t need to be that original, we can just do. What others do, you know, adapt it to our situation and try to find the best possibility that fits with our, again with the open canvas and the cards and all that sort of thing, but we can draw from that rather than needing to always be inventing everything kind of from scratch.

To that point, I’d be curious to hear the apps that we kind of took for inspiration or reference. I know I saw very often, you know, referencing, for example, the behavior finder as a bit of a kind of benchmark for here’s the baseline of what all Mac users are going to expect in terms of how basic interactions work. Yeah, what are some examples that we drew from.

00:14:53 - Speaker 1: Yeah, there were a lot of them. And I think part of it is that Muse cannot really be put into one category where we can just look at the other apps in that category and see how they do it. So yeah, we looked at the finder because Muse has a lot of sort of file browser style parts to it as well. We also looked at Sketch or Figma, sort of some of classic design apps, which have really set many new UI conventions over the last few years, I think.

00:15:20 - Speaker 2: Well, and importantly, they also pioneered the infinite canvas stuff. I guess even going back to like Adobe Illustrator, for example, I mean, certainly some of my early inspiration for wanting to make a sort of open canvas ideation tool came from watching how creative people often use Illustrator, where they have this just big open space and you can zoom out and you have a lot of iterations of one idea up in this corner and some iterations over here. So there is, even though we’re not a design tool or an illustration tool. Some of the precedents has been set on the open canvas we do borrow quite a bit from.

00:15:55 - Speaker 3: One other I’ll mention, even though that’s maybe just a different flavor of the finder, is actually the Mac OS desktop. I think that’s one of the areas of the operating system that actually comes closest to the experience of use the canvas.

It’s just kind of an open space where you have items, I think depending on how you have it configured, you can arrange them freely, drag them around.

And I was looking to that specifically when implementing multi-cart selection behavior, this is something that is also new in Muse2 and in use on the Mac specifically. I was surprised to see how we actually have quite an intuitive understanding on how selections should behave based on how our operating systems behave. So whenever we implemented something in the multi-cut selection space that didn’t feel quite right, it was usually because it was different from how the Mac OS desktop does it. So that was definitely a good inspiration on figuring out what kind of behavior users expect and what just feels right.

00:16:59 - Speaker 1: Yeah, and it’s really fascinating how especially sort of this really core part of the desktop experience or even MacOS, you know, it’s been around for decades and it really hasn’t changed and at least for me like I basically grew up with it, right? I don’t know any other way that it could work. And so it’s very ingrained in how I think these interactions should work. And that’s quite different from the iPad where I kind of know a time before the iPad and I also know how Apple started and how it got there very iteratively over just the last few years and like it’s still changing things, it’s moving things around every year.

Even app developers haven’t really agreed on a single way to do it and probably even Apple itself hasn’t really agreed on one way to do certain things on the iPad. And so in that way, I think it’s really helpful to just look at what Apple is doing on Mac OS as sort of the gold standard, even though they are, I think, especially with sort of the catalyst apps that Apple has been doing over the last few years, like there are also a few outliers of apps that just aren’t as well designed or aren’t as well fitted to the Mac OS system. And so since Muse is a catalyst app, you also have to like specifically look at Catalyst apps. I think where to me craft stands out as one example of just a really well designed catalyst app, and I think we looked to them quite a lot for just seeing what’s possible on Catalyst and like what’s a reasonable solution in between the Mac and the iPad.

00:18:30 - Speaker 2: Yeah, definitely. I think we have to give a big shout out slash some appreciation to Kraft.

They’re very good sort of comparable to us because they did start on iPad and then came to Mac, and they have sync between those.

They were an early mover on Catalyst, and I think probably, you know, we benefited from starting, you know, a year or two later than they did, and I think they faced a lot of the bugs as that technology was still pretty early and they wrote a nice long guide that I think we referenced quite a bit.

And on top of that, their team even was nice enough to answer a few questions we had, which I think usually came in the form of we go to look at how the preferences panel is implemented or the toolbar or something, and we go, oh wait, Catalyst doesn’t seem to support that API that normally would have elsewhere and then basically we’re able to ask their team and they said, oh yeah, you know, here’s how we did it.

And that was just really, really nice to have uh referencing back to our career episode, maybe where the advice is always talk to your elders, people who have, you know, forged the path before. And so, yeah, very appreciative to their help along the way.

00:19:35 - Speaker 3: Yeah, definitely, I’ll second that and yeah, for sure, Kraft has been a big inspiration for us and kind of the benchmark on the kind of quality that you can actually deliver building a catalyst app.

And on many occasions they’ve helped me with some quote level support and even just to get confirmation on an issue that I got stuck with to figure out is this really not possible? Is this potentially a bug in the system.

There’s just unfortunately quite little documentation out there on Catalyst stuff and not so many people are using it. So if you do run into some weird unexpected behavior, it’s often kind of hard to find materials to solve that and Just talking to the craft team about that and hearing that, yeah, we got stuck on the exact same thing. It seems to not be possible, what was just a good check for me.

00:20:28 - Speaker 2: So we mentioned using Finder and also the, I guess Finder-ish thing that is your Mac desktop is kind of one of our main inspirations or references for selections, and you, you mentioned the multiselect is new in use too, which was sort of an interesting fallout. We weren’t, I think, necessarily thinking the 2.0 product is going to include more powerful selections, but it’s more that once you’re on Mac. Where you just expect selections to work a particular way, and yeah, you expect them to be more powerful.

That’s just the nature of the desktop platform and so that pretty naturally led to implementing much more there, and then a lot of that does indeed benefit the iPad.

So maybe the starting place there is how do selections actually work in Muse.

00:21:16 - Speaker 1: Yeah, so, so far on the iPad, it really just work with the pencil, right? So we rely on the pencil on the iPad a lot and we have the selection tool for the pencil and you can use it basically to draw with a lesser selection around your content and then you can move that.

But we don’t really visually indicate that much what exactly is selected and you don’t really have a way to really One by one select cards. And so on the Mac, the selection system kind of requires that.

So if you select something in the finder or even on the Mac desktop, you know, you can very quickly select something just by like opening a rectangle with your mouse and everything within that gets highlighted and selected. And even from that stage, you could like hold down command and deselect things again like called shift and deselect a lot of things, move those around and Yeah, it’s like a very. Well thought out and established system that kind of really works exactly this way across everything on the Mac. Ideally, And yes, I think we really needed to follow that from use as well in order for it to feel native in any way to the Mac.

00:22:24 - Speaker 3: Yeah, and then on the Mac, even taking it a step further and allowing you to just single click a single card and select it this way so you don’t even need to necessarily draw a selection with your cursor around it, but the default behavior that happens when you click on a card is that it gets selected and if you click again, it gets deselected. We actually did have to do. A few compromises here because for some card types, you just expect, for example, if it’s a text card, you expect that clicking into it will make you able to edit the text, and that is in fact the case. So there were a few places where we just had to make the decision to sacrifice consistence for user expectation, but I think we landed in a pretty good place there.

00:23:15 - Speaker 1: And I didn’t even realize that consciously before, but on the Mac, basically anything you click on is first selected, right? Like if you click on an email, it gets selected. If you click on something on your desktop, it gets selected.

And so while on the iPad selection is really about bulk editing, like you can press the edit button on the iPad and then you can select multiple things if you want to edit multiple things. On the Mac, it’s really just used for everything, even if you just want to interact with a single item, like if you want to delete something, you first select it and then press the delete key or you press enter to rename it or something like that. And it even gets automatically selected if you move an item from one place to another. By doing so, you also selected. And so it’s really such a core interaction on the Mac while being this like special case on the iPad. And I think that was just a really big shift for us as well even in how we think about it.

00:24:13 - Speaker 2: Seeing how the selections came together also to me really emphasized this difference in Yeah, use case and setting for each platform.

It’s that, you know, with the iPad, you’re doing relaxed thinking, you’re reading, maybe annotating brainstorming with Mac, you’re doing productivity, heavy research and in particularly this kind of idea of bulk editing, which came up a lot, I feel, and you see that with the selections, which is that you can, for example, just the simple thing of Hitting command A to select everything so I can move it around a little bit, or yeah, selecting a whole screen or half a screen full of stuff and then you can fine tune the selection with command, click, and then you know cut that and move to another board or drag it to another window or something like that. You can do all of that stuff on iPad, but it’s just you have the smaller screen. You don’t have the same kind of speed and precision. You don’t have the keyboard shortcuts. So while it’s more intimate and natural and organic to touch things and use your pencil and stuff, it’s just not fast to do those big bulk edits. I guess intuitively I knew before, OK, the Mac would be better for that sort of thing, but I wouldn’t have really been able to say why, and now I feel like the selections and powerful selections, it really is the core of that.

00:25:29 - Speaker 1: Yeah, for sure, and it’s in combination with keyboard shortcuts as well, right? Like you select something and then you want to do something to it quickly and with the Mac, you have sort of the guarantee that every user has a way to quickly select something and has a keyboard to quickly manipulate those objects. And I feel like that is sort of the matric combination that we have and that makes the Mac so powerful and the iPad is kind of missing both of those sadly.

And so just because of that, it kind of becomes a very different platform that can’t have speed and efficiency sort of as the target, but sort of has to have strength and the direct manipulation and sort of the immersiveness you get through touch input.

00:26:11 - Speaker 2: One area I think is important or I’ve observed.

In my own just use of uses and others as well as this concept of culling, what I usually call culling, which is you probably dump a bunch of source material that you think is going to be inspiration or reference or as a starting place for your thinking process into a board, but often in the process of kind of sorting through it and trying to find the patterns and figure out what you want to do with it, there’s this element of, oh, you know, this thing doesn’t fit in. I don’t need it or I don’t want it or it’s a duplicate of something else. And that’s why I think that gesture we have on iPad, which is to throw something off the edge of the screen. So 3 of the 4 edges of the screen, if you swipe something in that direction, it just disappears. I think that’s a really important gesture to be so easy to do and top level because of that quick culling.

You can just dump a bunch of junk in because it’s OK. You can throw out the stuff you don’t want. But it’s very good for the kind of one-offs, so I’m going to do this one, this one, this one, and that’s pretty quick. Whereas, yeah, on desktop, we’re just used to, you select a couple of things and you hit the delete key, and that’s a very quick way to do it. And of course there the throwing something off the edge of the window, I don’t think it would even really make sense. That’s just not an interaction that is at all pleasing to try to do with a mouse, nor does that really fit with the paradigm at all. Whereas, yes, there is something that’s really well established, selected and hit delete, and that’s very fast in its own way.

00:27:37 - Speaker 1: Yeah, there were some tough decisions there, I would say of, OK, which elements of the iPad app can we just take over and apply on the Mac as well, or which things do we actually have to cut or find a different way to do it on the Mac since we don’t want to alienate our users or like take things away from them, basically. But we also have to be very careful not to just take what’s on the iPad out of convenience basically and just say, OK, it’s gonna work the same way on the Mac, it’s gonna be fine. And yeah, so weighing these was always, I think a difficult choice even for like tiny details.

00:28:10 - Speaker 2: Another thing that points to the importance of selections is kind of where they sit in our gesture space.

I don’t know if that’s the right terminology when you talk about Mac, but that brings me to another section that I’d love to hear you both talk a little bit about, which is, I know you put a whole bunch of time into questions like, How do you navigate into a board? Is it a single click? Is it a double click, or what happens when you click on open board space? Does that kind of turn into a little grabby hand and let you pan the board, which actually is what it does on iPad. If you just put your finger down and open board space and then move your finger, you start to pan.

But I think here we ended up with when you put down your cursor and click in open board space and then drag, you start to get a rectangle selection.

So we’re saying the selection is so important we want. That single click kind of default to actually be that. So I’d love to hear about the process of kind of coming up with the holistic idea for what the Mac gesture space is.

00:29:08 - Speaker 1: Yeah, that was really something that especially in the early parts, we really had to just try out a lot of different things, like basically for every possible basic interaction, like whether it’s double click or single click to open something, to edit something, to make a selection, you know, we basically had a list of choices to pick and combine those. And yeah, in the end, it’s a lot of sort of trying and giving it to a few users and seeing how they react. And yeah, then you just have to Sort of consider all the different parts of, OK, so you wanna fit into sort of the Macro as well, but you also wanna fit into what you’ve established on the iPad. You wanna make sure it just feels good to use and feels in line with how we want you to feel basically. And then I think we actually also have to think a bit about, OK, what other things do we actually want to build on the Mac in the future? Like, what do we want the Mac app to become at some point because we don’t want to box ourselves in by setting certain interactions know that in the future, you know, means we can’t do something else.

00:30:11 - Speaker 3: Yeah, and I think in many cases it’s actually trying out these different versions, even though we were unsure in the beginning what the right answer is. Once you try out one versus the other, you just immediately know because one feels wrong and the other one feels right.

So I think the example of zooming into a board with a single click felt wrong. It felt borrowed from the iPad where you tap to zoom into something is just a very well established pattern. Whereas on the Mac, opening a document is almost always a double click. So putting these two side by side, it was just immediately clear that one was much more well suited for the Mac.

And yeah, when it comes to scrolling the canvas. The good thing is that you actually, if you use a MacBook that has the trackpad touch input. That actually works with the two finger scroll just like you used to from almost any other scroll view, your browser or whatever.

00:31:11 - Speaker 2: As well as the pinch gestures, so you can pinch the zoom in on a board, pinch the zoom out. Yeah, you have the two finger pan, so I was surprised how natural that stuff was, but then you also can’t assume they have a trackpad. They might have a mouse or a trackball or something like that, so we couldn’t rely on those existing, but it’s nice to have those gestures as a bonus, right?

00:31:32 - Speaker 3: Yeah, exactly.

00:31:35 - Speaker 2: So something was notable to me about the creative process here was that we started with a debug menu that had a long list of everything you could do that was kind of in the command gesture space, and then you would have options for each one. You could play a video with single click or double click or maybe some other thing. You could zoom to a board with this, this, or this.

And then you go through a very granularly set your whole command space, but of course it’s very easy to get things that collided if you said, I’m gonna single click to select, but also single click the zoom or something like that, maybe weird stuff would happen.

And then eventually based on those experiments, I think Leonard, you boiled it down to like 2 or 3 kind of groups that naturally fit together, you know, maybe one. It was more double click oriented around actions, and that was more single click oriented, and then you could ask folks on the team or users or whatever to try out these different setups and say what feels right to you. And I remember testing those out a little bit, and the real test of it to me is once you get it and you’re really trying to do something, do you get lost in the flow? Does it just start to come naturally and you stop thinking about it versus I have to stop and think, what do I need to click or press or hold or whatever to get what I want to happen to happen here.

00:32:51 - Speaker 1: Yeah, right, that’s really the key. It’s kind of about how everything feels when you use it together, right? You can’t really look at any one single thing on its own.

00:33:01 - Speaker 2: Now did you find that the existence of a cursor hovering over the canvas, was that sort of a major change in the sense of how the user experiences it, or was it more kind of minor?

00:33:15 - Speaker 3: To me personally, I regained my appreciation for correct cursor, how would you even call it? Correct cursor shapes. I think this is something that you don’t notice when it’s working as expected, but you definitely do notice when it’s not like if you’re trying to resize a card and you actually have the normal arrow pointer. It just feels like something’s broken.

You expect there to be the little reset arrows pointing in different direction. I think same with kind of hovering over a card that you can click like a link card. You just expect there to be the little pointy head that indicates this is a link that you can click. And yeah, it was actually a bit of work to get all of that right, but I do appreciate the cursor and like the subtle ways in which it cues the user as to what would happen if they interacted with this element.

00:34:07 - Speaker 1: Yeah, and I would say it’s actually quite underappreciated how much of another dimension the cursor adds. Like it’s not just sort of a translation of the touch input, but to me it’s also about the interaction that happens before the touch input like a cursor, you can kind of move around before you commit to an interaction. And so the cursor shape can change, so you can see what’s about to happen. You can even show additional information if you want to unhover. But you can also use that to let the user just make more informed and precise decisions, I would say.

00:34:42 - Speaker 2: Queuing what the user can do reminds me of one really interesting call it subtree in the interaction space that you both went down, which is this, I guess I would call it this sort of ghost card that you get when you’re adding content. I’m not sure if that’s what you both call it, but I’d love to hear about the decision to go that way.

00:35:02 - Speaker 1: Yeah, I’m not sure what we call it. I, I don’t know that it needs a name, but yeah, it’s basically while on the iPad, when you add something, it immediately appears in your inbox on the left side of the screen and you just drag it to where you want it to be. On the Mac, it works quite differently where you add something and then it adds this little preview to your cursor and it follows you around and you can kind of figure out where you want to place it first and then when you click, you place it on the board.

00:35:28 - Speaker 3: And it even has a little additional hidden feature, uh, which is that you can actually click down to place it and then move your mouse with the button down to resize it in the same gesture.

So this is, I think one of the little things where we felt like this is something that feels news on the iPad like, even though it’s obviously like a cursor base and very Mac specific gesture, but A little thoughtful touch and you know, what if you want to make this image really big, you would have to click and then go to the bottom right corner and resize it and we wanted to make it easier to do both in one gesture. So this is what you can do on the Mac.

00:36:10 - Speaker 1: Yeah, and this is something that really isn’t possible on the iPad, right? Like the iPad in that way has a very simple input system, whereas on the Mac with the cursor, you know, you can very quickly, especially in combination with the keyboard, add something, you find where you want to place it. Confirm that and then also set the size of the content. Whereas on the iPad, that would be multiple steps and take quite a bit longer. So in that way, I think it kind of maps to what we said earlier about the Mac that it’s just a lot more about speed and precision while the iPad is sort of about this direct connection to the content, which, yeah, it’s lacking when you use the cursor on the Mac to add something.

00:36:53 - Speaker 2: One thing I think they both have in common, whether something goes in the inbox or whether you drag out a new board from the left side on iPad with that kind of extra gesture, or whether you’re moving the ghost card for the new board or the image or whatever it is around to find where you want to place it.

In all cases, we really try to avoid putting content on the user’s boards in a kind of a random place. Basically everything on your boards is things you have decided where it goes. I don’t know how much that’s a, you know, an intentional core principle that you always try to adhere to or something that just naturally came out, but it’s an interesting because it feels to me like there is a shared common principle or set of values, maybe like you were referencing earlier, but how it is implemented per platform is extremely different because the input devices are so different.

00:37:42 - Speaker 1: Yeah, that is quite intentional and that’s a big reason for why it works that way on the Mac as well. Yeah, I think since new is the spatial canvas and it is really directly about your content, it is really important that the user places everything. And, you know, Muse is not a linear text editor or something where we can just add new things at the bottom and the user will find them there. But since it is a special place. We feel it’s important that the user. Always has to say about where exactly something is.

00:38:13 - Speaker 2: And we talked about how a lot of what is in our kind of in-app custom UI Chrome on iPad, on Mac, you have the benefit of stuff that sits kind of outside the canvas, and that includes the toolbar, but then there’s that. Very ever present top menu bar that every app has. You just got the little apple in the corner and then the name of the app, and then you’ve got file edit, and so forth from there.

Now, those all seem very, I guess standardized to me. They seem pretty similar across apps, but I don’t know if there’s really good established conventions there or they just seem similar because apps borrow from each other or I don’t know, what was our process for coming up with the new menus.

00:38:57 - Speaker 1: Yeah, I would say it’s a bit of all of that. Like there are actually a surprising amount of sort of guidelines and just Apple documents about how you should structure and order your menus.

So I also think many apps don’t really follow it that closely or kind of make their own decisions. So in that way, it’s, you try to look at everything other apps are doing. You try to look at what Apple is doing, and then you need to figure out what is the right call for use. And I think for us, especially the challenging part was that we also have this iPad app which doesn’t have any of those menus and we still need them to work similarly and so we can’t rely on only those menus.

But it did actually mean that the iPad also benefits a lot from what we did on the Mac. Like, for example, we introduced a lot of new sort of context menus, right click menus for the Mac version where you can click on a card, click on the background of your canvas, and you get really a lot of useful options which we didn’t have before on the iPad.

And the interesting part of me was also that there isn’t really like a single menu on the Mac, right? Like you have the menu bar, you have the context menus in different places, and then you have keyboard shortcuts that need to map to those menu entries. And so in that way, you’re kind of trying to build up this whole system of different menus that all need to make sense and sort of have the same structure and order for different content types, but also, you know, the different places, the menu appears. And I think the iPad version of news also benefits from that a bit, especially in regards to keyboard shortcuts. Like we always wanted to do better keyboard shortcuts on the iPad, but it’s never really been a focus for us on the, on the iPad. But now on the Mac, it really is table stakes to have menu entries and then have keyboard shortcuts set for all of them. And so we’re trying to bring the same thing to the iPad as well and really try to build up this universe of actions that work the same way across iPad and Mac.

00:40:54 - Speaker 2: Are the keyboard short guts between the two exactly the same, or are there places where they’re different?

00:41:00 - Speaker 1: Yeah, we’re mostly trying to keep them the same. I think the difference is more that there are some. iPad specific shortcuts and some Mac specific shortcuts, like for one, of course, you know, the iPad has like inking or something and maybe you want to do special shortcuts for that. But there are also a lot of system shortcuts that Mac OS just takes for itself, basically, and there are a lot more of those on the Mac than on the iPad. And so we have to be very careful on the Mac, not to touch on any of those, but we could use them on the iPad if we want to.

00:41:30 - Speaker 3: One other thing about menus that I realized and I think wasn’t quite consciously aware of before is that they actually also a great way to teach users what’s actually possible to do in an app.

I think on the iPad we spend a lot of time thinking about how to best teach all of these complex gestures to our users.

We had a few different approaches of onboarding, even popping videos into the inbox that will show two hands, you know, zooming into a card while also carrying another card with them and Since we don’t have any of that on the Mac, I think actually just that’s for me how I often learn how to work with an app is to just click through all of the menus and see what the options are there and learn about the capabilities and how to navigate the app. So it’s basically almost like a little on boarding intro for free.

00:42:22 - Speaker 2: And maybe like a sort of a table of contents of what you should be able to expect to do, and not just the verbs, but also the nouns in many cases.

So I think of something like an audio, you know, I use audio editor tools as part of podcast audio editing, among other things. And so you might go into something like audacity, and you go through the menus and you see there’s a whole section on adding and removing marks. OK, what’s a mark? Well, it turns out this is a way to put a little marker in the audio and possibly give it a label.

But you might, if you’re new to that, now you know that this is a noun within the world of the application and maybe what you should learn about or what you should read about in the documentation, or you just try adding one and see, you can kind of infer from, you know, the name and what happened, what it is. So yeah, there’s a lot of discoverability in those standard menus.

00:43:11 - Speaker 1: Yeah, and even Apple is actually really explicit about that being one of the jobs of the menu.

And so they really encourage you to build your menu in that way that users can, when they first start the app, go through the menu and basically get an overview of everything that’s possible.

And yeah, that doesn’t really exist in a system like that on the iPad, which I think is partly because, you know, the iPad is not as much a pro device, but Yeah, for people that want to make it a pro device, that makes it a lot more difficult to sort of do this kind of onboarding themselves and come up with another way of teaching users about everything that’s in the app.

00:43:50 - Speaker 2: I also give a quick shout out to the Mac help menu which has a default search box.

Muse has this as well, and so you basically just search menus for what you’re looking for and definitely for a lot of apps that I use, for example, sophisticated programming editors or video editors where they have so many options and the menus are often many layers deep, and I can’t necessarily remember the keyboard shortcut for an action I use infrequently. But I can go into the little help menu and type in something there. It’s almost like a little command line or, you know, spotlight quick search kind of thing.

In addition to being a quick way to execute a command, they can help you discover. So I’m thinking now, even just going to use for Mac right now and I type in, let’s say I’m looking for the duplicate option or I want to remember what the keyboard shortcut is or where that’s located in the menu. And if I type in DUP, I see duplicate, but I also see duplicate Inc, which maybe I didn’t realize before that there was an option just for that, but I discovered that by using the school search. Now another huge area from my perspective is drag and drop, and that’s also important in the iOS iPad OS world, but way more so in the world of, yeah, pro app workflows and big screens with you’ve got multiple windows on screen at a time and you’re moving data between them. And actually through this process, I think of watching the two of you work on this application, I’ve rediscovered some of my love for native apps, so I found myself using a lot more, especially maybe smaller, just utility apps, something like Transmit for uploading stuff to S3 or Optimage or Forecast is a little podcast kind of compression tool. But all of these, the drag and drop jus