Sound Shapes for PS Vita: Jonathan Mak Demystifies Music Creation

Last week I finally had a chance to go hands-on with Sound Shapes, the upcoming PS Vita platformer-cum-music-studio that left critics reeling in its E3 debut. Coached by co-designer and Everyday Shooter creator Jonathan Mak and co-designer Shaw-Han Liem, I discovered a deceptively simple platformer that gave me the tools to build a series of game levels – and a corresponding song – in seconds flat.

Sound Shapes’ level creation and music composition are inseparable. As you use the PS Vita’s front touchscreen to place objects – platforms, lasers, obstacles – that you’ll interact with during gameplay, you’re simultaneously laying down beats and musical cues that will evolve into a full song when you play. During the creation process, you’ll use PS Vita’s rear touch panel to resize platforms and objects to further tweak your level’s gameplay or musical style. Where you go from there is entirely up to you: the final version of Sound Shapes will enable you to upload and share your creation via PSN for others to enjoy.

Sound Shapes is a strange but eye-opening experience that’s completely unlike any music-creation game — or platformer — I’ve played. After I set down the PS Vita, I was bursting with questions for both Jonathan Mak and Shaw-Han Liem. Read our full conversation below and leave your questions in the comments!

Jonathan Mak, co-designer of Sound Shapes (left); Shaw-Han Liem, co-designer

PlayStation.Blog: Jonathan, we didn’t hear from you for years after Everyday Shooter came out on PSN. What were you up? How long were you working on the Sound Shapes concept?

Jonathan Mak, co-designer, Sound Shapes: Shortly after Everyday Shooter, I met Shaw-Han Liem. I went to one of his shows, and I ended up showing him Everyday Shooter. We were sort of on the same wavelength — not necessarily video games and music, but…

Shaw-Han Liem, co-designer: …interactive music?

Jonathan Mak: That sort of thing. So we ended up collaborating on some stuff. We worked on visuals for shows, and later worked on a few gaming concepts. We spent two years prototyping concepts — I think we prototyped enough stuff to make another Everyday Shooter. But we kept upping the ante. While we had a bunch of cool ideas, we happened upon this prototype [gestures to Sound Shapes sitting unassumingly on the nearby table] and decided to pursue it. We decided to contact PlayStation, they showed us the early development hardware for PS Vita, and that’s how we got started.

PSB: Did you come up with the side-scrolling gameplay first, or the idea of creating a level that played music? What was the genesis of that concept?

Jonathan Mak: I think we came up with it partially out of frustration. We kept making prototypes for ways for people to actually make music, as opposed to games like Everyday Shooter where you can play musical sounds. We wanted players to actually make their own songs, which you can do in Sound Shapes.

I don’t know, I think it all came together at the same time. We chose platforming because it’s a well-known form of gameplay, where a shooter is kind of abstract — “I’m dodging bullets in space, weird!” With a platformer, you know you’re in this world and it makes more sense immediately. It seemed like a natural fit to introduce gamers to this idea of making music.

PSB: How did you work into the process, Shaw?

Shaw-Han Liem: We started collaborating on a pretty casual basis, just trying out different ideas. Some of them weren’t game-related at all, like “what can we project behind a rock show that will look really cool?” We wanted to find ways to use technology as a way to interact with and experience music.

Jonathan Mak: The interesting thing about this collaboration is that it’s not a case of me being the programmer and Shaw being just the music guy. We’re both capable of writing code and creating music….You can’t think of the gameplay first and then cram the music in; it doesn’t work that way.

Shaw-Han Liem: In order to make a game like this, we’ve realized that you need a musician who can make a game and a game person who can write music. Having gone through this process, I think it’s been a key thing. We both have access to the same toolsets and we can work off of each other.

PSB: Did you always plan for Sound Shapes to be a portable game?

Jonathan Mak: Because Everyday Shooter came out on the PS3, so we thought we might have a lead there, though we weren’t really thinking about a platform at the time. The PS Vita was a happy coincidence, because the touchscreen makes it a lot easier to compose and edit in the game.

PSB: How would you define Sound Shapes, if you had to? A music creator, a platformer…?

Jonthan Mak: It’s weird, because I had to do that a few weeks ago…and I couldn’t do it. [laughs] Um…it’s equal parts musical instrument and equal parts videogame. Sometimes I pick it up and I pretend it’s just a musical instrument — that there’s no game. It’s cool to sit there and make music, even using it in ways that it’s not intended.

Shaw-Han Liem: It’s sort of two things. If you’re a musician, it is a cool musical composition tool that also allows you to experiment with making these game worlds. If you’re a game player, it’s a platformer that also has this musical element and introduces you to the idea of writing a song. So depending on who you are, you might look at it in slightly different ways. We’ve never come up with a good, one-sentence way of summing it up…but once people try it, they get it.

Jonathan Mak: We’re hoping it’s a bridge for people. If you just play videogames, it would be a bridge for you into making music. If you’re a musician, it would be a bridge into making game levels. I think we really want to empower people to make music, though. As musicians ourselves, we take for granted how awesome it is to be able to write music. But it’s actually not that hard — it’s very simple.

Shaw-Han Liem: Just put a bunch of fingers on the screen [he places three fingers on the PS Vita touchscreen] and you already have a loop. That took three seconds and you have a drum beat. That’s the whole idea — to demystify making music. If someone who never thought of themselves as a musician tries out Sound Shapes and realizes they can do it, and then later picks up a drum machine and sequencer…that would be amazing.

PSB: So if I picked up Sound Shapes loved making music, where would I go next? What would be the next logical step, in terms of music creation?

Shaw-Han Liem: The musical logic, once you figure it out, is very similar to the way a drum machine works. So it’s basically a 16-step sequencer with an eight-note scale. As you’re playing through the levels, each game screen is like an instruction manual for how to make that music. It’s like you’re looking at the sheet music as you play the game, but you’re visualizing it differently. As you play, you’re learning how to write music. And once you grasp that concept, you can take it to other music software, drum machines, sequencers… Those all use the same concepts.

PSB: During E3, a lot of media raved about Sound Shapes; some have even said it’s one of PS Vita’s killer apps. Did you expect that reaction? What does that feel like?

Jonathan Mak: Uh..well, I didn’t think people would get it, or like it, so… [laughter] It was really awesome when it was the exact opposite of what I expected!

Shaw-Han Liem: Going into E3, we both had a certain amount of terror because we couldn’t describe it in a sentence, and because it’s a new idea and so different…

Jonathan Mak: …and it doesn’t have that neat one-liner. Like, “oh, it’s in HD!”

Shaw-Han Liem: What was a surprise was that everyone who has seen it has approached it with an open mind. Once we explain it, people seem to be enthusiastic.

Jonathan Mak: And to be honest, the version we showed at E3 was pretty barebones. It was just about introducing the premise, so we showed a very simplified version of what it’s going to be.

PSB: So if this is the framework, where does Sound Shapes go from here? Where will you focus your attention for the remaining development time?

Shaw-Han Liem: On a general level, the goal is to build out the tool set so that you have enough entities to build a really interesting world with a lot of gameplay. And to also add a lot sounds to allow totally different styles of music from one level to another. Basically, giving the tool more power — which will allow us to create cooler levels, but also giving more options to the players.

Jonathan Mak: From the gameplay point of view, there are some classic platform mechanics that aren’t in the game right now because we haven’t had time. If you like platformers, you’ll know there’s some obvious stuff we could put in there. And every time we add one entity, we exponentially increase the number of levels you can create.

PSB: Do you have a name for the, uh, protagonist?

Shaw-Han Liem: It never even occurred to me that it should have a name, but a lot of people have been asking…

Jonathan Mak: The key is that we didn’t want the avatar to steal the spotlight because the focus should be on the music that’s being created. So we made him a little bit low-key just to allow the music to come to the forefront.

PSB: In terms of a single-player experience, what’s your philosophy? Will I sit down with a campaign of sorts, or just a big, hot mess of levels to play?

Shaw-Han Liem: The metaphor that we’re using is that you’re sitting down with your record collection and you’ve got a bunch of levels – tracks – to leaf through. As you progress, you’re unlocking tunes, a new record… I guess people don’t sit down to listen to records anymore, but I still do! In terms of a “narrative,” that’s what we’re thinking. It’s about creating the atmosphere and interacting with music in a cool way.

PSB: What’s your vision for the online sharing element? Can I pass a level along to a bunch of friends and have them all leave their mark on it?

Jonathan Mak: Right now, it’s simple because we’re focusing on the core game. But you’ll be able to share levels with the community and download other people’s levels. There are a lot of good ideas kicking around…that we probably shouldn’t even talk about!

Shaw-Han Liem: The high-level goal is to capture the fun of making music — playing it for people, sharing it, making a record and sending it into the world to see what happens to it. If you can share your work and be proud of it, that’s going to encourage you to make more cool stuff.

Jonathan Mak: On the internet, people are like, “check out this song I found!” Once we get enough community-created levels, it would be cool if that happened to Sound Shapes. There might be a niche audience who really likes one guy’s levels.

PSB: Are you guys gamers?

Shaw-Han Liem: Five years ago, when I started this collaboration with Jon, I wouldn’t have considered myself a gamer. Around that time, people like Jon and some others introduced me to games that interacted with music in a cool way, like Rez. That’s what got me interested, and now I love Call of Duty. I don’t know what it was, but about a year ago I became a fiend.

Jonathan Mak: I play a lot of Tetris. I’ve always played Tetris. I hope to turn pro one day. Maybe…

PSB: What draws you to Tetris, Jonathan?

Jonathan Mak: I guess it’s a game that grew with me. I started playing it at eight years old, and I thought it was a piece of crap. But then I watched my brother’s friend play it like a speed run, so I started playing it that way. As I grew older, I started reading deeper meaning into Tetris: it only gave you certain shapes, so it’s kinda like life — you have to deal with what you’re given. Then came the poker craze in the 2000s, and I started applying that to Tetris: Even if you’re given crap, you can rearrange it in a way that’s fruitful, grab onto an opportunity. So it was all about probability and playing the odds .

Shaw-Han Liem: You should watch him play Tetris sometime, he’s like Rain Man.

Jonathan Mak: The funny thing with Sound Shapes is, I’ve never played platformers. It’s not a genre that I like and I was never good at them. But having worked on a platformer now, I can finally play and appreciate those games now. They used to be way too hard for me.

But yeah, it’s mostly Tetris. [laughter] There are a lot of games I want to play…I really want to play Journey when it comes out. I broke down and played a little Portal 2.