Hello! This is the story of ARISE, a mobile augmented reality open world RPG designed to motivate movement in pediatric patients.
About Me
I am the lead game designer & engineer at SpellBound AR. I graduated from the University of Michigan in 2020, and I’ve been making games for 5 years.
About SpellBound AR
SpellBound’s mission is to help pediatric patients by building games and tools for education, distraction, and rehabilitation.
Our business model relies on selling licenses to hospitals to use our products. In the future, we will grow our value by collecting patient data and reporting it back to hospitals, and by acquiring government approval so that our products can be prescribed to patients and paid for by health insurance companies.
SpellBound is based in Ann Arbor, Michigan and has 9 employees, but we do all of our work completely remote. We are largely funded by research projects, and we focus on building mobile applications that leverage augmented reality (AR) technology.
About ARISE
SpellBound’s biggest product is ARISE, a mobile game designed to motivate ambulation. It is most commonly used by pediatric patients in a post-surgical setting, but has also been used for physical therapy, occupational therapy, general entertainment, and more. The game has been in iterative development for 3 years, contains about 2 hours worth of engaging gameplay, and is available in both English and Spanish.
In the beginning of the game, a great storm destroys an underwater city and scatters all of its residents. Players will complete challenges to repair the city, and explore the real world to find augmented reality portals that contain the lost residents. Each portal contains unique games, systems, and stories for the player to complete in order to rescue the lost residents.
ARISE is built in Unity and uses AR Foundation for its augmented reality capabilities.
A Quick Crash Course in Augmented Reality (AR)
The first thing to know about AR is anchors. An anchor is a point in space tracked by the device, and that all AR content is positioned relative to some anchor.
In this example, the paper card depicting a phoenix is the anchor. This is known as an “image target.”
The 2 main types of augmented reality are marker-based and marker-less. Marker-based uses predefined anchors programmed into the application, and marker-less uses anchors that the application “finds” at runtime.
examples of Marker Based tracking
Image Targets
Using predefined images as anchors.
Pros:
– Easy to implement
– Easy for user
– Widely supported & reliable
Cons:
– Expensive to distribute
– Hard to maintain
– Weird camera angles
– AR size limited by target size
Object & Area Tracking
Using predefined 3D models or 3D room scans as anchors.
Pros:
– Easy for user
– Better camera angles
Cons:
– Expensive to implement
– Less scalable
Examples of markerless tracking
Position Tracking
Using the device camera, gyroscope, and accelerometer to calculate the position of the device relative to generated anchor points.
Pros:
– Easy to distribute & update
– Works for big content
– Widely supported & reliable
Cons:
– Harder for user
Surface Tracking
Finding horizontal and vertical surfaces in the environment.
Pros:
– Easy to distribute & update
– Works for big content
– Floor tracking is pretty reliable & widely supported
Cons:
– Harder for user
– Wall tracking is less reliable & less widely supported
– Suffers with shiny floors or bad lighting
SpellBound’s Early Experiments
Before ARISE, SpellBound built a few products to experiment with augmented reality and learn how it could fit into the medical market. I mentioned earlier that SpellBound helps patients through education, distraction, and rehabilitation, so here are examples of products we built in each of those:
Education:
An MRI simulator that demonstrates a procedure and teaches the looks and sounds of an MRI machine.
Distraction:
A series of mythologic creatures with interactions that could distract patients while they receive injections.
Rehabilitation:
A space-transforming mural that motivates ambulation in post surgery patients.
After building these products, we were most excited by the space-transformation rehabilitation solution. The problem of motivating movement was real, as we had seen crude solutions in several places (for example, some hospitals used toy areas and prize bins to motivate movement). Our initial target audience was post-surgery patients, ages 7-12, whose length of stay was 1-3 days. The key business opportunity here was a bigger game with a bigger price-tag, and the ability to track health data both in the hospital and at home.
However, one takeaway from the initial experiment was that murals are problematic. They are expensive to install, hard to change, and limit the opportunity for movement because of their static nature.
The First Prototype
The first version of ARISE was an underwater themed scavenger hunt where players ventured through hospital hallways to find 10″ wide wall-stickers. Each sticker contained a 3D model with a few animations, and the flow of movement through the game looked like this:
Although this app did motivate patients to move, testing showed that the effect was limited. The app had little guidance, so most players were confused and needed to rely on clinician support to play. Additionally, the lack of a story and deeper gameplay meant limited replay-ability and fading novelty.
After pondering the guidance problem, I learned the seemingly obvious lesson that augmented reality is a tool, not a game. During initial development I was so caught up in learning the technology that I thought “building augmented reality” was enough to make an app engaging and functional. After watching players be “deeply confused” with gameplay, I realized that the scavenger hunt was merely the base of an idea. Players needed directions, motivations, accomplishments, and world context, so I later implemented those things through quests, a story, a journal, and a map.
To address the problem of fading novelty and replay-ability, we focused on adding features to each sticker. We treated each sticker like a unique “location,” each of which needed a it’s own system or game, a reason to hang out, and a reason to return. These changes had to be implemented thoughtfully, because each location needed to encourage exploration while still supporting non-linear discovery. In other words, players should be able to go anywhere and find something to do, no matter where they are in the game. We didn’t want players being disappointed by walking to a location only to realize they couldn’t access it yet.
After dealing with those problems, we arrived at…
ARISE 2.0
The second version of ARISE now included a story, so it’s the first version that I consider “open world.” It had a hub-and-spoke model of gameplay, where a player’s hospital room was their home base (called “The Reef”), and from there they would embark on quests to the augmented reality locations in the halls (called “The Open Ocean”). The flow of movement through this version looked more like this:
The gameplay pillars that we focused on developing were exploration, customization, and community.
– Exploration was the source of movement.
– Customization helped the player make their game experience “theirs.”
– Community made the player feel welcome, invited, and immersed.
This version was more engaging and replay-able, but it still had problems. The wall stickers were vertical image targets with small diorama-like environments that were easy to look around in, but the reef was a horizontal target with a massive city that was hard to navigate. The first moments of the game were also awkward, so the initial onboarding was confusing and our first impression wasn’t great.
The horizontal target that we used for the reef was a 8.5×11″ piece of paper, and we tried to cram an entire city and a cast of characters onto it. Everything was tiny, so interactions were difficult to execute (i.e. tap on characters, move buildings). Unfortunately I don’t have a great photo of this, but this was what the Reef looked like. Imagine all this jammed onto a sheet of computer paper:
To explain the awkward first moments of the game, I’ll first share this idea:
“Open world games should put their open worlds first and do everything they can to encourage players to want to explore them”
When I think about captivating new players and encouraging exploration, something like this comes to mind:
It’s less than 5 minutes into the game and the player is already introduced to a vast land with unbounded limits and endless adventure. An introduction like this would be perfect for ARISE, but how did our game introduce itself? Well, it’s entirely in augmented reality, so it introduced itself with a camera feed of the player’s surroundings.
I don’t know about you, but I’m not really motivated to move by a live feed of my couch.
We considered a few solutions (i.e. Pokemon Go’s map interface), but ultimately decided to make the reef into a non-AR environment, similar to city building games like Clash of Clans or Animal Crossing.
At this point we learned that AR is not great for everything. For us, the best feature to leverage from augmented reality was the ability to tie gameplay to physical locations, because that’s what enabled and motivated movement. AR is less important during stationary moments like talking to characters and playing mini games, and forcing it into every aspect of the game often made things worse.
These lessons brought us to…
ARISE 3.0
In this version, the reef became a static scene that the player could navigate using their fingers:
The flow the flow of movement in this version was similar to that of the previous, with the main change being the removal of the reef image target. Players could now swap between the reef and the open ocean with the tap of a button.
Since we learned that augmented reality was strongest during exploration, we built features to make the open ocean environment more engaging to walk around in, such as a sand floor below and waves overhead. Players would navigate the game like this:
Moving around the reef to fix buildings and receive quests.
Exploring the AR open ocean to find portals.
Opening AR portals to meet characters and play games.
The format above is how the gameplay loop currently stands. Our recent efforts focus on implementing unique systems for each location, with the goal of adding replay-ability and motivating players to revisit older portals. Some examples of these systems include:
Painting
The octopus portal introduces painting, and after you make some artwork you can hang it up all over the ocean.
Music Sheets
The turtle portal introduces the piano, and afterwards you can explore the ocean to find new music sheets to play.
Treasure Hunting
The lobster portal introduces treasure hunting, and afterwards you can find and follow treasure maps to chests all over the ocean.
More examples of this system include photography, finding lost fish, shopping, & decorating. As more systems are added, the flow of movement in ARISE gets closer and closer to this:
And that’s how we got to the state of ARISE today! Thank you for joining me on this journey, and I have countless more lessons that I hope to share in posts to come.
Cheers,
– Chris
This essay was originally presented live at a monthly meeting of the Ann Arbor Indie Game Developers Association on Thursday, Sep 22, 2022. Click here to view the original slides.
Chris Monteferrante 