Download the final report (PDF) - Alicia Nachman

sandpaperleadSoftware and s/w Development

Oct 31, 2013 (3 years and 5 months ago)


Augmented Reality Social Gift Experience
Team Unity
Chandan Dasgupta • Alicia Nachman • Davide Pluda • Aneesh Tarun
Final Report
GoGifts is a Facebook application that enables you to send augmented reality gifts to your
friends anywhere in the world. For the course, we have developed a proof of concept of this
application that demonstrates a mobile, social, augmented reality experience using the Facebook
But why virtual gifts? According to a recent
article, “Virtual gifts are sort of a head
scratcher,” yet 60 million of them have been given on Facebook alone.
It's hard to imagine why
such a concept has become popular, yet social networks are increasingly joining in on this
phenomenon. In our research, we learned that in one social network focused on parenting,
members send virtual gifts to others as a thank you for good advice. Perhaps its popularity stems
from the fact that it takes advantage of the persuasive principal of reciprocity, where we feel
compelled to reciprocate when we receive a gift.
Our target experience is all about playing up the parts about gift-giving that people love. That
means creative freedom for the sender to design the gifting experience and the serendipity and
surprise factor for the receiver of the gift. Because Facebook already successfully uses the
concept of virtual gifts, we wanted to build on this existing phenomenon in our experience.
But the GoGifts experience is more than just another Facebook gift application. We wanted to
leave the experience open enough so that it could be appropriated by users in ways we might not
have seen in advance. As a mechanism of leaving objects out in the world for others to find, it is
truly a platform on which to build many different kinds of experiences, even outside the realm of
gifts. For some example scenarios, please see Appendix 1.
GoGifts • Team Unity
Snyder Chris. 11 December 2008. “What is the Value of a Virtual Gift?” Retrieved March 3, 2009 from
The target experience for sending gifts is to incorporate a full-fledged gift authoring tool into the
Facebook application. This would allow users to either choose from our set of 3D objects or
import their own. There would be additional options for adding animations, audio, and text
notes. As the application grows, there would be room for 3rd parties to develop content that
could be given away or for purchase.
The basic gift interface involves designing or choosing the gift, selecting the recipient(s), and
setting the gift location. The user could choose to send to one particular Facebook friend or an
entire network. Location is selected via a map. As positioning technologies improve, gifts would
be able to be left inside, in which case a more precise interface would need to be used.
By default, the gift is sent right away along with a simple Facebook notification. However,
advanced options would allow the user to specify the date and time that the gift should be
available, the expiration of the gift, and the type and number of notifications. The user could also
choose whether to place an order for the real gift online to be sent to the recipient upon approval.
As far as receiving the gift, the experience is meant to be shaped by how the sender has designed
it. With the default settings, the user would receive a Facebook notification telling them that one
of their friends sent them a gift using GoGifts and the area where it was located. Or the sender
might have chosen to skip the notification, in which case the recipient would only find out about
the gift once they were near the location.
Ideally, location-based alerts could be done through a system similar to geofencing, where SMS
alerts are sent when the target crosses a certain boundary. This would allow users to receive
alerts without having to be running a program first. The SMS would tell them about the gift point
them to Facebook to add the GoGifts application. All viewing of the gift would happen through
Facebook, with the experience targeted for a mobile device such as the iPhone or Android.
The third key aspect of the GoGifts experience is the ability to share the whole thing with your
network. After finding the gift, the recipient can take a snapshot of the merged view which is
automatically published to a special GoGifts photo album on your Facebook profile. Because of
GoGifts • Team Unity
the way that Facebook is designed, the photo would immediately be shared with your friends
through your status feed, and they could start commenting on it. Part of the GoGifts experience
would also be to rate the gift. As a gift sender, your social capital could grow as you sent more
gifts and maintained high ratings. All of these metrics could be shared on the GoGifts application
page. This is where the viral element would really take over and generate interest in the system.
With the persuasive principle of reciprocity in action, people on the receiving end of GoGifts
would soon become senders and the cycle would continue.
The two main applications that make up the software side of our prototype are a Facebook
application and a Unity3D application. We built our Facebook application using PHP and Flex
and accessing the associated client libraries. We used Google Map APIs to show a map, locate
the gift, and retrieve coordinates from it. The chosen gift index and the GPS coordinates are
written to a file on the server and later fetched by our second application developed under
Unity3D. Within the Unity application, we made use of VRPN (Virtual-Reality Peripheral
Network), a library that provides connection between our application and external devices.
Since we had to capture video from our camera we also used the Video Wrapper plug-in.
Figure 1: Software architecture
GoGifts • Team Unity
The Unity3D system is a standalone application that runs on a laptop that requires a webcam, a
GPS receiver, and an orientation sensor [figure 2]. The webcam used for our prototype was a
Logitec Pro 300 with a 640x480 resolution and a 14p framerate. The webcam is needed because
we want the user to see the real environment through our device. The GPS receiver used was a
Holux GPSlim 236 connected by Bluetooth to the laptop. It was needed to track the user location
and hence show information of the gift on the screen. The orientation sensor was a MotionNode.
Since it provides 3 degrees of freedom it has been very useful because according to the
orientation of the device we could show different perspective of our environment, hence a more
immersive augmented reality experience.
Figure 2. Hardware setup
The prototype for the AR GoGifts System consists of two main components. An application for
sending gifts to any friend in facebook and an application for viewing the gift received.
GoGifts • Team Unity
Component 1: Gift-Sender Application
The Gift Sender application has been implemented as a Facebook application that any Facebook
user can install and use. The application as been implemented using php for server side
functionality and Adobe Flex for the application's user interface along with the Google Map
The php code handles the initialization of the application in Facebook and providing a list of
friends to choose from to be the gift recipient [figure 3]. Currently this data is not made use of for
notifying the selected recipients as it does not make sense to notify any user without providing
access to the gift viewer application. Currently, the notification is sent to a pre-defined default
user [figure 4] whenever the gift is sent. But, the placeholder can be made use of to easily add
facebook and email notification based on the recipient.
Figure 3. Selecting the Gift Recipient
Figure 4. Gift Notification
The Flex component presents the user with a preset choice of 3D gifts to choose from [figure 5].
Also, the interface provides the user with a map [figure 6] which can be used to pin point to the
specific location where the gift has to be placed. Interaction with the map and selection of the gift
can be done in one of the three ways. The user can provide an address in the “Address” text area
and set the location for the gift. Also, she can locate the place where the gift needs to be placed
and click on the spot to set the location. Alternatively, she can drag & drop the marker to a
location and send the gift.
GoGifts • Team Unity
Figure 5. List of predefined gifts to choose from
Figure 6. Map Interface to place the gift in a specified location
GoGifts • Team Unity
The php code also encapsulates the data for consumption by the gift viewer application. This data
consists of the gift ID and the lat/long values of the gift location. Overall, the Facebook
application provides a simple and easy-to-use interface with which users can send gifts to anyone
in just a few simple steps.
Component 2: The Gift-Viewer Application
The Gift-Viewer application is mostly developed using the Unity3D Game Development tool. It
also makes use of the VideoWrapper and the VRPNTracker (Virtual Reality Peripheral Network
Tracker) plugins for Unity. Both these plugins have been developed at the Augmented
Environments Lab. VideoWrapper provides interface for any video camera to integrate with
Unity3D. And the VRPNTracker plugin provides abstraction for using any from of tracking
device needed for the Augmented Reality (AR) experience. For our application we are making
use of this plugin to enable reading of data from the GPS and Orientation sensor. Our prototype
provides a seamless interface which helps users track the gift's location and view the gift!
The viewer application queries Facebook data and determines whether any gift has been placed
for the user. If any gift exists, the user is notified of this [figure 7]. After this, the user is
presented with necessary information on screen to search for the gift. Firstly, the user is
presented with the distance information [figure 8]. The screen displays the distance from the
user's current location to that of the gift in meters. This information is not enough to help the
user to locate the gift as it could be placed anywhere in the radius of the distance specified. To
alleviate this, a bright 3D arrow [figure 9] is presented to the user which points towards the
direction of the gift in real-time (based on the user's current position and orientation.)  This
along with the distance information helps the user to locate the gift easily.
GoGifts • Team Unity
Figure 7. Gift Notification in the Gift-Viewer
Figure 8. Distance to the gift displayed on screen
Figure 9. 3D Navigation arrow overlaid on the screen
Due to the error in GPS data, the prototype has been developed such that it fetches the gift to the
user when she is approximately 20m from the gift location. This reduces the last-mile problem
GoGifts • Team Unity
where the user trying to locate the gift while it is jumping all around the location due to GPS-
induced errors. When the user is close enough to the gift, a colorful giftbox pops up in front of
the user increasing the anticipation of the gift [figure 10]. Then the actual gift is displayed to the
user augmented in real world [figure 11]. The gift is slowly rotating at the same spot so that the
user is able to view the gift clearly. The user can share her surprise and joy by taking a snapshot
of her current view in the Gift-Viewer. A snapshot tool has been provided for this purpose [figure
12]. It can also be invoked by the shortcut key ‘S’ and any point of the experience.
The snapshot is automatically uploaded to the user’s Facebook photo album. Thus, the user can
share her experience and let her friends and family and provide a platform for comments and
discussions on her unique experience.
Figure 10. A gift box indicating that the gift has been found
GoGifts • Team Unity
Figure 11. The Gift displayed in the user's augmented view
Figure 12. The Snapshot Tool
Overall, the prototype captures the essentials of our planned target experience. It manages to
provide a fun and unique experience for the user and would definitely make any gift feel special,
the AR way!
The gift authoring tool was out of scope for the prototype. Full Unity and Facebook integration
was also not possible, in particular the ability to run the Unity app from within Facebook. The
prototype also runs on a laptop instead of the target mobile device.
GoGifts • Team Unity
Our process really began at the point in the course where each person was responsible for
brainstorming multiple ideas for mobile, social, AR experiences. As a group, we evaluated our
individual ideas as well as generating some new concepts. We settled on AR gifts because of its
strong link to existing social network behavior in Facebook and its potential to become a
platform for many more user-defined AR experiences.
Once we settled on the general idea, the next step was all about brainstorming potential
characteristics and features that would define our desired experience.
Figure 13. Whiteboard session
Shown in figure 13 is part of our initial whiteboard brainstorming session, where we laid out a
very broad definition of our idea. We identified it really as a platform for many different kinds of
GoGifts • Team Unity
experiences. We also represented a rough idea of a temporal model of engagement. Figure 14
shows our thinking regarding potential features of this system.
Figure 14. Potential features
GoGifts • Team Unity
Figure 15. Early task analysis
As our ideas started gelling, we captured a rough task analysis by listing the steps involved in
both sending and receiving gifts, shown in Figure 15. This exercise also helped inform our
scenarios and storyboard. Next, we moved on to developing scenarios. We created four different
stories, each illustrating a different way that our experience could unfold. Two of our scenarios
cover the core experience that we planned to design, while another two explore "extension
scenarios," or possible ways that the idea could be appropriated by users in new ways that were
not directly supported in our design. These scenarios can be found in Appendix 1.
To develop the prototype, we first had to determine which parts of the experience would be built
and which would not. We took a staged approach to development to ensure that we stayed on
track. We first implemented a bare bones system with the minimal components necessary to
GoGifts • Team Unity
communicate our design. As we reached milestones, we added additional features as time
allowed. As it turned out, we were able to reach our stretch goals for the prototype.
Brainstorming and decision-making about the experience was done as a team, but we split out
development tasks to make the best use of time. Some tasks were done by an individual and some
by a subset of the team. The Facebook application was primarily developed by Alicia, while
Davide and Alicia worked together on the Unity snapshot feature. Davide also did much of the
upfront Unity user interface and networking research. Aneesh was primarily responsible for the
Unity development, with assistance from Davide. Chandan developed storyboards for our
proposal and final presentation, and also did upfront research with Davide on 3D objects for use
with Unity. We tried to split the work evenly and allow people to work on areas that they were
interested in.
Overall, our collaboration and development process went well. However, we ran into a few
issues that needed to be addressed. Our original plan to develop the gift interface in Unity was
not appropriate, so we moved it to Facebook. Also we ran were unable to authenticate the
Facebook user through Unity app for the purposed of uploading a photo to the profile. Instead
we had to use a static user ID. Since we have no expertise in creating/authoring 3D objects, we
made use of decent free 3d objects available online as gifts. In terms of gift viewing, inaccurate
GPS data was the biggest problem causing the gift to bounce around. To counteract this, we had
to set a threshold where we cut off the GPS and brought the gift right in front of user.
Before implementing additional features, it will be important to conduct some user testing to
determine the areas in need of improvement. In future iterations of GoGifts, we would like to
include a gift authoring tool that helps users better define the end-user experience of the gift
recipient. We would also add a full-fledged notification system based on the selected recipient.
In general, the app should be more integrated with the Facebook friends list and thus aware of
the sender and the recipient of the gift.
GoGifts • Team Unity
In terms of locating gifts, we would like to provide more context-aware information presented in
the gift viewer app that helps users locate the gift. In addition, it would be nice to enhance the 3D
objects such that the gift feels more realistic in the augmented view. Finally, one of the main
objectives for the future is deployment on a mobile platform to tap into the potential of this
GoGifts • Team Unity
Appendix 1
(1) Jack has a tendency to forget birthdays. He even forgets the birthday of his best friend, Tom,
which makes him very angry. Jack has been using his mobile phone calendar to set reminders but
then he does not always have time to buy the gift. Recently, Jack came across this app that allowed
the user to select/create meaningful gifts for friends, allowed the user to set the time when the
virtual gifts will be visible to their friends, and also facilitated ordering of that gift online. He
installs the app and decides to get something which Tom can use for the upcoming triathlon right
after his birthday. He selects a Schwinn mountain bike for Tom and to make things interesting,
puts the gift on the running track in the CRC. Tom practices on the tracks in CRC before a tri-
athlete event and he was sure to come across the gift there. So, Jack goes to the CRC track and
selects the gift for Tom and specifies the timeframe after which the gift should become visible to
Tom. He sets the notification mechanism such that Tom will get a message when he walks into
the CRC track on his birthday.
On March 14th, before Tom starts running on the tracks, he gets a message saying that Jack has
left “something” for him there. Surprised how Jack could have remembered his birthday, he
quickly scans the area with his mobile phone and sees the Schwinn. He is overjoyed to see the
bike as he was planning to get a similar one for himself that evening! He was also very happy that
Jack finally managed to remember his birthday and had given such a special gift! He immediately
took a picture of the bike and rated it as the “Best Gift ever”. Tom’s profile on Facebook starts
reflecting that Jack has given a Schwinn to Tom for his birthday and Tom decides to make this
publicly viewable so that everyone could see what he got for his birthday! He’s very happy and
gives Jack maximum friendship ratings. He uses the picture and contacts the vendor to approve
the online order that very instant. Jack gets a notification that Tom had approved the online order
and that the bike was on its way to Tom’s house!
(2) A silent protest is going on in the Tech campus against the new SGA president. Students are
unhappy over the shoddy way in which the elections had been handled this year and they suspect
GoGifts • Team Unity
that the process was not transparent. They are demanding that he step down from his position
immediately and make way for a re-election. The protesting students decide to take their protests
to the next level but do not want to disturb the decorum of the entire school. Some of the
students had recently heard about a gift giving app which let the users superimpose animations,
actions, or 3D artifacts on the real world. They decide to use this app by tweaking it to suit their
requirement. They install the app on their mobile phone and select/create the act of hurling
rotten tomatoes aimed at the president’s house. The group assembles in front of the president’s
house and “attach” these “gifts” to the location. They set the privacy settings such that the
“gifts” are visible to everybody in the Georgia Tech group on Facebook. If anybody in that group
walked past the house, they received a notification on their mobile phone about a “gift” in the
area. They could see the gift by installing the app and scanning the area with their mobile phone.
In the evening, when the president reached his street, he did not notice anything different with
his house but he got a notification that the Georgia Tech group had left a “gift” for him. He was a
bit surprised but curious as to what gift could be waiting for him. He installs the app quickly and
scans the area with his mobile phone. As soon as he focuses on his house, he sees loads and loads
of rotten tomatoes being thrown at it! He gets a message conveying that the “gifts” were given by
the Tech group to make sure that everybody knew what he was up to. The entire exercise draws a
lot of public attention and attracts the attention of previous SGA members. They decide to
intervene and hold meetings with the current board and the protesting students to reach a
consensus on the future course of action. The whole experience embarrasses the president so
much that he relents to the demands of the protestors and resigns.
(3) Jim and Karen had been dating for 2 years, and Jim was ready to pop the question. He had
always wanted to do something creative for the occasion, something they could tell their
grandchildren about. Jim worked in technical marketing and considered himself a total
technology geek. He was always the first adopter for the latest gadgets. He was the one who had
encouraged all of his friends to join Facebook. "Facebook! It's perfect." thought Jim. "I have to
find a way to propose to Karen through Facebook." He thought about the gifts feature in
Facebook where he could leave Karen a virtual ring (accompanied by a real one of course), but it
just didn't have the impact he wanted. Then he remembered reading an article about a new app
that allowed you to leave virtual gifts out in the real world. "This could be interesting," thought
Jim. He decided to turn the whole thing into a treasure hunt, and his plans were set. That Friday,
he asked Karen to meet him at one of their favorite restaurants. When Karen arrived, she heard
GoGifts • Team Unity
an alert on her iPhone. It was from Facebook telling her she had an alert from Jim that required
her to install a new app. The new app installed immediately, and Karen saw a message: "You have
a virtual gift in the area from Jim. Look through your phone to find it." Karen held up her phone
and scanned the area. There on a bench she saw a fluffy teddy bear and a note explaining where to
go next. "What is Jim up to?" she thought. Each location she went to held another gift, and
another clue. Finally, she arrived at their favorite spot in the park where they had their first kiss.
A candlelit table was set up, but Jim was nowhere to be seen. As Karen looked through her
iPhone, she saw a small jewelry box on the table. It began to open, and inside was a gorgeous
diamond engagement ring. As Karen looked up, Jim had appeared holding that very same ring
box in his hand. He barely had time to get down on one knee when Karen said "Yes!" By the
time the couple came home after their romantic dinner, all of their Facebook friends had seen the
ring on Karen's profile and flooded their walls with congratulations. Now this was something
they would remember forever.
(4) Sam, a local business owner in Boulder, CO had just started a new venture in the outdoor
apparel business. His store had just opened, and he had gone through the traditional marketing
channels to generate buzz on the store. He still felt like he wanted to tap into the power of a
social network like Facebook, which was chock full of his target demographic. His business
already had a Facebook page, but there had to be something more. Maybe something involving a
Facebook game. As Sam did some research, he found out about a new app that was all about
leaving virtual gifts for people out in the world. Sam immediately thought of a perfect way to
harness this concept for a marketing campaign. Through the app's simple map interface, he
chose locations in which to leave virtual coupons up to 75% off. He left them on popular hiking
and biking trails in the area. The better the coupon, the more remote the location. As a grand
prize, he left a virtual mountain bike on one of the trails, which he would give away to the first
finder. Sam sent out a blast to his store's Facebook network, and customers immediately sprang
into action. Hikers were trying new trails just to find the best coupons, and everyone wanted to
find the mountain bike. As people began finding the discounts, they showed up in their profiles,
spreading the word about the store and the promotion to their entire networks. Sam had never
seen anything like this, as people from a few towns over started coming to Boulder just to visit his
store. A few months later, the mountain bike was finally found, and given away. As far as Sam was
concerned, this was the most successful marketing campaign he had ever launched.
GoGifts • Team Unity