Web Services in

insidiousbehaviorSecurity

Nov 3, 2013 (3 years and 7 months ago)

64 views

Incorporating
Web Services in
Mobile
Applications

April 3, 2009


San Francisco

Slides available at

www.aduci.com/web2expo2009sf.ppt

Disclaimer

Web services and APIs of third parties referenced in this presentation
are owned by the third parties and governed by their agreements.
Before using any of the APIs or code referenced in this presentation
review the terms of use by the third party providing the API.





These slides are available under the Creative Commons Attribution
-
Noncommercial
-
Share Alike 3.0 License. You can blog, photograph,
and share this information with others.

Trademarks and brands are the property of their respective
owners.



Introduction
-

Convergence


Available APIs are growing daily

-
Currently 1229 (and growing) APIs registered at Programmable Web


Shifts in Consumer Expectations

-
Consumers now expect mobile devices to be connected to Internet
Sites and Services


Platform Development becoming easier and more standardized

-
iPhone SDK


Greater than 50,000 registered developers

-
Android


5% of smart phone web usage in just three months
1



Seeing extension of enterprise applications to mobile devices

-
Salesforce.com


1

Techcrunch, March 24, 2009, Erick Schonfeld


Introduction


Many Web Services and APIs were originally developed with server
to server or server to browser in mind, not mobile applications


Mobile platforms have their own set of challenges given:

-
Bandwidth

-
Memory and CPU Availability

-
Storage Capacity

-
Connectivity Options and Issues

-
Security

-
User Interaction and Display


Web Service


Mobile

Integration

Challenges


5.1.5. Unless we have provided you our express written
consent in advance, you are not permitted to use Amazon
Associates Web Service in connection with any handheld,
mobile or mobile phone application.


From the Amazon Web
Services
TM

Customer Agreement

http://
aws.amazon.com
/agreement/

API Developer Programs


Is mobile access allowed?







Other considerations:


Call Limitations (# per second, total per hour/day, pricing above)


Caching and Storage of Data


Persistence of Data


Length of time stored


Freshness of Data


Length of time before refreshing











Persistence Limitations







Refreshing Requirements









-
You may otherwise store Amazon Properties generally (other than
pricing or availability information) for caching purposes for up to 24
hours. However, you may store the following Amazon Properties for
caching purposes for up to 1 month:…

From the Amazon Web Services User Agreement

Caching and Storage Limits

You must refresh and re
-
display any Amazon Properties (other than
pricing or availability information) at least once every 24 hours or
once every month, as applicable, by making a call to AWS and
refreshing your Application’s contents immediately after the call.

From the Amazon Web Services User Agreement


Certification of your Mobile Application







Some API sets require certification on top of distributor certification


Determine Costs / Timeframe / Effort given a mobile app


Prepare for testing


typically looking for error handling, API abuse


Certification is a positive!


Credibility, validation, marketing


Keeps the neighborhood safe










“If any of your
Facebook

Platform Applications are client
-
resident
(including on a mobile device), you agree to furnish a copy of such
Facebook

Platform Applications and any supporting documentation
upon request for the purpose of verifying your compliance with this
Agreement; and …”

From the
Facebook

Developer Terms of Service

Application Verification


Mobile
Techniques

Authentication vs. Authorization


The Difference



Authentication from the API provider


API Key



Authorization from the user


authToken


Session Key

-
By providing combination of API Key and authToken can then
receive the session key

-
What is the shelf life of the session key





Authorization will commonly
affect user’s experience on
mobile

-
eBay authentication and
authorization screens




Last.fm
-

Sample Flow


http://www.lastfm.com/api/mobileauth


Get an API key


Request auth


auth.getMobileSession

-
Inputs:

-
username

-
authToken (md5(username + md5(password)))

-
api_key

-
api_sig


Returns session key


Make API call with sk (session key)



Note: Try to keep credentials in one area to facilitate later changes




Last.fm auth.getmobilesession


auth.getmobilesession




Result with new session key








http://ws.audioscrobbler.com/2.0/?method=auth.getmobilesession&api_key=be8c2
…&username=chudsonfm&authToken=4eb71…&api_sig=e93fb…

<lfm status="ok”>


<session>



<name>chudsonfm</name>



<key>142c0085138a4b72d1782215b1da770a</key
>



<subscriber>0</subscriber>


</session>

</lfm>

username (Required) : The last.fm username.

authToken (Required) : A 32
-
byte ASCII
hexadecimal MD5 hash of the user's Last.fm
credentials.

api_key (Required) : A Last.fm API key.

api_sig (Required) : A Last.fm method signature.

Last.fm


Call Execution


Now we can make the call with our sk (session key)


user.getinfo



Response

http://ws.audioscrobbler.com/2.0/?method=user.getinfo&api_key=be8c2…&sk=142c0…&api_sig=ccbd1
…cc9dd

<lfm status="ok”>


<user>



<id>17317105</id>

<name>chudsonfm</name>

<realname/>

<url>http://www.last.fm/user/chudsonfm</url
>

<image/>

<lang>en</lang>

<country/>

<age/>

<gender>n</gender>

<subscriber>0</subscriber>

<playcount>0</playcount>

<playlists>0</playlists>


</user>

</lfm>

sk (Required) : The session key returned by auth.getMobileSession
service.

api_key (Required) : Your 32
-
character API key.

api_sig (Required) : Your API method signature

Speed Traps


Could be on wifi, 3G or 1G based signal and bandwidth


Segment functionality / calls to prevent issues

-
Standard Apple test for approval is bandwidth usage when on cell
network


Polling

-
Create subset of calls and functionality when on slower bandwidth

-
Use notifications if possible

-
Modify polling interval based on need


Recovery from intermittent or lost connections


Test the use of APIs via public vendor API test tools on various
bandwidths


Large Data Set Handling


Decreasing returns for XML with large data sets and repetitive XML
sections


Even more the case with mobile devices given:

-
Communication challenges

-
Memory and processing limitations


Minimize large data sets:

-
Use Just
-
in
-
Time (JIT) methodology

-
Ask for only those elements that you require

-
Ask for only those items that you require

-
Store what you can locally instead of requesting the same data

Response Processing


Leverage Stream Parsing for mobile environments

-
Provides low memory overhead

-
But, requires more heavy lifting


iPhone SDK based on stream parsing


NSXMLParser class


Android has Xstream as the streaming parser




Response


Minimize to just
what is needed

Parse Stream


Stream Element
Based


Sub parsing for
specific items

Display / Store


Cache what
will be reused
(file, db,
memory)

iPhone Parsing


NSXMLParser delegate methods

-
5 key methods






-
Parser:didStartElement:

-
Parser:foundCharacters:

-
Parser:didEndElement:

-
Parser:didEndDocument:

-
Parser:parseErrorOccurred:

Method

Purpose

parser:didStartElement:

Start of element tag occurred with attributes

parser:foundCharacters:

Element data located which may be all or some

parser:didEndElement:

End of element tag located

parser:didEndDocument:

End of XML document occurred

parser:parseErrorOccurred:

Unrecoverable parsing error occurred


Typical didStartElement, foundCharacters, and didEndElement

Parsing


In long parsing streams provide feedback
when available


Example: Control4 My House

-
Home control from your iTouch/iPhone

-
During initial project load describes the
sections being processed

-
Stores those items that will not change
including house devices

-
But when providing current data,
requests from server current state

-
Blends stored information with real time
status information to provide display

Filtering

-
Types of Request Filtering

-
Selection criteria for narrowing data set returned

-
In general determine how efficient are the calls


reduce data waste

-
How much extra data, streaming, parsing is happening to access
the data you use? How much data is being dropped on the floor?


<?xml version="1.0" encoding="utf
-
8"?>

<FindItemsAdvancedRequest xmlns="urn:ebay:apis:eBLBaseComponents">


<!
--

Standard Input Fields
--
>


<MessageID> string </MessageID>


<!
--

Call
-
specific Input Fields
--
>


<BidCountMax> int </BidCountMax>


<BidCountMin> int </BidCountMin>


<CategoryHistogramMaxChildren> int </CategoryHistogramMaxChildren>


<CategoryHistogramMaxParents> int </CategoryHistogramMaxParents>


<CategoryID> string </CategoryID>


<CharityID> int </CharityID>


<Condition> ItemConditionCodeType </Condition>


<Currency> CurrencyCodeType </Currency>





<ShippingLocation> CountryCodeType </ShippingLocation>


<ShippingPostalCode> string </ShippingPostalCode>


<SortOrder> SortOrderCodeType </SortOrder>


<StoreName> string </StoreName>


<StoreSearch> StoreSearchCodeType </StoreSearch>

</FindItemsAdvancedRequest>


Paging


Provide paging functionality to users


Not all calls have paging available


Inform user of current location in set, easy access to next and
previous if applicable


Cache key information depending on

-
The user flow (detail page from results page)

-
The aging of the information (when does it expire)




Page Requests


Specify the page that you are requesting



<PageNumber>


How many items are on a page

-
Set the number of entries per page returned



<MaxEntries>


<?xml version="1.0" encoding="utf
-
8"?>

<
FindItemsAdvancedRequest

xmlns
="
urn:ebay:apis:eBLBaseComponents
">


<
PageNumber
>2</
PageNumber
>


<
QueryKeywords
>
iphone
</
QueryKeywords
>


<
MaxEntries
>20</
MaxEntries
>

</
FindItemsAdvancedRequest
>


http://developer.ebay.com/products/overview/


List Paging Example


Must cache of course the
current page


Remember though the
total pages can change
dynamically so check total
pages after subsequent
requests


<TotalItems> int </TotalItems>

<TotalPages> int </TotalPages>


<?xml version="1.0" encoding="utf
-
8"?>


<
FindItemsAdvancedResponse



xmlns
="
urn:ebay:apis:eBLBaseComponents
">


<Timestamp>2007
-
07
-
06T00:51:05.250Z</Timestamp>


<
Ack
>Success</
Ack
>


<Build>e525_core_APILW_4863605_R1</Build>


<
SearchResult
>


<
ItemArray
>


<Item>


<
ItemID
>123000268897</
ItemID
>


<
EndTime
>2009
-
07
-
23T09:26:21.000Z</
EndTime
>





</Item>


<Item>


<
ItemID
>123000268898</
ItemID
>


<
EndTime
>2009
-
07
-
24T09:26:21.000Z</
EndTime
>





</Item>


….


</
ItemArray
>


</
SearchResult
>


<
PageNumber
>2</
PageNumber
>


<
TotalPages
>6</
TotalPages
>


<
TotalItems
>103</
TotalItems
>





</
FindItemsAdvancedResponse
>

Display


Caching of information for future retrieval and navigation

-
Be careful of data aging depending on what is being shown


Execute web service parsing and display in background when
possible to prevent UI blocking

-
example


eBay item description and images


When parsing large data blocks, progress indications can be provided
from element name (sections of the XML)


Synchronous versus Asynchronous calls


Use JIT and Information on Demand to maximize small layouts and
minimize web service calls

Caching Options


To minimize API round trips leverage caching and storage

-
Cache in memory

-
Cache in onboard light weight database or file

-
Off device persistence


Memory Cache


Memory Intensive


Non
-
persistent


Eg
. Data key for
last viewed item

Onboard


Space Limitations


Persistence
across sessions


Eg
. User
Information

Off Device


Requires
Connection


Reuse across
users


Eg
. Statistics,
look up tables


Expanding Your
Options

Blending Multiple Results


What if you have to:

-
Blend results from multiple calls
in the same API

-
Blend results from multiple API
sources


This could take the form of:

-
Extra item information on top of
item details call





Note this is simplified so does not
show authentication and
authorization checking, encoding,
or the like.

API Service

Mobile View

Trigger

Request 1

Mobile App

Request 2

Result 1

Fulfill

Fulfill

Result 2

Blend

Outlet

Blending Multiple Services


Multiple Services

-
Dependent on multiple
services


Example:

-
Weather request from
additional WS based
on zip code returned
by latitude and
longitude from a
geoposition service

API Service
1

Mobile View

Trigger

Request 1

Mobile App

Request 2

Result 1

Fulfill

Result 2

Blend

Outlet

API Service
2

Fulfill

Proxy Server Solution


Provides layer of abstraction to consolidate or manage web services


Proxy Server retrieves data on demand or on scheduled basis


Allows for single consolidated call from the mobile device


Processes data and returns subset or stores data for retrieval later








Can also reduce API calls by caching similar results across user


Allows for extending business logic layer

Benefits


Reduces round trips


Custom Filtering


Custom Business Logic


Off device storage


Cache duplicate requests

Challenges


Another layer to maintain


Data aging


Data synchronization

Proxy Flow


Provides for single
round trip for mobile
application


Provides easy
maintenance of web
services business logic
layer


Easily modify

-
Business logic

-
API credentials

Proxy
Server

Mobile View

Trigger

Request

Mobile App

Request

Result

Result

Outlet

API
Services

Fulfill

Fulfill

Request

Result



Proxy Landscape

Web Service

Web Service

Proxy Server


Designing your
WS/API Destiny

Creating Web Services


Try to serve lowest common device


Balance flexibility with overhead in calls:

-
Filtering of criteria and paging of data functionality

-
Split out high traffic calls versus critical requests

-
Easy but secure authentication and authorization for both the
consumer and application


Benefit for mobile consumers, application programmers and web
service providers

-
Decreased round trips

-
Increased efficiency of calls and applications

-
Better use of call volume restrictions

-
User experience improves



Wrap Up

Provide user preference options that translate into better tuned API calls


-

number of items to show on a page as an example

This will translate to data efficiency, fewer callers and greater use of
data retrieved

Minimize the number of trips, combine in single calls when possible

Leverage caching and data storage when allowed

Balance data granularity with length of returns for parsing


Final thought to ponder


The mobile device that is typically serving as a web service client
could in the now or future be a web service provider…

Thank you



Slides available at:

www.aduci.com/web2expo2009sf.ppt

Chuck Hudson

chudson@aduci.com

aduci

Appendix A
-

Resources


API / Web Service Online Catalogues



-
Programmable Web

-
www.programmableweb.com/api



-
Web Service List

-
www.webservicelist.com


Appendix B


Code Resources

-
iPhone URL Connections

-
http://developer.apple.com/documentation/Cocoa/Conceptual/URLLoadingSystem
/Tasks/UsingNSURLConnection.html

-
iPhone Rest Wrapper

-
ihttp://github.com/akosma/iphonerestwrapper/tree/master

-
iPhone XML Parsing


Cocoa Stream Parser

-
http://developer.apple.com/documentation/Cocoa/Conceptual/XMLParsing/XMLPa
rsing.html

-
SOAP with iPhone examples

-
http://icodeblog.com/2008/11/03/iphone
-
programming
-
tutorial
-
intro
-
to
-
soap
-
web
-
services/

-
http://developer.omniture.com/node/321

-
Façade Server landscape


O’REILLY


Windows platform

-
http://www.ondotnet.com/pub/a/dotnet/2004/02/23/mobilewebserviceapps.html

-
Connecting to Web Services


Sean Sullivan

-
http://www.slideshare.net/sullis/connecting
-
to
-
web
-
services
-
on
-
android

Appendix C
-

Efficient XML Interchange (EXI)


W3C Working Draft 19 September 2008
-

http://www.w3.org/TR/2008/WD
-
exi
-
20080919/


“EXI is a very compact representation for the Extensible Markup
Language (XML) Information Set that is intended to simultaneously
optimize performance and the utilization of computational resources.”


Components of XML Complexity

1. size

2. total number of elements

3. number of unique elements

4. tree
-
height (nesting)

5. number of unique external references

6. total number of attributes

7. number of unique attributes

8. regularity, or "self
-
similarity"



Facebook

-
Notice the session key expiration

-
http://wiki.developers.facebook.com/index.php/Authorizing_Applications









Oauth


MPOAuthMobile


compiled in project library
-

Karl Adam

-
http://code.google.com/p/mpoauthconnection/wiki/MPOAuthMobile


<?xml version="1.0" encoding="UTF
-
8"?>

<
auth_getSession_response

xmlns
="http://api.facebook.com/1.0/"


xmlns:xsi
="http://www.w3.org/2001/XMLSchema
-
instance"


xsi:schemaLocation
="http://api.facebook.com/1.0/
http://api.facebook.com/1.0/facebook.xsd">



<
session_key
>5f34e11bfb97c762e439e6a5
-
8055</
session_key
>



<
uid
>8055</
uid
>


<expires>1243814400</expires>

</
auth_getSession_response
>

Appendix D
-

Other Auth Examples