Search Engine Optimization

thumbpinchInternet and Web Development

Nov 18, 2013 (3 years and 10 months ago)

84 views

Search Engine
Optimization

L.
Grewe

1

Why Search Engines?


An
estimated 77 percent+

of
Internet users

employ
search
engines

to find
Web sites



If you are
not listed

on
Google
or

Yahoo!
, you are
invisible


It
doesn’t matter

how
well known

your
brand is
!


Visitors will not guess your site name

but will
go through

Google

or

Yahoo!

just to be

safe


2

3

1.
How search
engines handle your
content

2.
How to improve
ranking of
web
pages
on search results


Objectives

4

Search Engine Producers


Google


Yahoo!


Bing


Ask.com


Major Search Engines and
Directories

5

Modified from Bruce Clay’s Search Engine Chart:
http://www.bruceclay.com/searchenginechart.pdf

Search Engine Optimization


Search Engine Optimization or SEO is
the task of designing (or
redesigning) you site (pages) to
appear higher in the list of results
from Search Engines.

6

7

Basics of Search Friendly Design

8


Keywords
: Use text that targets your
audience; and attracts search engines and
links from other sites


Navigation:

Give users and crawlers easy
access to content


Design Elements:

Make sure bells and
whistles don’t undermine SEO efforts


Page Rank:
Develop a linking strategy to
improve your site’s “popularity”

Basic SEO Concepts

9

Things Search Engines Like


Text (Keyword)


Words and phrases that match what
your target audience types into search
engines


Site Navigation


Site navigation and URL structure that
search crawlers can easily follow


Links to Your Site


How “popular” are you?

Basic Idea of how Web
Discovered


How does a Search Engine discover
the web……..



Much is proprietary



Actual algorithms can be complex
but, need to be fast!



Here is a peek into the concept….

10

Web crawlers (and robots)


Software that travel through a
website gathering information.



Search engines use these automated
SW robots to index a website.



Web crawlers are mainly used to copy all the visited pages for
later processing by a search engine that will
index

the downloaded
pages to provide fast searches. Crawlers can also be used for
automating maintenance tasks on a Web site, such as checking
links or validating
HTML

code. Also, crawlers can be used to
gather specific types of information from Web pages, such as
harvesting e
-
mail addresses (usually for
spam
).

11

Other names for Web Crawlers


Web crawlers are
ants
,
automatic indexers
,
bots
,
[1]

or
Web spiders
,
[2]

Web robots
,
[2]

or

especially in the
FOAF

community

Web
scutters
[3]
.






How it works:


it starts with a list of
URLs

to visit, called the
seeds
. As
the crawler
visits these URLs
, it
identifies all the
hyperlinks

in the page and adds them to the list
of URLs
to visit, called the
crawl frontier
. URLs from the frontier
are
recursively visited
according to a set of policies.

12

What Information Do Search
Engines Gather


No one know exactly

except each
Search Engine Company


People fear too much.



Some known things: keywords, links
to and from, content /word counts,
images, contact info (email, etc.),
media and ????

13

14

Managing Page Content

15


Search engines place “weight” on terms
according to where they are used

Keyword Prominence

Primary Text


Title tag


Body text


Anchor text


Text near the top of the
page


Headings and
Emphasized Text

Secondary Text


Alt text


Description tag


Domain name and
URL elements


16

What About Metadata?


Metadata is not a factor in most web
search engines


Except

for Title and META description tags,
crawlers ignore other META tags


Use metadata if important for your
agency’s mission or enterprise systems
(e.g., CMS, site search engine)

17

Keywords in Text


Choose one or two keywords or phases to
optimize for each page


Use language of your audience


Do not over use
-

avoid keyword stuffing


Keywords in links influence relevancy
of the pages your linking to


No “
click here
” links



18


Site’s Search Metrics



Online Advertising Tools


Yahoo!/Overture:
http://advertising.yahoo.com/


(Yahoo!
White paper

,
Case Study

,
Advertising Case Study
)


Google
:


https://adwords.google.com/o/Targeting/Explorer?__u=100000000
0&__c=1000000000&ideaRequestType=KEYWORD_IDEAS#search.n
one



Keyword Density


http://www.webjectives.com/keyword.htm


http://www.keyworddensity.com/



Google Webmaster Tools



Keyword Selection Tools

Other Keyword Selection tools


http://www.wordtracker.com/


Not directly related to web and
search engines


http://thesaurus.reference.com/thesaur
us/

19

Keyword Selection


Must relate to your business / site



Use Tools to help



DO you choose popular keywords or
more specific less popular (but, very
targeted keywords)?


Gardening or Butterfly Gardening ???



Dig Deep and then select

20

Chose your Keyword now what?


Put keywords in your site
prominently (think primary text)



Link to other pages that feature this
keyword / related information.



Have annotated media related to this
keyword(s).

21

22

Site Navigation

23


Plan how pages link to each other


Pages will not rank well if your site
does not have a navigation scheme


Navigation scheme must please
users
and

search engines


Create a crawler
-
friendly site map


Avoid dangling pages

Linking Basics

24


Keywords in links tell crawlers about
the pages to which you are linking


Keywords in links influence relevancy
of the page to which you are linking


Avoid “click here” links, instead create
links like:


Bad:
Click here

for more info on famous admirals.


Good:
Visit our
naval history

site for more info on
famous
admirals
.

Creating Links

25


Make a site with a clear hierarchy



Text Links


The most search engine friendly


Use for primary or secondary navigation


Every page should be reachable from a text link



If using graphic navigation,
use alt text



Avoid JavaScript, unless you provide a
secondary text navigation scheme

Site Navigation Schemes

26



Poor HTML coding



Image maps



Frames



JavaScript



Dynamic Pages



Flash

Problem Navigation Schemes

27


Create a Site Map


HTML page on your site with text links


Create a Sitemap


An XML file that lists your site’s URLs


Allows crawlers to analyzes information about
your site’s architecture to improve crawling


Google:
http://www.google.com/webmasters/


Yahoo:
http://siteexplorer.search.yahoo.com/


Site Maps Protocol:
http://www.sitemaps.org/


Help Crawlers Navigate

SiteMaps


The
Sitemaps

protocol allows a webmaster to inform
search engines about
URLs

on a website that are available
for crawling.


is (usually) an
XML

file that lists the URLs for a site.


This allows search engines to crawl the site more
intelligently.


Sitemaps are a URL inclusion protocol and complement
robots.txt
, a URL exclusion protocol.


Sitemaps are particularly beneficial on websites where:


some areas of the website are not available through the
browsable

interface, or


webmasters use rich
Ajax
,
Silverlight
, or
Flash

content that is
not normally processed by search engines.


28

Sitemaps


Google Sitemaps


http://www.electionsonline.us/sitemap.xml

for
example tells Google where to begin crawling
even without a site

29

<
urlset
>



<
url
>

<loc>http://www.electionsonline.us/</loc>

<
changefreq
>monthly</
changefreq
>

</
url
>


<
url
>

<loc>http://www.electionsonline.us/sitemap.htm</loc>

<
changefreq
>monthly</
changefreq
>

<priority>1.0</priority>

</
url
>


<
url
>

<loc>

http://www.electionsonline.us/products
-
and
-
prices/electionServices.htm

</loc>

<
changefreq
>monthly</
changefreq
>

</
url
>



…..MORE <
url
> ‘s


</
urlset
>

Sitemap XML protocol


Same for popular Search Engines:
Google, Yahoo!, Bing

30

31

Robots Exclusion


robots.txt
File


Place in server’s root directory


Two elements: User
-
agent, Disallow


Example:

User
-
agent: *

Disallow: /
cgi
-
bin
/

Disallow: /scripts/

Disallow: /images
/





Meta
-
Tag Robots Exclusion

<head><meta name=“robots” content=“no
index,
nofollow
”></head>

is a convention to prevent
cooperating


web spiders
/crawlers


and other
web robots

from


accessing all or part of a
website

which

is otherwise publicly viewable. The standard


is unrelated to, but can be

used in conjunction with,
Sitemaps
, a

robot
inclusion

standard for websites.

32

Managing Your
Popularity

33


A calculation assigned to a page based on the
number of links into the page


The more popular pages that link to you, the higher
your popularity rank


An inbound link is a vote for a page


An outbound link is a vote for a page you’re linking to


All search engines have different popularity
algorithms


Google’s algorithm is called Page Rank


Based on a scale from 0
-
10

What is Popularity?

34


Number and Popularity of Inbound
Links


Anchor Text


Others’ use of keywords in a link to your
site


Popularity is assigned per page, not for
the entire site


Popularity is not inherited


Need to deliberately link internal pages to
pass on PR

Popularity Factors

35


Get Listed in Yahoo!
and
DMOZ
(
www.dmoz.org



open directory
project)


Impact on popularity may vary


Helps to get noticed by
crawlers



Establish reciprocal link arrangements with
agencies covering similar topics


Reach out to state and local agencies


Syndicate your content


Reach out to professional communities of interest

Getting Links to Your Site

36

Links That Aren’t Links


Link has been created in a manner that search engines
can’t understand


Link actually points elsewhere, and not directly to your site


Example:

http://www.site.gov/ext/external.jsp?url=http://www.noaa.gov/


Link passes through a JSP program that logs the click
and re
-
directs the browser to noaa.gov


Search engines read this as an internal link to Site.gov,
not an outbound link to NOAA


Links from these sites will not help boost your search
ranking

37

Design Considerations

38


Implement bells and whistles
carefully


Use external Javascript and CSS to
decrease page load time


Avoid using Frames


Flash only pages


include skip link and
title and META description tag


Omit Session IDs when requestor is a
search engine crawler


Ensure design team understands
SEO concepts

General Considerations

39


Database
-
driven, created on the fly by asp,
cfm, php, jsp or cgi scripts


Dynamic sites are comprised of templates,
but usually without original content


When a page is viewed, the template loads
the content from the database


Parameters are added to the URL, which tells
the template to load specific content

Dynamic Pages

40

Example:

http://smithsonianstore.com/catalog/product.jsp?productId=1
4273&parentCategoryId=3151&categoryId=3152


URLs such as this are difficult for search
engines to index because they do not know
the parameters that define a unique page


The more parameters, the less likely pages
will be indexed


A database may continually feed data,
crashing your server and scaring off the
crawler

Dynamic Pages

41


Modify URLs so they don’t look like
dynamic pages, fewer parameters


Use URL re
-
write trick using
mod_rewrite (Apache)


Create static HTML pages

Search Friendly Dynamic Pages

42


Make sure PDFs contain actual text,
not images of text


Same rules for use of keywords and
phrases apply


Put the most important text in the
title, headlines


Minimize document size (> 100K)


Create optimized HTML pages for
PDFs

Optimizing PDFs

43

Site Maintenance

44


Plan
BEFORE

you embark on a re
-
design


Will the site architecture change?


Static to Dynamic?


How is the content changing?


Add SEO review as an activity to your project
plan


Make sure your contractor has SEO skills


Poor planning and execution can kill your search
rankings


Do not wait until after you re
-
designed!

Re
-
Designing Your Site

45


Try to keep the same filenames and
directory structure when redesigning



Recommended
: Set up HTTP 301 redirects
that point to the new site or pages



Not Recommended
: Add a meta
-
refresh tag
to your page header. This won’t remove
your original page from the search index

Changes to Site Architecture

46


Avoid using default 404 pages


Custom 404 pages are more user
-
friendly


Submitting all 404 URLs to the search
engine helpful, but not essential


Remember to change the HTTP status
code on custom error pages from 200
to 404


Removing Content

Updating Content


Usually when you update a page on your web site
it helps your site get a higher ranking



Not only does frequent updating get you a higher
ranking it will also assist search agents with
finding your page as new or newly updated.



So be sure to update your site as frequently as
possible (but, only appropriately).

47

48

Case Studies

49

50

Case Study: Business.gov

Term
: Advertising Regulations


#6 on Google


#8 on Yahoo


#2 on MSN


#1 on USA.gov


Page created March ‘07


Submitted to crawlers, listed in DMOZ
(
www.dmoz.org



open directory project)


0
-
78 inbound links

51

Case Study: Business.gov


Text Component
: page is well
-
optimized for “advertising” and
“regulations”


Link Component
: All text links


Popularity Component
: Inbound
links are using title submitted to
DMOZ


“advertising” and
“regulations”

52

Case Study: NOAA Fisheries

Term
: Fisheries


#1 on USA.gov


#2 on MSN


#2 on Yahoo


#3 on Google



Term
: Fish


Doesn’t show up on
the first page or
subsequent pages

53

54

Case Study: NOAA Fisheries


Text Component
: page is well
-
optimized for “fisheries” but not
“fish”


Link Component
: “pull
-
down”
menu may be a problem for some
crawlers. Add search friendly
navigation (e.g., site map)


Popularity Component
: Lots of
inbound links using “fisheries” (not
“fish”) in anchor text

55

Next Steps

1.
Visit WebContent.gov Guide to Search Engines:

http://www.usa.gov/webcontent/technology/search/search.shtml

2.
Conduct an SEO best practices review

3.
Conduct keyword analysis and determine your rank

4.
Set success measures

5.
Use Webmaster Tools offered by the search engines,
(e.g. Google Webmaster Tools, etc.)

Make SEO a routine in your web content management program

56

Resources and
Recommendations

57


Battelle, John.
The Search: How Google and Its Rivals Rewrote
the Rules of Business and Transformed our Culture
. New York.
Penguin, 2005.


Kent, Peter.
Search Engine Optimization for Dummies.

Hoboken,
NJ. Wiley, 2004.


Moran, Mike, Hunt, Bill.
Search Engine Marketing, Inc.

Upper
Saddle River, NJ. IBM Press, 2006.


Thurow
, Shari.
Search Engine Visibility.

Indianapolis, IN: New
Riders, 2003
.




FIND YOU OWN MORE recent publications


search in library


ACM/ IEEE portals.


Recommended Reading

58

SEO Forums & Events


Industry Forums

Webmaster World:
http://www.webmasterworld.com/

Search Engine Watch:
http://www.searchenginewatch.com/

Digital Point Forums:
http://forums.digitalpoint.com/

High Rankings:
http://www.highrankings.com/forum/


Industry Events

Search Engine Strategies:
http://www.searchenginestrategies.com/



Now…


Using tools monitor your site



Study your traffic reports to determine

those keywords which drive the most

traffic to your site. Then edit your site to

ensure they are near the top of the target

page and the target page is not

overwhelmed by other, less relevant,

content.

59

Some Additional SOE tools/info


http://www.seotoolset.com/tools/free_tools.html


http://www.seomoz.org/tools


http://seobook.com

60

The Next Step


Pay for it
-

Search Engine Marketing
(SEM)


Can be a “sponsored link”


Data Asset Optimization (DAO)




manage your media /data (images, video
for Search Engines


Youtube

links or ??


Social Media


Twitter,
Facebook
, MySpace and more


Mobile

61

The Next Step Continued


Press Releases (paid for on external
sites)


PR web sites that specialize in online press
releases can


help in taking the press releases outside of
your own site and


onto various web sites of relevance.


i.e.
http://prweb.com

62