JOURNEY TO NOSQL

deliriousattackInternet and Web Development

Dec 4, 2013 (3 years and 10 months ago)

113 views

JOURNEY TO
NOSQL
Will
Iverson
CTO, Dynacron Group

Background

Java since 1995

Consulting since 1999

Four
Java Books

Hibernate
: A J2EE Developer’s Guide
, 2004

CTO,
Dynacron Group, 2010-Present

Several
NoSQL

PoCs

Brian
and Dustin on details

Trends & Context

Hibernate

MyBatis

NoSQL

Big Data &
NoSQL
= related, but not same thing

Both are now famous!

Hibernate
ORM &
MyBatis

Hibernate

Map classes to tables

Use HQL/Criteria queries

Pitch: Don’t have to learn SQL!

Reality: Learn Java, SQL, and HQL!

Debugging very difficult

MyBatis

Write interfaces

Add SQL via
annotations

For more on Hibernate vs.
MyBatis
, see video of last talk on
SeaJUG.org
Most interesting:
decoupling application
from RDBMS

MyBatis
Decoupling?

Contract with data/persistence is just an interface

Easy to mock

Easy to understand

Easy to… replace…

NoSQL
http://cdi-
mdm.blogspot.com
/2011/07/
nosql
-
newsql
-and-
mdm.html

NoSQL
Popularity

Today’s Two Examples

MongoDB

Document Store

Order Example

Neo4j

Graph walking

Different Tools

…very complimentary
From http
://www.neo4j.org/develop/
example_data
… and the BBC

Final Thoughts
Full Stack JavaScript
Angular.js
or
Ember.js
Node.js
MongoDB

Final Thoughts
Favorite
(Personal) Stack Today?
Persistence:
MongoDB
Plain Old Java + Maven
Interfaces
in JSON, easy, fast
Jackson FTW
Add
Ratpack
(HTTP)
http://
www.ratpack-
framework.org
/
Angular.js

vs

Ember.js
?
FIGHT!
http://
pastordonblog.blogspot.com
/2010/09/
stephen
-hawking-is-probably-best-
known.html

MONGODB
High Level Overview
Prepared by Brian Kereszturi, Dynacron Group

MongoDB
: What is it?

Document-oriented


a scalable, high-performance,
open source
NoSQL

database”

Auto-
Sharding

Distributed writes

Replica Sets

High Availability

Distributed reads

Low cost of ownership

Commodity hardware

Minimal administration

MongoDB
: Data Structure

BSON (Binary JSON) Documents

Schema-less

Secondary
indexes

MongoDB
: Use Cases/
Strengths

Well-suited for large data sets

Craigslist (10TB, 5B records)

Wordnik
(3.5TB, 20B records)

Disney 1400 instances

Always fetch an object with sub-objects

High volume reads/writes

Latency as low as .1ms

Map Reduce

Aggregation

MongoDB
: Challenges

Data
modeling

What questions do I need to answer up front?

Selecting a Shard Key

Multi-datacenter replication

Time-delayed replicas

Hidden replicas

Elections

Arbiter

MongoDB
: Topology

NEO4J
Developer Overview

What is
Neo4j?
It’s a Graph Database!

Say What?

GRAPH
database

Nodes, relationships, properties

Shines with complex, highly connected data

Social
n
etworks

R
ecommendations

Path finding

Graph
DATABASE

Reliable: ACID Compliance, High availability

Scalable: 32B nodes and edges, 64B properties

Accessible: REST API, Embeddable on JVM



Querying

Cypher Query Language

Best for ad-hoc querying

SQL-like language

REST interface

Easy to copy-paste in email

“Prepared” statements

Traversal API

Best for high-performance querying

Custom JAX-RS plugin

Java code

More powerful

Lower latency

Clean REST interface








Cypher

Start-Match-Where-Return

START root=node(0)
RETURN root

START root=node(100)
MATCH (root)-[:has]->(child)
RETURN child

START me=
node:lookup
("name=
dustin
")
MATCH me-[
f:friend
]->(friend)
WHERE
friend.gender
=‘M’
AND
f.date
< ‘2012-01-01’
RETURN
friend.name
,
f.date

Traversal API

Domain Layers

Spring Data

Collaboration with
SpringSource

Annotation/
AspectJ
-driven


Qi4j, jo4neo, …


Performance

> 1 billion nodes, > 1 billion relationships, > 3
billion properties

< 10ms query time on average

< 100ms query time, 99
t
h
percentile

4000
req
/sec on 3 beefy servers

16-core, 256GB ram, 1.1TB SSD in Raid0+1

Demands

Practically begs for SSD

Not horizontally scalable

Add more machines for read scaling

Tuning is VERY important.

Order of magnitude speed increase letting memory-mapped
IO consume almost all system resources

Exploring Data

BEER GOOD

Wikipedia collection of Belgian Beers


Q&A