MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Total Order in Content-basedPublish/Subscribe Systems
Joint work with:
Vinod Muthusamy,
Hans-Arno Jacobsen
University of Toronto
http://www.padres.msrg.utoronto.ca
ICDCS 2012, June 19th 2012
Kaiwen Zhang
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
2
Outline
Background: publish/subscribe
Total order semantics
Related work: sequencers
Natural pairwise total order
Our two-phase solution
Performance evaluation
Comparison withGroup communication systems
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
3
Advertisement-Based Pub/Sub
3
broker
match &
forward
Advertisement path
Subscription path
Publication path
publisher
name = `IBM’
price = $40
subscriber
name = `IBM’price > $20
subscriber
name = `IBM’price > $30
name = ‘IBM’
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
4
Pairwise Total Order (PTO)for Pub/Sub
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
5
Application Scenarios
Little work on total order in pub/sub so far
Subscriber decoupling
Minimize overhead
External communication between peers
Comparing event streams
Consistency
Online games (non-commutative operations)
Monitoring (stream ordering)
Fairness
Stock quote (detecting event patterns)
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
6
Drawbacks of Sequencers
Sequencer networks [Lumezanu 2006]
Topic-based
Global knowledge
Sequencing overhead appliedto all pubs and subs
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
7
Per-Publisher Ordering usingFIFO Links
Per-Publisher TO
Maintaining order per publisher (i.e. FIFO order)
Per-Publisher TO is maintained usingFIFO links between overlay neighbours
Requires unique paths to each sub!
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
8
Natural Pairwise TO(N-PTO)
Using FIFO links, a common broker for allpublisher/subscriber pairs provides N-PTO.
Consider that broker as a “super-publisher”
Per-publisher TO is then enforced from that broker on
Consider P1 and P2 as internal to
the publisher B0
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
9
Our Solution
Uses FIFO links
N-PTO provided by a common broker
Two phases:
Conflict detection
Identify pairs of disjoint publisher/subscriber paths
Resolution
Defer delivery only to affected subscribers
Lightweight solution for content-based P/S
Uses local ads/subs broker knowledge(next-hops)
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
10
No Conflict
P2 doesn't match S4
No conflict
S1-S3 share same next-hop as P2
No conflicts detected at B1
S1-S3 have different
next hop from P2:
B3 is a common broker
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
11
Conflict Resolution
S4 forwarded
immediately
S1 has a different next-hop from
S2-3 and P2:
Their paths are disjoint!
Defer delivery to S1, wait for ack
Conflict for S3, defer until ack
Detects that P2 and S2
have different next-hops:
Send ack back
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
12
Properties of the Solution
System model:
Acyclic topology
FIFO links
Advertisement-based
Solution is safe under crash failures
Theorem: “Pairwise total order is impossible inpublish/subscribe systems under the presence offailures.”
Optimizations:
Progressive delivery
Faster detection
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
13
Evaluation
Implementation using PADRES in Java
Modular; requires only few changes to the core
Workload
Overlapping subscriptions; stress test
High conflict rates: 25-100%
Topology
24 to 96 brokers
800 subscribers, 240 publishers
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
14
Low Detection Overhead
Overall, still small
impact to non-conflict
subs
Overhead increases
with less conflict
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
15
Scalability over Baseline
Central sequencer
Our solution
No total order
Throughput saturated
for central sequencer
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
16
Performance over Spread(Group Communication Systems)
Our solution
Spread w/o ord.
Outperformed after
a certain size
Spread w/ TO
(token algorithm)
Overhead due to
TO
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
17
Conclusions & Future Work
Lightweight solution for content-based p/s
Local broker knowledge
Contained resolution overhead
Extension for cyclic topologies
Adjust routing paths
Pub/sub clustering and client mobility
Maximize natural ordering
MIDDLEWARE SYSTEMS
RESEARCH GROUP
MSRG.ORG
Kaiwen Zhang, Vinod Muthusamy, Hans-Arno Jacobsen
18
Thank you!