1
January 23, 2007 1
SDIMS: A Scalable Distributed
Information Management System
Courtesy of Praveen Yalagandula (UT Austin)
Varun Singhal
January 23, 2007 2
Goal
A Distributed Operating System Backbone
Information collection and management
Core functionality of large distributed systems
Monitor, query and react to changes in the system
Examples:
Benefits
Ease development of new services
Facilitate deployment
Avoid repetition of same task by different services
Optimize system performance
System administration and management
Service placement and location
Sensor monitoring and control
Distributed Denial-of-Service attack
detection
File location service
Multicast tree construction
Naming and request routing
…………
January 23, 2007 3
Contributions – SDIMS
Provides an important basic building block
Information collection and management
Satisfies key requirements
Scalability
• With both nodes and attributes
• Leverage Distributed Hash Tables (DHT)
Flexibility
• Enable applications to control the aggregation
• Provide flexible API: install, update and probe
Autonomy
• Enable administrators to control flow of information
• Build Autonomous DHTs
Robustness
• Handle failures gracefully
• Perform re-aggregation upon failures
– Lazy (by default) and On-dem and (optional)
January 23, 2007 4
Outline
SDIMS: a distributed operating system backbone
Aggregation abstraction
Our approach
Design
Prototype
Simulation and experimental results
Conclusions
January 23, 2007 5
Aggregation Abstraction
Attributes
Information at machines
Aggregation tree
Physical nodes are leaves
Each virtual node represents a logical group of nodes
• Administrative domains, groups within domains, etc.
Aggregation function, f, for attribute A
Computes the aggregated value A
i
for level-i subtree
• A
0
= locally stored value at the physical node or NULL
• A
i
= f(A
i-10
, A
i-11
, …, A
i-1k
) for virtual node with k children
Each virtual node is simulated by one or more machines
Example: Total users logged in the system
Attribute: numUsers
Aggregation Function: Summation
abc d
A
0
A
1
A
2
f(a,b) f(c,d)
f(f(a,b), f(c,d))
4122
54
9
Astrolabe [VanRenesse et al TOCS’03]
January 23, 2007 6
Outline
SDIMS: a distributed operating system backbone
Aggregation abstraction
Our approach
Design
Prototype
Simulation and experimental results
Conclusions
