Distributed System III
这一篇关于分布式文件系统的案例学习。
首先为什么需要DFS？
- Data sharing among multiple users
- User mobility
- Location transparency
- Backups and centralized management
那么DFS的一些Challenges:
- Heterogeneity (异质，不同的机器)
- Scale (大量用户)
- Security (…)
- Failures
- Concurrency
相对应的设计时需要考虑到的问题:
- Caching
- Consistency
- Naming
- Authentication and access control
Classical DFSs: NFS, AFS
| NFS | AFS | |
|---|---|---|
| Assumption | Client machine have disks, W/W and W/R sharing are rare (文件读写操作由相同用户在同一个机器上完成), Client machines are un-trusted | |
| Design/Goals | Stateless servers with smart clients. Portable access different OS. Low implementation cost. Small number of clients. Single administrative domain. | Global distributed file system, LARGE numbers of clients and servers, minimize/eliminate work per client operation |
| Architecture | Clients -Network-Servers. Use RPC to forward filesystem operation to server. | Cells correspond to adminsitrative groups, and broken into volumes(miniature file systems), Client machine has cell-server database |
| Cacheing strategy | write-through | write-back |
| Client-Side caching | Cache both clean and dirty file data and file attributes (30s + 60s). File attributes expire after 60s, data doesn’t expire. File data is checked against modified time. Dirty data buffered until file close or 30s.(in memory cache) | Callbacks: clients register with server that they have copies. Reconstruct callbacks from client if server crashes. Revalidate crashed client content from server. |
| Advantages | No network traffic can be done locally. | Lower server load. Better scalability. |
| Disadvantages | Data consistency guarantee poor. Sensitive to network latency. | Slower. |
| File Access Consistency | sometime within 30+60 seconds | session semantics for consistency |
| Results | NFS provides transparent, remote file access. Simple, portable, popular. Weak consistency semantics. Requires hefty server resources to scale. | Lower server load than NFS(read-only case commonly). May slower (disk than memory). |
| Name space construction | per-client linkage. Clients mount NFS where they want. | global name space. name space consistent across clients. |
| Location transparency | no transparency | transparency |
| Both | Central server is bottleneck. All W/R hit it at least once. | A single point of failure. Costly to make fast and reliable servers. |
| User authentication | All servers and all client workstations share the same name space. Server believes client workstation unconditionally. | Client need to authenticated themselves. |
| CAP | A, P | A, P |
- Client-side caching is a fundamental technique toimprove scalability and performance
- Timeouts and callbacks are common methods for providing (some forms of) consistency.
重要概念：CAP
CAP tradeoff: 对于DFS，consistency, availability, partition-resilience不可能同时拥有。
Modern DFSs: Coda, LBFS, etc
这节modern DFSs不算是重点，主要介绍了可以离线工作的CODA和面向低带宽网络的LBFS。CODA的重点思想是利用了Optimistic replica control和file hoarding来支持离线工作，但牺牲了Consistency。LBFS，顾名思义，在带宽低于90%传统带宽情况下也可以工作。LBFS的中心思想是将data不重叠分块，然后hash(SHA-1)每个chunk，如果data发生了改变，hash也会发生改变。这里分块的方法是Rabin fingerprint而不是按大小分块，可以尽量减少数据变更后重新hash的数据数量。另外简单介绍了Dropbox和Google两家的文件系统，有个trick是hash了数据库内容所以用户上传文件时可以避免重复上传文件，提升用户体验。顺便说一嘴，Dropbox使用AFS设计模式。
CODA
DFS for working in different places and without network connectivity.
CODA大概介绍了以下几点
- Puts scalability and availability beforedata consistency
- Assumes that inconsistent updates are very infrequent => detect conflicts when reintegrating
- Introduced disconnected operation mode byallowing cached data (weakly consistent), backedby a file hoarding database
- Limitations: Detects only W/W conflicts, no R/W(lazy consistency), no client-client sharing possible
LBFS
LBFS大概就是以下几点内容。
- Under normal circumstances, LBFS consumes90% less bandwidth than traditional file systems.
- Makes transparent remote file access a viable andless frustrating alternative to running interactiveprograms on remote machines.
- Key Ideas: Content based chunks definition, rabinfingerprints to deal with insertions/deletions,hashes to determine content changes…
参考资料: