红色有角F叔对《The Datacenter as a Computer》的笔记(3)

The Datacenter as a Computer
  • 书名: The Datacenter as a Computer
  • 作者: Urs Hoelzle/Luiz Andre Barroso
  • 副标题: An Introduction to the Design of Warehouse-Scale Machines
  • 页数: 120
  • 出版社: Morgan and Claypool Publishers
  • 出版年: 2009-5-29
  • Introduction
    They differ significantly from traditional datacenters: they belong to a single organization, use a relatively homegeneous hardware and system software platform, and share a common systems management layer.
    Most importantly, WSCs run a smaller number of very large applications(or Internet Services).
    The relentless demand for more computing capabilities makes cost efficiency a primary metric of interest in the design of WSCs.
    However, network switches with high port counts, which are needed to tie together WSC clusters, have a much different price structure and are more than 10 times more expensive (per 1Gps port) than comodity switches.
    A switch that has 10 times the bi-section bandwidth costs about 100 times as much. As a result of this cost discontinuity, the networking fabric of WSCs is often organized as the two level hierarchy despicted on Figure1.1
    每个 rack 一个低端交换机,上面一个高端的核心交换机。
    In such a network, programmers must be aware of the relatively scarce cluster-level bandwidth resources and try to exploit rack level networking locality.
    one can remove some of the cluster level networking bottlenecks by spending more money on the interconnect fabric. for example, Infiniband interconnects typically scales to a few thousand ports but can cost $500~$2000 per port.
    Alternatively, lower-cost fabrics can be formed from commodity Ethernet switches by building "fat tree" Clos network.
    现在该 clos 布局弄的比较多,局部性是需要适应的现状,也可以看做一个可以解决的问题。
    A large application that requires many more servers than can fit on a single rack must deal effectively with these large discrepancies in latency, bandwidth, and capacity.
    these discrepancies are much larger than those seen on a single machine, making it more difficult to program a WSC.
    key challenge for architects of WSCs is to smooth out these discrepancies in a cost efficient manner. conversely, a key challenge for software architects is to build cluster infrastructure and services that hide most of this complexity from application developers
    存储层次的异构是复杂性的一个来源
    Although this breakdown can vary significantly depending on how systems are configured for a given workload domain, the graph indicates that CPUs can no longer be the sole focus of en- ergy efficiency improvements because no one subsystem dominates the overall energy usage profile. .
    CPU 不再是耗能的主要来源
    2015-10-01 22:44:17 回应
  • Workloads and Software Infrastructure
    Typical Internet services exhibit a large amount of parallelism stemming from both data- and request-level parallelism. Usually, the problem is not to find parallelism but to manage and efficiently harness the explicit parallelism that is inherent in the application
    WSC 中的应用固有无限的数据级、请求级的并行。最困难的问题不再是挖掘并行,而是应对固有的并行。
    A beneficial side effect of this aggressive software deployment environment is that hardware architects are not necessarily burdened with having to provide good performance for immutable pieces of code. Instead, architects can consider the possibil- ity of significant software rewrites to take advantage of new hardware capabilities or devices.
    Homogeneity within a platform generation simplifies cluster-level scheduling and load balancing and reduces the maintenance burden for platforms software (kernels, drivers, etc.).
    Ideally, the cluster-level system software should provide a layer that hides most of that complexity from application-level software, although that goal may be difficult to accomplish for all types of applications.
    Although the plentiful thread-level parallelism and a more homogeneous computing platform help reduce software development complexity in Internet services compared to desktop systems, the scale, the need to operate under hardware failures, and the speed of workload churn have the opposite effect.
    简化复杂度的地方:1. 不必挖掘并行;2. 基础组件同构;
    增加复杂度的地方:1. 需要日常地应对硬件故障;2. 访问模式异构;
    Once the diverse requirements of multiple services are considered, it becomes clear that the datacenter must be a general-purpose computing system.
    2015-10-01 22:54:06 回应
  • Balanced Designs
    There is opportunity here to find solutions by software- hardware co-design, while being careful not to arrive at machines that are too complex to program.
    The most cost-efficient and balanced configuration for the hardware may be a match with the combined resource requirements of multiple workloads and not necessarily a perfect fit for any one workload
    Fungible resources tend to be more efficiently used.
    好像说在存储系统上,网络带宽和磁盘带宽是可以互换的,因而可以利用率更高。
    2015-10-03 11:01:58 回应

红色有角F叔的其他笔记  · · · · · ·  ( 全部654条 )

注定一战
1
美国反对美国
1
哲学·科学·常识
1
计算机组成(第6版)
2
图解TCP/IP(第5版)
1
沸腾十五年
2
重新理解创业
8
雄性衰落
3
股市真规则
1
资本和收入的性质
2
存在主义是一种人道主义
3
程序员的职业素养
1
何为良好生活
1
活出生命的意义
3
货币的教训
3
Docker——容器与容器云(第2版)
2
政治的人生
4
中国巨债
3
深入浅出React和Redux
5
历史的教训
4
聪明的投资者
8
Designing Data-Intensive Applications
4
投资中最简单的事
5
供给的逻辑
1
逃不开的经济周期
1
图解服务器端网络架构
1
斯坦福极简经济学
3
政治的逻辑
4
原则
5
大数据之路
1
在苍茫中传灯
4
巴菲特传(纪念版)
1
中产阶级如何保护自己的财富
1
指数基金投资指南
4
模式分类
2
深度学习
1
我看电商
2
数据挖掘导论
1
中国国家治理的制度逻辑
2
漫步华尔街
2
尽在双11:阿里巴巴技术演进与超越
2
共同基金常识
3
企业IT架构转型之道:阿里巴巴中台战略思想与架构实战
6
未来简史
2
MySQL DBA修炼之道
1
大国大城
2
计算广告
4
机器学习
1
集体智慧编程
1
重新定义公司
1
Hadoop应用架构
1
第二性
6
硅谷钢铁侠
1
大数据
5
经营的本质
1
人人都是产品经理
7
你凭什么做好互联网
4
Spark机器学习
2
聊聊架构
8
游戏引擎架构
1
美国大城市的死与生(纪念版)
5
给大家看的Photoshop讲座
1
技术的本质
5
我们房地产这些年
2
行动的勇气
2
合作的进化
5
马克斯·韦伯与德国政治:1890—1920
6
数据库索引设计与优化
1
精益企业
7
高可用MySQL
2
发布!软件的设计与部署
2
项目管理艺术
2
右派国家
5
现实感
4
领域驱动设计
11
从0到1
1
高效程序员的45个习惯
1
可扩展的艺术
3
空之境界 上
1
成为技术领导者
1
改革的逻辑
3
修改代码的艺术
9
恰如其分的软件架构
7
软件开发者路线图
3
实现领域驱动设计
1
21世纪资本论
9
持续交付
16
构建之法
6
黑格尔导论
19
极端的年代
1
微服务设计
10
Site Reliability Engineering
5
测试驱动的面向对象软件开发
3
城市的胜利
2
对知识的恐惧
5
ZeroMQ
6
现代经济学主要流派
7
数学之美
2
程序员的思维修炼
1
大教堂与集市
1
一切坚固的东西都烟消云散了
5
兜售繁荣
1
数据科学与工程技术丛书
1
政治的细节(第10版)
8
发展研究指南(第二版)
2
代码大全(第2版)
2
企业应用架构模式
9
无情的革命
6
新教伦理与资本主义精神
3
人类简史
7
Understanding MySQL Internals
2
他改变了中国
1
态度改变与社会影响
4
复杂
2
民主新论
19
人件
2
国家的常识
4
乌合之众
3
Web Operations
2
个人印象
4
湖上闲思录
2
自由及其背叛
7
C++语言的设计与演化
8
百年中国经济史笔记
1
改变
4
创新与企业家精神
5
Cassandra
3
不敢止步
4
意志力
2
通向财务自由之路
1
制造同意
6
美国种族简史
4
NoSQL Distilled
4
理解专业程序员
2
一个自由主义者的良知
4
政治经济学要义
2
施瓦辛格健身全书
2
房地产的繁荣与萧条
5
为学十六法
2
Akka in Action
1
Java虚拟机并发编程
3
软件工艺
3
面向模式的软件架构,卷3
1
动物精神
4
非理性繁荣
10
MongoDB权威指南
2
海量数据库解决方案
1
Erlang/OTP并发编程实战
1
学术与政治
12
Java并发编程实战
16
论中国
3
金融炼金术
4
多处理器编程的艺术
1
Effective java 中文版(第2版)
1
中國近代史(下冊)
6
系统之美
6
压力下的角逐
2
古代东方史
1
Go 语言程序设计
1
Remote
1
深入Linux内核架构
2
中國近代史(上冊)
3
隐秩序
1
空之境界(上下集合售)
1
开放社会
4
中国近代史八种
5
喀提林阴谋 朱古达战争
1
政治秩序的起源
5
现代性的后果
2
失去的胜利
9
了不起的盖茨比
5
许倬云说历史:台湾四百年
2
大规模分布式存储系统
1
C++网络编程(卷1)
2
在约定的场所
1
中国的宗教
2
了不起的盖茨比
1
希腊罗马名人传(全三册)
2
自私的基因
2
学龠
1
中国政治思想史
4
列克星敦的幽灵
1
人月神话
2
现代体系结构上的UNIX系统
1
虚拟机
2
朱熹的历史世界
1