红色有角F叔对《ZeroMQ》的笔记(6)

红色有角F叔
红色有角F叔 (次元の呪い)

在读 ZeroMQ

ZeroMQ
  • 书名: ZeroMQ
  • 作者: Pieter Hintjens
  • 副标题: Messaging for Many Applications
  • 页数: 516
  • 出版社: O'Reilly Media
  • 出版年: 2013-3-28
  • I/O Threads
    We said that ØMQ does I/O in a background thread. One I/O thread (for all sockets) is sufficient for all but the most extreme applications. When you create a new context, it starts with one I/O thread. The general rule of thumb is to allow one I/O thread per gigabyte of data in or out per second. To raise the number of I/O threads, use the zmq_ctx_set() call before creating any sockets
    2016-03-19 21:19:26 回应
  • Handing Errors and ETERM
    ZeroMQ's error handling philosophy is a mix of fail-fast and resilience. Processes, we believe, should be as vulnerable as possible to internal errors, and as robust as possible against external attacks and errors. To give an analogy, a living cell will self-destruct if it detects a single internal error, yet it will resist attack from the outside by all means possible.
    If it is unclear whether a fault is internal or external, that is a design flaw to be fixed.
    2016-03-19 22:25:02 回应
  • Multithreading with ZeroMQ
    If there's one lesson we've learned from 30+ years of concurrent programming, it is: just don't share state.
    You should follow some rules to write happy multithreaded code with ZeroMQ: - Isolate data privately within its thread and never share data in multiple threads. The only exception to this are ZeroMQ contexts, which are threadsafe. - Stay away from the classic concurrency mechanisms like as mutexes, critical sections, semaphores, etc. These are an anti-pattern in ZeroMQ applications. - Create one ZeroMQ context at the start of your process, and pass that to all threads that you want to connect via inproc sockets. - Use attached threads to create structure within your application, and connect these to their parent threads using PAIR sockets over inproc. The pattern is: bind parent socket, then create child thread which connects its socket. - Use detached threads to simulate independent tasks, with their own contexts. Connect these over tcp. Later you can move these to stand-alone processes without changing the code significantly. - All interaction between threads happens as ZeroMQ messages, which you can define more or less formally. - Don't share ZeroMQ sockets between threads. ZeroMQ sockets are not threadsafe. Technically it's possible to migrate a socket from one thread to another but it demands skill. The only place where it's remotely sane to share sockets between threads are in language bindings that need to do magic like garbage collection on sockets.
    inproc 的 socket 看起来比较像 golang 的 channel;里面提到 "attached thread" / "PAIR sockets over inproc." 目测用起来会比较像协程或者管道。
    ZeroMQ uses native OS threads rather than virtual "green" threads. The advantage is that you don't need to learn any new threading API, and that ZeroMQ threads map cleanly to your operating system. You can use standard tools like Intel's ThreadChecker to see what your application is doing.
    gevent 这类“绿色线程”的协程库太 overrate 了,不过 python 生态圈也没有什么其他选择。
    2016-03-19 23:05:18 1人推荐 回应
  • Zero-copy
    You should think about using zero-copy in the specific case where you are sending large blocks of memory (thousands of bytes), at a high frequency. For short messages, or for lower message rates, using zero-copy will make your code messier and more complex with no measurable benefit. Like all optimizations, use this when you know it helps, and measure before and after.
    2016-03-20 12:55:13 回应
  • High-Water Marks
    ZeroMQ uses the concept of HWM (high-water mark) to define the capacity of its internal pipes. Each connection out of a socket or into a socket has its own pipe, and HWM for sending, and/or receiving, depending on the socket type. Some sockets (PUB, PUSH) only have send buffers. Some (SUB, PULL, REQ, REP) only have receive buffers. Some (DEALER, ROUTER, PAIR) have both send and receive buffers.
    In ZeroMQ v2.x, the HWM was infinite by default. This was easy but also typically fatal for high-volume publishers. In ZeroMQ v3.x, it's set to 1,000 by default, which is more sensible
    When your socket reaches its HWM, it will either block or drop data depending on the socket type. PUB and ROUTER sockets will drop data if they reach their HWM, while other socket types will block.
    2016-03-20 14:33:07 回应
  • Missing Message Problem Solver
    - If you start the SUB socket (i.e., establish a connection to a PUB socket) after the PUB socket has started sending out data, you will lose whatever it published before the connection was made. If this is a problem, set up your architecture so the SUB socket starts first, then the PUB socket starts publishing. - Even if you synchronize a SUB and PUB socket, you may still lose messages. It's due to the fact that internal queues aren't created until a connection is actually created. If you can switch the bind/connect direction so the SUB socket binds, and the PUB socket connects, you may find it works more as you'd expect.
    如果 SUB socket 的建立在 PUB 发送数据之后,会丢消息。应对:通过某种同步手段先建立 SUB socket 再允许 PUB socket 发送消息;
    即使先建立 SUB socket 再建立 PUB socket,依然会丢消息。应对:让 SUB 作服务端来 bind,PUB 作客户端来 connect。
    If you're using PUSH sockets, you'll find that the first PULL socket to connect will grab an unfair share of messages.
    If you're using inproc, make sure both sockets are in the same context.
    2016-03-20 14:52:34 回应

红色有角F叔的其他笔记  · · · · · ·  ( 全部654条 )

注定一战
1
美国反对美国
1
哲学·科学·常识
1
计算机组成(第 6 版)
2
图解TCP/IP(第5版)
1
沸腾十五年
2
重新理解创业
8
雄性衰落
3
股市真规则
1
资本和收入的性质
2
存在主义是一种人道主义
3
程序员的职业素养
1
何为良好生活
1
活出生命的意义
3
货币的教训
3
Docker——容器与容器云(第2版)
2
政治的人生
4
中国巨债
3
深入浅出React和Redux
5
历史的教训
4
聪明的投资者
8
Designing Data-Intensive Applications
4
投资中最简单的事
5
供给的逻辑
1
逃不开的经济周期
1
图解服务器端网络架构
1
斯坦福极简经济学
3
政治的逻辑
4
原则
5
大数据之路
1
在苍茫中传灯
4
巴菲特传(纪念版)
1
中产阶级如何保护自己的财富
1
指数基金投资指南
4
模式分类
2
深度学习
1
我看电商
2
数据挖掘导论
1
中国国家治理的制度逻辑
2
漫步华尔街
2
尽在双11:阿里巴巴技术演进与超越
2
共同基金常识
3
企业IT架构转型之道:阿里巴巴中台战略思想与架构实战
6
未来简史
2
MySQL DBA修炼之道
1
大国大城
2
计算广告
4
机器学习
1
集体智慧编程
1
重新定义公司
1
Hadoop应用架构
1
第二性
6
硅谷钢铁侠
1
大数据
5
经营的本质
1
人人都是产品经理
7
你凭什么做好互联网
4
Spark机器学习
2
聊聊架构
8
游戏引擎架构
1
美国大城市的死与生(纪念版)
5
给大家看的Photoshop讲座
1
技术的本质
5
我们房地产这些年
2
行动的勇气
2
合作的进化
5
马克斯·韦伯与德国政治:1890—1920
6
数据库索引设计与优化
1
精益企业
7
高可用MySQL
2
发布!软件的设计与部署
2
项目管理艺术
2
右派国家
5
现实感
4
领域驱动设计
11
从0到1
1
高效程序员的45个习惯
1
可扩展的艺术
3
空之境界 上
1
成为技术领导者
1
改革的逻辑
3
修改代码的艺术
9
恰如其分的软件架构
7
软件开发者路线图
3
实现领域驱动设计
1
21世纪资本论
9
持续交付
16
构建之法
6
黑格尔导论
19
极端的年代
1
微服务设计
10
Site Reliability Engineering
5
测试驱动的面向对象软件开发
3
城市的胜利
2
对知识的恐惧
5
现代经济学主要流派
7
数学之美
2
程序员的思维修炼
1
大教堂与集市
1
一切坚固的东西都烟消云散了
5
兜售繁荣
1
数据科学与工程技术丛书
1
政治的细节(第10版)
8
发展研究指南(第二版)
2
代码大全(第2版)
2
企业应用架构模式
9
The Datacenter as a Computer
3
无情的革命
6
新教伦理与资本主义精神
3
人类简史
7
Understanding MySQL Internals
2
他改变了中国
1
态度改变与社会影响
4
复杂
2
民主新论
19
人件
2
国家的常识
4
乌合之众
3
Web Operations
2
个人印象
4
湖上闲思录
2
自由及其背叛
7
C++语言的设计与演化
8
百年中国经济史笔记
1
改变
4
创新与企业家精神
5
Cassandra
3
不敢止步
4
意志力
2
通向财务自由之路
1
制造同意
6
美国种族简史
4
NoSQL Distilled
4
理解专业程序员
2
一个自由主义者的良知
4
政治经济学要义
2
施瓦辛格健身全书
2
房地产的繁荣与萧条
5
为学十六法
2
Akka in Action
1
Java虚拟机并发编程
3
软件工艺
3
面向模式的软件架构,卷3
1
动物精神
4
非理性繁荣
10
MongoDB权威指南
2
海量数据库解决方案
1
Erlang/OTP并发编程实战
1
学术与政治
12
Java并发编程实战
16
论中国
3
金融炼金术
4
多处理器编程的艺术
1
Effective java 中文版(第2版)
1
中國近代史(下冊)
6
系统之美
6
压力下的角逐
2
古代东方史
1
Go 语言程序设计
1
Remote
1
深入Linux内核架构
2
中國近代史(上冊)
3
隐秩序
1
空之境界(上下集合售)
1
开放社会
4
中国近代史八种
5
喀提林阴谋 朱古达战争
1
政治秩序的起源
5
现代性的后果
2
失去的胜利
9
了不起的盖茨比
5
许倬云说历史:台湾四百年
2
大规模分布式存储系统
1
C++网络编程(卷1)
2
在约定的场所
1
中国的宗教
2
了不起的盖茨比
1
希腊罗马名人传(全三册)
2
自私的基因
2
学龠
1
中国政治思想史
4
列克星敦的幽灵
1
人月神话
2
现代体系结构上的UNIX系统
1
虚拟机
2
朱熹的历史世界
1