From a code readability perspective, this solution is ugly. It requires dozens of lines of code and four passes through the data. However, we expect that it will avoid memory errors on the executors and complete faster than the `groupByKey` or secondary sort solutions. This is because the data on each column should be mostly distinct doubles, and thus the shuffle should be fairly efficient. The final two `mapPartitions` routines involve reducing the data and can be achieved through iterator-to-iterator transformations, so we expect them to scale well. Indeed, on randomly distributed test data with many records, this solution outperforms the other implementations of the Goldilocks problem by orders of magnitude. (查看原文)
0 有用 不会飞的超人 2019-10-12 00:18:05
简略
1 有用 Gabriel Fu 2018-08-09 13:40:24
目前看到的最全面最有用的讲解spark的书籍
0 有用 阿凡达弟弟 2022-05-31 11:34:33
吃透那几个例子对spark理解算是非常深入了。
0 有用 冰镇绿豆汤 2019-06-14 21:39:38
进阶提升，查漏补缺
0 有用 我霸天君不服 2023-05-17 00:42:08 四川
翻译垃圾，纯机器翻译。内容也一般，没讲什么东西