分享

Flink学习

1、Lazy Evaluation
All Flink programs are executed lazily: When the program’s main method is executed, the data loading and transformations do not happen directly. Rather, each operation is created and added to the program’s plan. The operations are actually executed when the execution is explicitly triggered by an execute() call on the execution environment. Whether the program is executed locally or on a cluster depends on the type of execution environment
The lazy evaluation lets you construct sophisticated programs that Flink executes as one holistically planned unit.

所有的Flink程序执行时懒洋洋地:程序的主要方法是执行,数据加载和转换不直接发生。相反,每个操作被创建并添加到程序的计划中.。操作实际上执行时执行明确的执行环境的execute()呼叫触发。该程序在本地还是在群集上执行取决于执行环境的类型.

2、Specifying Keys
Some transformations (join, coGroup, keyBy, groupBy) require that a key be defined on a collection of elements. Other transformations (Reduce, GroupReduce, Aggregate, Windows) allow data being grouped on a key before they are applied.
分组的key被定义为一个集合中的一个元素,可以是单个字段,也可以是组合字段(多个字段组成的Tuple)

Image (1).png
3、Accumulators


4、并行机制

(1)operator的并发

Image [2].png

(2)Execution Environment Level 执行环境的并发
Image [5].png

6、Iteration Operators
Image [6].png

Image [7].png

7、Broadcast Variables和Distributed Cache
Image [8].png
Image [9].png

Image [1].png

本帖被以下淘专辑推荐:

已有(1)人评论

跳转到指定楼层
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条