As per the official documentation of Flink https://ci.apache.org/projects/flink/flink-docs-release-1.3/concepts/runtime.html and https://ci.apache.org/projects/flink/flink-docs-release-1.3/internals/components.html, the following are some of the processes and basic components of Flink.
Processes: There are two types of processes in Flink, as shown in the following figure:
- JobManagers (Master): It is responsible for distributing the task among the TaskManagers, maintaining check points and for recovery in case of any failure. To achieve high availability, we need to set up more than one JobManager where one acts as a leader and the other as standby. The leader and standby JobManager are always in sync and in case of leader failure, the standby JobManager becomes the leader.
- TaskManagers (workers): TaskManagers are responsible for executing the tasks, maintaining buffers for operators, and so on. TaskManager is a JVM process and subtasks are executed in multiple...