关注 spark技术分享,
撸spark源码 玩spark最佳实践

LogicalPlanVisitor — Base Visitor for Computing Statistics of Logical Plan

LogicalPlanVisitor — Contract for Computing Statistic Estimates and Query Hints of Logical Plan

LogicalPlanVisitor is the contract that uses the visitor design pattern to scan a logical query plan and compute estimates of plan statistics and query hints.

Tip
Read about the visitor design pattern in Wikipedia.

LogicalPlanVisitor defines visit method that dispatches computing the statistics of a logical plan to the corresponding handler methods.

Note
T stands for the type of a result to be computed (while visiting the query plan tree) and is currently always Statistics only.

The concrete LogicalPlanVisitor is chosen per spark.sql.cbo.enabled configuration property. When turned on (i.e. true), LogicalPlanStats uses BasicStatsPlanVisitor while SizeInBytesOnlyStatsPlanVisitor otherwise.

Note
spark.sql.cbo.enabled configuration property is off, i.e. false by default.
Table 1. LogicalPlanVisitors
LogicalPlanVisitor Description

BasicStatsPlanVisitor

SizeInBytesOnlyStatsPlanVisitor

Table 2. LogicalPlanVisitor’s Logical Operators and Their Handlers
Logical Operator Handler

Aggregate

visitAggregate

Distinct

visitDistinct

Except

visitExcept

Expand

visitExpand

Filter

visitFilter

Generate

visitGenerate

GlobalLimit

visitGlobalLimit

Intersect

visitIntersect

Join

visitJoin

LocalLimit

visitLocalLimit

Pivot

visitPivot

Project

visitProject

Repartition

visitRepartition

RepartitionByExpression

visitRepartitionByExpr

ResolvedHint

visitHint

Sample

visitSample

ScriptTransformation

visitScriptTransform

Union

visitUnion

Window

visitWindow

Other logical operators

default

赞(0) 打赏
未经允许不得转载:spark技术分享 » LogicalPlanVisitor — Base Visitor for Computing Statistics of Logical Plan
分享到: 更多 (0)

关注公众号:spark技术分享

联系我们联系我们

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

微信扫一扫打赏