spark sql get result time larger than compute Duration

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

spark sql get result time larger than compute Duration

wkhapy_1
get result 1.67s
<http://apache-spark-user-list.1001560.n3.nabble.com/file/t3966/wpLV3.png>

compute cost 0.2s
<http://apache-spark-user-list.1001560.n3.nabble.com/file/t3966/Kl0VG.png>

below is sql select event_date, dim ,concat_ws('|',collect_list(result))
result from ( select event_day event_date , '' dim
,concat_ws(',',result,event) result from ( select event_day
,event,count(uid) result from (select uid,event_day ,event ,uid from
usereventattr1 where ( city ='a' ) and ( event='WENJUANWANG__SUBMIT' ) )
usereventattrchild group by event ,event_day union all select event_day
,event,count(uid) result from (select uid,event_day ,event ,uid from
usereventattr1 where ( city ='a' ) and ( event='WECHAT__SUBSCRIBE' ) )
usereventattrchild group by event ,event_day ) xx) ab group by
dim,event_date

explain get result also cost 1.4s
<http://apache-spark-user-list.1001560.n3.nabble.com/file/t3966/6p0Em.png>
explain get result also cost 1.4s

anybody kown why get result time large than compute 1.4s



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]