spark-ec2 script with Tachyon

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
nit
Reply | Threaded
Open this post in threaded view
|

spark-ec2 script with Tachyon

nit
This post was updated on .
Hi,

It seems that spark-ec2 script deploys Tachyon module along with other setup.
I am trying to use .persist(OFF_HEAP) for RDD persistence, but on worker I see this error
--
 Failed to connect (2) to master localhost/127.0.0.1:19998 : java.net.ConnectException: Connection refused
--

From netstat I see that worker is connected to master node on port 19998
--
Proto Recv-Q Send-Q Local Address               Foreign Address             State
tcp        0      0 ip-10-16-132-190.ec2.:49239 ip-10-158-45-248.ec2.:19998 ESTABLISHED
--

Does Tachyon on EC2 work out of the box? or does it requite further configuration ?

Am I supposed to set  "spark.tachyonStore.url" to Masters IP ?
mrm
Reply | Threaded
Open this post in threaded view
|

Re: spark-ec2 script with Tachyon

mrm
Hi,

Did you manage to figure this out? I would appreciate if you could share the answer.