File Not Found: /tmp/spark-events in Spark 3.0

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

File Not Found: /tmp/spark-events in Spark 3.0

ArtemisDev
While launching a spark job from Zeppelin against a standalone spark
cluster (Spark 3.0 with multiple workers without hadoop), we have
encountered a Spark interpreter exception caused by a I/O File Not Found
exception due to the non-existence of the /tmp/spark-events directory. 
We had to create the /tmp/spark-events directory manually in order to
resolve the problem.

As a reference, the same notebook code run on Spark 2.4.6 (also a
standalone cluster) without any problems.

What is /tmp/spark-events for and is there anyway to pre-define this
directory as some config parameter so we don't end up manually add it in
/tmp?

Thanks!

-- ND


---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: File Not Found: /tmp/spark-events in Spark 3.0

Jeff Evans
This should only be needed if the spark.eventLog.enabled property was set to true.  Is it possible the job configuration is different between your two environments?

On Mon, Jun 29, 2020 at 9:21 AM ArtemisDev <[hidden email]> wrote:
While launching a spark job from Zeppelin against a standalone spark
cluster (Spark 3.0 with multiple workers without hadoop), we have
encountered a Spark interpreter exception caused by a I/O File Not Found
exception due to the non-existence of the /tmp/spark-events directory. 
We had to create the /tmp/spark-events directory manually in order to
resolve the problem.

As a reference, the same notebook code run on Spark 2.4.6 (also a
standalone cluster) without any problems.

What is /tmp/spark-events for and is there anyway to pre-define this
directory as some config parameter so we don't end up manually add it in
/tmp?

Thanks!

-- ND


---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: File Not Found: /tmp/spark-events in Spark 3.0

Jinhan Xin
In reply to this post by ArtemisDev
Hi,

First, the '/tmp/spark-events' is the default storage location of spark
eventLog, but the log will be stored in it only when the
'spark.eventLog.enabled' is true, which your spark 2.4.6 may set to false.
So you can try to set false and the error may disappear.

Second, I suggest enable eventLog and you can set the storage location by
set  'spark.eventLog.dir' to a fileSystem or local path, in case you want to
check the log later.(can simplely use spark-history-server)

Regards,
Jinhan



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: File Not Found: /tmp/spark-events in Spark 3.0

lafeier
In reply to this post by ArtemisDev

This could be the result of you not setting the location of eventLog properly. By default, it's/TMP/Spark-Events, and since the files in the/TMP directory are cleaned up regularly, you could have this problem.

------------------ Original ------------------
From: "Xin Jinhan"<[hidden email]>;
Date: Thu, Jul 2, 2020 08:39 PM
To: "user"<[hidden email]>;
Subject: Re: File Not Found: /tmp/spark-events in Spark 3.0

Hi,

First, the /tmp/spark-events is the default storage location of spark
eventLog, but the log is stored only when you set the
'spark.eventLog.enabled=true', which maybe your spark 2.4.6 set to false. So
you can just set it to false and the error will disappear.
 
Second, I suggest to open the eventLog and you can specify the log location
with 'spark.eventLog.dir' either a filesystem or local one, because you
maybe to check the log later.(can simplely use spark-history-server)

Regards
Jinhan



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: File Not Found: /tmp/spark-events in Spark 3.0

ArtemisDev

Thank you all for the responses.  I believe the user shouldn't be worried about creating the log dir explicitly.  The event logging should behave like other logs (e.g. master or slave) that the directory should be automatically created if not exist.

-- ND

On 7/2/20 9:19 AM, Zero wrote:

This could be the result of you not setting the location of eventLog properly. By default, it's/TMP/Spark-Events, and since the files in the/TMP directory are cleaned up regularly, you could have this problem.

------------------ Original ------------------
From: "Xin Jinhan"[hidden email];
Date: Thu, Jul 2, 2020 08:39 PM
To: "user"[hidden email];
Subject: Re: File Not Found: /tmp/spark-events in Spark 3.0

Hi,

First, the /tmp/spark-events is the default storage location of spark
eventLog, but the log is stored only when you set the
'spark.eventLog.enabled=true', which maybe your spark 2.4.6 set to false. So
you can just set it to false and the error will disappear.
 
Second, I suggest to open the eventLog and you can specify the log location
with 'spark.eventLog.dir' either a filesystem or local one, because you
maybe to check the log later.(can simplely use spark-history-server)

Regards
Jinhan



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]