how to manage HBase connections in Executors of Spark Streaming ?

[big data]

Hi,

Does any best practices about how to manage Hbase connections with kerberos authentication in Spark Streaming (YARN) environment?

Want to now how executors manage the HBase connections,how to create them, close them and refresh Kerberos expires.

Thanks.

[Kevin Pis]

1. the issue about that Kerberos expires.
1. You don’t need to care aboubt usually, you can use the local keytab at every node in the Hadoop cluster.
2. If there don’t have the keytab in your Hadoop cluster, you will need update your keytab in every executor periodically。
2. best practices about how to manage Hbase connections with kerberos authentication, the demo.java is the code about how to get the hbase connection.

 

 

 

 

From: big data <[hidden email]>
Date: Tuesday, November 24, 2020 at 1:58 PM
To: "[hidden email]" <[hidden email]>
Subject: how to manage HBase connections in Executors of Spark Streaming ?

 

Hi,

Does any best practices about how to manage Hbase connections with kerberos authentication in Spark Streaming (YARN) environment?

Want to now how executors manage the HBase connections,how to create them, close them and refresh Kerberos expires.

Thanks.

---------------------------------------------------------------------
To unsubscribe e-mail: [hidden email]