Apache Spark User List

This forum is an archive for the mailing list user@spark.apache.org (more options) Messages posted here will be sent to this mailing list.
1234 ... 382
Topics (13348)
Replies Last Post Views
Does Spark SQL has match_recognize? by kant kodali
1
by Yeikel
Spark 2.4.3 on Kubernetes Client mode fails by David Aspegren
0
by David Aspegren
Scaling Kafka Streaming to Thousands of Partitions by hclc
0
by hclc
conflict with multiple jobs writing to different partitions but same baseDir by Koert Kuipers
0
by Koert Kuipers
Spark standalone and Pandas UDF from custom archive by Riccardo Ferrari
3
by Riccardo Ferrari
Writing to multiple Kafka partitions from Spark by femibyte
0
by femibyte
unsubcribe by NikhilP
0
by NikhilP
Spark hive table dependent on parquet version too low by 李斌松
0
by 李斌松
[spark on yarn] spark on yarn without DFS by Huizhe Wang
7
by Achilleus 003
Executors idle, driver heap exploding and maxing only 1 cpu core by ashic
1
by Nicholas Hakobian
unsubscribe by Mun, Woyou - US
0
by Mun, Woyou - US
PySpark Streaming “PicklingError: Could not serialize object” when use transform operator and checkpoint enabled by Xilang Yan
0
by Xilang Yan
[pyspark 2.3+] how to dynamically determine DataFrame partitions while writing by rishishah.star
1
by rishishah.star
[pyspark 2.3+] repartition followed by window function by rishishah.star
1
by Shraddha Shah
Similar Narrow Transformations should be chanined? by nicks29
0
by nicks29
Specifying yarn queue w/ Livy Controller Service by Varun Rao
0
by Varun Rao
Java heap error by Kumar sp
0
by Kumar sp
Structred Streaming Error by khajaasmath786
2
by khajaasmath786
[Spark K8] Kube2Iam Annotation Support by Chandu Kavar
0
by Chandu Kavar
How does number of partitions in DataFrame get decided while reading from HIVE by Shivam Sharma
0
by Shivam Sharma
Offsets out of order - Spark Datasource V2 by Cressy, Taylor
0
by Cressy, Taylor
run new spark version on old spark cluster ? by Nicolas Paris-2
9
by Nicolas Paris-2
Streaming job, catch exceptions by bsikander
12
by bsikander
double quota is automaticly added when sinking as csv by 杨浩
1
by Akshay Bhardwaj
NoClassDefFoundError by Sachit Murarka
0
by Sachit Murarka
How does dynamic allocation decide spark executor cores? by Pooja Agrawal
0
by Pooja Agrawal
spark checkpoint between 2 jobs and HDFS ramfs with storage policy by Julien Laurenceau
0
by Julien Laurenceau
[pyspark 2.3] count followed by write on dataframe by rishishah.star
1
by Keith Chapman
High level explanation of dropDuplicates by Yeikel
1
by Nicholas Hakobian
Spark-YARN | Scheduling of containers by Akshay Bhardwaj
4
by Hariharan
Fetching LinkedIn data into PySpark using OAuth2.0 by Aakash Basu-2
0
by Aakash Basu-2
Watermark handling on initial query start (Structured Streaming) by Joe Ammann
0
by Joe Ammann
spark 2.4.3 build fails using java 8 and scala 2.11 with NumberFormatException: Not a version: 9 by Bulldog20630405
1
by Bulldog20630405
Access to live data of cached dataFrame by Tomas Bartalos
2
by Tomas Bartalos
[PSA] Sharing our Experiences With Kubernetes by Matt Cheah
1
by Ramandeep Singh
1234 ... 382