-
Notifications
You must be signed in to change notification settings - Fork 184
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Error: Java heap space #312
Comments
@ObadaM are you running this on a local machine or on a cluster? If on a local machine, you are unlikely to see much speed up from using Spark, and we recommend local mode. But if on a cluster, or if you are running Spark on a single machine just to try it out, you can try adding this line to your bash profile (assuming you're on Mac OS X or Linux)
|
@freeman-lab I tried your solution but it did not work as well.
and the only way that it changed was from running Spark from terminal.
Yes I am running it on a single machine just to be familiar with the whole spark thunder environment then I will apply it on a cluster. |
I found the solution. first I downloaded the source code of spark then I built it again with sbt, after than i created spark-env.sh in conf file with the following:
I tried the pre-built version of spark with this before but it did not work. |
HI everyone,
I am facing a problem while I am trying to load a tif file with the help from spark. It gives me the error:
Caused by: java.lang.OutOfMemoryError: Java heap space
1- I tried to update "spark-env.sh" and "spark-defaults.conf" with
but I got the same error.
2- I tried to make it like this :
sc = SparkContext("local","app1","--executor-memory 20G")
I got the same error.
3- I tried to add a config variable
I got error with conf, _AttributeError: 'SparkConf' object has no attribute 'get_object_id'
I would appreciate any kind of help.
The text was updated successfully, but these errors were encountered: