Environment

Runtime Information

NameValue
Java Home/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.amzn2.0.2.x86_64/jre
Java Version1.8.0_312 (Red Hat, Inc.)
Scala Versionversion 2.10.5

Spark Properties

NameValue
spark.executor.extraJavaOptions-Djava.io.tmpdir=/data/tmp
spark.driver.host172.30.1.237
spark.eventLog.enabledtrue
spark.driver.port43037
spark.repl.class.urihttp://172.30.1.237:41809
spark.jarsfile:/home/ec2-user/jars-hug/spark-redis2.9.jar,file:/home/ec2-user/jars-hug/mysql-connector-java-5.1.45-bin.jar,file:/home/ec2-user/jars-hug/spark-cassandra-connector-1.6.1-s_2.10
spark.app.nameSpark shell
spark.scheduler.modeFIFO
spark.driver.memory512m
spark.history.fs.cleaner.enabledtrue
spark.executor.iddriver
spark.driver.extraJavaOptions-Djava.io.tmpdir=/data/tmp
spark.submit.deployModeclient
spark.masterspark://172.30.1.237:7077
spark.history.fs.cleaner.interval3d
spark.authenticatetrue
spark.history.fs.cleaner.maxAge20d
spark.authenticate.secret8cb24aaff5e31901f19063655325f4c708c6f3f76c0338ce4a12e8a85060855cbcf56bf5e9f4c427b4d9c51a376451bd99721933df6215e514cdf5f9ee1bc8e9
spark.executor.memory2g
spark.local.dir/data/tmp
spark.eventLog.dir/data/spark/eventlog
spark.eventLog.compresstrue
spark.executor.cores2
spark.externalBlockStore.folderNamespark-2c935a42-ed4e-41d5-abf1-6a7f2e7efb2e
spark.app.idapp-20220705084453-0001
spark.cassandra.connection.host172.30.1.253,172.30.1.38,172.30.1.97

System Properties

NameValue
java.io.tmpdir/data/tmp
line.separator
path.separator:
sun.management.compilerHotSpot 64-Bit Tiered Compilers
SPARK_SUBMITtrue
sun.cpu.endianlittle
java.specification.version1.8
java.vm.specification.nameJava Virtual Machine Specification
java.vendorRed Hat, Inc.
java.vm.specification.version1.8
user.home/home/ec2-user
file.encoding.pkgsun.io
sun.nio.ch.bugLevel
sun.arch.data.model64
sun.boot.library.path/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.amzn2.0.2.x86_64/jre/lib/amd64
user.dir/home/ec2-user/jars-hug
java.library.path/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
sun.cpu.isalist
os.archamd64
java.vm.version25.312-b07
java.endorsed.dirs/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.amzn2.0.2.x86_64/jre/lib/endorsed
java.runtime.version1.8.0_312-b07
java.vm.infomixed mode
java.ext.dirs/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.amzn2.0.2.x86_64/jre/lib/ext:/usr/java/packages/lib/ext
java.runtime.nameOpenJDK Runtime Environment
file.separator/
java.class.version52.0
scala.usejavacptrue
java.specification.nameJava Platform API Specification
sun.boot.class.path/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.amzn2.0.2.x86_64/jre/lib/resources.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.amzn2.0.2.x86_64/jre/lib/rt.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.amzn2.0.2.x86_64/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.amzn2.0.2.x86_64/jre/lib/jsse.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.amzn2.0.2.x86_64/jre/lib/jce.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.amzn2.0.2.x86_64/jre/lib/charsets.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.amzn2.0.2.x86_64/jre/lib/jfr.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.amzn2.0.2.x86_64/jre/classes
file.encodingUTF-8
user.timezoneUTC
java.specification.vendorOracle Corporation
sun.java.launcherSUN_STANDARD
os.version4.14.152-127.182.amzn2.x86_64
sun.os.patch.levelunknown
java.vm.specification.vendorOracle Corporation
user.countryUS
sun.jnu.encodingUTF-8
user.languageen
java.vendor.urlhttps://www.redhat.com/
java.awt.printerjobsun.print.PSPrinterJob
java.awt.graphicsenvsun.awt.X11GraphicsEnvironment
awt.toolkitsun.awt.X11.XToolkit
os.nameLinux
java.vm.vendorRed Hat, Inc.
java.vendor.url.bughttps://forums.aws.amazon.com/
user.nameec2-user
java.vm.nameOpenJDK 64-Bit Server VM
sun.java.commandorg.apache.spark.deploy.SparkSubmit --deploy-mode client --conf spark.driver.memory=512m --conf spark.cassandra.connection.host=172.30.1.253,172.30.1.38,172.30.1.97 --class org.apache.spark.repl.Main --name Spark shell --jars /home/ec2-user/jars-hug/spark-redis2.9.jar,/home/ec2-user/jars-hug/mysql-connector-java-5.1.45-bin.jar,/home/ec2-user/jars-hug/spark-cassandra-connector-1.6.1-s_2.10 --executor-cores 2 --executor-memory 2g spark-shell -i:/home/ec2-user/daily_wilson/wilson6
java.home/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.312.b07-1.amzn2.0.2.x86_64/jre
java.version1.8.0_312
sun.io.unicode.encodingUnicodeLittle

Classpath Entries

ResourceSource
/home/ec2-user/spark/lib/datanucleus-api-jdo-3.2.6.jarSystem Classpath
http://172.30.1.237:43129/jars/mysql-connector-java-5.1.45-bin.jarAdded By User
/home/ec2-user/spark/lib/datanucleus-rdbms-3.2.9.jarSystem Classpath
/home/ec2-user/spark/conf/System Classpath
/home/ec2-user/spark/lib/datanucleus-core-3.2.10.jarSystem Classpath
http://172.30.1.237:43129/jars/spark-redis2.9.jarAdded By User
/home/ec2-user/spark/lib/spark-assembly-1.6.2-hadoop2.6.0.jarSystem Classpath
/home/ec2-user/installation/hadoop-2.7.2/etc/hadoop/System Classpath