Samza - Deploying a Samza job from HDFS

Deploying a Samza job from HDFS

This tutorial uses hello-samza to illustrate how to run a Samza job if you want to publish the Samza job’s .tar.gz package to HDFS.

Upload the package

hadoop fs -put ./target/hello-samza-1.1.0-dist.tar.gz /path/for/tgz

Add HDFS configuration

Put the hdfs-site.xml file of your cluster into ~/.samza/conf directory (The same place as the yarn-site.xml). If you set HADOOP_CONF_DIR, put the hdfs-site.xml in your configuration directory if the hdfs-site.xml is not there.

Change properties file

Change the yarn.package.path in the properties file to your HDFS location.

yarn.package.path=hdfs://<hdfs name node ip>:<hdfs name node port>/path/to/tgz

Then you should be able to run the Samza job as described in hello-samza.