Skip to content

Latest commit

 

History

History
94 lines (58 loc) · 1.82 KB

hadoop.md

File metadata and controls

94 lines (58 loc) · 1.82 KB
Raw

hadoop/util.sh

Provides Hadoop2 utility functions

Overview

Function to restart hadoop services on the cluster master

This may be used if you're using a different version of Java, for example

Index

restart_master_services()

Function to restart hadoop services on the cluster master

This may be used if you're using a different version of Java, for example

Example

restart_master_services

Function has no arguments.

restart_worker_services()

Function to restart hadoop services on the cluster workers

This only restarts the datanode service since the nodemanager is started after the bootstrap is run

Example

restart_worker_services

Function has no arguments.

restart_hadoop_services()

Generic function to restart hadoop services

Example

restart_hadoop_services

Function has no arguments.

use_java8()

Use Java 8 for hadoop daemons and jobs

By default, the hadoop daemons and jobs on Qubole clusters run on Java 7. Use this function if you would like to use Java 8. This is only required if your cluster:

  1. is in AWS, and
  2. is running Hive or Spark < 2.2

Example

use_java8

Function has no arguments.

wait_until_namenode_running()

Wait until namenode is out of safe mode

Example

wait_until_namenode_running 25 5

Arguments

  • $1 (int): Number of attempts function will make to get namenode out of safemode. Defaults to 50
  • $2 (int): Number of seconds each attempt will sleep for, waiting for namenode to come out of sleep mode. Defaults to 5