< Back
New-AzureRmHDInsightStreamingMapReduceJobDefinition
Post
NAME New-AzureRmHDInsightStreamingMapReduceJobDefinition
SYNOPSIS
Creates a Streaming MapReduce job object.
SYNTAX
New-AzureRmHDInsightStreamingMapReduceJobDefinition [-Arguments <String[]>] [-CommandEnvironment <Hashtable>] [-DefaultProfile
<IAzureContextContainer>] [-Defines <Hashtable>] [-File <String>] [-Files <String[]>] -InputPath <String> [-Mapper <String>] [-OutputPath
<String>] [-Reducer <String>] [-StatusFolder <String>] [<CommonParameters>]
DESCRIPTION
The New-AzureRmHDInsightStreamingMapReduceJobDefinition cmdlet defines a Streaming MapReduce job object for use with an Azure HDInsight cluster.
PARAMETERS
-Arguments <String[]>
Specifies an array of arguments for the job. The arguments are passed as command-line arguments to each task.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-CommandEnvironment <Hashtable>
Specifies an array of command-line environment variables to set when a job runs on worker nodes.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-DefaultProfile <IAzureContextContainer>
The credentials, account, tenant, and subscription used for communication with azure
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-Defines <Hashtable>
Specifies Hadoop configuration values to set for when the job runs.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-File <String>
Specifies the path to a file that contains a query to run. You can use this parameter instead of the Query parameter.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-Files <String[]>
Specifies a collection of files that are associated with a Hive job.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-InputPath <String>
Specifies the path to the input files.
Required? true
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-Mapper <String>
Specifies a Mapper file name.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-OutputPath <String>
Specifies the path for the job output.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-Reducer <String>
Specifies a Reducer file name.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-StatusFolder <String>
Specifies the location of the folder that contains standard outputs and error outputs for a job.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
<CommonParameters>
This cmdlet supports the common parameters: Verbose, Debug,
ErrorAction, ErrorVariable, WarningAction, WarningVariable,
OutBuffer, PipelineVariable, and OutVariable. For more information, see
about_CommonParameters (http://go.microsoft.com/fwlink/?LinkID=113216).
INPUTS
None
This cmdlet does not accept any input.
OUTPUTS
Microsoft.Azure.Commands.HDInsight.Models.AzureHDInsightStreamingMapReduceJobDefinition
NOTES
Example 1: Create a Streaming MapReduce job definition
PS C:\\># Cluster info
PS C:\\>$clusterName = "your-hadoop-001"
PS C:\\>$clusterCreds = Get-Credential
# Streaming MapReduce job details
PS C:\\>$statusFolder = "tempStatusFolder/"
PS C:\\>$query = "SHOW TABLES"
PS C:\\>New-AzureRmHDInsightStreamingMapReduceJobDefinition -StatusFolder $statusFolder `
-Query $query `
| Start-AzureRmHDInsightJob `
-ClusterName $clusterName `
-ClusterCredential $clusterCreds
This command creates a Streaming MapReduce job definition.
RELATED LINKS
Online Version: https://docs.microsoft.com/en-us/powers ... definition
Start-AzureRmHDInsightJob
SYNOPSIS
Creates a Streaming MapReduce job object.
SYNTAX
New-AzureRmHDInsightStreamingMapReduceJobDefinition [-Arguments <String[]>] [-CommandEnvironment <Hashtable>] [-DefaultProfile
<IAzureContextContainer>] [-Defines <Hashtable>] [-File <String>] [-Files <String[]>] -InputPath <String> [-Mapper <String>] [-OutputPath
<String>] [-Reducer <String>] [-StatusFolder <String>] [<CommonParameters>]
DESCRIPTION
The New-AzureRmHDInsightStreamingMapReduceJobDefinition cmdlet defines a Streaming MapReduce job object for use with an Azure HDInsight cluster.
PARAMETERS
-Arguments <String[]>
Specifies an array of arguments for the job. The arguments are passed as command-line arguments to each task.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-CommandEnvironment <Hashtable>
Specifies an array of command-line environment variables to set when a job runs on worker nodes.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-DefaultProfile <IAzureContextContainer>
The credentials, account, tenant, and subscription used for communication with azure
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-Defines <Hashtable>
Specifies Hadoop configuration values to set for when the job runs.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-File <String>
Specifies the path to a file that contains a query to run. You can use this parameter instead of the Query parameter.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-Files <String[]>
Specifies a collection of files that are associated with a Hive job.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-InputPath <String>
Specifies the path to the input files.
Required? true
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-Mapper <String>
Specifies a Mapper file name.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-OutputPath <String>
Specifies the path for the job output.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-Reducer <String>
Specifies a Reducer file name.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-StatusFolder <String>
Specifies the location of the folder that contains standard outputs and error outputs for a job.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
<CommonParameters>
This cmdlet supports the common parameters: Verbose, Debug,
ErrorAction, ErrorVariable, WarningAction, WarningVariable,
OutBuffer, PipelineVariable, and OutVariable. For more information, see
about_CommonParameters (http://go.microsoft.com/fwlink/?LinkID=113216).
INPUTS
None
This cmdlet does not accept any input.
OUTPUTS
Microsoft.Azure.Commands.HDInsight.Models.AzureHDInsightStreamingMapReduceJobDefinition
NOTES
Example 1: Create a Streaming MapReduce job definition
PS C:\\># Cluster info
PS C:\\>$clusterName = "your-hadoop-001"
PS C:\\>$clusterCreds = Get-Credential
# Streaming MapReduce job details
PS C:\\>$statusFolder = "tempStatusFolder/"
PS C:\\>$query = "SHOW TABLES"
PS C:\\>New-AzureRmHDInsightStreamingMapReduceJobDefinition -StatusFolder $statusFolder `
-Query $query `
| Start-AzureRmHDInsightJob `
-ClusterName $clusterName `
-ClusterCredential $clusterCreds
This command creates a Streaming MapReduce job definition.
RELATED LINKS
Online Version: https://docs.microsoft.com/en-us/powers ... definition
Start-AzureRmHDInsightJob