< Back
Invoke-AzureRmHDInsightHiveJob
Post
NAME Invoke-AzureRmHDInsightHiveJob
SYNOPSIS
Submits a Hive query to an HDInsight cluster and retrieves query results in one operation.
SYNTAX
Invoke-AzureRmHDInsightHiveJob [-Arguments <String[]>] [-DefaultContainer <String>] [-DefaultProfile <IAzureContextContainer>]
[-DefaultStorageAccountKey <String>] [-DefaultStorageAccountName <String>] [-Defines <Hashtable>] [-File <String>] [-Files <String[]>] [-JobName
<String>] [-Query <String>] [-RunAsFileJob] [-StatusFolder <String>] [<CommonParameters>]
DESCRIPTION
The Invoke-AzureRmHDInsightHiveJob cmdlet submits a Hive query to an Azure HDInsight cluster and retrieves query results in one operation. Use the
Use-AzureRmHDInsightCluster cmdlet before calling Invoke-AzureRmHDInsightHiveJob to specify which cluster will be used for the query.
PARAMETERS
-Arguments <String[]>
Specifies an array of arguments for the job. The arguments are passed as command-line arguments to each task.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-DefaultContainer <String>
Specifies the name of the default container in the default Azure Storage account that an HDInsight cluster uses.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-DefaultProfile <IAzureContextContainer>
The credentials, account, tenant, and subscription used for communication with azure
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-DefaultStorageAccountKey <String>
Specifies the account key for the default storage account that the HDInsight cluster uses.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-DefaultStorageAccountName <String>
Specifies the name of the default storage account that the HDInsight cluster uses.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-Defines <Hashtable>
Specifies Hadoop configuration values to set when a job runs.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-File <String>
Specifies the path to a file in Azure Storage that contains the query to run. You can use this parameter instead of the Query parameter.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-Files <String[]>
Specifies a collection of files that are required for a Hive job.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-JobName <String>
Specifies the name of a Hive job. If you do not specify this parameter, this cmdlet uses the default value: "Hive: <first 100 characters of
Query>".
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-Query <String>
Specifies the Hive query.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-RunAsFileJob [<SwitchParameter>]
Indicates that this cmdlet creates a file in the default Azure storage account in which to store a query. This cmdlet submits the job that
references this file as a script to run.
You can use this functionality to handle special characters such as percent sign (%) that would fail on a job submission through Templeton,
because Templeton interprets a query with a percent sign as a URL parameter.
Required? false
Position? named
Default value False
Accept pipeline input? False
Accept wildcard characters? false
-StatusFolder <String>
Specifies the location of the folder that contains standard outputs and error outputs for a job.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
<CommonParameters>
This cmdlet supports the common parameters: Verbose, Debug,
ErrorAction, ErrorVariable, WarningAction, WarningVariable,
OutBuffer, PipelineVariable, and OutVariable. For more information, see
about_CommonParameters (http://go.microsoft.com/fwlink/?LinkID=113216).
INPUTS
None
This cmdlet does not accept any input.
OUTPUTS
System.String
NOTES
Example 1: Submit a Hive query to an Azure HDInsight cluster
PS C:\\># Primary storage account info
PS C:\\> $storageAccountResourceGroupName = "Group"
PS C:\\> $storageAccountName = "yourstorageacct001"
PS C:\\> $storageAccountKey = (Get-AzureRmStorageAccountKey -ResourceGroupName $storageAccountResourceGroupName -Name $storageAccountName)[0].value
PS C:\\> $storageContainer = "container001"
# Cluster info
PS C:\\> $clusterName = "your-hadoop-001"
PS C:\\> $clusterCreds = Get-Credential
# Hive job details
PS C:\\> $statusFolder = "tempStatusFolder/"
PS C:\\> $query = "SHOW TABLES"
PS C:\\> Use-AzureRmHDInsightCluster `
-ClusterCredential $clusterCreds `
-ClusterName $clusterName
PS C:\\> Invoke-AzureRmHDInsightHiveJob -StatusFolder $statusFolder `
-Query $query `
-DefaultContainer $storageAccountContainer `
-DefaultStorageAccountName "$storageAccountName.blob.core.windows.net" `
-DefaultStorageAccountKey $storageAccountKey
This command submits the query SHOW TABLES to the cluster named your-hadoop-001.
RELATED LINKS
Online Version: https://docs.microsoft.com/en-us/powers ... ghthivejob
Use-AzureRmHDInsightCluster
SYNOPSIS
Submits a Hive query to an HDInsight cluster and retrieves query results in one operation.
SYNTAX
Invoke-AzureRmHDInsightHiveJob [-Arguments <String[]>] [-DefaultContainer <String>] [-DefaultProfile <IAzureContextContainer>]
[-DefaultStorageAccountKey <String>] [-DefaultStorageAccountName <String>] [-Defines <Hashtable>] [-File <String>] [-Files <String[]>] [-JobName
<String>] [-Query <String>] [-RunAsFileJob] [-StatusFolder <String>] [<CommonParameters>]
DESCRIPTION
The Invoke-AzureRmHDInsightHiveJob cmdlet submits a Hive query to an Azure HDInsight cluster and retrieves query results in one operation. Use the
Use-AzureRmHDInsightCluster cmdlet before calling Invoke-AzureRmHDInsightHiveJob to specify which cluster will be used for the query.
PARAMETERS
-Arguments <String[]>
Specifies an array of arguments for the job. The arguments are passed as command-line arguments to each task.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-DefaultContainer <String>
Specifies the name of the default container in the default Azure Storage account that an HDInsight cluster uses.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-DefaultProfile <IAzureContextContainer>
The credentials, account, tenant, and subscription used for communication with azure
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-DefaultStorageAccountKey <String>
Specifies the account key for the default storage account that the HDInsight cluster uses.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-DefaultStorageAccountName <String>
Specifies the name of the default storage account that the HDInsight cluster uses.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-Defines <Hashtable>
Specifies Hadoop configuration values to set when a job runs.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-File <String>
Specifies the path to a file in Azure Storage that contains the query to run. You can use this parameter instead of the Query parameter.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-Files <String[]>
Specifies a collection of files that are required for a Hive job.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-JobName <String>
Specifies the name of a Hive job. If you do not specify this parameter, this cmdlet uses the default value: "Hive: <first 100 characters of
Query>".
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-Query <String>
Specifies the Hive query.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
-RunAsFileJob [<SwitchParameter>]
Indicates that this cmdlet creates a file in the default Azure storage account in which to store a query. This cmdlet submits the job that
references this file as a script to run.
You can use this functionality to handle special characters such as percent sign (%) that would fail on a job submission through Templeton,
because Templeton interprets a query with a percent sign as a URL parameter.
Required? false
Position? named
Default value False
Accept pipeline input? False
Accept wildcard characters? false
-StatusFolder <String>
Specifies the location of the folder that contains standard outputs and error outputs for a job.
Required? false
Position? named
Default value None
Accept pipeline input? False
Accept wildcard characters? false
<CommonParameters>
This cmdlet supports the common parameters: Verbose, Debug,
ErrorAction, ErrorVariable, WarningAction, WarningVariable,
OutBuffer, PipelineVariable, and OutVariable. For more information, see
about_CommonParameters (http://go.microsoft.com/fwlink/?LinkID=113216).
INPUTS
None
This cmdlet does not accept any input.
OUTPUTS
System.String
NOTES
Example 1: Submit a Hive query to an Azure HDInsight cluster
PS C:\\># Primary storage account info
PS C:\\> $storageAccountResourceGroupName = "Group"
PS C:\\> $storageAccountName = "yourstorageacct001"
PS C:\\> $storageAccountKey = (Get-AzureRmStorageAccountKey -ResourceGroupName $storageAccountResourceGroupName -Name $storageAccountName)[0].value
PS C:\\> $storageContainer = "container001"
# Cluster info
PS C:\\> $clusterName = "your-hadoop-001"
PS C:\\> $clusterCreds = Get-Credential
# Hive job details
PS C:\\> $statusFolder = "tempStatusFolder/"
PS C:\\> $query = "SHOW TABLES"
PS C:\\> Use-AzureRmHDInsightCluster `
-ClusterCredential $clusterCreds `
-ClusterName $clusterName
PS C:\\> Invoke-AzureRmHDInsightHiveJob -StatusFolder $statusFolder `
-Query $query `
-DefaultContainer $storageAccountContainer `
-DefaultStorageAccountName "$storageAccountName.blob.core.windows.net" `
-DefaultStorageAccountKey $storageAccountKey
This command submits the query SHOW TABLES to the cluster named your-hadoop-001.
RELATED LINKS
Online Version: https://docs.microsoft.com/en-us/powers ... ghthivejob
Use-AzureRmHDInsightCluster