< Back

Add-AzureRmHDInsightConfigValues

Tue Jan 29, 2019 9:47 pm

NAME Add-AzureRmHDInsightConfigValues



SYNOPSIS

Adds a Hadoop configuration value customization and/or a Hive shared library customization to a cluster configuration object.





SYNTAX

Add-AzureRmHDInsightConfigValues [-Config] <AzureHDInsightConfig> [-Core <Hashtable>] [-DefaultProfile <IAzureContextContainer>] [-HBaseEnv

<Hashtable>] [-HBaseSite <Hashtable>] [-Hdfs <Hashtable>] [-HiveEnv <Hashtable>] [-HiveSite <Hashtable>] [-MapRed <Hashtable>] [-OozieEnv

<Hashtable>] [-OozieSite <Hashtable>] [-RServer <Hashtable>] [-Spark2Defaults <Hashtable>] [-Spark2ThriftConf <Hashtable>] [-Storm <Hashtable>]

[-Tez <Hashtable>] [-WebHCat <Hashtable>] [-Yarn <Hashtable>] [<CommonParameters>]



Add-AzureRmHDInsightConfigValues [-Config] <AzureHDInsightConfig> [-Core <Hashtable>] [-DefaultProfile <IAzureContextContainer>] [-HBaseEnv

<Hashtable>] [-HBaseSite <Hashtable>] [-Hdfs <Hashtable>] [-HiveEnv <Hashtable>] [-HiveSite <Hashtable>] [-MapRed <Hashtable>] [-OozieEnv

<Hashtable>] [-OozieSite <Hashtable>] [-RServer <Hashtable>] [-SparkDefaults <Hashtable>] [-SparkThriftConf <Hashtable>] [-Storm <Hashtable>]

[-Tez <Hashtable>] [-WebHCat <Hashtable>] [-Yarn <Hashtable>] [<CommonParameters>]





DESCRIPTION

The Add-AzureRmHDInsightConfigValues cmdlet adds a Hadoop configuration value customization, such as core-site.xml or hive-site.xml, and/or a Hive

shared library customization to the HDInsight configuration object created by the New-AzureRmHDInsightClusterConfig cmdlet.





PARAMETERS

-Config <AzureHDInsightConfig>

Specifies the HDInsight cluster configuration object that this cmdlet modifies. This object is created by the

New-AzureRmHDInsightClusterConfig cmdlet.



Required? true

Position? 0

Default value None

Accept pipeline input? True (ByValue)

Accept wildcard characters? false



-Core <Hashtable>

Specifies the Core Site configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-DefaultProfile <IAzureContextContainer>

The credentials, account, tenant, and subscription used for communication with azure



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-HBaseEnv <Hashtable>

Specifies the HBase Env configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-HBaseSite <Hashtable>

Specifies the HBase Site configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-Hdfs <Hashtable>

Specifies the HDFS configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-HiveEnv <Hashtable>

Specifies the Hive Env configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-HiveSite <Hashtable>

Specifies the Hive Site configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-MapRed <Hashtable>

Specifies the MapRed Site configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-OozieEnv <Hashtable>

Specifies the Oozie Env configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-OozieSite <Hashtable>

Specifies the Oozie Site configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-RServer <Hashtable>

Specifies the RServer configurations. Valid only for RServer clusters.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-Spark2Defaults <Hashtable>

Specifies the Spark2 Defaults configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-Spark2ThriftConf <Hashtable>

Specifies the Spark2 Thrift SparkConf configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-SparkDefaults <Hashtable>

Specifies the Spark Defaults configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-SparkThriftConf <Hashtable>

Specifies the Spark Thrift SparkConf configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-Storm <Hashtable>

Specifies the Storm Site configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-Tez <Hashtable>

Specifies the Tez Site configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-WebHCat <Hashtable>

Specifies the WebHCat Site configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



-Yarn <Hashtable>

Specifies the YARN Site configurations of this HDInsight cluster.



Required? false

Position? named

Default value None

Accept pipeline input? False

Accept wildcard characters? false



<CommonParameters>

This cmdlet supports the common parameters: Verbose, Debug,

ErrorAction, ErrorVariable, WarningAction, WarningVariable,

OutBuffer, PipelineVariable, and OutVariable. For more information, see

about_CommonParameters (http://go.microsoft.com/fwlink/?LinkID=113216).



INPUTS

AzureHDInsightConfig

Parameter 'Config' accepts value of type 'AzureHDInsightConfig' from the pipeline





OUTPUTS

Microsoft.Azure.Commands.HDInsight.Models.AzureHDInsightConfig







NOTES









Example 1: Add a custom configuration value to the cluster configuration object



PS C:\\># Primary storage account info

PS C:\\> $storageAccountResourceGroupName = "Group"

PS C:\\> $storageAccountName = "yourstorageacct001"

PS C:\\> $storageAccountKey = (Get-AzureRmStorageAccountKey -ResourceGroupName $storageAccountResourceGroupName -Name $storageAccountName)[0].value



PS C:\\> $storageContainer = "container001"



# Cluster configuration info

PS C:\\> $location = "East US 2"

PS C:\\> $clusterResourceGroupName = "Group"

PS C:\\> $clusterName = "your-hadoop-001"

PS C:\\> $clusterCreds = Get-Credential



# If the cluster's resource group doesn't exist yet, run:

# New-AzureRmResourceGroup -Name $clusterResourceGroupName -Location $location



# Config values

PS C:\\> $coreConfigs = @{"io.file.buffer.size"="300000"}

PS C:\\> $mapRedConfigs = @{"mapred.map.max.attempts"="2"}



# Create the cluster

PS C:\\> New-AzureRmHDInsightClusterConfig `

| Add-AzureRmHDInsightConfigValues `

-Core $coreConfigs `

-MapRed $mapRedConfigs `

| New-AzureRmHDInsightCluster `

-ClusterType Hadoop `

-OSType Windows `

-ClusterSizeInNodes 4 `

-ResourceGroupName $clusterResourceGroupName `

-ClusterName $clusterName `

-HttpCredential $clusterCreds `

-Location $location `

-DefaultStorageAccountName "$storageAccountName.blob.core.windows.net" `

-DefaultStorageAccountKey $storageAccountKey `

-DefaultStorageContainer $storageAccountContainer



This command adds a Hadoop configuration value to the cluster named your-hadoop-001.







RELATED LINKS

Online Version: https://docs.microsoft.com/en-us/powers ... nfigvalues

New-AzureRmHDInsightClusterConfig