gcloud beta metastore services create - create a Dataproc Metastore service
gcloud beta metastore services create (SERVICE : --location=LOCATION) [--async] [--data-catalog-sync] [--database-type=DATABASE_TYPE; default="mysql"] [--encryption-kms-key=ENCRYPTION_KMS_KEY] [--endpoint-protocol=ENDPOINT_PROTOCOL; default="thrift"] [--hive-metastore-version=HIVE_METASTORE_VERSION] [--labels=[KEY=VALUE,...]] [--port=PORT; default=9083] [--release-channel=RELEASE_CHANNEL; default="stable"] [--tier=TIER] [--auxiliary-versions=[AUXILIARY_VERSIONS,...] | --auxiliary-versions-from-file=AUXILIARY_VERSIONS_FROM_FILE] [--consumer-subnetworks=[CONSUMER_SUBNETWORKS,...] | --network=NETWORK | --network-config-from-file=NETWORK_CONFIG_FROM_FILE] [--hive-metastore-configs=KEY=VALUE | --hive-metastore-configs-from-file=HIVE_METASTORE_CONFIGS_FROM_FILE] [--kerberos-principal=KERBEROS_PRINCIPAL --keytab=KEYTAB --krb5-config=KRB5_CONFIG] [--maintenance-window-day=MAINTENANCE_WINDOW_DAY --maintenance-window-hour=MAINTENANCE_WINDOW_HOUR] [GCLOUD_WIDE_FLAG ...]
(BETA) Create a new Dataproc Metastore service with the given name and configurations.
If run asynchronously with --async, exits after printing one operation name that can be used to poll the status of the creation via:
gcloud metastore operations describe
To create a Dataproc Metastore service with the name my-metastore-service in location us-central using a non default port 9090, run:
$ gcloud beta metastore services create my-metastore-service \ --location=us-central1 --port=9090
To create a Dataproc Metastore service with the name my-metastore-service in location us-central using a non default network foo, run:
$ gcloud beta metastore services create my-metastore-service \ --location=us-central1 --network=foo
- Service resource - Arguments and flags that specify the Dataproc Metastore
service you want to create. The arguments in this group can be used to specify the attributes of this resource. (NOTE) Some attributes are not given arguments in this group but can be set in other ways. To set the project attribute:
- —
provide the argument service on the command line with a fully specified name;
- —
set the property core/project;
- —
provide the argument --project on the command line.
This must be specified.
- SERVICE
ID of the service or fully qualified identifier for the service. To set the service attribute:
provide the argument service on the command line.
This positional argument must be specified if any of the other arguments in this group are specified.
- --location=LOCATION
The location of the Dataproc Metastore service.
If not specified, will use default metastore/location.
To set the location attribute:
provide the argument service on the command line with a fully specified name;
provide the argument --location on the command line;
set the property metastore/location.
- --async
Return immediately, without waiting for the operation in progress to complete.
- --data-catalog-sync
A boolean flag to determine whether Dataproc Metastore metadata sync to Data Catalog should be enabled, false if unspecified. Mutually exclusive with flag --encryption-kms-key.
- --database-type=DATABASE_TYPE; default="mysql"
The type of database the Dataproc Metastore service will store data in. DATABASE_TYPE must be one of:
- mysql
MYSQL database type is a Dataproc Metastore service backed by MySQL CloudSQL.
- spanner
SPANNER database type is a Dataproc Metastore service backed by Cloud Spanner.
- --encryption-kms-key=ENCRYPTION_KMS_KEY
The name of the Cloud KMS key to use for customer data encryption. Cannot be changed once the service is created. Mutually exclusive with flag --data-catalog-sync. This can be provided as a fully-qualified URL, or a relative name in the form projects/{project_id}/locations/{location}/keyRings/{key_ring_id}/cryptoKeys/{crypto_key_id}.
- --endpoint-protocol=ENDPOINT_PROTOCOL; default="thrift"
The protocol to use for the metastore service endpoint. If unspecified, defaults to THRIFT. ENDPOINT_PROTOCOL must be one of:
- grpc
The modernized GRPC protocol.
- thrift
The legacy Apache THRIFT protocol.
- --hive-metastore-version=HIVE_METASTORE_VERSION
The Hive metastore schema version. The supported versions of a location are listed via:
gcloud metastore locations describe
If unspecified, the default version chosen by the server will be used.
- --labels=[KEY=VALUE,...]
List of label KEY=VALUE pairs to add.
Keys must start with a lowercase character and contain only hyphens (-), underscores (_), lowercase characters, and numbers. Values must contain only hyphens (-), underscores (_), lowercase characters, and numbers.
- --port=PORT; default=9083
The TCP port on which the Metastore service will listen. If unspecified, the default port 9083 will be used.
- --release-channel=RELEASE_CHANNEL; default="stable"
The release channel of the service. RELEASE_CHANNEL must be one of:
- canary
The CANARY release channel contains the newest features, which may be unstable and subject to unresolved issues with no known workarounds. Services using the CANARY release channel are not subject to any SLAs.
- stable
The STABLE release channel contains features that are considered stable and have been validated for production use.
- --tier=TIER
The tier of the service. TIER must be one of:
- developer
The developer tier provides limited scalability and no fault tolerance. Good for low-cost proof-of-concept.
- enterprise
The enterprise tier provides multi-zone high availability, and sufficient scalability for enterprise-level Dataproc Metastore workloads.
- Auxiliary versions configuration for the Dataproc Metastore service. When
specified, a secondary Hive metastore service is created along with the primary service.
At most one of these can be specified:
- --auxiliary-versions=[AUXILIARY_VERSIONS,...]
Comma-separated list of auxiliary Hive metastore versions to deploy. Auxiliary Hive metastore versions must be less than the primary Hive metastore service's version.
- --auxiliary-versions-from-file=AUXILIARY_VERSIONS_FROM_FILE
Path to a YAML file containing the auxiliary versions configuration for Dataproc Metastore instance. The file should contain a unique auxiliary service name and auxiliary version that is lower than the primary service version. The primary version's Hive metastore configs are applied to the auxiliary version. Additional Hive metastore configs can be specified for the auxiliary version using "config_overrides". If a specified config property has already been overridden in the primary Hive metastore version's configs, the auxiliary version's override takes precedence. The contents of the file should be structured as follows:
YAML:
- name: aux-service1 version: x.y.z config_overrides: key1: value1 key2: value2 ... - name: aux-service2 version: x.y.z config_overrides: key1: value1 key2: value2 ... ...
- Network configuration for the metastore service.
At most one of these can be specified:
- --consumer-subnetworks=[CONSUMER_SUBNETWORKS,...]
The list of subnetworks from which the service can be accessed. This can be the subnetwork's ID, fully-qualified URL, or relative name in the form projects/{project_id}/regions/{region_id}/subnetworks/{subnetwork}.
- --network=NETWORK
The name of the network on which the service can be accessed. This can be the network's ID, fully-qualified URL, or relative name in the form projects/{project_id}/global/networks/{network}. If "network", "consumer-subnetworks" and "network-config" are unspecified, the "default" network will be used.
- --network-config-from-file=NETWORK_CONFIG_FROM_FILE
Path to a YAML file containing the network configuration for Dataproc Metastore instance. The contents of the file should be structured as follows:
YAML: consumers:
subnetwork: projects/{project_id}/regions/{region_id}/subnetworks/{subnetwork_id}
subnetwork: projects/{project_id}/regions/{region_id}/subnetworks/{subnetwork_id} ...
- Configuration properties specific to running Hive metastore software as the
metastore service.
At most one of these can be specified:
- --hive-metastore-configs=KEY=VALUE
A mapping of Hive metastore configuration key-value pairs to apply to the Hive metastore.
- --hive-metastore-configs-from-file=HIVE_METASTORE_CONFIGS_FROM_FILE
Path to a XML file containing a mapping of Hive metastore configuration key-value pairs to apply to the Hive metastore.
For example:
hive-site.xml <configuration> <property> <name>hive.metastore.warehouse.dir</name> <value>${test.warehouse.dir}</value> <description></description> </property> </configuration>
- Information used to configure the Hive metastore service as a service principal
in a Kerberos realm.
- --kerberos-principal=KERBEROS_PRINCIPAL
A Kerberos principal that exists in the KDC to authenticate as. A typical principal is of the form "primary/instance@REALM", but there is no exact format.
This flag argument must be specified if any of the other arguments in this group are specified.
- --keytab=KEYTAB
A Kerberos keytab file that can be used to authenticate a service principal with a Kerberos Key Distribution Center. This is a Secret Manager secret version, and can be fully-qualified URL, or relative name in the form projects/{project_id}/secrets/{secret_id}/versions/{version_id}.
This flag argument must be specified if any of the other arguments in this group are specified.
- --krb5-config=KRB5_CONFIG
A Cloud Storage URI that specifies the path to a krb5.conf file. It is of the form gs://{bucket_name}/path/krb5.conf, although the file does not need to be named krb5.conf explicitly.
This flag argument must be specified if any of the other arguments in this group are specified.
- The one hour maintenance window that specifies when Dataproc Metastore may
perform system maintenance operation to the service, in UTC time.
- --maintenance-window-day=MAINTENANCE_WINDOW_DAY
The day of week when the window starts, e.g., sun. MAINTENANCE_WINDOW_DAY must be one of: fri, mon, sat, sun, thu, tue, wed.
This flag argument must be specified if any of the other arguments in this group are specified.
- --maintenance-window-hour=MAINTENANCE_WINDOW_HOUR
The hour of day (0-23) when the window starts.
This flag argument must be specified if any of the other arguments in this group are specified.
These flags are available to all commands: --access-token-file, --account, --billing-project, --configuration, --flags-file, --flatten, --format, --help, --impersonate-service-account, --log-http, --project, --quiet, --trace-token, --user-output-enabled, --verbosity.
Run $ gcloud help for details.
This command uses the metastore/v1beta API. The full documentation for this API can be found at: https://cloud.google.com/dataproc-metastore/docs
This command is currently in beta and might change without notice. These variants are also available:
$ gcloud metastore services create
$ gcloud alpha metastore services create