@datafire/google_dataproc

v6.0.0

Published

3 years ago

DataFire integration for Cloud Dataproc API

Downloads

0High
0Medium
0Low

datafire

@datafire/google_dataproc

Client library for Cloud Dataproc API

Installation and Usage

npm install --save @datafire/google_dataproc

let google_dataproc = require('@datafire/google_dataproc').create({
  access_token: "",
  refresh_token: "",
  client_id: "",
  client_secret: "",
  redirect_uri: ""
});

.then(data => {
  console.log(data);
});

Description

Manages Hadoop-based clusters and jobs on Google Cloud Platform.

Actions

oauthCallback

Exchange the code passed to your redirect URI for an access_token

google_dataproc.oauthCallback({
  "code": ""
}, context)

Input

input object
- code required string

Output

output object
- access_token string
- refresh_token string
- token_type string
- scope string
- expiration string

oauthRefresh

Exchange a refresh_token for an access_token

google_dataproc.oauthRefresh(null, context)

Input

This action has no parameters

Output

output object
- access_token string
- refresh_token string
- token_type string
- scope string
- expiration string

dataproc.projects.regions.clusters.list

Lists all regions/{region}/clusters in a project alphabetically.

google_dataproc.dataproc.projects.regions.clusters.list({
  "projectId": "",
  "region": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project that the cluster belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- filter string: Optional. A filter constraining the clusters to list. Filters are case-sensitive and have the following syntax:field = value AND field = value ...where field is one of status.state, clusterName, or labels.[KEY], and [KEY] is a label key. value can be * to match all values. status.state can be one of the following: ACTIVE, INACTIVE, CREATING, RUNNING, ERROR, DELETING, or UPDATING. ACTIVE contains the CREATING, UPDATING, and RUNNING states. INACTIVE contains the DELETING and ERROR states. clusterName is the name of the cluster provided at creation time. Only the logical AND operator is supported; space-separated items are treated as having an implicit AND operator.Example filter:status.state = ACTIVE AND clusterName = mycluster AND labels.env = staging AND labels.starred = *
- pageSize integer: Optional. The standard List page size.
- pageToken string: Optional. The standard List page token.
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output ListClustersResponse

dataproc.projects.regions.clusters.create

Creates a cluster in a project. The returned Operation.metadata will be ClusterOperationMetadata (https://cloud.google.com/dataproc/docs/reference/rpc/google.cloud.dataproc.v1beta2#clusteroperationmetadata).

google_dataproc.dataproc.projects.regions.clusters.create({
  "projectId": "",
  "region": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project that the cluster belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- requestId string: Optional. A unique id used to identify the request. If the server receives two CreateClusterRequest requests with the same id, then the second request will be ignored and the first google.longrunning.Operation created and stored in the backend is returned.It is recommended to always set this value to a UUID (https://en.wikipedia.org/wiki/Universally_unique_identifier).The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). The maximum length is 40 characters.
- body Cluster
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Operation

dataproc.projects.regions.clusters.delete

Deletes a cluster in a project. The returned Operation.metadata will be ClusterOperationMetadata (https://cloud.google.com/dataproc/docs/reference/rpc/google.cloud.dataproc.v1beta2#clusteroperationmetadata).

google_dataproc.dataproc.projects.regions.clusters.delete({
  "projectId": "",
  "region": "",
  "clusterName": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project that the cluster belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- clusterName required string: Required. The cluster name.
- clusterUuid string: Optional. Specifying the cluster_uuid means the RPC should fail (with error NOT_FOUND) if cluster with specified UUID does not exist.
- requestId string: Optional. A unique id used to identify the request. If the server receives two DeleteClusterRequest requests with the same id, then the second request will be ignored and the first google.longrunning.Operation created and stored in the backend is returned.It is recommended to always set this value to a UUID (https://en.wikipedia.org/wiki/Universally_unique_identifier).The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). The maximum length is 40 characters.
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Operation

dataproc.projects.regions.clusters.get

Gets the resource representation for a cluster in a project.

google_dataproc.dataproc.projects.regions.clusters.get({
  "projectId": "",
  "region": "",
  "clusterName": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project that the cluster belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- clusterName required string: Required. The cluster name.
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Cluster

dataproc.projects.regions.clusters.patch

Updates a cluster in a project. The returned Operation.metadata will be ClusterOperationMetadata (https://cloud.google.com/dataproc/docs/reference/rpc/google.cloud.dataproc.v1beta2#clusteroperationmetadata).

google_dataproc.dataproc.projects.regions.clusters.patch({
  "projectId": "",
  "region": "",
  "clusterName": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project the cluster belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- clusterName required string: Required. The cluster name.
- gracefulDecommissionTimeout string: Optional. Timeout for graceful YARN decomissioning. Graceful decommissioning allows removing nodes from the cluster without interrupting jobs in progress. Timeout specifies how long to wait for jobs in progress to finish before forcefully removing nodes (and potentially interrupting jobs). Default timeout is 0 (for forceful decommission), and the maximum allowed timeout is 1 day (see JSON representation of Duration (https://developers.google.com/protocol-buffers/docs/proto3#json)).Only supported on Dataproc image versions 1.2 and higher.
- requestId string: Optional. A unique id used to identify the request. If the server receives two UpdateClusterRequest requests with the same id, then the second request will be ignored and the first google.longrunning.Operation created and stored in the backend is returned.It is recommended to always set this value to a UUID (https://en.wikipedia.org/wiki/Universally_unique_identifier).The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). The maximum length is 40 characters.
- updateMask string: Required. Specifies the path, relative to Cluster, of the field to update. For example, to change the number of workers in a cluster to 5, the update_mask parameter would be specified as config.worker_config.num_instances, and the PATCH request body would specify the new value, as follows: { "config":{ "workerConfig":{ "numInstances":"5" } } } Similarly, to change the number of preemptible workers in a cluster to 5, the update_mask parameter would be config.secondary_worker_config.num_instances, and the PATCH request body would be set as follows: { "config":{ "secondaryWorkerConfig":{ "numInstances":"5" } } } Note: currently only the following fields can be updated: Mask Purpose labels Updates labels config.worker_config.num_instances Resize primary worker group config.secondary_worker_config.num_instances Resize secondary worker group config.lifecycle_config.auto_delete_ttl Reset MAX TTL duration config.lifecycle_config.auto_delete_time Update MAX TTL deletion timestamp config.lifecycle_config.idle_delete_ttl Update Idle TTL duration config.autoscaling_config.policy_uri Use, stop using, or change autoscaling policies
- body Cluster
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Operation

dataproc.projects.regions.clusters.diagnose

Gets cluster diagnostic information. The returned Operation.metadata will be ClusterOperationMetadata (https://cloud.google.com/dataproc/docs/reference/rpc/google.cloud.dataproc.v1beta2#clusteroperationmetadata). After the operation completes, Operation.response contains Empty.

google_dataproc.dataproc.projects.regions.clusters.diagnose({
  "projectId": "",
  "region": "",
  "clusterName": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project that the cluster belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- clusterName required string: Required. The cluster name.
- body DiagnoseClusterRequest
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Operation

dataproc.projects.regions.clusters.start

Starts a cluster in a project.

google_dataproc.dataproc.projects.regions.clusters.start({
  "projectId": "",
  "region": "",
  "clusterName": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project the cluster belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- clusterName required string: Required. The cluster name.
- body StartClusterRequest
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Operation

dataproc.projects.regions.clusters.stop

Stops a cluster in a project.

google_dataproc.dataproc.projects.regions.clusters.stop({
  "projectId": "",
  "region": "",
  "clusterName": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project the cluster belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- clusterName required string: Required. The cluster name.
- body StopClusterRequest
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Operation

dataproc.projects.regions.jobs.list

Lists regions/{region}/jobs in a project.

google_dataproc.dataproc.projects.regions.jobs.list({
  "projectId": "",
  "region": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project that the job belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- clusterName string: Optional. If set, the returned jobs list includes only jobs that were submitted to the named cluster.
- filter string: Optional. A filter constraining the jobs to list. Filters are case-sensitive and have the following syntax:field = value AND field = value ...where field is status.state or labels.[KEY], and [KEY] is a label key. value can be * to match all values. status.state can be either ACTIVE or NON_ACTIVE. Only the logical AND operator is supported; space-separated items are treated as having an implicit AND operator.Example filter:status.state = ACTIVE AND labels.env = staging AND labels.starred = *
- jobStateMatcher string (values: ALL, ACTIVE, NON_ACTIVE): Optional. Specifies enumerated categories of jobs to list. (default = match ALL jobs).If filter is provided, jobStateMatcher will be ignored.
- pageSize integer: Optional. The number of results to return in each response.
- pageToken string: Optional. The page token, returned by a previous call, to request the next page of results.
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output ListJobsResponse

dataproc.projects.regions.jobs.delete

Deletes the job from the project. If the job is active, the delete fails, and the response returns FAILED_PRECONDITION.

google_dataproc.dataproc.projects.regions.jobs.delete({
  "projectId": "",
  "region": "",
  "jobId": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project that the job belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- jobId required string: Required. The job ID.
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Empty

dataproc.projects.regions.jobs.get

Gets the resource representation for a job in a project.

google_dataproc.dataproc.projects.regions.jobs.get({
  "projectId": "",
  "region": "",
  "jobId": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project that the job belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- jobId required string: Required. The job ID.
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Job

dataproc.projects.regions.jobs.patch

Updates a job in a project.

google_dataproc.dataproc.projects.regions.jobs.patch({
  "projectId": "",
  "region": "",
  "jobId": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project that the job belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- jobId required string: Required. The job ID.
- updateMask string: Required. Specifies the path, relative to Job, of the field to update. For example, to update the labels of a Job the update_mask parameter would be specified as labels, and the PATCH request body would specify the new value. Note: Currently, labels is the only field that can be updated.
- body Job
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Job

dataproc.projects.regions.jobs.cancel

Starts a job cancellation request. To access the job resource after cancellation, call regions/{region}/jobs.list (https://cloud.google.com/dataproc/docs/reference/rest/v1beta2/projects.regions.jobs/list) or regions/{region}/jobs.get (https://cloud.google.com/dataproc/docs/reference/rest/v1beta2/projects.regions.jobs/get).

google_dataproc.dataproc.projects.regions.jobs.cancel({
  "projectId": "",
  "region": "",
  "jobId": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project that the job belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- jobId required string: Required. The job ID.
- body CancelJobRequest
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Job

dataproc.projects.regions.jobs.submit

Submits a job to a cluster.

google_dataproc.dataproc.projects.regions.jobs.submit({
  "projectId": "",
  "region": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project that the job belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- body SubmitJobRequest
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Job

dataproc.projects.regions.jobs.submitAsOperation

Submits job to a cluster.

google_dataproc.dataproc.projects.regions.jobs.submitAsOperation({
  "projectId": "",
  "region": ""
}, context)

Input

input object
- projectId required string: Required. The ID of the Google Cloud Platform project that the job belongs to.
- region required string: Required. The Dataproc region in which to handle the request.
- body SubmitJobRequest
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Operation

dataproc.projects.regions.workflowTemplates.delete

Deletes a workflow template. It does not cancel in-progress workflows.

google_dataproc.dataproc.projects.regions.workflowTemplates.delete({
  "name": ""
}, context)

Input

input object
- name required string: Required. The resource name of the workflow template, as described in https://cloud.google.com/apis/design/resource_names. For projects.regions.workflowTemplates.delete, the resource name of the template has the following format: projects/{project_id}/regions/{region}/workflowTemplates/{template_id} For projects.locations.workflowTemplates.instantiate, the resource name of the template has the following format: projects/{project_id}/locations/{location}/workflowTemplates/{template_id}
- version integer: Optional. The version of workflow template to delete. If specified, will only delete the template if the current server version matches specified version.
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Empty

dataproc.projects.regions.workflowTemplates.get

Retrieves the latest workflow template.Can retrieve previously instantiated template by specifying optional version parameter.

google_dataproc.dataproc.projects.regions.workflowTemplates.get({
  "name": ""
}, context)

Input

input object
- name required string: Required. The resource name of the workflow template, as described in https://cloud.google.com/apis/design/resource_names. For projects.regions.workflowTemplates.get, the resource name of the template has the following format: projects/{project_id}/regions/{region}/workflowTemplates/{template_id} For projects.locations.workflowTemplates.get, the resource name of the template has the following format: projects/{project_id}/locations/{location}/workflowTemplates/{template_id}
- version integer: Optional. The version of workflow template to retrieve. Only previously instantiated versions can be retrieved.If unspecified, retrieves the current version.
- pageSize integer: The standard list page size.
- pageToken string: The standard list page token.
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output WorkflowTemplate

dataproc.projects.regions.workflowTemplates.update

Updates (replaces) workflow template. The updated template must contain version that matches the current server version.

google_dataproc.dataproc.projects.regions.workflowTemplates.update({
  "name": ""
}, context)

Input

input object
- name required string: Output only. The resource name of the workflow template, as described in https://cloud.google.com/apis/design/resource_names. For projects.regions.workflowTemplates, the resource name of the template has the following format: projects/{project_id}/regions/{region}/workflowTemplates/{template_id} For projects.locations.workflowTemplates, the resource name of the template has the following format: projects/{project_id}/locations/{location}/workflowTemplates/{template_id}
- body WorkflowTemplate
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output WorkflowTemplate

dataproc.projects.regions.operations.cancel

Starts asynchronous cancellation on a long-running operation. The server makes a best effort to cancel the operation, but success is not guaranteed. If the server doesn't support this method, it returns google.rpc.Code.UNIMPLEMENTED. Clients can use Operations.GetOperation or other methods to check whether the cancellation succeeded or whether the operation completed despite cancellation. On successful cancellation, the operation is not deleted; instead, it becomes an operation with an Operation.error value with a google.rpc.Status.code of 1, corresponding to Code.CANCELLED.

google_dataproc.dataproc.projects.regions.operations.cancel({
  "name": ""
}, context)

Input

input object
- name required string: The name of the operation resource to be cancelled.
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Empty

dataproc.projects.regions.workflowTemplates.instantiate

Instantiates a template and begins execution.The returned Operation can be used to track execution of workflow by polling operations.get. The Operation will complete when entire workflow is finished.The running workflow can be aborted via operations.cancel. This will cause any inflight jobs to be cancelled and workflow-owned clusters to be deleted.The Operation.metadata will be WorkflowMetadata (https://cloud.google.com/dataproc/docs/reference/rpc/google.cloud.dataproc.v1beta2#workflowmetadata). Also see Using WorkflowMetadata (https://cloud.google.com/dataproc/docs/concepts/workflows/debugging#using_workflowmetadata).On successful completion, Operation.response will be Empty.

google_dataproc.dataproc.projects.regions.workflowTemplates.instantiate({
  "name": ""
}, context)

Input

input object
- name required string: Required. The resource name of the workflow template, as described in https://cloud.google.com/apis/design/resource_names. For projects.regions.workflowTemplates.instantiate, the resource name of the template has the following format: projects/{project_id}/regions/{region}/workflowTemplates/{template_id} For projects.locations.workflowTemplates.instantiate, the resource name of the template has the following format: projects/{project_id}/locations/{location}/workflowTemplates/{template_id}
- body InstantiateWorkflowTemplateRequest
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Operation

dataproc.projects.regions.autoscalingPolicies.list

Lists autoscaling policies in the project.

google_dataproc.dataproc.projects.regions.autoscalingPolicies.list({
  "parent": ""
}, context)

Input

input object
- parent required string: Required. The "resource name" of the region or location, as described in https://cloud.google.com/apis/design/resource_names. For projects.regions.autoscalingPolicies.list, the resource name of the region has the following format: projects/{project_id}/regions/{region} For projects.locations.autoscalingPolicies.list, the resource name of the location has the following format: projects/{project_id}/locations/{location}
- pageSize integer: Optional. The maximum number of results to return in each response. Must be less than or equal to 1000. Defaults to 100.
- pageToken string: Optional. The page token, returned by a previous call, to request the next page of results.
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output ListAutoscalingPoliciesResponse

dataproc.projects.regions.autoscalingPolicies.create

Creates new autoscaling policy.

google_dataproc.dataproc.projects.regions.autoscalingPolicies.create({
  "parent": ""
}, context)

Input

input object
- parent required string: Required. The "resource name" of the region or location, as described in https://cloud.google.com/apis/design/resource_names. For projects.regions.autoscalingPolicies.create, the resource name has the following format: projects/{project_id}/regions/{region} For projects.locations.autoscalingPolicies.create, the resource name has the following format: projects/{project_id}/locations/{location}
- body AutoscalingPolicy
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output AutoscalingPolicy

dataproc.projects.regions.workflowTemplates.list

Lists workflows that match the specified filter in the request.

google_dataproc.dataproc.projects.regions.workflowTemplates.list({
  "parent": ""
}, context)

Input

input object
- parent required string: Required. The resource name of the region or location, as described in https://cloud.google.com/apis/design/resource_names. For projects.regions.workflowTemplates,list, the resource name of the region has the following format: projects/{project_id}/regions/{region} For projects.locations.workflowTemplates.list, the resource name of the location has the following format: projects/{project_id}/locations/{location}
- pageSize integer: Optional. The maximum number of results to return in each response.
- pageToken string: Optional. The page token, returned by a previous call, to request the next page of results.
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output ListWorkflowTemplatesResponse

dataproc.projects.regions.workflowTemplates.create

Creates new workflow template.

google_dataproc.dataproc.projects.regions.workflowTemplates.create({
  "parent": ""
}, context)

Input

input object
- parent required string: Required. The resource name of the region or location, as described in https://cloud.google.com/apis/design/resource_names. For projects.regions.workflowTemplates,create, the resource name of the region has the following format: projects/{project_id}/regions/{region} For projects.locations.workflowTemplates.create, the resource name of the location has the following format: projects/{project_id}/locations/{location}
- body WorkflowTemplate
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output WorkflowTemplate

dataproc.projects.regions.workflowTemplates.instantiateInline

Instantiates a template and begins execution.This method is equivalent to executing the sequence CreateWorkflowTemplate, InstantiateWorkflowTemplate, DeleteWorkflowTemplate.The returned Operation can be used to track execution of workflow by polling operations.get. The Operation will complete when entire workflow is finished.The running workflow can be aborted via operations.cancel. This will cause any inflight jobs to be cancelled and workflow-owned clusters to be deleted.The Operation.metadata will be WorkflowMetadata (https://cloud.google.com/dataproc/docs/reference/rpc/google.cloud.dataproc.v1#workflowmetadata). Also see Using WorkflowMetadata (https://cloud.google.com/dataproc/docs/concepts/workflows/debugging#using_workflowmetadata).On successful completion, Operation.response will be Empty.

google_dataproc.dataproc.projects.regions.workflowTemplates.instantiateInline({
  "parent": ""
}, context)

Input

input object
- parent required string: Required. The resource name of the region or location, as described in https://cloud.google.com/apis/design/resource_names. For projects.regions.workflowTemplates,instantiateinline, the resource name of the region has the following format: projects/{project_id}/regions/{region} For projects.locations.workflowTemplates.instantiateinline, the resource name of the location has the following format: projects/{project_id}/locations/{location}
- instanceId string: Deprecated. Please use request_id field instead.
- requestId string: Optional. A tag that prevents multiple concurrent workflow instances with the same tag from running. This mitigates risk of concurrent instances started due to retries.It is recommended to always set this value to a UUID (https://en.wikipedia.org/wiki/Universally_unique_identifier).The tag must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). The maximum length is 40 characters.
- body WorkflowTemplate
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Operation

dataproc.projects.regions.workflowTemplates.getIamPolicy

Gets the access control policy for a resource. Returns an empty policy if the resource exists and does not have a policy set.

google_dataproc.dataproc.projects.regions.workflowTemplates.getIamPolicy({
  "resource": ""
}, context)

Input

input object
- resource required string: REQUIRED: The resource for which the policy is being requested. See the operation documentation for the appropriate value for this field.
- options.requestedPolicyVersion integer: Optional. The policy format version to be returned.Valid values are 0, 1, and 3. Requests specifying an invalid value will be rejected.Requests for policies with any conditional bindings must specify version 3. Policies without any conditional bindings may specify any valid value or leave the field unset.To learn which resources support conditions in their IAM policies, see the IAM documentation (https://cloud.google.com/iam/help/conditions/resource-policies).
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Policy

dataproc.projects.regions.workflowTemplates.setIamPolicy

Sets the access control policy on the specified resource. Replaces any existing policy.Can return NOT_FOUND, INVALID_ARGUMENT, and PERMISSION_DENIED errors.

google_dataproc.dataproc.projects.regions.workflowTemplates.setIamPolicy({
  "resource": ""
}, context)

Input

input object
- resource required string: REQUIRED: The resource for which the policy is being specified. See the operation documentation for the appropriate value for this field.
- body SetIamPolicyRequest
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output Policy

dataproc.projects.regions.workflowTemplates.testIamPermissions

Returns permissions that a caller has on the specified resource. If the resource does not exist, this will return an empty set of permissions, not a NOT_FOUND error.Note: This operation is designed to be used for building permission-aware UIs and command-line tools, not for authorization checking. This operation may "fail open" without warning.

google_dataproc.dataproc.projects.regions.workflowTemplates.testIamPermissions({
  "resource": ""
}, context)

Input

input object
- resource required string: REQUIRED: The resource for which the policy detail is being requested. See the operation documentation for the appropriate value for this field.
- body TestIamPermissionsRequest
- $.xgafv string (values: 1, 2): V1 error format.
- access_token string: OAuth access token.
- alt string (values: json, media, proto): Data format for response.
- callback string: JSONP
- fields string: Selector specifying which fields to include in a partial response.
- key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
- oauth_token string: OAuth 2.0 token for the current user.
- prettyPrint boolean: Returns response with indentations and line breaks.
- quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
- upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
- uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

output TestIamPermissionsResponse

Definitions

AcceleratorConfig

AcceleratorConfig object: Specifies the type and number of accelerator cards attached to the instances of an instance group (see GPUs on Compute Engine (https://cloud.google.com/compute/docs/gpus/)).
- acceleratorCount integer: The number of the accelerator cards of this type exposed to this instance.
- acceleratorTypeUri string: Full URL, partial URI, or short name of the accelerator type resource to expose to this instance. See Compute Engine AcceleratorTypes (https://cloud.google.com/compute/docs/reference/beta/acceleratorTypes)Examples * https://www.googleapis.com/compute/beta/projects/[project_id]/zones/us-east1-a/acceleratorTypes/nvidia-tesla-k80 * projects/[project_id]/zones/us-east1-a/acceleratorTypes/nvidia-tesla-k80 * nvidia-tesla-k80Auto Zone Exception: If you are using the Dataproc Auto Zone Placement (https://cloud.google.com/dataproc/docs/concepts/configuring-clusters/auto-zone#using_auto_zone_placement) feature, you must use the short name of the accelerator type resource, for example, nvidia-tesla-k80.

AutoscalingConfig

AutoscalingConfig object: Autoscaling Policy config associated with the cluster.
- policyUri string: Optional. The autoscaling policy used by the cluster.Only resource names including projectid and location (region) are valid. Examples: https://www.googleapis.com/compute/v1/projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id] projects/[project_id]/locations/[dataproc_region]/autoscalingPolicies/[policy_id]Note that the policy must be in the same project and Dataproc region.

AutoscalingPolicy

AutoscalingPolicy object: Describes an autoscaling policy for Dataproc cluster autoscaler.
- basicAlgorithm BasicAutoscalingAlgorithm
- id string: Required. The policy id.The id must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), and hyphens (-). Cannot begin or end with underscore or hyphen. Must consist of between 3 and 50 characters.
- name string: Output only. The "resource name" of the autoscaling policy, as described in https://cloud.google.com/apis/design/resource_names. For projects.regions.autoscalingPolicies, the resource name of the policy has the following format: projects/{project_id}/regions/{region}/autoscalingPolicies/{policy_id} For projects.locations.autoscalingPolicies, the resource name of the policy has the following format: projects/{project_id}/locations/{location}/autoscalingPolicies/{policy_id}
- secondaryWorkerConfig InstanceGroupAutoscalingPolicyConfig
- workerConfig InstanceGroupAutoscalingPolicyConfig

BasicAutoscalingAlgorithm

BasicAutoscalingAlgorithm object: Basic algorithm for autoscaling.
- cooldownPeriod string: Optional. Duration between scaling events. A scaling period starts after the update operation from the previous event has completed.Bounds: 2m, 1d. Default: 2m.
- yarnConfig BasicYarnAutoscalingConfig

BasicYarnAutoscalingConfig

BasicYarnAutoscalingConfig object: Basic autoscaling configurations for YARN.
- gracefulDecommissionTimeout string: Required. Timeout for YARN graceful decommissioning of Node Managers. Specifies the duration to wait for jobs to complete before forcefully removing workers (and potentially interrupting jobs). Only applicable to downscaling operations.Bounds: 0s, 1d.
- scaleDownFactor number: Required. Fraction of average YARN pending memory in the last cooldown period for which to remove workers. A scale-down factor of 1 will result in scaling down so that there is no available memory remaining after the update (more aggressive scaling). A scale-down factor of 0 disables removing workers, which can be beneficial for autoscaling a single job. See How autoscaling works for more information.Bounds: 0.0, 1.0.
- scaleDownMinWorkerFraction number: Optional. Minimum scale-down threshold as a fraction of total cluster size before scaling occurs. For example, in a 20-worker cluster, a threshold of 0.1 means the autoscaler must recommend at least a 2 worker scale-down for the cluster to scale. A threshold of 0 means the autoscaler will scale down on any recommended change.Bounds: 0.0, 1.0. Default: 0.0.
- scaleUpFactor number: Required. Fraction of average YARN pending memory in the last cooldown period for which to add workers. A scale-up factor of 1.0 will result in scaling up so that there is no pending memory remaining after the update (more aggressive scaling). A scale-up factor closer to 0 will result in a smaller magnitude of scaling up (less aggressive scaling). See How autoscaling works for more information.Bounds: 0.0, 1.0.
- scaleUpMinWorkerFraction number: Optional. Minimum scale-up threshold as a fraction of total cluster size before scaling occurs. For example, in a 20-worker cluster, a threshold of 0.1 means the autoscaler must recommend at least a 2-worker scale-up for the cluster to scale. A threshold of 0 means the autoscaler will scale up on any recommended change.Bounds: 0.0, 1.0. Default: 0.0.

Binding

Binding object: Associates members with a role.
- condition Expr
- members array: Specifies the identities requesting access for a Cloud Platform resource. members can have the following values: allUsers: A special identifier that represents anyone who is on the internet; with or without a Google account. allAuthenticatedUsers: A special identifier that represents anyone who is authenticated with a Google account or a service account. user:{emailid}: An email address that represents a specific Google account. For example, [email protected] . serviceAccount:{emailid}: An email address that represents a service account. For example, [email protected]. group:{emailid}: An email address that represents a Google group. For example, [email protected]. deleted:user:{emailid}?uid={uniqueid}: An email address (plus unique identifier) representing a user that has been recently deleted. For example, [email protected]?uid=123456789012345678901. If the user is recovered, this value reverts to user:{emailid} and the recovered user retains the role in the binding. deleted:serviceAccount:{emailid}?uid={uniqueid}: An email address (plus unique identifier) representing a service account that has been recently deleted. For example, [email protected]?uid=123456789012345678901. If the service account is undeleted, this value reverts to serviceAccount:{emailid} and the undeleted service account retains the role in the binding. deleted:group:{emailid}?uid={uniqueid}: An email address (plus unique identifier) representing a Google group that has been recently deleted. For example, [email protected]?uid=123456789012345678901. If the group is recovered, this value reverts to group:{emailid} and the recovered group retains the role in the binding. domain:{domain}: The G Suite domain (primary) that represents all the users of that domain. For example, google.com or example.com.
  - items string
- role string: Role that is assigned to members. For example, roles/viewer, roles/editor, or roles/owner.

CancelJobRequest

CancelJobRequest object: A request to cancel a job.

Cluster

Cluster object: Describes the identifying information, config, and status of a cluster of Compute Engine instances.
- clusterName string: Required. The cluster name. Cluster names within a project must be unique. Names of deleted clusters can be reused.
- clusterUuid string: Output only. A cluster UUID (Unique Universal Identifier). Dataproc generates this value when it creates the cluster.
- config ClusterConfig
- labels object: Optional. The labels to associate with this cluster. Label keys must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). Label values may be empty, but, if present, must contain 1 to 63 characters, and must conform to RFC 1035 (https://www.ietf.org/rfc/rfc1035.txt). No more than 32 labels can be associated with a cluster.
- metrics ClusterMetrics
- projectId string: Required. The Google Cloud Platform project ID that the cluster belongs to.
- status ClusterStatus
- statusHistory array: Output only. The previous cluster status.
  - items ClusterStatus

ClusterConfig

ClusterConfig object: The cluster config.
- autoscalingConfig AutoscalingConfig
- configBucket string: Optional. A Cloud Storage bucket use

Published

Vulnerabilities

Links

Maintainers

Keywords

Readme

@datafire/google_dataproc

Installation and Usage

Description

Actions

oauthCallback

Input

Output

oauthRefresh

Input

Output

dataproc.projects.regions.clusters.list

Input

Output

dataproc.projects.regions.clusters.create

Input

Output

dataproc.projects.regions.clusters.delete

Input

Output

dataproc.projects.regions.clusters.get

Input

Output

dataproc.projects.regions.clusters.patch

Input

Output

dataproc.projects.regions.clusters.diagnose

Input

Output

dataproc.projects.regions.clusters.start

Input

Output

dataproc.projects.regions.clusters.stop

Input

Output

dataproc.projects.regions.jobs.list

Input

Output

dataproc.projects.regions.jobs.delete

Input

Output

dataproc.projects.regions.jobs.get

Input

Output

dataproc.projects.regions.jobs.patch

Input

Output

dataproc.projects.regions.jobs.cancel

Input

Output

dataproc.projects.regions.jobs.submit

Input

Output

dataproc.projects.regions.jobs.submitAsOperation

Input

Output

dataproc.projects.regions.workflowTemplates.delete

Input

Output

dataproc.projects.regions.workflowTemplates.get

Input

Output

dataproc.projects.regions.workflowTemplates.update

Input

Output

dataproc.projects.regions.operations.cancel

Input

Output

dataproc.projects.regions.workflowTemplates.instantiate

Input

Output

dataproc.projects.regions.autoscalingPolicies.list

Input

Output

dataproc.projects.regions.autoscalingPolicies.create