npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2024 – Pkg Stats / Ryan Hefner

@datafire/google_dataflow

v6.0.0

Published

DataFire integration for Dataflow API

Downloads

6

Vulnerabilities

Powered byPowered by Snyk
  • 0High
  • 0Medium
  • 0Low

Links

Git

Maintainers

datafiredatafire

Keywords

Readme

@datafire/google_dataflow

Client library for Dataflow API

Installation and Usage

npm install --save @datafire/google_dataflow
let google_dataflow = require('@datafire/google_dataflow').create({
  access_token: "",
  refresh_token: "",
  client_id: "",
  client_secret: "",
  redirect_uri: ""
});

.then(data => {
  console.log(data);
});

Description

Manages Google Cloud Dataflow projects on Google Cloud Platform.

Actions

oauthCallback

Exchange the code passed to your redirect URI for an access_token

google_dataflow.oauthCallback({
  "code": ""
}, context)

Input

  • input object
    • code required string

Output

  • output object
    • access_token string
    • refresh_token string
    • token_type string
    • scope string
    • expiration string

oauthRefresh

Exchange a refresh_token for an access_token

google_dataflow.oauthRefresh(null, context)

Input

This action has no parameters

Output

  • output object
    • access_token string
    • refresh_token string
    • token_type string
    • scope string
    • expiration string

dataflow.projects.workerMessages

Send a worker_message to the service.

google_dataflow.dataflow.projects.workerMessages({
  "projectId": ""
}, context)

Input

  • input object
    • projectId required string: The project to send the WorkerMessages to.
    • body SendWorkerMessagesRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.jobs.list

List the jobs of a project. To list the jobs of a project in a region, we recommend using projects.locations.jobs.list with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints). To list the all jobs across all regions, use projects.jobs.aggregated. Using projects.jobs.list is not recommended, as you can only get the list of jobs that are running in us-central1.

google_dataflow.dataflow.projects.jobs.list({
  "projectId": ""
}, context)

Input

  • input object
    • projectId required string: The project which owns the jobs.
    • filter string (values: UNKNOWN, ALL, TERMINATED, ACTIVE): The kind of filter to use.
    • location string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains this job.
    • pageSize integer: If there are many jobs, limit response to at most this many. The actual number of jobs returned will be the lesser of max_responses and an unspecified server-defined limit.
    • pageToken string: Set this to the 'next_page_token' field of a previous response to request additional results in a long list.
    • view string (values: JOB_VIEW_UNKNOWN, JOB_VIEW_SUMMARY, JOB_VIEW_ALL, JOB_VIEW_DESCRIPTION): Level of information requested in response. Default is JOB_VIEW_SUMMARY.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.jobs.create

Creates a Cloud Dataflow job. To create a job, we recommend using projects.locations.jobs.create with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints). Using projects.jobs.create is not recommended, as your job will always start in us-central1.

google_dataflow.dataflow.projects.jobs.create({
  "projectId": ""
}, context)

Input

  • input object
    • projectId required string: The ID of the Cloud Platform project that the job belongs to.
    • location string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains this job.
    • replaceJobId string: Deprecated. This field is now in the Job message.
    • view string (values: JOB_VIEW_UNKNOWN, JOB_VIEW_SUMMARY, JOB_VIEW_ALL, JOB_VIEW_DESCRIPTION): The level of information requested in response.
    • body Job
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.jobs.get

Gets the state of the specified Cloud Dataflow job. To get the state of a job, we recommend using projects.locations.jobs.get with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints). Using projects.jobs.get is not recommended, as you can only get the state of jobs that are running in us-central1.

google_dataflow.dataflow.projects.jobs.get({
  "projectId": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: The ID of the Cloud Platform project that the job belongs to.
    • jobId required string: The job ID.
    • location string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains this job.
    • view string (values: JOB_VIEW_UNKNOWN, JOB_VIEW_SUMMARY, JOB_VIEW_ALL, JOB_VIEW_DESCRIPTION): The level of information requested in response.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.jobs.update

Updates the state of an existing Cloud Dataflow job. To update the state of an existing job, we recommend using projects.locations.jobs.update with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints). Using projects.jobs.update is not recommended, as you can only update the state of jobs that are running in us-central1.

google_dataflow.dataflow.projects.jobs.update({
  "projectId": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: The ID of the Cloud Platform project that the job belongs to.
    • jobId required string: The job ID.
    • location string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains this job.
    • body Job
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.jobs.debug.getConfig

Get encoded debug configuration for component. Not cacheable.

google_dataflow.dataflow.projects.jobs.debug.getConfig({
  "projectId": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: The project id.
    • jobId required string: The job id.
    • body GetDebugConfigRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.jobs.debug.sendCapture

Send encoded debug capture data for component.

google_dataflow.dataflow.projects.jobs.debug.sendCapture({
  "projectId": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: The project id.
    • jobId required string: The job id.
    • body SendDebugCaptureRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.jobs.messages.list

Request the job status. To request the status of a job, we recommend using projects.locations.jobs.messages.list with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints). Using projects.jobs.messages.list is not recommended, as you can only request the status of jobs that are running in us-central1.

google_dataflow.dataflow.projects.jobs.messages.list({
  "projectId": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: A project id.
    • jobId required string: The job to get messages about.
    • endTime string: Return only messages with timestamps < end_time. The default is now (i.e. return up to the latest messages available).
    • location string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the job specified by job_id.
    • minimumImportance string (values: JOB_MESSAGE_IMPORTANCE_UNKNOWN, JOB_MESSAGE_DEBUG, JOB_MESSAGE_DETAILED, JOB_MESSAGE_BASIC, JOB_MESSAGE_WARNING, JOB_MESSAGE_ERROR): Filter to only get messages with importance >= level
    • pageSize integer: If specified, determines the maximum number of messages to return. If unspecified, the service may choose an appropriate default, or may return an arbitrarily large number of results.
    • pageToken string: If supplied, this should be the value of next_page_token returned by an earlier call. This will cause the next page of results to be returned.
    • startTime string: If specified, return only messages with timestamps >= start_time. The default is the job creation time (i.e. beginning of messages).
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.jobs.getMetrics

Request the job status. To request the status of a job, we recommend using projects.locations.jobs.getMetrics with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints). Using projects.jobs.getMetrics is not recommended, as you can only request the status of jobs that are running in us-central1.

google_dataflow.dataflow.projects.jobs.getMetrics({
  "projectId": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: A project id.
    • jobId required string: The job to get metrics for.
    • location string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the job specified by job_id.
    • startTime string: Return only metric data that has changed since this time. Default is to return all information about all metrics for the job.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.jobs.workItems.lease

Leases a dataflow WorkItem to run.

google_dataflow.dataflow.projects.jobs.workItems.lease({
  "projectId": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: Identifies the project this worker belongs to.
    • jobId required string: Identifies the workflow job this worker belongs to.
    • body LeaseWorkItemRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.jobs.workItems.reportStatus

Reports the status of dataflow WorkItems leased by a worker.

google_dataflow.dataflow.projects.jobs.workItems.reportStatus({
  "projectId": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: The project which owns the WorkItem's job.
    • jobId required string: The job which the WorkItem is part of.
    • body ReportWorkItemStatusRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.jobs.snapshot

Snapshot the state of a streaming job.

google_dataflow.dataflow.projects.jobs.snapshot({
  "projectId": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: The project which owns the job to be snapshotted.
    • jobId required string: The job to be snapshotted.
    • body SnapshotJobRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.jobs.aggregated

List the jobs of a project across all regions.

google_dataflow.dataflow.projects.jobs.aggregated({
  "projectId": ""
}, context)

Input

  • input object
    • projectId required string: The project which owns the jobs.
    • filter string (values: UNKNOWN, ALL, TERMINATED, ACTIVE): The kind of filter to use.
    • location string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains this job.
    • pageSize integer: If there are many jobs, limit response to at most this many. The actual number of jobs returned will be the lesser of max_responses and an unspecified server-defined limit.
    • pageToken string: Set this to the 'next_page_token' field of a previous response to request additional results in a long list.
    • view string (values: JOB_VIEW_UNKNOWN, JOB_VIEW_SUMMARY, JOB_VIEW_ALL, JOB_VIEW_DESCRIPTION): Level of information requested in response. Default is JOB_VIEW_SUMMARY.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.workerMessages

Send a worker_message to the service.

google_dataflow.dataflow.projects.locations.workerMessages({
  "projectId": "",
  "location": ""
}, context)

Input

  • input object
    • projectId required string: The project to send the WorkerMessages to.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the job.
    • body SendWorkerMessagesRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.flexTemplates.launch

Launch a job with a FlexTemplate.

google_dataflow.dataflow.projects.locations.flexTemplates.launch({
  "projectId": "",
  "location": ""
}, context)

Input

  • input object
    • projectId required string: Required. The ID of the Cloud Platform project that the job belongs to.
    • location required string: Required. The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) to which to direct the request. E.g., us-central1, us-west1.
    • body LaunchFlexTemplateRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.jobs.list

List the jobs of a project. To list the jobs of a project in a region, we recommend using projects.locations.jobs.list with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints). To list the all jobs across all regions, use projects.jobs.aggregated. Using projects.jobs.list is not recommended, as you can only get the list of jobs that are running in us-central1.

google_dataflow.dataflow.projects.locations.jobs.list({
  "projectId": "",
  "location": ""
}, context)

Input

  • input object
    • projectId required string: The project which owns the jobs.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains this job.
    • filter string (values: UNKNOWN, ALL, TERMINATED, ACTIVE): The kind of filter to use.
    • pageSize integer: If there are many jobs, limit response to at most this many. The actual number of jobs returned will be the lesser of max_responses and an unspecified server-defined limit.
    • pageToken string: Set this to the 'next_page_token' field of a previous response to request additional results in a long list.
    • view string (values: JOB_VIEW_UNKNOWN, JOB_VIEW_SUMMARY, JOB_VIEW_ALL, JOB_VIEW_DESCRIPTION): Level of information requested in response. Default is JOB_VIEW_SUMMARY.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.jobs.create

Creates a Cloud Dataflow job. To create a job, we recommend using projects.locations.jobs.create with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints). Using projects.jobs.create is not recommended, as your job will always start in us-central1.

google_dataflow.dataflow.projects.locations.jobs.create({
  "projectId": "",
  "location": ""
}, context)

Input

  • input object
    • projectId required string: The ID of the Cloud Platform project that the job belongs to.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains this job.
    • replaceJobId string: Deprecated. This field is now in the Job message.
    • view string (values: JOB_VIEW_UNKNOWN, JOB_VIEW_SUMMARY, JOB_VIEW_ALL, JOB_VIEW_DESCRIPTION): The level of information requested in response.
    • body Job
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.jobs.get

Gets the state of the specified Cloud Dataflow job. To get the state of a job, we recommend using projects.locations.jobs.get with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints). Using projects.jobs.get is not recommended, as you can only get the state of jobs that are running in us-central1.

google_dataflow.dataflow.projects.locations.jobs.get({
  "projectId": "",
  "location": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: The ID of the Cloud Platform project that the job belongs to.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains this job.
    • jobId required string: The job ID.
    • view string (values: JOB_VIEW_UNKNOWN, JOB_VIEW_SUMMARY, JOB_VIEW_ALL, JOB_VIEW_DESCRIPTION): The level of information requested in response.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.jobs.update

Updates the state of an existing Cloud Dataflow job. To update the state of an existing job, we recommend using projects.locations.jobs.update with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints). Using projects.jobs.update is not recommended, as you can only update the state of jobs that are running in us-central1.

google_dataflow.dataflow.projects.locations.jobs.update({
  "projectId": "",
  "location": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: The ID of the Cloud Platform project that the job belongs to.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains this job.
    • jobId required string: The job ID.
    • body Job
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.jobs.debug.getConfig

Get encoded debug configuration for component. Not cacheable.

google_dataflow.dataflow.projects.locations.jobs.debug.getConfig({
  "projectId": "",
  "location": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: The project id.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the job specified by job_id.
    • jobId required string: The job id.
    • body GetDebugConfigRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.jobs.debug.sendCapture

Send encoded debug capture data for component.

google_dataflow.dataflow.projects.locations.jobs.debug.sendCapture({
  "projectId": "",
  "location": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: The project id.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the job specified by job_id.
    • jobId required string: The job id.
    • body SendDebugCaptureRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.jobs.getExecutionDetails

Request detailed information about the execution status of the job. EXPERIMENTAL. This API is subject to change or removal without notice.

google_dataflow.dataflow.projects.locations.jobs.getExecutionDetails({
  "projectId": "",
  "location": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: A project id.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the job specified by job_id.
    • jobId required string: The job to get execution details for.
    • pageSize integer: If specified, determines the maximum number of stages to return. If unspecified, the service may choose an appropriate default, or may return an arbitrarily large number of results.
    • pageToken string: If supplied, this should be the value of next_page_token returned by an earlier call. This will cause the next page of results to be returned.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.jobs.messages.list

Request the job status. To request the status of a job, we recommend using projects.locations.jobs.messages.list with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints). Using projects.jobs.messages.list is not recommended, as you can only request the status of jobs that are running in us-central1.

google_dataflow.dataflow.projects.locations.jobs.messages.list({
  "projectId": "",
  "location": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: A project id.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the job specified by job_id.
    • jobId required string: The job to get messages about.
    • endTime string: Return only messages with timestamps < end_time. The default is now (i.e. return up to the latest messages available).
    • minimumImportance string (values: JOB_MESSAGE_IMPORTANCE_UNKNOWN, JOB_MESSAGE_DEBUG, JOB_MESSAGE_DETAILED, JOB_MESSAGE_BASIC, JOB_MESSAGE_WARNING, JOB_MESSAGE_ERROR): Filter to only get messages with importance >= level
    • pageSize integer: If specified, determines the maximum number of messages to return. If unspecified, the service may choose an appropriate default, or may return an arbitrarily large number of results.
    • pageToken string: If supplied, this should be the value of next_page_token returned by an earlier call. This will cause the next page of results to be returned.
    • startTime string: If specified, return only messages with timestamps >= start_time. The default is the job creation time (i.e. beginning of messages).
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.jobs.getMetrics

Request the job status. To request the status of a job, we recommend using projects.locations.jobs.getMetrics with a [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints). Using projects.jobs.getMetrics is not recommended, as you can only request the status of jobs that are running in us-central1.

google_dataflow.dataflow.projects.locations.jobs.getMetrics({
  "projectId": "",
  "location": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: A project id.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the job specified by job_id.
    • jobId required string: The job to get metrics for.
    • startTime string: Return only metric data that has changed since this time. Default is to return all information about all metrics for the job.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.jobs.snapshots.list

Lists snapshots.

google_dataflow.dataflow.projects.locations.jobs.snapshots.list({
  "projectId": "",
  "location": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: The project ID to list snapshots for.
    • location required string: The location to list snapshots in.
    • jobId required string: If specified, list snapshots created from this job.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.jobs.stages.getExecutionDetails

Request detailed information about the execution status of a stage of the job. EXPERIMENTAL. This API is subject to change or removal without notice.

google_dataflow.dataflow.projects.locations.jobs.stages.getExecutionDetails({
  "projectId": "",
  "location": "",
  "jobId": "",
  "stageId": ""
}, context)

Input

  • input object
    • projectId required string: A project id.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the job specified by job_id.
    • jobId required string: The job to get execution details for.
    • stageId required string: The stage for which to fetch information.
    • endTime string: Upper time bound of work items to include, by start time.
    • pageSize integer: If specified, determines the maximum number of work items to return. If unspecified, the service may choose an appropriate default, or may return an arbitrarily large number of results.
    • pageToken string: If supplied, this should be the value of next_page_token returned by an earlier call. This will cause the next page of results to be returned.
    • startTime string: Lower time bound of work items to include, by start time.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.jobs.workItems.lease

Leases a dataflow WorkItem to run.

google_dataflow.dataflow.projects.locations.jobs.workItems.lease({
  "projectId": "",
  "location": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: Identifies the project this worker belongs to.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the WorkItem's job.
    • jobId required string: Identifies the workflow job this worker belongs to.
    • body LeaseWorkItemRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.jobs.workItems.reportStatus

Reports the status of dataflow WorkItems leased by a worker.

google_dataflow.dataflow.projects.locations.jobs.workItems.reportStatus({
  "projectId": "",
  "location": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: The project which owns the WorkItem's job.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the WorkItem's job.
    • jobId required string: The job which the WorkItem is part of.
    • body ReportWorkItemStatusRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.jobs.snapshot

Snapshot the state of a streaming job.

google_dataflow.dataflow.projects.locations.jobs.snapshot({
  "projectId": "",
  "location": "",
  "jobId": ""
}, context)

Input

  • input object
    • projectId required string: The project which owns the job to be snapshotted.
    • location required string: The location that contains this job.
    • jobId required string: The job to be snapshotted.
    • body SnapshotJobRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.snapshots.list

Lists snapshots.

google_dataflow.dataflow.projects.locations.snapshots.list({
  "projectId": "",
  "location": ""
}, context)

Input

  • input object
    • projectId required string: The project ID to list snapshots for.
    • location required string: The location to list snapshots in.
    • jobId string: If specified, list snapshots created from this job.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.snapshots.delete

Deletes a snapshot.

google_dataflow.dataflow.projects.locations.snapshots.delete({
  "projectId": "",
  "location": "",
  "snapshotId": ""
}, context)

Input

  • input object
    • projectId required string: The ID of the Cloud Platform project that the snapshot belongs to.
    • location required string: The location that contains this snapshot.
    • snapshotId required string: The ID of the snapshot.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.snapshots.get

Gets information about a snapshot.

google_dataflow.dataflow.projects.locations.snapshots.get({
  "projectId": "",
  "location": "",
  "snapshotId": ""
}, context)

Input

  • input object
    • projectId required string: The ID of the Cloud Platform project that the snapshot belongs to.
    • location required string: The location that contains this snapshot.
    • snapshotId required string: The ID of the snapshot.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.sql.validate

Validates a GoogleSQL query for Cloud Dataflow syntax. Will always confirm the given query parses correctly, and if able to look up schema information from DataCatalog, will validate that the query analyzes properly as well.

google_dataflow.dataflow.projects.locations.sql.validate({
  "projectId": "",
  "location": ""
}, context)

Input

  • input object
    • projectId required string: Required. The ID of the Cloud Platform project that the job belongs to.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) to which to direct the request.
    • query string: The sql query to validate.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.templates.create

Creates a Cloud Dataflow job from a template.

google_dataflow.dataflow.projects.locations.templates.create({
  "projectId": "",
  "location": ""
}, context)

Input

  • input object
    • projectId required string: Required. The ID of the Cloud Platform project that the job belongs to.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) to which to direct the request.
    • body CreateJobFromTemplateRequest
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.templates.get

Get the template associated with a template.

google_dataflow.dataflow.projects.locations.templates.get({
  "projectId": "",
  "location": ""
}, context)

Input

  • input object
    • projectId required string: Required. The ID of the Cloud Platform project that the job belongs to.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) to which to direct the request.
    • gcsPath string: Required. A Cloud Storage path to the template from which to create the job. Must be valid Cloud Storage URL, beginning with 'gs://'.
    • view string (values: METADATA_ONLY): The view to retrieve. Defaults to METADATA_ONLY.
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: API key. Your API key identifies your project and provides you with API access, quota, and reports. Required unless you provide an OAuth 2.0 token.
    • oauth_token string: OAuth 2.0 token for the current user.
    • prettyPrint boolean: Returns response with indentations and line breaks.
    • quotaUser string: Available to use for quota purposes for server-side applications. Can be any arbitrary string assigned to a user, but should not exceed 40 characters.
    • upload_protocol string: Upload protocol for media (e.g. "raw", "multipart").
    • uploadType string: Legacy upload protocol for media (e.g. "media", "multipart").

Output

dataflow.projects.locations.templates.launch

Launch a template.

google_dataflow.dataflow.projects.locations.templates.launch({
  "projectId": "",
  "location": ""
}, context)

Input

  • input object
    • projectId required string: Required. The ID of the Cloud Platform project that the job belongs to.
    • location required string: The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) to which to direct the request.
    • dynamicTemplate.gcsPath string: Path to dynamic template spec file on GCS. The file must be a Json serialized DynamicTemplateFieSpec object.
    • dynamicTemplate.stagingLocation string: Cloud Storage path for staging dependencies. Must be a valid Cloud Storage URL, beginning with gs://.
    • gcsPath string: A Cloud Storage path to the template from which to create the job. Must be valid Cloud Storage URL, beginning with 'gs://'.
    • validateOnly boolean: If true, the request is validated but not actually executed. Defaults to false.
    • body LaunchTemplateParameters
    • $.xgafv string (values: 1, 2): V1 error format.
    • access_token string: OAuth access token.
    • alt string (values: json, media, proto): Data format for response.
    • callback string: JSONP
    • fields string: Selector specifying which fields to include in a partial response.
    • key string: