@datafire/amazonaws_firehose
v5.0.0
Published
DataFire integration for Amazon Kinesis Firehose
Downloads
25
Readme
@datafire/amazonaws_firehose
Client library for Amazon Kinesis Firehose
Installation and Usage
npm install --save @datafire/amazonaws_firehose
let amazonaws_firehose = require('@datafire/amazonaws_firehose').create({
accessKeyId: "",
secretAccessKey: "",
region: ""
});
amazonaws_firehose.CreateDeliveryStream({
"DeliveryStreamName": ""
}).then(data => {
console.log(data);
});
Description
Amazon Kinesis Firehose API Reference Amazon Kinesis Firehose is a fully managed service that delivers real-time streaming data to destinations such as Amazon Simple Storage Service (Amazon S3), Amazon Elasticsearch Service (Amazon ES), and Amazon Redshift.
Actions
CreateDeliveryStream
amazonaws_firehose.CreateDeliveryStream({
"DeliveryStreamName": ""
}, context)
Input
- input
object
- DeliveryStreamName required DeliveryStreamName
- DeliveryStreamType DeliveryStreamType
- ElasticsearchDestinationConfiguration ElasticsearchDestinationConfiguration
- ExtendedS3DestinationConfiguration ExtendedS3DestinationConfiguration
- KinesisStreamSourceConfiguration KinesisStreamSourceConfiguration
- RedshiftDestinationConfiguration RedshiftDestinationConfiguration
- S3DestinationConfiguration S3DestinationConfiguration
- SplunkDestinationConfiguration SplunkDestinationConfiguration
Output
- output CreateDeliveryStreamOutput
DeleteDeliveryStream
amazonaws_firehose.DeleteDeliveryStream({
"DeliveryStreamName": ""
}, context)
Input
- input
object
- DeliveryStreamName required DeliveryStreamName
Output
- output DeleteDeliveryStreamOutput
DescribeDeliveryStream
amazonaws_firehose.DescribeDeliveryStream({
"DeliveryStreamName": ""
}, context)
Input
- input
object
- DeliveryStreamName required DeliveryStreamName
- ExclusiveStartDestinationId DestinationId
- Limit DescribeDeliveryStreamInputLimit
Output
- output DescribeDeliveryStreamOutput
ListDeliveryStreams
amazonaws_firehose.ListDeliveryStreams({}, context)
Input
- input
object
- DeliveryStreamType DeliveryStreamType
- ExclusiveStartDeliveryStreamName DeliveryStreamName
- Limit ListDeliveryStreamsInputLimit
Output
- output ListDeliveryStreamsOutput
PutRecord
amazonaws_firehose.PutRecord({
"DeliveryStreamName": "",
"Record": {
"Data": ""
}
}, context)
Input
- input
object
- DeliveryStreamName required DeliveryStreamName
- Record required Record
Output
- output PutRecordOutput
PutRecordBatch
amazonaws_firehose.PutRecordBatch({
"DeliveryStreamName": "",
"Records": []
}, context)
Input
- input
object
- DeliveryStreamName required DeliveryStreamName
- Records required PutRecordBatchRequestEntryList
Output
- output PutRecordBatchOutput
UpdateDestination
amazonaws_firehose.UpdateDestination({
"DeliveryStreamName": "",
"CurrentDeliveryStreamVersionId": "",
"DestinationId": ""
}, context)
Input
- input
object
- CurrentDeliveryStreamVersionId required DeliveryStreamVersionId
- DeliveryStreamName required DeliveryStreamName
- DestinationId required DestinationId
- ElasticsearchDestinationUpdate ElasticsearchDestinationUpdate
- ExtendedS3DestinationUpdate ExtendedS3DestinationUpdate
- RedshiftDestinationUpdate RedshiftDestinationUpdate
- S3DestinationUpdate S3DestinationUpdate
- SplunkDestinationUpdate SplunkDestinationUpdate
Output
- output UpdateDestinationOutput
Definitions
AWSKMSKeyARN
- AWSKMSKeyARN
string
BooleanObject
- BooleanObject
boolean
BucketARN
- BucketARN
string
BufferingHints
- BufferingHints
object
: Describes hints for the buffering to perform before delivering data to the destination. Please note that these options are treated as hints, and therefore Kinesis Firehose may choose to use different values when it is optimal.- IntervalInSeconds IntervalInSeconds
- SizeInMBs SizeInMBs
CloudWatchLoggingOptions
- CloudWatchLoggingOptions
object
: Describes the Amazon CloudWatch logging options for your delivery stream.- Enabled BooleanObject
- LogGroupName LogGroupName
- LogStreamName LogStreamName
ClusterJDBCURL
- ClusterJDBCURL
string
CompressionFormat
- CompressionFormat
string
(values: UNCOMPRESSED, GZIP, ZIP, Snappy)
ConcurrentModificationException
- ConcurrentModificationException
object
: Another modification has already happened. Fetch VersionId again and use it to update the destination.- message ErrorMessage
CopyCommand
- CopyCommand
object
: Describes a COPY command for Amazon Redshift.- CopyOptions CopyOptions
- DataTableColumns DataTableColumns
- DataTableName required DataTableName
CopyOptions
- CopyOptions
string
CreateDeliveryStreamInput
- CreateDeliveryStreamInput
object
- DeliveryStreamName required DeliveryStreamName
- DeliveryStreamType DeliveryStreamType
- ElasticsearchDestinationConfiguration ElasticsearchDestinationConfiguration
- ExtendedS3DestinationConfiguration ExtendedS3DestinationConfiguration
- KinesisStreamSourceConfiguration KinesisStreamSourceConfiguration
- RedshiftDestinationConfiguration RedshiftDestinationConfiguration
- S3DestinationConfiguration S3DestinationConfiguration
- SplunkDestinationConfiguration SplunkDestinationConfiguration
CreateDeliveryStreamOutput
- CreateDeliveryStreamOutput
object
- DeliveryStreamARN DeliveryStreamARN
Data
- Data
string
DataTableColumns
- DataTableColumns
string
DataTableName
- DataTableName
string
DeleteDeliveryStreamInput
- DeleteDeliveryStreamInput
object
- DeliveryStreamName required DeliveryStreamName
DeleteDeliveryStreamOutput
- DeleteDeliveryStreamOutput
object
DeliveryStartTimestamp
- DeliveryStartTimestamp
string
DeliveryStreamARN
- DeliveryStreamARN
string
DeliveryStreamDescription
- DeliveryStreamDescription
object
: Contains information about a delivery stream.- CreateTimestamp Timestamp
- DeliveryStreamARN required DeliveryStreamARN
- DeliveryStreamName required DeliveryStreamName
- DeliveryStreamStatus required DeliveryStreamStatus
- DeliveryStreamType required DeliveryStreamType
- Destinations required DestinationDescriptionList
- HasMoreDestinations required BooleanObject
- LastUpdateTimestamp Timestamp
- Source SourceDescription
- VersionId required DeliveryStreamVersionId
DeliveryStreamName
- DeliveryStreamName
string
DeliveryStreamNameList
- DeliveryStreamNameList
array
- items DeliveryStreamName
DeliveryStreamStatus
- DeliveryStreamStatus
string
(values: CREATING, DELETING, ACTIVE)
DeliveryStreamType
- DeliveryStreamType
string
(values: DirectPut, KinesisStreamAsSource)
DeliveryStreamVersionId
- DeliveryStreamVersionId
string
DescribeDeliveryStreamInput
- DescribeDeliveryStreamInput
object
- DeliveryStreamName required DeliveryStreamName
- ExclusiveStartDestinationId DestinationId
- Limit DescribeDeliveryStreamInputLimit
DescribeDeliveryStreamInputLimit
- DescribeDeliveryStreamInputLimit
integer
DescribeDeliveryStreamOutput
- DescribeDeliveryStreamOutput
object
- DeliveryStreamDescription required DeliveryStreamDescription
DestinationDescription
- DestinationDescription
object
: Describes the destination for a delivery stream.- DestinationId required DestinationId
- ElasticsearchDestinationDescription ElasticsearchDestinationDescription
- ExtendedS3DestinationDescription ExtendedS3DestinationDescription
- RedshiftDestinationDescription RedshiftDestinationDescription
- S3DestinationDescription S3DestinationDescription
- SplunkDestinationDescription SplunkDestinationDescription
DestinationDescriptionList
- DestinationDescriptionList
array
- items DestinationDescription
DestinationId
- DestinationId
string
ElasticsearchBufferingHints
- ElasticsearchBufferingHints
object
: Describes the buffering to perform before delivering data to the Amazon ES destination.- IntervalInSeconds ElasticsearchBufferingIntervalInSeconds
- SizeInMBs ElasticsearchBufferingSizeInMBs
ElasticsearchBufferingIntervalInSeconds
- ElasticsearchBufferingIntervalInSeconds
integer
ElasticsearchBufferingSizeInMBs
- ElasticsearchBufferingSizeInMBs
integer
ElasticsearchDestinationConfiguration
- ElasticsearchDestinationConfiguration
object
: Describes the configuration of a destination in Amazon ES.- BufferingHints ElasticsearchBufferingHints
- CloudWatchLoggingOptions CloudWatchLoggingOptions
- DomainARN required ElasticsearchDomainARN
- IndexName required ElasticsearchIndexName
- IndexRotationPeriod ElasticsearchIndexRotationPeriod
- ProcessingConfiguration ProcessingConfiguration
- RetryOptions ElasticsearchRetryOptions
- RoleARN required RoleARN
- S3BackupMode ElasticsearchS3BackupMode
- S3Configuration required S3DestinationConfiguration
- TypeName required ElasticsearchTypeName
ElasticsearchDestinationDescription
- ElasticsearchDestinationDescription
object
: The destination description in Amazon ES.- BufferingHints ElasticsearchBufferingHints
- CloudWatchLoggingOptions CloudWatchLoggingOptions
- DomainARN ElasticsearchDomainARN
- IndexName ElasticsearchIndexName
- IndexRotationPeriod ElasticsearchIndexRotationPeriod
- ProcessingConfiguration ProcessingConfiguration
- RetryOptions ElasticsearchRetryOptions
- RoleARN RoleARN
- S3BackupMode ElasticsearchS3BackupMode
- S3DestinationDescription S3DestinationDescription
- TypeName ElasticsearchTypeName
ElasticsearchDestinationUpdate
- ElasticsearchDestinationUpdate
object
: Describes an update for a destination in Amazon ES.- BufferingHints ElasticsearchBufferingHints
- CloudWatchLoggingOptions CloudWatchLoggingOptions
- DomainARN ElasticsearchDomainARN
- IndexName ElasticsearchIndexName
- IndexRotationPeriod ElasticsearchIndexRotationPeriod
- ProcessingConfiguration ProcessingConfiguration
- RetryOptions ElasticsearchRetryOptions
- RoleARN RoleARN
- S3Update S3DestinationUpdate
- TypeName ElasticsearchTypeName
ElasticsearchDomainARN
- ElasticsearchDomainARN
string
ElasticsearchIndexName
- ElasticsearchIndexName
string
ElasticsearchIndexRotationPeriod
- ElasticsearchIndexRotationPeriod
string
(values: NoRotation, OneHour, OneDay, OneWeek, OneMonth)
ElasticsearchRetryDurationInSeconds
- ElasticsearchRetryDurationInSeconds
integer
ElasticsearchRetryOptions
- ElasticsearchRetryOptions
object
: Configures retry behavior in case Kinesis Firehose is unable to deliver documents to Amazon ES.- DurationInSeconds ElasticsearchRetryDurationInSeconds
ElasticsearchS3BackupMode
- ElasticsearchS3BackupMode
string
(values: FailedDocumentsOnly, AllDocuments)
ElasticsearchTypeName
- ElasticsearchTypeName
string
EncryptionConfiguration
- EncryptionConfiguration
object
: Describes the encryption for a destination in Amazon S3.- KMSEncryptionConfig KMSEncryptionConfig
- NoEncryptionConfig NoEncryptionConfig
ErrorCode
- ErrorCode
string
ErrorMessage
- ErrorMessage
string
ExtendedS3DestinationConfiguration
- ExtendedS3DestinationConfiguration
object
: Describes the configuration of a destination in Amazon S3.- BucketARN required BucketARN
- BufferingHints BufferingHints
- CloudWatchLoggingOptions CloudWatchLoggingOptions
- CompressionFormat CompressionFormat
- EncryptionConfiguration EncryptionConfiguration
- Prefix Prefix
- ProcessingConfiguration ProcessingConfiguration
- RoleARN required RoleARN
- S3BackupConfiguration S3DestinationConfiguration
- S3BackupMode S3BackupMode
ExtendedS3DestinationDescription
- ExtendedS3DestinationDescription
object
: Describes a destination in Amazon S3.- BucketARN required BucketARN
- BufferingHints required BufferingHints
- CloudWatchLoggingOptions CloudWatchLoggingOptions
- CompressionFormat required CompressionFormat
- EncryptionConfiguration required EncryptionConfiguration
- Prefix Prefix
- ProcessingConfiguration ProcessingConfiguration
- RoleARN required RoleARN
- S3BackupDescription S3DestinationDescription
- S3BackupMode S3BackupMode
ExtendedS3DestinationUpdate
- ExtendedS3DestinationUpdate
object
: Describes an update for a destination in Amazon S3.- BucketARN BucketARN
- BufferingHints BufferingHints
- CloudWatchLoggingOptions CloudWatchLoggingOptions
- CompressionFormat CompressionFormat
- EncryptionConfiguration EncryptionConfiguration
- Prefix Prefix
- ProcessingConfiguration ProcessingConfiguration
- RoleARN RoleARN
- S3BackupMode S3BackupMode
- S3BackupUpdate S3DestinationUpdate
HECAcknowledgmentTimeoutInSeconds
- HECAcknowledgmentTimeoutInSeconds
integer
HECEndpoint
- HECEndpoint
string
HECEndpointType
- HECEndpointType
string
(values: Raw, Event)
HECToken
- HECToken
string
IntervalInSeconds
- IntervalInSeconds
integer
InvalidArgumentException
- InvalidArgumentException
object
: The specified input parameter has a value that is not valid.- message ErrorMessage
KMSEncryptionConfig
- KMSEncryptionConfig
object
: Describes an encryption key for a destination in Amazon S3.- AWSKMSKeyARN required AWSKMSKeyARN
KinesisStreamARN
- KinesisStreamARN
string
KinesisStreamSourceConfiguration
- KinesisStreamSourceConfiguration
object
: The stream and role ARNs for a Kinesis stream used as the source for a delivery stream.- KinesisStreamARN required KinesisStreamARN
- RoleARN required RoleARN
KinesisStreamSourceDescription
- KinesisStreamSourceDescription
object
: Details about a Kinesis stream used as the source for a Kinesis Firehose delivery stream.- DeliveryStartTimestamp DeliveryStartTimestamp
- KinesisStreamARN KinesisStreamARN
- RoleARN RoleARN
LimitExceededException
- LimitExceededException
object
: You have already reached the limit for a requested resource.- message ErrorMessage
ListDeliveryStreamsInput
- ListDeliveryStreamsInput
object
- DeliveryStreamType DeliveryStreamType
- ExclusiveStartDeliveryStreamName DeliveryStreamName
- Limit ListDeliveryStreamsInputLimit
ListDeliveryStreamsInputLimit
- ListDeliveryStreamsInputLimit
integer
ListDeliveryStreamsOutput
- ListDeliveryStreamsOutput
object
- DeliveryStreamNames required DeliveryStreamNameList
- HasMoreDeliveryStreams required BooleanObject
LogGroupName
- LogGroupName
string
LogStreamName
- LogStreamName
string
NoEncryptionConfig
- NoEncryptionConfig
string
(values: NoEncryption)
NonNegativeIntegerObject
- NonNegativeIntegerObject
integer
Password
- Password
string
Prefix
- Prefix
string
ProcessingConfiguration
- ProcessingConfiguration
object
: Describes a data processing configuration.- Enabled BooleanObject
- Processors ProcessorList
Processor
- Processor
object
: Describes a data processor.- Parameters ProcessorParameterList
- Type required ProcessorType
ProcessorList
- ProcessorList
array
- items Processor
ProcessorParameter
- ProcessorParameter
object
: Describes the processor parameter.- ParameterName required ProcessorParameterName
- ParameterValue required ProcessorParameterValue
ProcessorParameterList
- ProcessorParameterList
array
- items ProcessorParameter
ProcessorParameterName
- ProcessorParameterName
string
(values: LambdaArn, NumberOfRetries, RoleArn, BufferSizeInMBs, BufferIntervalInSeconds)
ProcessorParameterValue
- ProcessorParameterValue
string
ProcessorType
- ProcessorType
string
(values: Lambda)
PutRecordBatchInput
- PutRecordBatchInput
object
- DeliveryStreamName required DeliveryStreamName
- Records required PutRecordBatchRequestEntryList
PutRecordBatchOutput
- PutRecordBatchOutput
object
- FailedPutCount required NonNegativeIntegerObject
- RequestResponses required PutRecordBatchResponseEntryList
PutRecordBatchRequestEntryList
- PutRecordBatchRequestEntryList
array
- items Record
PutRecordBatchResponseEntry
- PutRecordBatchResponseEntry
object
: Contains the result for an individual record from a PutRecordBatch request. If the record is successfully added to your delivery stream, it receives a record ID. If the record fails to be added to your delivery stream, the result includes an error code and an error message.- ErrorCode ErrorCode
- ErrorMessage ErrorMessage
- RecordId PutResponseRecordId
PutRecordBatchResponseEntryList
- PutRecordBatchResponseEntryList
array
PutRecordInput
- PutRecordInput
object
- DeliveryStreamName required DeliveryStreamName
- Record required Record
PutRecordOutput
- PutRecordOutput
object
- RecordId required PutResponseRecordId
PutResponseRecordId
- PutResponseRecordId
string
Record
- Record
object
: The unit of data in a delivery stream.- Data required Data
RedshiftDestinationConfiguration
- RedshiftDestinationConfiguration
object
: Describes the configuration of a destination in Amazon Redshift.- CloudWatchLoggingOptions CloudWatchLoggingOptions
- ClusterJDBCURL required ClusterJDBCURL
- CopyCommand required CopyCommand
- Password required Password
- ProcessingConfiguration ProcessingConfiguration
- RetryOptions RedshiftRetryOptions
- RoleARN required RoleARN
- S3BackupConfiguration S3DestinationConfiguration
- S3BackupMode RedshiftS3BackupMode
- S3Configuration required S3DestinationConfiguration
- Username required Username
RedshiftDestinationDescription
- RedshiftDestinationDescription
object
: Describes a destination in Amazon Redshift.- CloudWatchLoggingOptions CloudWatchLoggingOptions
- ClusterJDBCURL required ClusterJDBCURL
- CopyCommand required CopyCommand
- ProcessingConfiguration ProcessingConfiguration
- RetryOptions RedshiftRetryOptions
- RoleARN required RoleARN
- S3BackupDescription S3DestinationDescription
- S3BackupMode RedshiftS3BackupMode
- S3DestinationDescription required S3DestinationDescription
- Username required Username
RedshiftDestinationUpdate
- RedshiftDestinationUpdate
object
: Describes an update for a destination in Amazon Redshift.- CloudWatchLoggingOptions CloudWatchLoggingOptions
- ClusterJDBCURL ClusterJDBCURL
- CopyCommand CopyCommand
- Password Password
- ProcessingConfiguration ProcessingConfiguration
- RetryOptions RedshiftRetryOptions
- RoleARN RoleARN
- S3BackupMode RedshiftS3BackupMode
- S3BackupUpdate S3DestinationUpdate
- S3Update S3DestinationUpdate
- Username Username
RedshiftRetryDurationInSeconds
- RedshiftRetryDurationInSeconds
integer
RedshiftRetryOptions
- RedshiftRetryOptions
object
: Configures retry behavior in case Kinesis Firehose is unable to deliver documents to Amazon Redshift.- DurationInSeconds RedshiftRetryDurationInSeconds
RedshiftS3BackupMode
- RedshiftS3BackupMode
string
(values: Disabled, Enabled)
ResourceInUseException
- ResourceInUseException
object
: The resource is already in use and not available for this operation.- message ErrorMessage
ResourceNotFoundException
- ResourceNotFoundException
object
: The specified resource could not be found.- message ErrorMessage
RoleARN
- RoleARN
string
S3BackupMode
- S3BackupMode
string
(values: Disabled, Enabled)
S3DestinationConfiguration
- S3DestinationConfiguration
object
: Describes the configuration of a destination in Amazon S3.- BucketARN required BucketARN
- BufferingHints BufferingHints
- CloudWatchLoggingOptions CloudWatchLoggingOptions
- CompressionFormat CompressionFormat
- EncryptionConfiguration EncryptionConfiguration
- Prefix Prefix
- RoleARN required RoleARN
S3DestinationDescription
- S3DestinationDescription
object
: Describes a destination in Amazon S3.- BucketARN required BucketARN
- BufferingHints required BufferingHints
- CloudWatchLoggingOptions CloudWatchLoggingOptions
- CompressionFormat required CompressionFormat
- EncryptionConfiguration required EncryptionConfiguration
- Prefix Prefix
- RoleARN required RoleARN
S3DestinationUpdate
- S3DestinationUpdate
object
: Describes an update for a destination in Amazon S3.- BucketARN BucketARN
- BufferingHints BufferingHints
- CloudWatchLoggingOptions CloudWatchLoggingOptions
- CompressionFormat CompressionFormat
- EncryptionConfiguration EncryptionConfiguration
- Prefix Prefix
- RoleARN RoleARN
ServiceUnavailableException
- ServiceUnavailableException
object
: The service is unavailable, back off and retry the operation. If you continue to see the exception, throughput limits for the delivery stream may have been exceeded. For more information about limits and how to request an increase, see Amazon Kinesis Firehose Limits.- message ErrorMessage
SizeInMBs
- SizeInMBs
integer
SourceDescription
- SourceDescription
object
: Details about a Kinesis stream used as the source for a Kinesis Firehose delivery stream.- KinesisStreamSourceDescription KinesisStreamSourceDescription
SplunkDestinationConfiguration
- SplunkDestinationConfiguration
object
: Describes the configuration of a destination in Splunk.- CloudWatchLoggingOptions CloudWatchLoggingOptions
- HECAcknowledgmentTimeoutInSeconds HECAcknowledgmentTimeoutInSeconds
- HECEndpoint required HECEndpoint
- HECEndpointType required HECEndpointType
- HECToken required HECToken
- ProcessingConfiguration ProcessingConfiguration
- RetryOptions SplunkRetryOptions
- S3BackupMode SplunkS3BackupMode
- S3Configuration required S3DestinationConfiguration
SplunkDestinationDescription
- SplunkDestinationDescription
object
: Describes a destination in Splunk.- CloudWatchLoggingOptions CloudWatchLoggingOptions
- HECAcknowledgmentTimeoutInSeconds HECAcknowledgmentTimeoutInSeconds
- HECEndpoint HECEndpoint
- HECEndpointType HECEndpointType
- HECToken HECToken
- ProcessingConfiguration ProcessingConfiguration
- RetryOptions SplunkRetryOptions
- S3BackupMode SplunkS3BackupMode
- S3DestinationDescription S3DestinationDescription
SplunkDestinationUpdate
- SplunkDestinationUpdate
object
: Describes an update for a destination in Splunk.- CloudWatchLoggingOptions CloudWatchLoggingOptions
- HECAcknowledgmentTimeoutInSeconds HECAcknowledgmentTimeoutInSeconds
- HECEndpoint HECEndpoint
- HECEndpointType HECEndpointType
- HECToken HECToken
- ProcessingConfiguration ProcessingConfiguration
- RetryOptions SplunkRetryOptions
- S3BackupMode SplunkS3BackupMode
- S3Update S3DestinationUpdate
SplunkRetryDurationInSeconds
- SplunkRetryDurationInSeconds
integer
SplunkRetryOptions
- SplunkRetryOptions
object
: Configures retry behavior in case Kinesis Firehose is unable to deliver documents to Splunk or if it doesn't receive an acknowledgment from Splunk.- DurationInSeconds SplunkRetryDurationInSeconds
SplunkS3BackupMode
- SplunkS3BackupMode
string
(values: FailedEventsOnly, AllEvents)
Timestamp
- Timestamp
string
UpdateDestinationInput
- UpdateDestinationInput
object
- CurrentDeliveryStreamVersionId required DeliveryStreamVersionId
- DeliveryStreamName required DeliveryStreamName
- DestinationId required DestinationId
- ElasticsearchDestinationUpdate ElasticsearchDestinationUpdate
- ExtendedS3DestinationUpdate ExtendedS3DestinationUpdate
- RedshiftDestinationUpdate RedshiftDestinationUpdate
- S3DestinationUpdate S3DestinationUpdate
- SplunkDestinationUpdate SplunkDestinationUpdate
UpdateDestinationOutput
- UpdateDestinationOutput
object
Username
- Username
string