Using data transformation in KDF with a Lambda function
KDF provides the ability to transform ingested records inline through integration with the AWS Lambda service, which allows KDF to invoke a Lambda function (called a Lambda transform) to do custom processing as long as the code adheres to a data transformation and status model. By default, data transformation is disabled and needs to be enabled in the delivery stream configuration. The following diagram illustrates how data transformation works in KDF:
Figure 5.3 – Data transformation with Lambda invocations
Once enabled, the incoming records are buffered up to 3 megabytes (MB) by default. The buffering size can be adjusted using the ProcessingConfiguration API using the BufferSizeInMBs processor parameter (the AWS Console only supports the BufferSizeInMBs and BufferIntervalInSeconds parameters) available as textboxes and lets you choose from a drop-down list of Lambda functions available in the...
