Connectors are ways to simplify the way that users can consume or send data to remote endpoints.
A connector should be specific to a remote set of endpoints for an application (for example Slack, Shopify or Twitter).
These are run as AWS Lambda instances, and can be directly called by an ApiOpenStudio API resource. This means that ApiOpenStudio can generate whatever data it wants to send to the Docker instance, process the results, and then do whatever it wants with the results data before generating its response.
A project has been created at gitlab.com/apiopenstudio/processors/nodejs_lambda_template, to allow you create a docker in NodeJS.
However, it is just as easy to use AWS SAM CLI. Both approaches will have the desired end result of a deployable Lambda image that can be used by ApiOpenStudio.
AWS SAM
allows you to quickly and easily create a development environment, test and deploy in any language that you like.
A sample repository has been created to allow you to develop on NodeJS on a local docker, and then deply that to AWS as a Lambda.
cd </path/to/my/clone>
docker build --platform linux/amd64 -t my-processor:test .
docker run --platform linux/amd64 -d -p 9000:8080 my-processor:test
curl "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'
See the documentation for deploying your docker using AWS SAM CLI.
To upload the image to Amazon ECR and create the Lambda function, see
Working with Lambda container images.
Run the get-login-password command to authenticate the Docker CLI to your Amazon ECR registry.
aws ecr get-login-password --region <region> | docker login --username AWS --password-stdin <registryId>.dkr.ecr.<region>.amazonaws.com
or (with AWS profile)
aws ecr get-login-password --profile my_profile | docker login --username AWS --password-stdin 111122223333.dkr.ecr.us-east-1.amazonaws.com
Create a repository in Amazon ECR using the create-repository command.
aws ecr create-repository --repository-name <repository-name> --region <region> --image-scanning-configuration scanOnPush=true --image-tag-mutability MUTABLE
or (with AWS profile)
aws ecr create-repository \
--profile my_profile \
--repository-name hello-world \
--region us-east-1 \
--image-scanning-configuration \
scanOnPush=true \
--image-tag-mutability MUTABLE
Note: The Amazon ECR repository must be in the same AWS Region as the Lambda function.
If successful, you see a response like this:
{
"repository": {
"repositoryArn": "arn:aws:ecr:us-east-1:111122223333:repository/hello-world",
"registryId": "111122223333",
"repositoryName": "hello-world",
"repositoryUri": "111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world",
"createdAt": "2023-03-09T10:39:01+00:00",
"imageTagMutability": "MUTABLE",
"imageScanningConfiguration": {
"scanOnPush": true
},
"encryptionConfiguration": {
"encryptionType": "AES256"
}
}
}
Copy the repositoryUri
from the output in the previous step.
Build and tag to tag your local image into your Amazon ECR repository as the latest version. In this command:
docker-image:test
with the name and tag of your Docker image.<ECRrepositoryUri>
with the repositoryUri
that you copied. Make sure to include :latest
at the end of the URI.docker build -t docker-image:test -t <ECRrepositoryUri>:latest .
Example:
docker build \
--platform=linux/amd64 \
-t hello-world:test \
-t 111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest .
Run the docker push command to deploy your local image to the Amazon ECR repository. Make sure to include :latest at the end of the repository URI.
docker push <ECRrepositoryUri>:latest
Example
docker push 111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest
Create an execution role for the function, if you don't already have one. You need the Amazon Resource Name (ARN) of the role in the next step.
Create the Lambda function. For ImageUri, specify the repository URI from earlier. Make sure to include :latest at the end of the URI.
aws lambda create-function \
--function-name <image_name> \
--package-type Image \
--code ImageUri=<ECRrepositoryUri>:<tag> \
--role arn:aws:iam::<registryId>:role/lambda-ex
Example (with profile)
aws lambda create-function \
--function-name hello-world \
--package-type Image \
--code ImageUri=111122223333.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest \
--role arn:aws:iam::111122223333:role/lambda-ex \
--region us-east-1 \
--profile my_profile
Note: You can create a function using an image in a different AWS account, as long as the image is in the same Region as the Lambda function. For more information, see Amazon ECR cross-account permissions.
Invoke the function.
aws lambda invoke --function-name hello-world response.json
aws lambda invoke --function-name hello-world --profile my_profile response.json
You should see a response like this:
{
"StatusCode": 200,
"ExecutedVersion": "$LATEST"
}
To see the output of the function, check the response.json
file.
For an introduction to ApiOpenStudio rewsources, please see Quick Start
A processor (lambda_function
) has been created to send data to a Lambda instance and consume the response. You can send any number or size of inputs to your Lambda. The Lambda will be triggered and the responce will be sent back to ApiOpenStudio for further prcessing.
Once ApiOpenStudio Reverse Proxy is set up, create a new resource to belong to the account/application of your choosing (you can work with apiopenstudio/core if you want, but make sure that you have granted your self developer
role for core
):
Upload a new resource
name: Test lambda function
description: test lambda function.
uri: test/lambda
method: post
appid: <my_app_id>
ttl: 0
process:
processor: lambda_function
id: test lambda function
function_name:
processor: var_post
id: Lambda function name
key: function_name
iam_key:
processor: var_post
id: IAM key
key: key
iam_secret:
processor: var_post
id: IAM secret
key: secret
aws_region:
processor: var_post
id: AWS region
key: region
data:
processor: var_post
id: Input data
key: data
This will process [POST] requests to [my_dev_domain]/[account]/[application]/test/lambda.
The following post varaiables are accepted:
function_name
- The Lambda function name.iam_key
- Your IAM key.iam_secret
- Your IAM Secret.aws_region
- The region that the Lambda is hosted.data
- any input data for the Lambda to process.When you call this resource, you should see the same results as when youn tested locally.
To extend the Hello World example into a more realistic scenario, you will need to send data into the Lambda to process.
This is in the data
input in the lambda_function
processor.
The data
input take 0 to many inputs. It assumes that the input is an associative array of key/value pairs, therefore you should ideally transform each input into a key/value pair using the var_field
processor. The individual inputs can then be accessed withing the lambda function, by referencing the name of the input. A sample configured in this way will appear like this inside the Lambda input value:
{
"my_param1": "foobar",
"my next parameter": {
"question": "what is the meaning of life",
"answer": 42
}
}