これは、なにをしたくて書いたもの?
AWS SAM+LocalStackを使って、Amazon SNSの通知を受け取るAWS Lambda関数を書いてみようかなと。
AWS Lambda関数は、Node.js+TypeScriptで作成します。
Amazon SNSのトピックからの通知をAmazon Lambda関数で受け取る
Amazon SNSのトピックからの通知をAWS Lambdaで受け取る例は、AWS Lambdaのドキュメントに書かれています。
Amazon SNS でのAWS Lambdaの使用 - AWS Lambda
また、AWS Lambdaを使ったものではありませんが、AWS SDKを使ってAmazon SNSトピックをサブスクライブする例はAmazon SNSの
ドキュメントに記載されています。
AWS SDK を使用して、Amazon SNS トピックから通知を受け取る Lambda 関数をサブスクライブする - Amazon Simple Notification Service
今回は、AWS Lambdaを使います。
AWS SAMでイベントのタイプにSNS
を指定すると、AWS Lambda関数をAmazon SNSトピックにサブスクライブするように設定して
くれるようなので、簡単に使えそうです。
SNS - AWS Serverless Application Model
こちらで試してみましょう。
環境
今回の環境は、こちらです。
$ python3 --version Python 3.8.10 $ localstack --version 1.0.1
LocalStackを起動。
$ LAMBDA_EXECUTOR=docker-reuse localstack start
AWS CLIおよびAWS SAM CLI、加えてLocalStackの提供ツール。
$ awslocal --version aws-cli/2.7.18 Python/3.9.11 Linux/5.4.0-122-generic exe/x86_64.ubuntu.20 prompt/off $ samlocal --version SAM CLI, version 1.53.0
アプリケーションを作成する際に使用するNode.jsのバージョン。
$ node --version v16.16.0 $ npm --version 8.11.0
AWS Lambda関数としても、Node.js 16で動かすことにします。
AWS SAMプロジェクトを作成する
まずは、AWS SAMプロジェクトを作成します。ランタイムはNode.js 16、テンプレートはquick-start-sns
を選択します。
$ samlocal init --name sam-lambda-sns --runtime nodejs16.x --app-template quick-start-sns --package-type Zip --no-tracing
プロジェクト内に移動。
$ cd sam-lambda-sns
プロジェクト内の構成。
$ tree . ├── README.md ├── __tests__ │ └── unit │ └── handlers │ └── sns-payload-logger.test.js ├── buildspec.yml ├── events │ └── event-sns.json ├── package.json ├── src │ └── handlers │ └── sns-payload-logger.js └── template.yaml 6 directories, 7 files
生成された各ファイルも見ておきましょう。
package.json
{ "name": "replaced-by-user-input", "description": "replaced-by-user-input", "version": "0.0.1", "private": true, "dependencies": {}, "devDependencies": { "jest": "^26.6.3" }, "scripts": { "test": "jest" } }
src/handlers/sns-payload-logger.js
/** * A Lambda function that logs the payload received from SNS. */ exports.snsPayloadLoggerHandler = async (event, context) => { // All log statements are written to CloudWatch by default. For more information, see // https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-logging.html console.info(event); }
__tests__/unit/handlers/sns-payload-logger.test.js
// Import all functions from sns-payload-logger.js const snsPayloadLogger = require('../../../src/handlers/sns-payload-logger.js'); describe('Test for sns-payload-logger', function () { // This test invokes the sns-payload-logger Lambda function and verifies that the received payload is logged it('Verifies the payload is logged', async() => { // Mock console.log statements so we can verify them. For more information, see // https://jestjs.io/docs/en/mock-functions.html console.info = jest.fn() // Create a sample payload with SNS message format var payload = { TopicArn: "arn:aws:sns:us-west-2:123456789012:SimpleTopic", Message: "This is a notification from SNS", Subject: "SNS Notification" } await snsPayloadLogger.snsPayloadLoggerHandler(payload, null) // Verify that console.info has been called with the expected payload expect(console.info).toHaveBeenCalledWith(payload) }); });
events/event-sns.json
{ "TopicArn": "arn:aws:sns:us-west-2:123456789012:SimpleTopic", "Message": "This is a notification from SNS", "Subject": "SNS Notification" }
template.yaml
# This is the SAM template that represents the architecture of your serverless application # https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-template-basics.html # The AWSTemplateFormatVersion identifies the capabilities of the template # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/format-version-structure.html AWSTemplateFormatVersion: 2010-09-09 Description: >- sam-lambda-sns # Transform section specifies one or more macros that AWS CloudFormation uses to process your template # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/transform-section-structure.html Transform: - AWS::Serverless-2016-10-31 # Resources declares the AWS resources that you want to include in the stack # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resources-section-structure.html Resources: # This is an SNS Topic with all default configuration properties. To learn more about the available options, see # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-sns-topic.html SimpleTopic: Type: AWS::SNS::Topic # This is the Lambda function definition associated with the source code: sns-payload-logger.js. For all available properties, see # https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction SNSPayloadLogger: Type: AWS::Serverless::Function Properties: Description: A Lambda function that logs the payload of messages sent to an associated SNS topic. Runtime: nodejs16.x Architectures: - x86_64 Handler: src/handlers/sns-payload-logger.snsPayloadLoggerHandler # This property associates this Lambda function with the SNS topic defined above, so that whenever the topic # receives a message, the Lambda function is invoked Events: SNSTopicEvent: Type: SNS Properties: Topic: !Ref SimpleTopic MemorySize: 128 Timeout: 100 Policies: # Give Lambda basic execution Permission to the helloFromLambda - AWSLambdaBasicExecutionRole
buildspec.yml
version: 0.2 phases: install: commands: # Install all dependencies (including dependencies for running tests) - npm install pre_build: commands: # Discover and run unit tests in the '__tests__' directory - npm run test # Remove all unit tests to reduce the size of the package that will be ultimately uploaded to Lambda - rm -rf ./__tests__ # Remove all dependencies not needed for the Lambda deployment package (the packages from devDependencies in package.json) - npm prune --production build: commands: # Use AWS SAM to package the application by using AWS CloudFormation - aws cloudformation package --template template.yaml --s3-bucket $S3_BUCKET --output-template template-export.yml artifacts: type: zip files: - template-export.yml
構成はだいたいわかりましたね。
TypeScriptプロジェクトに変更する
ここから、TypeScriptに変えていきます。
$ npm i -D typescript $ npm i -D -E prettier $ npm i -D @types/node@v16 @types/aws-lambda
TypeScriptの設定。
tsconfig.json
{ "compilerOptions": { "target": "esnext", "module": "commonjs", "lib": ["esnext"], "baseUrl": "./src", "outDir": "dist", "strict": true, "forceConsistentCasingInFileNames": true, "noFallthroughCasesInSwitch": true, "noImplicitOverride": true, "noImplicitReturns": true, "noPropertyAccessFromIndexSignature": true, "esModuleInterop": true }, "include": [ "src" ] }
tsconfig.typecheck.json
{ "extends": "./tsconfig", "compilerOptions": { "baseUrl": "./", "noEmit": true }, "include": [ "src", "__tests__" ] }
.prettierrc.json
{ "singleQuote": true }
デフォルトでpackage.json
に記載されているJestは古いので、アンインストール。
$ npm uninstall jest
あらためて、Jestをインストールします。
$ npm i -D jest @types/jest esbuild esbuild-jest
依存関係はこんな感じになりました。
"devDependencies": { "@types/aws-lambda": "^8.10.101", "@types/jest": "^28.1.6", "@types/node": "^16.11.45", "esbuild": "^0.14.49", "esbuild-jest": "^0.5.0", "jest": "^28.1.3", "prettier": "2.7.1", "typescript": "^4.7.4" },
jest.config.js
module.exports = { testEnvironment: 'node', transform: { "^.+\\.tsx?$": "esbuild-jest" } };
package.json
のscripts
要素は、こんな感じにしました。
"scripts": { "build": "tsc --project . && cp package*.json dist", "typecheck": "tsc --project ./tsconfig.typecheck.json", "typecheck:watch": "tsc --project ./tsconfig.typecheck.json --watch", "format": "prettier --write src __tests__", "test": "jest" }
Amazon SNSのトピックから通知を受け取るAWS Lambda関数。
src/handlers/sns-payload-logger.ts
import { Context, SNSEvent } from 'aws-lambda'; export const snsPayloadLoggerHandler = async ( event: SNSEvent, context: Context ): Promise<void> => { console.info(JSON.stringify(event, null, 2)); };
受け取ったメッセージは、標準出力に書き出しているだけです。
テストコード。
__tests__/unit/handlers/sns-payload-logger.test.ts
import { Context, SNSEvent } from 'aws-lambda'; import { snsPayloadLoggerHandler } from '../../../src/handlers/sns-payload-logger'; test('Test for sns-payload-logger, Verifies the payload is logged', async () => { console.info = jest.fn(); const payload: SNSEvent = { Records: [ { EventSource: 'aws:sns', EventVersion: '1.0', EventSubscriptionArn: 'arn:aws:sns:us-east-1::ExampleTopic', Sns: { Type: 'Notification', MessageId: '95df01b4-ee98-5cb9-9903-4c221d41eb5e', TopicArn: 'arn:aws:sns:us-east-1:123456789012:ExampleTopic', Subject: 'example subject', Message: 'example message', Timestamp: '1970-01-01T00:00:00.000Z', SignatureVersion: '1', Signature: 'EXAMPLE', SigningCertUrl: 'EXAMPLE', UnsubscribeUrl: 'EXAMPLE', MessageAttributes: { Test: { Type: 'String', Value: 'TestString', }, TestBinary: { Type: 'Binary', Value: 'TestBinary', }, }, }, }, ], }; await snsPayloadLoggerHandler(payload, {} as Context); expect(console.info).toHaveBeenCalledWith(JSON.stringify(payload, null, 2)); });
もともと生成されたJavaScriptのテストコードとはpayload
が変わっています。もとはこうだったのですが
// Create a sample payload with SNS message format var payload = { TopicArn: "arn:aws:sns:us-west-2:123456789012:SimpleTopic", Message: "This is a notification from SNS", Subject: "SNS Notification" }
実際に動かしてみると、このフォーマットではなかったのでAWS SAM CLIでイベントのデータを生成してこちらを設定しました。
$ samlocal local generate-event sns notification { "Records": [ { "EventSource": "aws:sns", "EventVersion": "1.0", "EventSubscriptionArn": "arn:aws:sns:us-east-1::ExampleTopic", "Sns": { "Type": "Notification", "MessageId": "95df01b4-ee98-5cb9-9903-4c221d41eb5e", "TopicArn": "arn:aws:sns:us-east-1:123456789012:ExampleTopic", "Subject": "example subject", "Message": "example message", "Timestamp": "1970-01-01T00:00:00.000Z", "SignatureVersion": "1", "Signature": "EXAMPLE", "SigningCertUrl": "EXAMPLE", "UnsubscribeUrl": "EXAMPLE", "MessageAttributes": { "Test": { "Type": "String", "Value": "TestString" }, "TestBinary": { "Type": "Binary", "Value": "TestBinary" } } } } ] }
ソースコードは書いたので、もともと生成されていたJavaScriptコードは削除します。
$ rm src/handlers/sns-payload-logger.js __tests__/unit/handlers/sns-payload-logger.test.js
テストを実行して確認。
$ npm test > replaced-by-user-input@0.0.1 test > jest PASS __tests__/unit/handlers/sns-payload-logger.test.ts ✓ Test for sns-payload-logger, Verifies the payload is logged (3 ms) Test Suites: 1 passed, 1 total Tests: 1 passed, 1 total Snapshots: 0 total Time: 0.35 s, estimated 1 s Ran all test suites.
OKですね。
LocalStackにデプロイする
ソースコードができたところで、今度はLocalStackにデプロイします。
が、その前にtemplate.yaml
を修正します。CodeUri
を追加して、Handler
のパスを修正
SNSPayloadLogger: Type: AWS::Serverless::Function Properties: Description: A Lambda function that logs the payload of messages sent to an associated SNS topic. Runtime: nodejs16.x Architectures: - x86_64 CodeUri: dist/ #Handler: src/handlers/sns-payload-logger.snsPayloadLoggerHandler Handler: sns-payload-logger.snsPayloadLoggerHandler
Transform
を修正。この必要性は、LocalStack固有な気もしますが…。
#Transform: #- AWS::Serverless-2016-10-31 Transform: AWS::Serverless-2016-10-31
Outputs
も追加しておきます。
Outputs: SNSPayloadLogger: Description: "SNS Payload Logger Handler Function ARN" Value: !GetAtt SNSPayloadLogger.Arn SimpleTopic: Description: "SNS Topic ARN" Value: !GetAtt SimpleTopic.Arn
ここまでできたら、ビルドして
$ npm run build
デプロイ。
$ yes | samlocal sync --stack-name $(uuidgen) --region us-east-1 --no-dependency-layer
デプロイが完了しました。
2022-07-27 00:12:41 - Waiting for stack create/update to complete CloudFormation events from stack operations (refresh every 0.5 seconds) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ResourceStatus ResourceType LogicalResourceId ResourceStatusReason ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- CREATE_COMPLETE AWS::CloudFormation::Stack SimpleTopic - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLoggerRole - UPDATE_COMPLETE AWS::CloudFormation::Stack SimpleTopic - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLogger - UPDATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLoggerRole - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLoggerSNSTopicEventPermission - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLoggerSNSTopicEvent - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLogger - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLoggerSNSTopicEventPermission - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLoggerSNSTopicEvent - CREATE_COMPLETE AWS::CloudFormation::Stack 43b7e1dc-57de-4062-a65d-30bc0cd1b0c4 - ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- CloudFormation outputs from deployed stack --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Outputs --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Key SNSPayloadLogger Description SNS Payload Logger Handler Function ARN Value arn:aws:lambda:us-east-1:000000000000:function:43b7e1dc-57de-4062-a65d-30bc0cd1b0c4-SNSPayloadLogger-432b3f6f Key SimpleTopic Description SNS Topic ARN Value arn:aws:sns:us-east-1:000000000000:43b7e1dc-57de-4062-a65d-30bc0cd1b0c4-SimpleTopic-2af58778 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Stack creation succeeded. Sync infra completed.
Outputs
でも出力されていますが、作成されたAmazon SNSトピックを確認。
$ awslocal sns list-topics { "Topics": [ { "TopicArn": "arn:aws:sns:us-east-1:000000000000:43b7e1dc-57de-4062-a65d-30bc0cd1b0c4-SimpleTopic-2af58778" } ] }
トピックのARNを取得。
$ TOPIC_ARN=$(awslocal sns list-topics --query 'Topics[-1].TopicArn' --output text)
メッセージを送ってみます。
$ awslocal sns publish --topic-arn $TOPIC_ARN --message 'Hello World' { "MessageId": "350ed5b5-44f4-46e7-8393-a5ec686577c3" }
結果を確認したいのですが…なぜかAmazon CloudWatch Logsには保存されていません…。
Dockerコンテナでログを見ると、確認できました。
$ docker container logs -f localstack_main_lambda_arn_aws_lambda_us-east-1_000000000000_function_43b7e1dc-57de-4062-a 65d-30bc0cd1b0c4-SNSPayloadLogger-432b3f6f Lambda API listening on port 9001... START RequestId: c0e10353-dd24-1d86-f1de-cde423ba87df Version: $LATEST 2022-07-26T15:13:35.520Z c0e10353-dd24-1d86-f1de-cde423ba87df INFO { "Records": [ { "EventSource": "aws:sns", "EventVersion": "1.0", "EventSubscriptionArn": "arn:aws:sns:us-east-1:000000000000:43b7e1dc-57de-4062-a65d-30bc0cd1b0c4-SimpleTopic-2af58778:2a8a4ef6-f5ae-4d87-a9e2-6400e4d7992b", "Sns": { "Type": "Notification", "MessageId": "350ed5b5-44f4-46e7-8393-a5ec686577c3", "TopicArn": "arn:aws:sns:us-east-1:000000000000:43b7e1dc-57de-4062-a65d-30bc0cd1b0c4-SimpleTopic-2af58778", "Subject": null, "Message": "Hello World", "Timestamp": "2022-07-26T15:13:33.930Z", "SignatureVersion": "1", "Signature": "EXAMPLEpH+..", "SigningCertUrl": "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-000000000.pem", "UnsubscribeUrl": "http://localhost:4566/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:000000000000:43b7e1dc-57de-4062-a65d-30bc0cd1b0c4-SimpleTopic-2af58778:2a8a4ef6-f5ae-4d87-a9e2-6400e4d7992b", "MessageAttributes": {} } } ] } END RequestId: c0e10353-dd24-1d86-f1de-cde423ba87df REPORT RequestId: c0e10353-dd24-1d86-f1de-cde423ba87df Init Duration: 262.49 ms Duration: 26.21 ms Billed Duration: 27 ms Memory Size: 1536 MB Max Memory Used: 43 MB
もうひとつメッセージを送ってみましょう。
$ awslocal sns publish --topic-arn $TOPIC_ARN --message 'Hello SNS' { "MessageId": "b3e235a6-470a-49e8-a8a2-0bcf69ef91aa" }
結果。
START RequestId: baf3f2c1-9c47-146d-dc97-32853af51a59 Version: $LATEST 2022-07-26T15:14:21.630Z baf3f2c1-9c47-146d-dc97-32853af51a59 INFO { "Records": [ { "EventSource": "aws:sns", "EventVersion": "1.0", "EventSubscriptionArn": "arn:aws:sns:us-east-1:000000000000:43b7e1dc-57de-4062-a65d-30bc0cd1b0c4-SimpleTopic-2af58778:2a8a4ef6-f5ae-4d87-a9e2-6400e4d7992b", "Sns": { "Type": "Notification", "MessageId": "b3e235a6-470a-49e8-a8a2-0bcf69ef91aa", "TopicArn": "arn:aws:sns:us-east-1:000000000000:43b7e1dc-57de-4062-a65d-30bc0cd1b0c4-SimpleTopic-2af58778", "Subject": null, "Message": "Hello SNS", "Timestamp": "2022-07-26T15:14:21.618Z", "SignatureVersion": "1", "Signature": "EXAMPLEpH+..", "SigningCertUrl": "https://sns.us-east-1.amazonaws.com/SimpleNotificationService-000000000.pem", "UnsubscribeUrl": "http://localhost:4566/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:us-east-1:000000000000:43b7e1dc-57de-4062-a65d-30bc0cd1b0c4-SimpleTopic-2af58778:2a8a4ef6-f5ae-4d87-a9e2-6400e4d7992b", "MessageAttributes": {} } } ] } END RequestId: baf3f2c1-9c47-146d-dc97-32853af51a59 REPORT RequestId: baf3f2c1-9c47-146d-dc97-32853af51a59 Duration: 4.23 ms Billed Duration: 5 ms Memory Size: 1536 MB Max Memory Used: 43 MB
OKですね。
これで確認できた感じです。
template.yaml
のファイル全体も載せておきましょう。
# This is the SAM template that represents the architecture of your serverless application # https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-template-basics.html # The AWSTemplateFormatVersion identifies the capabilities of the template # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/format-version-structure.html AWSTemplateFormatVersion: 2010-09-09 Description: >- sam-lambda-sns # Transform section specifies one or more macros that AWS CloudFormation uses to process your template # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/transform-section-structure.html #Transform: #- AWS::Serverless-2016-10-31 Transform: AWS::Serverless-2016-10-31 # Resources declares the AWS resources that you want to include in the stack # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resources-section-structure.html Resources: # This is an SNS Topic with all default configuration properties. To learn more about the available options, see # https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-sns-topic.html SimpleTopic: Type: AWS::SNS::Topic # This is the Lambda function definition associated with the source code: sns-payload-logger.js. For all available properties, see # https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction SNSPayloadLogger: Type: AWS::Serverless::Function Properties: Description: A Lambda function that logs the payload of messages sent to an associated SNS topic. Runtime: nodejs16.x Architectures: - x86_64 CodeUri: dist/ #Handler: src/handlers/sns-payload-logger.snsPayloadLoggerHandler Handler: sns-payload-logger.snsPayloadLoggerHandler # This property associates this Lambda function with the SNS topic defined above, so that whenever the topic # receives a message, the Lambda function is invoked Events: SNSTopicEvent: Type: SNS Properties: Topic: !Ref SimpleTopic MemorySize: 128 Timeout: 100 Policies: # Give Lambda basic execution Permission to the helloFromLambda - AWSLambdaBasicExecutionRole Outputs: SNSPayloadLogger: Description: "SNS Payload Logger Handler Function ARN" Value: !GetAtt SNSPayloadLogger.Arn SimpleTopic: Description: "SNS Topic ARN" Value: !GetAtt SimpleTopic.Arn
ハマったこと
ここからは、ハマったことを書いていきます。
sam syncの時に「--no-dependency-layer」オプションを付与しないとエラーになる
デプロイ時にこんなコマンドを使っていましたが。
$ yes | samlocal sync --stack-name $(uuidgen) --region us-east-1 --no-dependency-layer
最後にある--no-dependency-layer
というオプションを外して実行すると
$ yes | samlocal sync --stack-name $(uuidgen) --region us-east-1
一見デプロイに成功したかに見えますが
2022-07-23 02:03:44 - Waiting for stack create/update to complete CloudFormation events from stack operations (refresh every 0.5 seconds) ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ResourceStatus ResourceType LogicalResourceId ResourceStatusReason ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- CREATE_COMPLETE AWS::CloudFormation::Stack SimpleTopic - CREATE_COMPLETE AWS::CloudFormation::Stack AwsSamAutoDependencyLayerNestedStack - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLoggerRole - UPDATE_COMPLETE AWS::CloudFormation::Stack SimpleTopic - UPDATE_COMPLETE AWS::CloudFormation::Stack AwsSamAutoDependencyLayerNestedStack - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLogger - UPDATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLoggerRole - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLoggerSNSTopicEventPermission - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLoggerSNSTopicEvent - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLogger - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLoggerSNSTopicEventPermission - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLoggerSNSTopicEvent - CREATE_COMPLETE AWS::CloudFormation::Stack 503fadda-2e83-4341-ac04-641734a2bfb6 - ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- CloudFormation outputs from deployed stack ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Outputs ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Key SNSPayloadLogger Description SNS Payload Logger Handler Function ARN Value arn:aws:lambda:us-east-1:000000000000:function:503fadda-2e83-4341-ac04-641734a2bfb6-SNSPayloadLogger-9b5cd1ef Key SimpleTopic Description SNS Topic ARN Value arn:aws:sns:us-east-1:000000000000:503fadda-2e83-4341-ac04-641734a2bfb6-SimpleTopic-e5d8f43a ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Stack creation succeeded. Sync infra completed.
裏で盛大にエラーになっています。
2022-07-22T17:03:44.392 WARN --- [ Thread-163] l.u.c.template_deployer : Unexpected resource type Lambda::LayerVersion when resolving references of resource SNSPayloadLogger0faa899dDepLayer2f27043c81: {"Type": "AWS::Lambda::LayerVersion", "DeletionPolicy": "Delete", "Properties": {"Content": {"S3Bucket": "aws-sam-cli-managed-default-samclisourcebucket-fc361bf1", "S3Key": "3f31d69f659baa13ce3c80a0c664563e"}, "Description": "Auto created layer for dependencies of function SNSPayloadLogger", "LayerName": "503fadda-2e83-4383a3a404-SNSPayloadLogger0faa899d-DepLayer", "CompatibleRuntimes": ["nodejs16.x"]}, "LogicalResourceId": "SNSPayloadLogger0faa899dDepLayer2f27043c81", "PhysicalResourceId": null}. To find out if Lambda::LayerVersion is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation 2022-07-22T17:03:44.392 WARN --- [ Thread-163] l.u.c.template_deployer : Unable to deploy resource type "Lambda::LayerVersion": {"Type": "AWS::Lambda::LayerVersion", "DeletionPolicy": "Delete", "Properties": {"Content": {"S3Bucket": "aws-sam-cli-managed-default-samclisourcebucket-fc361bf1", "S3Key": "3f31d69f659baa13ce3c80a0c664563e"}, "Description": "Auto created layer for dependencies of function SNSPayloadLogger", "LayerName": "503fadda-2e83-4383a3a404-SNSPayloadLogger0faa899d-DepLayer", "CompatibleRuntimes": ["nodejs16.x"]}, "LogicalResourceId": "SNSPayloadLogger0faa899dDepLayer2f27043c81", "PhysicalResourceId": null} 2022-07-22T17:03:44.420 WARN --- [ asgi_gw_2] l.u.c.template_deployer : Unexpected resource type Lambda::LayerVersion when resolving references of resource SNSPayloadLogger0faa899dDepLayer2f27043c81: {"Type": "AWS::Lambda::LayerVersion", "DeletionPolicy": "Delete", "Properties": {"Content": {"S3Bucket": "aws-sam-cli-managed-default-samclisourcebucket-fc361bf1", "S3Key": "3f31d69f659baa13ce3c80a0c664563e"}, "Description": "Auto created layer for dependencies of function SNSPayloadLogger", "LayerName": "503fadda-2e83-4383a3a404-SNSPayloadLogger0faa899d-DepLayer", "CompatibleRuntimes": ["nodejs16.x"]}, "LogicalResourceId": "SNSPayloadLogger0faa899dDepLayer2f27043c81", "PhysicalResourceId": null}. To find out if Lambda::LayerVersion is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation 2022-07-22T17:03:44.433 WARN --- [ Thread-163] l.u.c.template_deployer : Unexpected resource type Lambda::LayerVersion when resolving references of resource SNSPayloadLogger0faa899dDepLayer2f27043c81: {"Type": "AWS::Lambda::LayerVersion", "DeletionPolicy": "Delete", "Properties": {"Content": {"S3Bucket": "aws-sam-cli-managed-default-samclisourcebucket-fc361bf1", "S3Key": "3f31d69f659baa13ce3c80a0c664563e"}, "Description": "Auto created layer for dependencies of function SNSPayloadLogger", "LayerName": "503fadda-2e83-4383a3a404-SNSPayloadLogger0faa899d-DepLayer", "CompatibleRuntimes": ["nodejs16.x"]}, "LogicalResourceId": "SNSPayloadLogger0faa899dDepLayer2f27043c81", "PhysicalResourceId": null}. To find out if Lambda::LayerVersion is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation 2022-07-22T17:03:44.434 WARN --- [ Thread-163] l.u.c.template_deployer : Unable to deploy resource type "Lambda::LayerVersion": {"Type": "AWS::Lambda::LayerVersion", "DeletionPolicy": "Delete", "Properties": {"Content": {"S3Bucket": "aws-sam-cli-managed-default-samclisourcebucket-fc361bf1", "S3Key": "3f31d69f659baa13ce3c80a0c664563e"}, "Description": "Auto created layer for dependencies of function SNSPayloadLogger", "LayerName": "503fadda-2e83-4383a3a404-SNSPayloadLogger0faa899d-DepLayer", "CompatibleRuntimes": ["nodejs16.x"]}, "LogicalResourceId": "SNSPayloadLogger0faa899dDepLayer2f27043c81", "PhysicalResourceId": null} 2022-07-22T17:03:44.524 WARN --- [ asgi_gw_2] l.u.c.template_deployer : Unexpected resource type Lambda::LayerVersion when resolving references of resource SNSPayloadLogger0faa899dDepLayer2f27043c81: {"Type": "AWS::Lambda::LayerVersion", "DeletionPolicy": "Delete", "Properties": {"Content": {"S3Bucket": "aws-sam-cli-managed-default-samclisourcebucket-fc361bf1", "S3Key": "3f31d69f659baa13ce3c80a0c664563e"}, "Description": "Auto created layer for dependencies of function SNSPayloadLogger", "LayerName": "503fadda-2e83-4383a3a404-SNSPayloadLogger0faa899d-DepLayer", "CompatibleRuntimes": ["nodejs16.x"]}, "LogicalResourceId": "SNSPayloadLogger0faa899dDepLayer2f27043c81", "PhysicalResourceId": null}. To find out if Lambda::LayerVersion is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation 2022-07-22T17:03:44.528 WARN --- [ Thread-157] l.u.c.template_deployer : Unable to extract reference attribute "Outputs.SNSPayloadLogger0faa899dDepLayer" from resource: {'StackId': 'arn:aws:cloudformation:us-east-1:000000000000:stack/503fadda-2e83-4341-ac04-64173-AwsSamAutoDependencyLaye-82d64898/a911ab69', 'StackName': '503fadda-2e83-4341-ac04-64173-AwsSamAutoDependencyLaye-82d64898', 'Parameters': [], 'CreationTime': datetime.datetime(2022, 7, 22, 17, 3, 44, 328000, tzinfo=tzlocal()), 'StackStatus': 'CREATE_COMPLETE', 'StackStatusReason': 'Deployment succeeded', 'Capabilities': [], 'Outputs': [{'OutputKey': 'SNSPayloadLogger0faa899dDepLayer'}], 'Tags': []} {'DeletionPolicy': 'Delete', 'Metadata': {'CreatedBy': 'AWS SAM CLI sync command', 'SamResourceId': 'AwsSamAutoDependencyLayerNestedStack'}, 'Properties': {'TemplateURL': 'http://localhost:4566/aws-sam-cli-managed-default-samclisourcebucket-fc361bf1/065e8683100ad572c9dc075b8632a1ea.template', 'StackName': '503fadda-2e83-4341-ac04-64173-AwsSamAutoDependencyLaye-82d64898'}, 'Type': 'AWS::CloudFormation::Stack', 'LogicalResourceId': 'AwsSamAutoDependencyLayerNestedStack', 'PhysicalResourceId': 'arn:aws:cloudformation:us-east-1:000000000000:stack/503fadda-2e83-4341-ac04-64173-AwsSamAutoDependencyLaye-82d64898/a911ab69', '_state_': {'StackId': 'arn:aws:cloudformation:us-east-1:000000000000:stack/503fadda-2e83-4341-ac04-64173-AwsSamAutoDependencyLaye-82d64898/a911ab69', 'StackName': '503fadda-2e83-4341-ac04-64173-AwsSamAutoDependencyLaye-82d64898', 'Parameters': [], 'CreationTime': datetime.datetime(2022, 7, 22, 17, 3, 44, 328000, tzinfo=tzlocal()), 'StackStatus': 'CREATE_COMPLETE', 'StackStatusReason': 'Deployment succeeded', 'Capabilities': [], 'Outputs': [{'OutputKey': 'SNSPayloadLogger0faa899dDepLayer'}], 'Tags': []}}
これはsam sync
を使った時にAwsSamAutoDependencyLayerNestedStack
というLambdaレイヤーを作成するのですが、Lambdaレイヤーは
LocalStack Proでしかサポートしていないからです。
AWS SAM Accelerateによるサーバーレス開発の加速 | Amazon Web Services ブログ
こんなことを言われています。
To find out if Lambda::LayerVersion is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation
template.yamlのTransform
template.yaml
で、Transform
を以下のように変更しました。
#Transform: #- AWS::Serverless-2016-10-31 Transform: AWS::Serverless-2016-10-31
これ、変更しないとどうなるかというと
Transform: - AWS::Serverless-2016-10-31
sam sync
時に
$ yes | samlocal sync --stack-name $(uuidgen) --region us-east-1 --no-dependency-layer
盛大に失敗します。
2022-07-23 02:09:20 - Waiting for stack create/update to complete CloudFormation events from stack operations (refresh every 0.5 seconds) ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- ResourceStatus ResourceType LogicalResourceId ResourceStatusReason ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- CREATE_COMPLETE AWS::CloudFormation::Stack SimpleTopic - CREATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLogger - UPDATE_COMPLETE AWS::CloudFormation::Stack SimpleTopic - UPDATE_COMPLETE AWS::CloudFormation::Stack SNSPayloadLogger - CREATE_COMPLETE AWS::CloudFormation::Stack f88c5c65-6de1-4354-94fc-e4636dd7844d - ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- CloudFormation outputs from deployed stack ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Outputs ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Key SNSPayloadLogger Description SNS Payload Logger Handler Function ARN Traceback (most recent call last): File "/path/to/venv/bin/samlocal", line 41, in <module> main.cli() File "/path/to/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__ return self.main(*args, **kwargs) File "/path/to/venv/lib/python3.8/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/path/to/venv/lib/python3.8/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/path/to/venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/path/to/venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/path/to/venv/lib/python3.8/site-packages/click/decorators.py", line 73, in new_func return ctx.invoke(f, obj, *args, **kwargs) File "/path/to/venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/path/to/venv/lib/python3.8/site-packages/samcli/lib/telemetry/metric.py", line 166, in wrapped raise exception # pylint: disable=raising-bad-type File "/path/to/venv/lib/python3.8/site-packages/samcli/lib/telemetry/metric.py", line 124, in wrapped return_value = func(*args, **kwargs) File "/path/to/venv/lib/python3.8/site-packages/samcli/lib/cli_validation/image_repository_validation.py", line 92, in wrapped return func(*args, **kwargs) File "/path/to/venv/lib/python3.8/site-packages/samcli/lib/telemetry/metric.py", line 88, in wrapped return func(*args, **kwargs) File "/path/to/venv/lib/python3.8/site-packages/samcli/lib/utils/version_checker.py", line 41, in wrapped actual_result = func(*args, **kwargs) File "/path/to/venv/lib/python3.8/site-packages/samcli/cli/main.py", line 87, in wrapper return func(*args, **kwargs) File "/path/to/venv/lib/python3.8/site-packages/samcli/commands/_utils/cdk_support_decorators.py", line 38, in wrapped return func(*args, **kwargs) File "/path/to/venv/lib/python3.8/site-packages/samcli/commands/sync/command.py", line 170, in cli do_cli( File "/path/to/venv/lib/python3.8/site-packages/samcli/commands/sync/command.py", line 320, in do_cli execute_infra_contexts(build_context, package_context, deploy_context) File "/path/to/venv/lib/python3.8/site-packages/samcli/commands/sync/command.py", line 344, in execute_infra_contexts deploy_context.run() File "/path/to/venv/lib/python3.8/site-packages/samcli/commands/deploy/deploy_context.py", line 163, in run return self.deploy( File "/path/to/venv/lib/python3.8/site-packages/samcli/commands/deploy/deploy_context.py", line 278, in deploy result = self.deployer.sync( File "/path/to/venv/lib/python3.8/site-packages/samcli/lib/deploy/deployer.py", line 596, in sync self.wait_for_execute(stack_name, "CREATE", False) File "/path/to/venv/lib/python3.8/site-packages/samcli/lib/deploy/deployer.py", line 493, in wait_for_execute self._display_stack_outputs(outputs) File "/path/to/venv/lib/python3.8/site-packages/samcli/commands/_utils/table_print.py", line 74, in wrap result = func(*args, **kwargs) File "/path/to/venv/lib/python3.8/site-packages/samcli/lib/deploy/deployer.py", line 617, in _display_stack_outputs columns=["{k:<{0}}{v:<{0}}".format(MIN_OFFSET, k=k, v=v)], TypeError: unsupported format string passed to NoneType.__format__
LocalStack側を見ると、Serverless::Function
というリソースタイプがわかっていないようです。
2022-07-22T17:09:21.055 WARN --- [ Thread-153] l.u.c.template_deployer : Unexpected resource type Serverless::Function when resolving references of resource SNSPayloadLogger: {"Type": "AWS::Serverless::Function", "Properties": {"Description": "A Lambda function that logs the payload of messages sent to an associated SNS topic.", "Runtime": "nodejs16.x", "Architectures": ["x86_64"], "CodeUri": "s3://aws-sam-cli-managed-default-samclisourcebucket-7992d0cc/dc5ed1147fb133006d2d390b4a56be95", "Handler": "sns-payload-logger.snsPayloadLoggerHandler", "Events": {"SNSTopicEvent": {"Type": "SNS", "Properties": {"Topic": "arn:aws:sns:us-east-1:000000000000:f88c5c65-6de1-4354-94fc-e4636dd7844d-SimpleTopic-74053aac"}}}, "MemorySize": 128, "Timeout": 100, "Policies": ["AWSLambdaBasicExecutionRole"]}, "Metadata": {"SamResourceId": "SNSPayloadLogger"}, "LogicalResourceId": "SNSPayloadLogger", "PhysicalResourceId": null}. To find out if Serverless::Function is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation 2022-07-22T17:09:21.055 WARN --- [ Thread-153] l.u.c.template_deployer : Unable to deploy resource type "Serverless::Function": {"Type": "AWS::Serverless::Function", "Properties": {"Description": "A Lambda function that logs the payload of messages sent to an associated SNS topic.", "Runtime": "nodejs16.x", "Architectures": ["x86_64"], "CodeUri": "s3://aws-sam-cli-managed-default-samclisourcebucket-7992d0cc/dc5ed1147fb133006d2d390b4a56be95", "Handler": "sns-payload-logger.snsPayloadLoggerHandler", "Events": {"SNSTopicEvent": {"Type": "SNS", "Properties": {"Topic": "arn:aws:sns:us-east-1:000000000000:f88c5c65-6de1-4354-94fc-e4636dd7844d-SimpleTopic-74053aac"}}}, "MemorySize": 128, "Timeout": 100, "Policies": ["AWSLambdaBasicExecutionRole"]}, "Metadata": {"SamResourceId": "SNSPayloadLogger"}, "LogicalResourceId": "SNSPayloadLogger", "PhysicalResourceId": null} 2022-07-22T17:09:21.102 WARN --- [ Thread-153] l.u.c.template_deployer : Unexpected resource type Serverless::Function when resolving references of resource SNSPayloadLogger: {"Type": "AWS::Serverless::Function", "Properties": {"Description": "A Lambda function that logs the payload of messages sent to an associated SNS topic.", "Runtime": "nodejs16.x", "Architectures": ["x86_64"], "CodeUri": "s3://aws-sam-cli-managed-default-samclisourcebucket-7992d0cc/dc5ed1147fb133006d2d390b4a56be95", "Handler": "sns-payload-logger.snsPayloadLoggerHandler", "Events": {"SNSTopicEvent": {"Type": "SNS", "Properties": {"Topic": "arn:aws:sns:us-east-1:000000000000:f88c5c65-6de1-4354-94fc-e4636dd7844d-SimpleTopic-74053aac"}}}, "MemorySize": 128, "Timeout": 100, "Policies": ["AWSLambdaBasicExecutionRole"]}, "Metadata": {"SamResourceId": "SNSPayloadLogger"}, "LogicalResourceId": "SNSPayloadLogger", "PhysicalResourceId": null}. To find out if Serverless::Function is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation 2022-07-22T17:09:21.102 WARN --- [ Thread-153] l.u.c.template_deployer : Unable to deploy resource type "Serverless::Function": {"Type": "AWS::Serverless::Function", "Properties": {"Description": "A Lambda function that logs the payload of messages sent to an associated SNS topic.", "Runtime": "nodejs16.x", "Architectures": ["x86_64"], "CodeUri": "s3://aws-sam-cli-managed-default-samclisourcebucket-7992d0cc/dc5ed1147fb133006d2d390b4a56be95", "Handler": "sns-payload-logger.snsPayloadLoggerHandler", "Events": {"SNSTopicEvent": {"Type": "SNS", "Properties": {"Topic": "arn:aws:sns:us-east-1:000000000000:f88c5c65-6de1-4354-94fc-e4636dd7844d-SimpleTopic-74053aac"}}}, "MemorySize": 128, "Timeout": 100, "Policies": ["AWSLambdaBasicExecutionRole"]}, "Metadata": {"SamResourceId": "SNSPayloadLogger"}, "LogicalResourceId": "SNSPayloadLogger", "PhysicalResourceId": null} 2022-07-22T17:09:21.434 INFO --- [ asgi_gw_2] localstack.request.aws : AWS cloudformation.DescribeStackEvents => 200 2022-07-22T17:09:21.478 WARN --- [ asgi_gw_1] l.u.c.template_deployer : Unexpected resource type Serverless::Function when resolving references of resource SNSPayloadLogger: {"Type": "AWS::Serverless::Function", "Properties": {"Description": "A Lambda function that logs the payload of messages sent to an associated SNS topic.", "Runtime": "nodejs16.x", "Architectures": ["x86_64"], "CodeUri": "s3://aws-sam-cli-managed-default-samclisourcebucket-7992d0cc/dc5ed1147fb133006d2d390b4a56be95", "Handler": "sns-payload-logger.snsPayloadLoggerHandler", "Events": {"SNSTopicEvent": {"Type": "SNS", "Properties": {"Topic": "arn:aws:sns:us-east-1:000000000000:f88c5c65-6de1-4354-94fc-e4636dd7844d-SimpleTopic-74053aac"}}}, "MemorySize": 128, "Timeout": 100, "Policies": ["AWSLambdaBasicExecutionRole"]}, "Metadata": {"SamResourceId": "SNSPayloadLogger"}, "LogicalResourceId": "SNSPayloadLogger", "PhysicalResourceId": null}. To find out if Serverless::Function is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation 2022-07-22T17:09:21.488 INFO --- [ asgi_gw_1] localstack.request.aws : AWS cloudformation.DescribeStacks => 200 2022-07-22T17:09:21.578 WARN --- [ asgi_gw_2] l.u.c.template_deployer : Unexpected resource type Serverless::Function when resolving references of resource SNSPayloadLogger: {"Type": "AWS::Serverless::Function", "Properties": {"Description": "A Lambda function that logs the payload of messages sent to an associated SNS topic.", "Runtime": "nodejs16.x", "Architectures": ["x86_64"], "CodeUri": "s3://aws-sam-cli-managed-default-samclisourcebucket-7992d0cc/dc5ed1147fb133006d2d390b4a56be95", "Handler": "sns-payload-logger.snsPayloadLoggerHandler", "Events": {"SNSTopicEvent": {"Type": "SNS", "Properties": {"Topic": "arn:aws:sns:us-east-1:000000000000:f88c5c65-6de1-4354-94fc-e4636dd7844d-SimpleTopic-74053aac"}}}, "MemorySize": 128, "Timeout": 100, "Policies": ["AWSLambdaBasicExecutionRole"]}, "Metadata": {"SamResourceId": "SNSPayloadLogger"}, "LogicalResourceId": "SNSPayloadLogger", "PhysicalResourceId": null}. To find out if Serverless::Function is supported in LocalStack Pro, please check out our docs at https://docs.localstack.cloud/aws/cloudformation 2022-07-22T17:09:21.584 INFO --- [ asgi_gw_2] localstack.request.aws : AWS cloudformation.DescribeStacks => 200
同じくAWS SAMで作成したAmazon API Gatewayを使ったプロジェクトだとこういうことにはならなかったので、それぞれで生成された
template.yaml
を見比べてこの差異に気づきました。
ログがAmazon CloudWatch Logsに出力されない
これは気づくまでちょっとかかりました。ロググループ自体は作られていましたからね。
ログが見れなくて、ふとコンテナ側を直接見てはどうだろう?と思って試したら見れました、という話でした。
LocalStackの問題だと思うのですが、どこかで直らないでしょうかね…?
まとめ
LocalStack上で、Amazon SNSの通知を受け取るAWS Lambda関数を作成してみました。
AWS Lambda関数とSNSというかLocalStackにだいぶハマった感じがしますが、とりあえず動かせてよかったかなと思います。