AWS Lambda Error：An error occurred (UnauthorizedOperation) when calling the AuthorizeSecurityGroupIngress operation: You are not authorized to perform this operation.

Problem

當我嘗試使用 Lambda function，增加一條 rule 至指定 security group 時，出現以下錯誤：

Test Event Name
MyTest

Response
null

Function Logs
START RequestId: 7f5624bf-430f-4c65-a51f-9060e990b159 Version: $LATEST
An error occurred (UnauthorizedOperation) when calling the AuthorizeSecurityGroupIngress operation: You are not authorized to perform this operation. User: arn:aws:sts::102832830373:assumed-role/AddSGFunction-role-shx93f9g/AddSGFunction is not authorized to perform: ec2:AuthorizeSecurityGroupIngress on resource: arn:aws:ec2:us-east-1:102832830373:security-group/sg-056a6d3af28d29d47 because no identity-based policy allows the ec2:AuthorizeSecurityGroupIngress action. Encoded authorization failure message: u7vWnAW1hpW1M7YZt5LVRybW3WXafRneSDK4jpCisbXWFiI5yS7DAZQBJNuXqUMKsRSSn-pTBiotcINMCxvnICsMzd9e7D61fZgGGwrrnrsSPcwnC6V-SH7pDmEtw_rD8cIHhN1CQIByIzn3waZ0bwQqB7ggufrlDZlf4pVWU860dhL89jes5EP8XAW-cuXnoz156F-11Us2ZToRlSHIFGbsTQhxCjaLTIkKFyLnd45mkgF_24a8VKjqUXz1jcEAfZEM3FgQeCoG7BBKqk9Z_3S-ODAPoBQ4NuGYFfYKTqIGOVx_QEV3HlqC9QEXJ1ylRWoZ2aU94KESR54ak4yk9U6bwZGMz_Y8Lxw2UcQBfI43sdvGQW8Ga6G8yGMLO9qjaFqgRkVBrUZIjpprl0vEo5pN96m8mzdmqdELDR0KUV69VZkLzfevef44zP2Bwo5JaQKhFyzb00eiaZj3AuLSbf7Hyzawp5DKnDa7xasZkHU60sVbDDPnAyWQ98D9felWps-tafPaWVO49rg50UQKAz2lSwDKFLT-BIluOph8ruuJO_0YZyiQJmn3TYMEq3x2uR7IUsYTdugmYAJzmzNHzdIy5YxBPGx7SEvoEX9C_xZIloJLSWkO
END RequestId: 7f5624bf-430f-4c65-a51f-9060e990b159
REPORT RequestId: 7f5624bf-430f-4c65-a51f-9060e990b159	Duration: 3407.09 ms	Billed Duration: 3408 ms	Memory Size: 128 MB	Max Memory Used: 89 MB	Init Duration: 408.87 ms

Request ID
7f5624bf-430f-4c65-a51f-9060e990b159

Root Cause

此 Lambda function 不具備增添 security group rule 的權限

How-To

增添以下權限 ec2:AuthorizeSecurityGroupIngress

Test Result

執行結果

Test Event Name
MyTest

Response
null

Function Logs
START RequestId: 266304a2-53ce-40f0-8918-7782fb6bb3cc Version: $LATEST
Ingress Successfully Set {'Return': True, 'SecurityGroupRules': [{'SecurityGroupRuleId': 'sgr-063e134cb604a00a2', 'GroupId': 'sg-056a6d3af28d29d47', 'GroupOwnerId': '102832830373', 'IsEgress': False, 'IpProtocol': 'tcp', 'FromPort': 80, 'ToPort': 80, 'CidrIpv4': '0.0.0.0/0'}], 'ResponseMetadata': {'RequestId': '58ea616d-28bb-4d65-b544-d61ff7834a33', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '58ea616d-28bb-4d65-b544-d61ff7834a33', 'cache-control': 'no-cache, no-store', 'strict-transport-security': 'max-age=31536000; includeSubDomains', 'content-type': 'text/xml;charset=UTF-8', 'content-length': '719', 'date': 'Sun, 10 Mar 2024 02:11:17 GMT', 'server': 'AmazonEC2'}, 'RetryAttempts': 0}}
END RequestId: 266304a2-53ce-40f0-8918-7782fb6bb3cc
REPORT RequestId: 266304a2-53ce-40f0-8918-7782fb6bb3cc	Duration: 3649.45 ms	Billed Duration: 3650 ms	Memory Size: 128 MB	Max Memory Used: 89 MB	Init Duration: 324.16 ms

Request ID
266304a2-53ce-40f0-8918-7782fb6bb3cc

確認 security group rule

延伸問題

若 Lambda function 要 revoke security group rule，需賦予以下權限，範例：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "ec2:RevokeSecurityGroupIngress",
            "Resource": "arn:aws:ec2:region:account-id:security-group/sg-xxxxxxxx"
        }
    ]
}

實際例子：

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "ec2:RevokeSecurityGroupIngress",
            "Resource": "arn:aws:ec2:us-east-1:102832830373:security-group/sg-056a6d3af28d29d47"
        }
    ]
}

Using AWS CLI to Encrypt and Decrypt

上傳一個 plain text file 到 AWS CloudShell，並印出內容

[cloudshell-user@ip-10-130-78-240 ~]$ cat ExamplePlaintextFile.txt
abc
123
test

建立一個 AWS KMS Key 如下

透過 AWS CLI 加密原 plain text file，並產生一個 EncryptedFile.txt，並印出 EncryptedFile.txt 內容

[cloudshell-user@ip-10-130-78-240 ~]$ aws kms encrypt --key-id alias/AlbertKey --plaintext fileb://ExamplePlaintextFile.txt --output text --query CiphertextBlob | base64 --decode > EncryptedFile.txt
[cloudshell-user@ip-10-130-78-240 ~]$ cat EncryptedFile.txt
00Y0Tom%`He.08Ó~SGe`VCn5j0h     *H
             "N^";';P䫒}XU'Jڋ?oC

透過 AWS CLI 解密 EncryptedFile.txt 至 DecryptedFile.txt，並印出 DecryptedFile.txt

[cloudshell-user@ip-10-130-78-240 ~]$ aws kms decrypt --ciphertext-blob fileb://EncryptedFile.txt --output text --query Plaintext | base64 --decode > DecryptedFile.txt
[cloudshell-user@ip-10-130-78-240 ~]$ cat DecryptedFile.txt
abc
123
test

How to read Parameter Store values from Lambda function

已在 Parameter Store 建立兩個參數

已建立以下 Lambda function Code

import boto3
import json

def lambda_handler(event, context):
    # 初始化 SSM 客戶端
    ssm = boto3.client('ssm')
    
    # 參數名稱
    parameters_to_get = [
        '/Dev/DBServer/MySQL/db-user',
        '/Dev/DBServer/MySQL/db-pwd'
    ]
    
    # 從 Parameter Store 獲取參數值
    response = ssm.get_parameters(
        Names=parameters_to_get,
        WithDecryption=True  # 如果參數是加密的，設置為 True
    )
    
    # 初始化一個變數來存儲參數名稱和值
    parameters_values = {}
    
    # 檢查並讀取每個參數的值
    for param in response['Parameters']:
        parameters_values[param['Name']] = param['Value']
    
    # 列印參數值
    print(f"Parameter Values: {json.dumps(parameters_values)}")
    
    return {
        'statusCode': 200,
        'body': json.dumps(f"Parameter Values: {parameters_values}")
    }

賦予 Lambda 有讀取 SSM 的權限

執行結果

Test Event Name
MyTest

Response
{
  "statusCode": 200,
  "body": "\"Parameter Values: {'/Dev/DBServer/MySQL/db-pwd': 'mypassword', '/Dev/DBServer/MySQL/db-user': 'admin'}\""
}

Function Logs
START RequestId: 7c95489a-f09d-44b8-9b5e-25968ca0b318 Version: $LATEST
Parameter Values: {"/Dev/DBServer/MySQL/db-pwd": "mypassword", "/Dev/DBServer/MySQL/db-user": "admin"}
END RequestId: 7c95489a-f09d-44b8-9b5e-25968ca0b318
REPORT RequestId: 7c95489a-f09d-44b8-9b5e-25968ca0b318	Duration: 2379.66 ms	Billed Duration: 2380 ms	Memory Size: 128 MB	Max Memory Used: 78 MB	Init Duration: 301.53 ms

Request ID
7c95489a-f09d-44b8-9b5e-25968ca0b318

Create AWS Lambda from CloudFormation

假設已準備好 yaml file，內容如下：

AWSTemplateFormatVersion: '2010-09-09'
Description: 'AWS CloudFormation Sample: AWS Lambda Function with Python.'

Resources:
  HelloWorldFunction:
    Type: 'AWS::Lambda::Function'
    Properties:
      Handler: index.lambda_handler
      Role: !GetAtt LambdaExecutionRole.Arn
      Runtime: python3.8
      Code:
        ZipFile: |
          def lambda_handler(event, context):
              return {
                  'statusCode': 200,
                  'body': 'Hello, World!'
              }
      Timeout: 5

  LambdaExecutionRole:
    Type: 'AWS::IAM::Role'
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: 'sts:AssumeRole'
      Policies:
        - PolicyName: 'lambda-basic-execution'
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - 'logs:CreateLogGroup'
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvents'
                Resource: 'arn:aws:logs:*:*:*'

確認 CloudFormation 執行結果：

確認 CloudFormation 已建立的 AWS resources：

確認所建立的 Lambda Function：

確認 Lambda 執行結果符合預期：

Test Event Name
Test

Response
{
  "statusCode": 200,
  "body": "Hello World!"
}

Function Logs
START RequestId: 665b5e07-c386-4b89-b59c-0a6e00979de0 Version: $LATEST
END RequestId: 665b5e07-c386-4b89-b59c-0a6e00979de0
REPORT RequestId: 665b5e07-c386-4b89-b59c-0a6e00979de0	Duration: 2.07 ms	Billed Duration: 3 ms	Memory Size: 128 MB	Max Memory Used: 39 MB	Init Duration: 129.94 ms

Request ID
665b5e07-c386-4b89-b59c-0a6e00979de0

AWS Lambda Alias

假設我有一支 Lambda function，內容如下：

import json

def lambda_handler(event, context):
    # TODO implement
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda! (Version 1)')
    }

並成功完成 Test：

Test Event Name
MyTest

Response
{
  "statusCode": 200,
  "body": "\"Hello from Lambda! (Version 1)\""
}

Function Logs
START RequestId: f0a719c8-1ffb-499d-942f-206f73dd0045 Version: $LATEST
END RequestId: f0a719c8-1ffb-499d-942f-206f73dd0045
REPORT RequestId: f0a719c8-1ffb-499d-942f-206f73dd0045	Duration: 11.58 ms	Billed Duration: 12 ms	Memory Size: 128 MB	Max Memory Used: 34 MB

Request ID
f0a719c8-1ffb-499d-942f-206f73dd0045

透過 AWS CLI 發佈第一個版本：

[cloudshell-user@ip-10-132-51-15 ~]$ aws lambda publish-version --function-name HelloWorldFunction
{
    "FunctionName": "HelloWorldFunction",
    "FunctionArn": "arn:aws:lambda:us-east-1:827888540138:function:HelloWorldFunction:1",
    "Runtime": "python3.12",
    "Role": "arn:aws:iam::827888540138:role/service-role/HelloWorldFunction-role-2pj1z9b2",
    "Handler": "lambda_function.lambda_handler",
    "CodeSize": 277,
    "Description": "",
    "Timeout": 3,
    "MemorySize": 128,
    "LastModified": "2024-02-19T00:51:48.000+0000",
    "CodeSha256": "FGaiUQRyAxEPhQmLcV9dUlGl1qIvgba7Ae1zCInXC3g=",
    "Version": "1",
    "TracingConfig": {
        "Mode": "PassThrough"
    },
    "RevisionId": "d42bcc86-34a7-4ce4-a739-16e6d785c3af",
    "State": "Active",
    "LastUpdateStatus": "Successful",
    "PackageType": "Zip",
    "Architectures": [
        "x86_64"
    ],
    "EphemeralStorage": {
        "Size": 512
    },
    "SnapStart": {
        "ApplyOn": "None",
        "OptimizationStatus": "Off"
:...skipping...

修改 Lambda function，程式內容如下：

import json

def lambda_handler(event, context):
    # TODO implement
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda! (Version 2)')
    }

發佈第二個版本：

[cloudshell-user@ip-10-132-51-15 ~]$ aws lambda publish-version --function-name HelloWorldFunction
{
    "FunctionName": "HelloWorldFunction",
    "FunctionArn": "arn:aws:lambda:us-east-1:827888540138:function:HelloWorldFunction:2",
    "Runtime": "python3.12",
    "Role": "arn:aws:iam::827888540138:role/service-role/HelloWorldFunction-role-2pj1z9b2",
    "Handler": "lambda_function.lambda_handler",
    "CodeSize": 276,
    "Description": "",
    "Timeout": 3,
    "MemorySize": 128,
    "LastModified": "2024-02-19T01:02:46.000+0000",
    "CodeSha256": "xzDR5dGezW2VtWOvVraE1yY6kIQXIKy7UBS/1WDMeyw=",
    "Version": "2",
    "TracingConfig": {
        "Mode": "PassThrough"
    },
    "RevisionId": "299ba2c8-c055-4314-9b29-bbd9e9332e96",
    "State": "Active",
    "LastUpdateStatus": "Successful",
    "PackageType": "Zip",
    "Architectures": [
        "x86_64"
    ],
    "EphemeralStorage": {
        "Size": 512
    },
    "SnapStart": {

建立 Alias，PROD 指向 Version 1、DEV 指向 Version 2

[cloudshell-user@ip-10-132-51-15 ~]$ aws lambda create-alias --function-name HelloWorldFunction --name PROD --function-version 1
{
    "AliasArn": "arn:aws:lambda:us-east-1:827888540138:function:HelloWorldFunction:PROD",
    "Name": "PROD",
    "FunctionVersion": "1",
    "Description": "",
    "RevisionId": "dfc0cdd9-5bf8-4483-bd0b-0cc098e4a70e"
}
[cloudshell-user@ip-10-132-51-15 ~]$ aws lambda create-alias --function-name HelloWorldFunction --name DEV --function-version 2
{
    "AliasArn": "arn:aws:lambda:us-east-1:827888540138:function:HelloWorldFunction:DEV",
    "Name": "DEV",
    "FunctionVersion": "2",
    "Description": "",
    "RevisionId": "2526ec87-e93b-4189-828a-bab954fcf019"
}

執行 PROD alais，結果如下：

[cloudshell-user@ip-10-132-51-15 ~]$ aws lambda invoke --function-name "HelloWorldFunction:PROD" outputfile.txt
{
    "StatusCode": 200,
    "ExecutedVersion": "1"
}

執行 DEV alais，結果如下：

[cloudshell-user@ip-10-132-51-15 ~]$ aws lambda invoke --function-name "HelloWorldFunction:DEV" outputfile.txt
{
    "StatusCode": 200,
    "ExecutedVersion": "2"
}

Version 2 完成測試後，將 PROD 指向 Version 2：

[cloudshell-user@ip-10-132-51-15 ~]$ aws lambda update-alias --function-name "HelloWorldFunction" --name "PROD" --function-version "2"
{
    "AliasArn": "arn:aws:lambda:us-east-1:827888540138:function:HelloWorldFunction:PROD",
    "Name": "PROD",
    "FunctionVersion": "2",
    "Description": "",
    "RevisionId": "9b3c5ce4-4c46-4bba-a5bf-ceb8291e090d"
}

執行 PROD alais，結果如下：

[cloudshell-user@ip-10-132-51-15 ~]$ aws lambda invoke --function-name "HelloWorldFunction:PROD" outputfile.txt
{
    "StatusCode": 200,
    "ExecutedVersion": "2"
}

如何透過 AWS Lambda 啟動/關閉 EC2

前置作業

假設我已經建立好一台 EC2

已準備一支用 Python 語法寫的 AWS Lambda function（當 EC2 是 Running 狀態時，就 Stop 它；當 EC2 是 Stopped 狀態時，就 Start 它）

import boto3

# 初始化 EC2 客戶端
ec2 = boto3.client('ec2')

def lambda_handler(event, context):
    # 指定要操作的 EC2 實例 ID
    instance_id = 'i-0ebe0819d2efb0cff'

    # 獲取實例的當前狀態
    response = ec2.describe_instances(InstanceIds=[instance_id])
    state = response['Reservations'][0]['Instances'][0]['State']['Name']

    # 根據實例的當前狀態進行操作
    if state == 'running':
        # 如果實例正在運行，則停止它
        stop_response = ec2.stop_instances(InstanceIds=[instance_id])
        action = 'stopped'
    elif state == 'stopped':
        # 如果實例已停止，則啟動它
        start_response = ec2.start_instances(InstanceIds=[instance_id])
        action = 'started'
    else:
        # 如果實例處於其他狀態，不進行操作
        return {
            'statusCode': 400,
            'body': f'Instance {instance_id} is in {state} state, no action taken.'
        }

    # 返回操作結果
    return {
        'statusCode': 200,
        'body': f'Instance {instance_id} has been {action} successfully.'
    }

賦予 AWS Lambda 操作 EC2 的權限

執行測試：(Running => Stopped)

EC2 執行前

Lambda 執行結果

EC2 執行後

執行測試：(Stopped => Running)

EC2 執行前

Lambda 執行結果

EC2 執行後

AWS S3 + CloudFront + OAI

步驟 1: Create S3 bucket

步驟 2: Upload a image file to  S3 bucket

步驟 3: Create CloudFront distribution 與使用 Origin Access Identity (OAI) 保護 S3 content

步驟 4: Update S3 bucket policy

步驟 5: 等候幾分鐘，待 CloudFront distribution 完成 Deploy

步驟 6: 測試 CloudFront distribution

如何在 EC2 安裝 AWS SDK for Python (Boto3)

步驟 1: 安裝 Python

[ec2-user@ip-172-31-28-226 ~]$ sudo yum install python3 python3-pip -y

步驟 2: 驗證安裝

ec2-user@ip-172-31-28-226 ~]$ python3 --version
Python 3.9.16

步驟 3: 安裝 AWS SDK for Python (Boto3)

[ec2-user@ip-172-31-28-226 ~]$ pip3 install boto3
Defaulting to user installation because normal site-packages is not writeable
Collecting boto3
  Downloading boto3-1.34.40-py3-none-any.whl (139 kB)
     |████████████████████████████████| 139 kB 5.7 MB/s            
Collecting botocore<1.35.0,>=1.34.40
  Downloading botocore-1.34.40-py3-none-any.whl (12.0 MB)
     |████████████████████████████████| 12.0 MB 41.5 MB/s            
Collecting s3transfer<0.11.0,>=0.10.0
  Downloading s3transfer-0.10.0-py3-none-any.whl (82 kB)
     |████████████████████████████████| 82 kB 378 kB/s              
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /usr/lib/python3.9/site-packages (from boto3) (0.10.0)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/lib/python3.9/site-packages (from botocore<1.35.0,>=1.34.40->boto3) (2.8.1)
Requirement already satisfied: urllib3<1.27,>=1.25.4 in /usr/lib/python3.9/site-packages (from botocore<1.35.0,>=1.34.40->boto3) (1.25.10)
Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.40->boto3) (1.15.0)
Installing collected packages: botocore, s3transfer, boto3
Successfully installed boto3-1.34.40 botocore-1.34.40 s3transfer-0.10.0

步驟 4: 配置 AWS CLI (請自行設定自己的 AK 與 SK)

[ec2-user@ip-172-31-28-226 ~]$ aws configure
AWS Access Key ID [None]: AKIASHY4YDRZJLQSTQNU
AWS Secret Access Key [None]: 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
Default region name [None]: us-east-1
Default output format [None]: json

步驟 5: 寫一個 Python 程式來列出 S3 bucket

[ec2-user@ip-172-31-28-226 ~]$ vi list_s3_buckets.py 
[ec2-user@ip-172-31-28-226 ~]$ cat list_s3_buckets.py 
import boto3

def list_buckets():
    # 創建一個 S3 服務客戶端
    s3 = boto3.client('s3')
    
    # 列出所有buckets
    response = s3.list_buckets()
    
    # 輸出 bucket name
    print("S3 Buckets:")
    for bucket in response['Buckets']:
        print(f"- {bucket['Name']}")

if __name__ == '__main__':
    list_buckets()

步驟 6: 執行 Python 程式

[ec2-user@ip-172-31-28-226 ~]$ python3 list_s3_buckets.py
S3 Buckets:
- albert-buckert-154142252146

如何安裝與驗證 AWS CLI

安裝 AWS CLI

1. 下載 AWS CLI

[ec2-user@ip-172-31-28-226 ~]$ curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 57.3M  100 57.3M    0     0   260M      0 --:--:-- --:--:-- --:--:--  261M

2. 解壓縮下載檔案

[ec2-user@ip-172-31-28-226 ~]$ unzip awscliv2.zip

3. 安裝 AWS CLI

[ec2-user@ip-172-31-28-226 ~]$ sudo ./aws/install
You can now run: /usr/local/bin/aws --version

4. 驗證安裝結果

[ec2-user@ip-172-31-28-226 ~]$ aws --version
aws-cli/2.15.19 Python/3.11.6 Linux/6.1.75-99.163.amzn2023.x86_64 exe/x86_64.amzn.2023 prompt/off

5. 配置 AWS CLI

- AWS Access Key ID 和 AWS Secret Access Key：你可以在 AWS 管理控制台的 IAM 服務中建立一個新的使用者或使用現有的使用者憑證。
- Default region name：輸入你想要操作的 AWS 區域的代碼，例如 us-east-1。
- Default output format：輸入你偏好的輸出格式，常見的選項包括 json、text 和 table。

[ec2-user@ip-172-31-28-226 ~]$ aws configure
AWS Access Key ID [None]: AKIASHY4YDRZJLQSTQNU
AWS Secret Access Key [None]: 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
Default region name [None]: us-east-1
Default output format [None]: json

透過 AWS CLI 進行 S3 操作

1. Create S3 bucket

[ec2-user@ip-172-31-28-226 ~]$ aws s3 mb s3://albert-buckert-154142252146 --region us-east-1
make_bucket: albert-buckert-154142252146

2. List S3 bucket

[ec2-user@ip-172-31-28-226 ~]$ aws s3 ls
2024-02-13 10:02:24 albert-buckert-154142252146

3. Create a file and upload it to S3 bucket

[ec2-user@ip-172-31-28-226 ~]$ echo "Hello World" > hello.txt
[ec2-user@ip-172-31-28-226 ~]$ cat hello.txt
Hello World
[ec2-user@ip-172-31-28-226 ~]$ aws s3 cp hello.txt s3://albert-buckert-154142252146
upload: ./hello.txt to s3://albert-buckert-154142252146/hello.txt