lifelog and tech

표로 간단 비교:

AWS S3 복제를 사용하면 데이터 중복, 규정 준수 및 재해 복구와 같은 다양한 목적을 위해 동일하거나 다른 AWS 리전에 있는 버킷 간에 객체를 복제할 수 있습니다. AWS S3에서 사용할 수 있는 복제 유형에는 CRR(Cross-Region Replication)과 SRR(Same-Region Replication)의 두 가지 유형이 있습니다.

CRR(Cross-Region Replication)은 서로 다른 AWS 리전의 버킷 간에 객체를 복제하는 데 사용됩니다. CRR은 버킷의 모든 객체 또는 접두사 또는 태그를 기반으로 객체의 하위 집합을 복제할 수 있습니다. CRR은 여러 대상 버킷에 대한 복제를 지원하며 재해 복구 및 데이터 지역성에 사용할 수 있습니다.

동일 리전 복제(SRR)는 동일한 AWS 리전의 버킷 간에 객체를 복제하는 데 사용됩니다. SRR은 버킷의 모든 객체 또는 접두사 또는 태그를 기반으로 객체의 하위 집합을 복제할 수 있습니다. SRR은 데이터 중복, 규정 준수 및 고가용성을 위해 사용할 수 있습니다.

CRR과 SRR의 주요 차이점은 복제 대상입니다. CRR은 서로 다른 AWS 지역의 버킷에 객체를 복제하고 SRR은 동일한 AWS 지역의 버킷에 객체를 복제합니다. 이 차이는 각 복제 유형의 사용 사례에 영향을 미칩니다.

CRR은 재해 복구 및 데이터 지역성이 필요한 조직에 적합합니다. 개체를 다른 지역의 버킷에 복제함으로써 조직은 지역 중단이 발생한 경우에도 데이터를 사용할 수 있도록 보장할 수 있습니다. CRR은 데이터를 특정 지역에 저장해야 한다는 규정 요구 사항을 준수하는 데에도 사용할 수 있습니다.

반면 SRR은 데이터 중복성과 규정 준수가 필요한 조직에 적합합니다. 개체를 동일한 지역의 버킷에 복제함으로써 조직은 서비스 중단이 있거나 버킷을 사용할 수 없게 되더라도 데이터를 사용할 수 있도록 보장할 수 있습니다. SRR은 또한 데이터를 여러 위치에 저장해야 한다는 규정 요구 사항을 준수하는 데 사용할 수 있습니다.

요약하면 CRR과 SRR은 모두 AWS S3에서 데이터 관리를 위한 중요한 도구입니다. CRR은 재해 복구 및 데이터 지역성에 사용되는 반면 SRR은 데이터 중복 및 규정 준수에 사용됩니다. CRR과 SRR 사이의 선택은 조직의 특정 요구 사항과 요구 사항에 따라 다릅니다.

AWS S3 replication allows you to replicate objects between buckets in the same or different AWS regions for various purposes, such as data redundancy, compliance, and disaster recovery. There are two types of replication available in AWS S3: Cross-Region Replication (CRR) and Same-Region Replication (SRR).

CrossRegion Replication (CRR) is used to replicate objects between buckets in different AWS regions. CRR can replicate all objects in a bucket or just a subset of objects based on prefix or tags. CRR supports replication to multiple destination buckets and can be used for disaster recovery and data locality.

SameRegion Replication (SRR) is used to replicate objects between buckets in the same AWS region. SRR can replicate all objects in a bucket or just a subset of objects based on prefix or tags. SRR can be used for data redundancy, compliance, and high availability.

The main difference between CRR and SRR is the destination of replication. CRR replicates objects to buckets in different AWS regions, while SRR replicates objects to buckets in the same AWS region. This difference affects the use cases of each replication type.

CRR is suitable for organizations that require disaster recovery and data locality. By replicating objects to buckets in different regions, organizations can ensure that their data is available even in the event of a regional outage. CRR can also be used to comply with regulatory requirements that mandate data must be stored in specific geographic regions.

SRR, on the other hand, is suitable for organizations that require data redundancy and compliance. By replicating objects to buckets in the same region, organizations can ensure that their data is available even if there is a service disruption or if a bucket becomes unavailable. SRR can also be used to comply with regulatory requirements that mandate data must be stored in multiple locations.

In summary, CRR and SRR are both important tools for data management in AWS S3. CRR is used for disaster recovery and data locality, while SRR is used for data redundancy and compliance. The choice between CRR and SRR depends on the organization's specific needs and requirements.

기본적인 아마존 S3 버킷이 지원하는 오브젝트 클래스들을 정리한다.

우선 영어버전:

한글 버전:

Amazon S3 Glacier Flexible Retrieval과 AWS S3 Glacier Deep Archive의 주요 차이점은 사용 사례와 검색 시간입니다.

Amazon S3 Glacier Flexible Retrieval은 일반적으로 연간 1~2회 비동기식으로 자주 액세스하지 않는 데이터를 보관하는 데 적합한 저비용 스토리지 옵션입니다. S3 Glacier Instant Retrieval에 비해 최대 10% 저렴한 비용을 제공하지만 검색 시간은 몇 분에서 몇 시간까지 길어질 수 있습니다. 이 스토리지 클래스는 장기간 보존을 위해 많은 양의 데이터를 저장해야 하고 더 긴 검색 시간을 허용할 수 있는 조직에 이상적입니다.

반면 AWS S3 Glacier Deep Archive는 1년에 한두 번만 액세스하는 장기 데이터 아카이빙을 위해 설계되었습니다. 모든 S3 스토리지 클래스 중에서 가장 비용이 저렴한 스토리지 클래스이지만 검색 시간이 가장 길어 최대 12시간이 소요될 수 있습니다. 이 스토리지 클래스는 규정 준수 및 규정상의 이유로 데이터를 보존해야 하고 데이터에 즉시 액세스할 필요가 없는 금융 서비스, 의료 및 공공 부문과 같은 고객에게 적합합니다.

요약하면 S3 Glacier Flexible Retrieval은 가끔 비동기식으로 액세스할 수 있는 데이터를 보관하는 데 적합하고 S3 Glacier Deep Archive는 거의 액세스하지 않고 가장 긴 검색 시간을 견딜 수 있는 데이터를 보관하는 데 적합합니다.

Firstly, Create IAM Role with below permission for replication job.

Trust relationships

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "s3.amazonaws.com",
                    "batchoperations.s3.amazonaws.com"
                ]
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

IAM Policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:ListBucket",
                "s3:GetReplicationConfiguration",
                "s3:GetObjectVersionForReplication",
                "s3:GetObjectVersionAcl",
                "s3:GetObjectVersionTagging",
                "s3:GetObjectRetention",
                "s3:GetObjectLegalHold"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::source-bucket-A",
                "arn:aws:s3:::source-bucket-A/*",
                "arn:aws:s3:::destination-bucket-B",
                "arn:aws:s3:::destination-bucket-B/*"
            ]
        },
        {
            "Action": [
                "s3:ReplicateObject",
                "s3:ReplicateDelete",
                "s3:ReplicateTags",
                "s3:ObjectOwnerOverrideToBucketOwner"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::source-bucket-A/*",
                "arn:aws:s3:::destination-bucket-B/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:InitiateReplication",
                "s3:GetReplicationConfiguration",
                "s3:PutInventoryConfiguration"
            ],
            "Resource": [
                "arn:aws:s3:::source-bucket-A",
                "arn:aws:s3:::source-bucket-B/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::source-bucket-A/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:Decrypt",
                "kms:GenerateDataKey"
            ],
            "Resource": [
                "arn:aws:kms:ap-northeast-2:[Source A Account ID]:key/[keystrings]"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "kms:GenerateDataKey",
                "kms:Encrypt"
            ],
            "Resource": [
                "arn:aws:kms:ap-northeast-2:[Destination B Account ID]:key/[keystrings]"
            ]
        }
    ]
}

The action of KMS will depend on the encryption setting of the S3 bucket.

Due to the requirement of the setting, a replication rule of S3 bucket management in source account A has been created with below.

{
    "Sid": "S3PolicyStmt-DO-NOT-MODIFY-1670218832531",
    "Effect": "Allow",
    "Principal": {
        "AWS": "arn:aws:iam::[Source A Account ID]:root"
    },
    "Action": [
        "s3:GetBucketVersioning",
        "s3:PutBucketVersioning",
        "s3:ReplicateObject",
        "s3:ReplicateDelete"
    ],
    "Resource": [
        "arn:aws:s3:::destination-bucket-B",
        "arn:aws:s3:::destination-bucket-B/*"
    ]
}

Below is an included version of above permission to change object ownership to destination bucket owner.

{
    "Sid": "S3PolicyStmt-DO-NOT-MODIFY-1670219041656",
    "Effect": "Allow",
    "Principal": {
        "AWS": "arn:aws:iam::[Source Account A]:root"
    },
    "Action": [
        "s3:GetBucketVersioning",
        "s3:PutBucketVersioning",
        "s3:ReplicateObject",
        "s3:ReplicateDelete",
        "s3:ObjectOwnerOverrideToBucketOwner"
    ],
    "Resource": [
        "arn:aws:s3:::destination-bucket-B",
        "arn:aws:s3:::destination-bucket-B/*"
    ]
}

The KMS Policy also needs to allow source account as below.

{
    "Sid": "Enable cross account encrypt access for S3 Cross Region Replication",
    "Effect": "Allow",
    "Principal": {
        "AWS": "arn:aws:iam::[Source Account A]:root"
    },
    "Action": [
        "kms:Encrypt"
    ],
    "Resource": "*"
}