Amazon S3 supports both identity-based policies and resource-based policies (referred to as bucket policies). It also supports an access control list (ACL) that is independent of IAM policies and permissions.
The Amazon S3 data model is a flat structure: You create a bucket, and the bucket store objects. There is no hierarchy of subbuckets or subfolders.
You can change the storage class of an object in S3 to any other storage class by making a copy of the object using the PUT Object - Copy API. However, you can't use PUT Object - Copy to copy objects that are stored in the S3 Glacier or S3 Glacier Deep Archive storage classes.
Amazon S3 provides a highly durable storage infrastructure designed for mission-critical and primary data storage. Objects are redundantly stored on multiple devices across multiple facilities in an Amazon S3 Region.
Storage Class
In a versioning-enabled bucket, you can't change the storage class of a specific version of an object. When you copy it, Amazon S3 gives it a new version ID.
When setting up a replication configuration, you can set the storage class for replicated objects to any other storage class. However, you can't replicate objects that are stored in the S3 Glacier or S3 Glacier Deep Archive storage classes.
S3 Standard
The default storage class. If you don't specify the storage class when you upload an object, Amazon S3 assigns the S3 Standard storage class.
Reduced Redundancy
The Reduced Redundancy Storage (RRS) storage class is designed for noncritical, reproducible data that can be stored with less redundancy than the S3 Standard storage class.
S3 Intelligent-Tiering
The S3 Intelligent-Tiering designed to optimize storage costs by automatically moving data to the most cost-effective access tier, without operational overhead. It is the perfect storage class when you want to optimize storage costs for data that has unknown or changing access patterns.
There are no retrieval fees, but a small monthly object monitoring and automation fee for S3 Intelligent-Tiering.
The S3 Intelligent-Tiering storage class is suitable for objects larger than 128 KB that you plan to store for at least 30 days. Smaller objects can be stored, but they are always charged at the frequent access tier rates in the S3 Intelligent-Tiering storage class. If you delete an object before the end of the 30-day minimum storage duration period, you are charged for 30 days.
S3 Standard-IA
The S3 Standard-IA are designed for long-lived and infrequently accessed data that requires an access in millisecond.
This storage class stores the object data redundantly across multiple geographically separated Availability Zones (similar to the S3 Standard storage class), offering greater availability and resiliency than the S3 One Zone-IA class.
Amazon S3 charges a retrieval fee for objects in S3 Standard-IA.
S3 Standard-IA has a minimum capacity charge per object (128 KB), and a 30 days retentions period.
S3 One Zone-IA
The S3 One Zone-IA are designed for long-lived and infrequently accessed data that requires an access in millisecond.
This storage class stores the object data in only one Availability Zone, which makes it less expensive than S3 Standard-IA. However, the data is not resilient to the physical loss of the Availability Zone resulting from disasters.
Amazon S3 charges a retrieval fee for objects in S3 One Zone-IA.
Storage classes for Archiving objects
These classes are used for archives where portions of the data might that rarely needs to be accessed.
You must first restore the objects in these classes before you can access them. Restoring make a temporary copy of the object. The restored object copy is in the S3 Reduced Redundancy storage class and available only for the duration you specify in the restore request. After that, Amazon S3 deletes the temporary copy, and the object remains archived in archiving storage classes.
When you archive objects to or deleting objects in these classes by using S3 Lifecycle management, Amazon S3 transitions these objects asynchronously. There might be a delay between the transition/deletion date in the Lifecycle configuration rule and the date of the physical transition/deletion. You are charged Amazon S3 Glacier prices based on the transition date specified in the rule.
The objects that are stored in these classes are visible and available only through Amazon S3. They are not available through the separate Amazon S3 Glacier service.
For each object archived, Amazon S3 uses 8 KB of storage for the name of the object and other metadata. You are charged Amazon S3 Standard rates for this additional storage. Amazon S3 also uses 32 KB of storage for index and related metadata. This extra data is necessary to identify and restore your object. You are charged S3 Glacier or S3 Glacier Deep Archive rates for this additional storage.
Each object that you transition to these storage classes constitutes one transition request. There is a cost for each such request.
If you are archiving small objects, consider aggregating many small objects into a smaller number of large objects to reduce transition request costs.
S3 Glacier
Data stored in the S3 Glacier storage class has a minimum storage duration period of 90 days and can be accessed in as little as 1-5 minutes using expedited retrieval.
Has a default retrieval time of 3-5 hours.
If you have deleted, overwritten, or transitioned to a different storage class an object before the 90-day minimum, you are charged for 90 days.
S3 Glacier Deep Archive
Data stored in the S3 Glacier Deep Archive storage class has a minimum storage duration period of 180 days and a default retrieval time of 12 hours.
If you have deleted, overwritten, or transitioned to a different storage class an object before the 180-day minimum, you are charged for 180 days.
You can reduce S3 Glacier Deep Archive retrieval costs by using bulk retrieval, which returns data within 48 hours.
S3 Outposts storage class
You can create S3 buckets on your AWS Outposts and store and retrieve objects on-premises. The S3 Outposts storage class is only available for objects stored in buckets on AWS Outposts.
Objects stored in the S3 Outposts (OUTPOSTS) storage class are always encrypted using server-side encryption with Amazon S3 managed encryption keys (SSE-S3). You can also explicitly choose to encrypt objects stored in the S3 Outposts storage class using server-side encryption with customer-provided encryption keys (SSE-C).
AWS S3 Glacier Vault
A vault is a container for storing archives. You can store an unlimited number of archives in a vault.
Vault are specific to particular AWS Regions. When you create a vault, you specify a vault name and the AWS Region in which you want to create the vault.
Vault Lock Policy
S3 Glacier Vault Lock allows you to easily deploy and enforce compliance controls for individual S3 Glacier vaults with a vault lock policy.
A vault lock policy can be locked to prevent future changes, providing strong enforcement for your compliance controls.
Vault Access Policy
You use a vault access policy to implement access controls that are not compliance related, temporary, and subject to frequent modification.
Vault lock and vault access policies can be used together. For example, you can implement time-based data retention rules in the vault lock policy (deny deletes), and grant read access to designated third parties or your business partners (allow reads).
Storage Lifecycle
An S3 Lifecycle configuration is a set of rules that define actions that Amazon S3 applies to a group of objects. There are two types of actions:
- Transition actions—Define when objects transition to another storage class.
- Expiration actions—Define when objects expire. Amazon S3 deletes expired objects on your behalf.
Amazon S3 supports a waterfall model for transitioning between storage classes, as shown in the following diagram. Transitions supported by the lifecycle are shown by arrows in the diagram. By contrast, transitions not shown by arrows in the diagram are not supported by S3 lifecycle.
, you can transition only objects that are at least 30 days noncurrent to S3 Standard-IA or S3 One Zone-IA storage.
You should not specify a Lifecycle rule to move or delete an object from S3 Intelligent-Tiering, S3 Standard-IA, or S3 One Zone-IA when the object is less than 30 days in the storage class, because The S3 Intelligent-Tiering, S3 Standard-IA, and S3 One Zone-IA storage classes have a minimum 30-day storage charge.
You should not specify a Lifecycle rule to move or delete an object from S3 Glacier when the object is less than 90 days in the storage class, because The S3 Glacier has a minimum 90-day storage charge.
You should not specify a Lifecycle rule to move or delete an object from S3 Glacier Deep Archive when the object is less than 180 days in the storage class, because The S3 Glacier Deep Archive has a minimum 180-day storage charge.
S3 Inventory
Amazon S3 inventory is one of the tools Amazon S3 provides to help manage your storage. You can use it to audit and report on the replication and encryption status of your objects for business, compliance, and regulatory needs.
You can configure what object metadata to include in the inventory, whether to list all object versions or only current versions, where to store the inventory list file output, and whether to generate the inventory on a daily or weekly basis. You can also specify that the inventory list file be encrypted.
Source Bucket - The bucket that the inventory lists the objects for.
Destination Bucket - The bucket where the inventory list file is stored.
Destination bucket must be in the same AWS Region as the source bucket.
Destination bucket can be the same as the source bucket, or be owned by a different account than the account that owns the source bucket.
The inventory list provides eventual consistency for PUTs of both new objects and overwrites, and DELETEs.
You can query Amazon S3 inventory using standard SQL by using Amazon Athena.
Bucket
A bucket is a container for objects stored in Amazon S3. They organize the Amazon S3 namespace at the highest level.
An Amazon S3 bucket name is globally unique, and the namespace is shared by all AWS accounts. You can configure buckets so that they are created in a specific AWS Region. After you create a bucket, you can't change its name or Region.
You can't create a bucket from within another bucket.
Amazon S3 Block Public Access settings can override ACLs and bucket policies so that you can enforce uniform limits on public access to these resources. Turning on all four settings for Block Public Access for your account helps you block all public access for all current and future buckets.
Bucket names can consist only of lowercase letters, numbers, dots (.), and hyphens (-), and must begin and end with a letter or number. Bucket names must not be formatted as an IP address. Buckets used with Amazon S3 Transfer Acceleration can't have dots (.) in their names. If you include dots in a bucket's name, you can't use virtual-host-style addressing over HTTPS, unless you perform your own certificate validation. This limitation doesn't affect buckets used for static website hosting, because static website hosting is only available over HTTP.
When a bucket is configured to be a Requester Pays bucket, the requester instead of the bucket owner pays the cost of the request and the data download from the bucket. The bucket owner always pays the cost of storing data. Requester Pays buckets do not support anonymous requests, BitTorrent, SOAP requests, and use as the target bucket for end-user logging. When the requester assumes an IAM role before making their request, the account to which the role belongs is charged for the request.
Bucket ownership is not transferable to another account.
Virtual Hosting of Buckets
Object
Objects are the fundamental entities stored in Amazon S3. An object is uniquely identified within a bucket by a key (name) and a version ID. The combination of a bucket, key, and version ID uniquely identify each object. Every object in Amazon S3 can be uniquely addressed through the combination of the web service endpoint, bucket name, key, and optionally, a version. For example, in the URL https://doc.s3.amazonaws.com/2006-03-01/AmazonS3.wsdl, doc is the name of the bucket and 2006-03-01/AmazonS3.wsdl is the key.
There is no limit to the number of objects that you can store in a bucket.
Objects consist of key, version ID, value, metadata, subresources, and Access control information.
- A key is the unique identifier for an object within a bucket. You use the object key to retrieve the object. The Amazon S3 console use delimiter
/in keys to supports a concept of folder - The version ID is a string that Amazon S3 generates when you add an object to a bucket. A key and a version ID uniquely identify an object in a bucket.
- The object value can be any sequence of bytes. It is opaque to Amazon S3.
- The metadata is a set of name-value pairs that describe the object. You can assign user-defined metadata to your objects. AWS S3 can also assigns system-metadata to these objects.
You can set object metadata in Amazon S3 at the time you upload the object. After you upload the object, you cannot modify object metadata. - Amazon S3 uses the subresource mechanism to store object-specific additional information. Subresources are subordinates to objects, they are always associated with some other entity such as an object or a bucket. Subresources include:
-
acl: Theaclsubresource contains a list of grants identifying the grantees and the permissions granted. -
torrent: Amazon S3 uses the torrent subresource to return the torrent file associated with the specific object. -
restore: Supports temporarily restoring an archived object.
-
Presigned URL
All objects and buckets are private by default. However, you can use a presigned URL to optionally share objects or enable your customers/users to upload objects to buckets without AWS security credentials or permissions. When you create a presigned URL, you associate it with a specific action. You can share the URL, and anyone with access to it can perform the action embedded in the URL as if they were the original signing user.
Object locking
Object Lock works only in versioned buckets.
With S3 Object Lock, you can store objects using a write-once-read-many (WORM) model. Object Lock can help prevent objects from being deleted or overwritten for a fixed amount of time or indefinitely.
S3 buckets with S3 Object Lock can't be used as destination buckets for server access logs.
You cannot turn on Object Lock for an existing bucket by your own.
If you create a bucket with Object Lock enabled, Amazon S3 automatically enables versioning for the bucket. You can't disable Object Lock or suspend versioning for the bucket.
The default retention setting includes a different retention mode and a retention period.
Retention Mode
Governance mode
In governance mode, you can still grant some special users permission to overwrite or delete an object version, or alter its lock settings unless they have special permissions.Compliance mode
In compliance mode, a protected object version can't be overwritten or deleted by any user, including the root user in your AWS account. When an object is locked in compliance mode, its retention mode can't be changed, and its retention period can't be shortened.
Retention Period
Retention period specifies a fixed period of time during which an object remains locked. During this period, object is WORM-protected and can't be overwritten or deleted.
Amazon S3 stores a timestamp in the object version's metadata to indicate when the retention period expires.
If you apply a default retention period to a bucket, Amazon S3 creates the object with a retention period that matches the bucket default. After the object is created, its retention period is independent from the bucket's default retention period. Changing a bucket's default retention period doesn't change the existing retention period for any objects in that bucket.
Legal Hold
Legal hold provides the same protection as a retention period, but it has no expiration date. Instead, a legal hold remains in place until you explicitly remove it. Legal holds are independent from retention periods.
Retention periods and legal holds apply to individual object versions. An object version can have both a retention period and a legal hold, one but not the other, or neither. Different versions of a single object can have different retention modes and periods.
Amazon S3 stores the lock information in the metadata for that object version. Placing a retention period or legal hold on an object protects only the version specified in the request. It doesn't prevent new versions of the object from being created. If you put an object into a bucket that has the same key name as an existing protected object, Amazon S3 creates a new version of that object, stores it in the bucket as requested, and reports the request as completed successfully. The existing protected version of the object remains locked according to its retention configuration.
Versioning
Legal hold — Provides the same protection as a retention period, but it has no expiration date.
Versioning in Amazon S3 is a means of keeping multiple variants of an object in the same bucket. You can use the S3 Versioning feature to preserve, retrieve, and restore every version of every object stored in your buckets.
If you notice a significant increase in the number of HTTP 503-slow down responses received for Amazon S3 PUT or DELETE object requests to a bucket that has S3 Versioning enabled, you might have one or more objects in the bucket for which there are millions of versions.
Updating an object version's metadata, as occurs when you place or alter an Object Lock, doesn't overwrite the object version or reset its Last-Modified timestamp.
Version ID
Regardless of whether you enable versioning, each object in your bucket has a version ID. If you don't enable S3 Versioning, Amazon S3 sets the value of the version ID to null.
Only Amazon S3 generates version IDs, and they cannot be edited.
Three versioning states of a bucket
The versioning state applies to all (never some) of the objects in that bucket.
After you enable a bucket for versioning, objects added in it are always versioned and given a unique version ID. Objects that are already stored in the bucket are unchanged. The version IDs (null), contents, and permissions remain the same.
When you enable versioning, existing objects in your bucket do not change. What changes is how Amazon S3 handles the objects in future requests.
When you PUT/POST/COPY an object an object with a key that is existing in a versioning-enabled bucket, the noncurrent version is not overwritten. Amazon S3 generates a new version ID for the new object, and store the new object in the bucket.
When you DELETE an object, all versions remain in the bucket and Amazon S3 inserts a delete marker as the current version of the object. Indeed, the delete marker has the same object key as the deleted object.
When you GET an object, Amazon S3 forwards the most recently stored version, or 404 for the delete marker, to you. However, you can GET a noncurrent version of an object by specifying its version ID. .
You can permanently delete an object by specifying the version you want to delete, and Amazon S3 doesn't insert a delete marker in this case. Only the owner of an Amazon S3 bucket can permanently delete a version.
Versioning-suspended
When you suspend versioning, existing objects in your bucket do not change. What changes is how Amazon S3 handles the objects in future requests.
When you PUT/POST/COPY an object,
- If there is not a matched key in the bucket, you create the new object with a null version ID.
- If the object exists in the bucket and its version ID is not null, you put another layer with version ID equals null on its version stack.
- If the object exists in the bucket and its version ID is null, you overwrite its null version.
When you GET an object you get the current version of the object.
When you DELETE an object, S3 removes any object with the requested key and a null version ID. It doesn't remove anything if there isn't a null version of the object in the bucket. Then it inserts a delete marker with the same key and a null version ID as the current version of the object.
Even in a versioning-suspended bucket, the bucket owner can permanently delete a specified version.
Region
You can choose the geographical AWS Region where Amazon S3 will store the buckets that you create. Objects stored in a Region never leave the Region unless you explicitly transfer them to another Region.
Data Consistency Model
Object Consistency Model
Amazon S3 provides strong read-after-write consistency for PUTs and DELETEs of objects in your Amazon S3 bucket in all AWS Regions. If a PUT request is successful, your data is safely stored. Any read (GET or LIST) that is initiated following the receipt of a successful PUT response will return the data written by the PUT.
Amazon S3 does not support object locking for concurrent writers. If the second write begins before the first write has received an acknowledgement, AWS S3 will determine them as simultaneous, and will internally uses last-writer-wins semantics to determine which write takes precedence. The request with the latest timestamp wins.
Updates are key-based. There is no way to make atomic updates across keys.
Updates to a single key are atomic.
Bucket Consistency Model
Bucket configurations have an eventual consistency model. Specifically:
- If you delete a bucket and immediately list all buckets, the deleted bucket might still appear in the list.
- If you enable versioning on a bucket for the first time, it might take a short amount of time for the change to be fully propagated.
Identification Management
Buckets and objects are Amazon S3 resources. By default, only the resource owner can access these resources. The resource owner refers to the AWS account that creates the resource.
The bucket owner does not have permissions on the objects that other accounts own. The bucket owner pays the bills. The bucket owner can deny access to any objects, or delete any objects in the bucket, regardless of who owns them. The bucket owner can archive any objects or restore archived objects regardless of who owns them.
Bucket Policy
Bucket policies provide centralized access control to buckets and objects based on a variety of conditions. Unlike access control lists (described later), which can add (grant) permissions only on individual objects, policies can either add or deny permissions across all (or a subset) of objects within a bucket.
Only the bucket owner is allowed to associate a policy with a bucket.
User Policy
Access Point
Access points are named network endpoints that are attached to buckets that you can use to perform S3 object operations. They simplify managing data access at scale for shared datasets in S3.
Each access point has distinct permissions and network controls that applies for any request that is made through that access point. It also enforces a customized access point policy that works in conjunction with the bucket policy that is attached to the underlying bucket. It can also have customized block public access settings.
You can configure any access point to accept requests only from a virtual private cloud (VPC) .
You can only use access points to perform operations on objects.
Access Control List
Request authorization
Use CORS
Cross-origin resource sharing (CORS) defines a way for client web applications that are loaded in one domain to interact with resources in a different domain. With CORS support, you can build rich client-side web applications with Amazon S3 and selectively allow cross-origin access to your Amazon S3 resources.
You can configure your bucket to explicitly enable cross-origin requests for approaching javascript and fonts stored in the bucket from any location.
Encryption (for protecting data at rest in S3)
Server-side encryption
Request Amazon S3 to encrypt an object before saving it on disks in its data centers and decrypt it when you download the objects.
You can't apply different types of server-side encryption to the same object simultaneously.
Server-side encryption encrypts only the object data, not object metadata.
To encrypt your existing Amazon S3 objects with a single request, you can copy the existing unencrypted objects and write the new encrypted objects to the same bucket by Amazon S3 Batch Operations.
If you need server-side encryption for all of the objects that are stored in a bucket, use a bucket policy to deny all permissions to upload an object unless the request includes the x-amz-server-side-encryption header.
{
"Version": "2012-10-17",
"Id": "PutObjectPolicy",
"Statement": [
{
"Sid": "DenyIncorrectEncryptionHeader",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::awsexamplebucket1/*",
"Condition": {
"StringNotEquals": {
"s3:x-amz-server-side-encryption": "AES256"
}
}
},
{
"Sid": "DenyUnencryptedObjectUploads",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::awsexamplebucket1/*",
"Condition": {
"Null": {
"s3:x-amz-server-side-encryption": "true"
}
}
}
]
}
You can set the default encryption behavior for an S3 bucket so that all new objects are encrypted when they are stored in the bucket. The objects are encrypted using server-side encryption with either:
Amazon S3-managed keys (SSE-S3)
When you use Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3), each object is encrypted with a unique key. As an additional safeguard, it encrypts the key itself with a master key that it regularly rotates.
Only SSE-S3, but not SSE-KMS default encryption is supported for server access log destination buckets.
Customer master keys (CMKs) stored in AWS Key Management Service (AWS KMS) (SSE-KMS)
You can create and manage customer managed CMKs or use AWS managed CMKs.
There are separate permissions for the use of a CMK that provides added protection against unauthorized access of your objects in Amazon S3. SSE-KMS also provides you with an audit trail that shows when your CMK was used and by whom.
Amazon S3 only supports symmetric CMKs and not asymmetric CMKs.
When you use SSE-KMS encryption with an S3 bucket, the AWS KMS CMK must be in the same Region as the bucket.
The data keys used to encrypt your data are also encrypted and stored alongside the data that they protect.
Reducing the cost of SSE-KMS with Amazon S3 Bucket Keys
When you configure your bucket to use default encryption with SSE-KMS, you can also enable S3 Bucket Keys to decrease request traffic from Amazon S3 to AWS Key Management Service (AWS KMS) and reduce the cost of encryption.
When you use SSE-KMS to protect your data without an S3 Bucket Key, Amazon S3 uses an individual AWS KMS data key for every object. It makes a call to AWS KMS every time a request is made against a KMS-encrypted object.
When you configure your bucket to use an S3 Bucket Key for SSE-KMS on new objects, AWS KMS generates a bucket-level key that is used to create unique data keys for objects in the bucket. This bucket key is used for a time-limited period within Amazon S3, further reducing the traffic and cost for Amazon S3 to make requests to AWS KMS.
Amazon S3 will only share an S3 Bucket Key for objects encrypted by the same AWS KMS customer master key (CMK).
Server-Side Encryption with Customer-Provided Keys (SSE-C).
Using server-side encryption with customer-provided encryption keys (SSE-C) allows you to set your own encryption keys. With the encryption key you provide as part of your request, Amazon S3 manages the encryption as it writes to disks and decryption when you access your objects.
When you retrieve an object, you must provide the same encryption key as part of your request. Amazon S3 does not store the encryption key you provide. Instead, it stores a randomly salted HMAC value of the encryption key to validate future requests. The salted HMAC value cannot be used to derive the value of the encryption key or to decrypt the contents of the encrypted object. That means if you lose the encryption key, you lose the object.
Because you manage encryption keys on the client side, you manage any additional safeguards, such as key rotation, on the client side.
You must use HTTPS for requests with SSE-C keys.
If your bucket is versioning-enabled, each object version you upload using this feature can have its own encryption key.
Client-side encryption:
Encrypt data client-side and upload the encrypted data to Amazon S3.
Use a customer master key (CMK) stored in AWS Key Management Service (AWS KMS)
When uploading an object, the client first sends a request to AWS KMS for a new symmetric key that it can use to encrypt the object data. AWS KMS returns two versions of a randomly generated data key:
- A plaintext version of the data key that the client uses to encrypt the object data.
- A cipher blob of the same data key that the client uploads to Amazon S3 as object metadata.
When downloading an object, the client downloads the encrypted object from Amazon S3 along with the cipher blob version of the data key stored as object metadata. The client then sends the cipher blob to AWS KMS to get the plaintext version of the data key so that it can decrypt the object data.
The client can use CMK Id to get an existing CMK for all encryptions instead of generating random new data keys.
Use a master key that you store within your application
When uploading an object, the client calls the Amazon S3 encryption client to generate a one-time-use symmetric data encryption key locally and encrypt the object data by it. Then the client encrypts the data encryption key by the master key provided by your application. The client uploads the object and encrypted data encryption key as part of the object metadata.
When downloading an object, the client downloads the encrypted object from Amazon S3, decrypts the encrypted data encryption key by the master key in application, and then use the data encryption key to decrypt the object.
A separate data key should be generated for each object.
The client-side master key that you provide can be either a symmetric key or a public/private key pair.
Transfer Acceleration
Amazon S3 Transfer Acceleration is a bucket-level feature that enables fast, easy, and secure transfers of files over long distances between your client and an S3 bucket. Transfer Acceleration takes advantage of the globally distributed edge locations in Amazon CloudFront. As the data arrives at an edge location, the data is routed to Amazon S3 over an optimized network path.
Transfer Acceleration is only supported on virtual-hosted style requests.
After you enable Transfer Acceleration on a bucket, it might take up to 20 minutes before the data transfer speed to the bucket increases.
You must be the bucket owner to set the transfer acceleration state.
Event Notification
Logging
Replication
When you enable default encryption for a replication destination bucket,
- If objects in the source bucket are not encrypted, the replica objects in the destination bucket are encrypted using the default encryption settings of the destination bucket. This results in the
ETagof the source object being different from theETagof the replica object. You must update applications that use theETagto accommodate for this difference. - If objects in the source bucket are encrypted using SSE-S3 or SSE-KMS, the replica objects in the destination bucket use the same encryption as the source object encryption. The default encryption settings of the destination bucket are not used.
Cross-Region replication (CRR) is used to copy objects across Amazon S3 buckets in different AWS Regions for disaster recovery. Both source and destination buckets for CRR must have versioning enabled.
BitTorrent
BitTorrent is an open, peer-to-peer protocol for distributing files. You can use the BitTorrent protocol to retrieve any publicly-accessible object in Amazon S3.
The default distribution mechanism for Amazon S3 data is via client/server download. In client/server distribution, the entire object is transferred point-to-point from Amazon S3 to every authorized user who requests that object. Indeed, the costs of client/server distribution increase linearly as the number of users downloading objects increases.
BitTorrent addresses this problem by recruiting the very clients that are downloading the object as distributors themselves: Each client downloads some pieces of the object from Amazon S3 and some from other clients, while simultaneously uploading pieces of the same object to other interested "peers." The benefit for publishers is that for large, popular files the amount of data actually supplied by Amazon S3 can be substantially lower than what it would have been serving the same clients via client/server download. Less data transferred means lower costs for the publisher of the object.
The data transfer savings achieved from use of BitTorrent can vary widely depending on how popular your object is. Less popular objects require heavier use of the "seeder" to serve clients, and thus the difference between BitTorrent distribution costs and client/server distribution costs might be small for such objects. In particular, if only one client is ever downloading a particular object at a time, the cost of BitTorrent delivery will be the same as direct download.
There is no extra charge for use of BitTorrent with Amazon S3. Data transfer via the BitTorrent protocol is metered at the same rate as client/server delivery. To be precise, whenever a downloading BitTorrent client requests a "piece" of an object from the Amazon S3 "seeder," charges accrue just as if an anonymous request for that piece had been made using the REST or SOAP protocol.
Amazon S3 does not support the BitTorrent protocol in AWS Regions launched after May 30, 2016.
Athena
Athena is a serverless, interactive query service which enables you to analyse and query data located in S3 using standard SQL.
Pay per query / per TB scanned.
Used to analyze log data stored in S3.
Macie
Macie is a security service which uses machine learning and NLP to discover, classify, and protect sensitive data, such as personally identifiable information (passport number, driver license, etc.) stored in S3.
Dashboards, reports, and alerting.
Can be used to analyze CloudTrail logs for suspicious API activities, and used in PCI-DSS compliance to prevent ID theft.
