How to Migrate from Azure Blob Storage to Amazon S3 Using Agentless AWS DataSync

Summary

Migrating data from Azure Blob Storage to Amazon Simple Storage Service a common requirement for organizations consolidating on AWS or adopting multi-cloud storage strategies. AWS DataSync makes this agentless migration simple — no VMs to deploy, no agents to manage. Whether you’re consolidating storage on AWS, reducing cross-cloud egress costs, or unlocking AWS-native analytics and ML services, DataSync Enhanced mode handles the transfer directly over HTTPS.

Beyond Azure Blob Storage, DataSync Enhanced mode also supports agentless transfers from Google Cloud Storage, Wasabi, DigitalOcean Spaces, and other cloud providers. This blog focuses on the Azure Blob to S3 pattern as a reference implementation you can adapt for any supported source.This post walks through the end-to-end process: generating and securing SAS tokens, configuring least-privilege IAM roles, setting up source and destination locations, executing the transfer, and verifying the outcome.

Why migrate from Azure Blob Storage to Amazon S3?

Whether you have already decided to consolidate on AWS or are still evaluating the move, here are the key reasons organizations choose to migrate their object data from Azure Blob Storage to S3:

• Reduce cross-cloud data transfer costs — When your compute workloads already run on AWS, pulling data from Azure on every request incurs egress fees and adds latency. Storing data in S3 eliminates these recurring costs.
• Unlock native AWS ecosystem integration — Data in S3 is immediately accessible to 200+ AWS services — query with Athena, train models with SageMaker, trigger workflows with Lambda, or build generative AI with Bedrock — without additional data movement.
• Simplify operations with a single storage platform — Managing credentials, networking, IAM policies, and monitoring across two cloud providers adds operational overhead. Consolidating on S3 gives you one control plane for access, encryption, lifecycle, and compliance.
• Optimize storage costs automatically — S3 offers Intelligent-Tiering (automatic cost optimization based on access patterns), Glacier storage classes (up to 95% savings for archival data), and Lifecycle policies — capabilities that work without manual intervention.
• Improve performance for analytics and ML — Co-locating data with AWS compute eliminates cross-cloud latency. S3 delivers consistent throughput for data-intensive workloads like model training, ETL pipelines, and real-time analytics.
• Strengthen security and compliance posture — Unified IAM policies, CloudTrail audit logging, S3 Object Lock for immutability, and Macie for sensitive data discovery — all managed from a single security framework.

Why AWS DataSync?

DataSync is a fully managed data transfer service purpose-built for moving large datasets quickly and securely. Unlike custom scripts or open-source tools that require you to handle retries, parallelism, integrity checks, and credential management yourself, DataSync automates all of this out of the box.With Enhanced mode, DataSync takes this a step further — it eliminates the need to deploy any agent infrastructure. For cross-cloud transfers to S3, Enhanced mode connects directly to your source storage over HTTPS and processes data in parallel (listing, preparing, transferring, and verifying simultaneously). There is no VM to provision in Azure, no agent to patch or monitor, and no network configuration to maintain in the source environment.

Prerequisites

Azure prerequisites:

• An Azure storage account with one or more Blob containers holding the data to be migrated.
• Appropriate rights on the Azure storage account to generate SAS tokens (account-level or container-level).
• SAS Token with read and list permissions for each Blob container. Make sure that your SAS doesn’t expire before you expect to finish your transfer.
• Objects to be transferred must be in the hot or cool access tier. Objects in the archive tier must be rehydrated before transfer. See Azure access tiers documentation.

AWS prerequisites:

• An AWS account and an IAM principal with permissions to create and configure DataSync, S3, AWS Secrets Manager, IAM, and CloudWatch Logs resources.
• SAS Token stored in Secrets Manager.
• An S3 target bucket in a Region supported by DataSync.
• An IAM role to read the SAS token stored in Secrets Manager and permissions to list and write to the S3 bucket, plus CloudWatch Logs permissions for transfer logging.

Limitations

• DataSync doesn’t support using a SAS token to access only a specific folder in your Azure Blob Storage container.
• You can’t provide DataSync a user delegation SAS token for accessing your blob storage.
• DataSync will not transfer Azure Blob Storage objects larger than 5 TB or objects with metadata larger than 2 KB when transferring to S3. To check for oversized objects or metadata in your source container, you can use Azure Storage Explorer or the Azure CLI (az storage blob list) to audit blob sizes and metadata before starting the transfer. See: Creating a Blob with Azure Storage
• DataSync does not support the Azure Blob cold access tier as a source.

Architecture

Source technology stack

Microsoft Azure Blob Storage serves as the source for this migration. Azure Blob Storage is Microsoft’s object storage solution for the cloud, capable of storing text, binary data, and documents. It also supports Azure Data Lake Storage Gen2 for data analytics workloads.

Target technology stack

S3 serves as the destination for the migrated data. S3 is an object storage service offering industry-leading scalability, data availability, security, and performance. It integrates natively with the broader AWS ecosystem including analytics services like Amazon Athena, machine learning services like Amazon SageMaker, and compute services like Lambda. The migration leverages DataSync (Enhanced mode), Secrets Manager for credential storage, AWS Identity and Access Management (IAM) for access control, and CloudWatch Logs for transfer monitoring and auditing.

Target architecture

Azure Infrastructure:

• Blob Containers — One or more Azure Blob Storage containers holding the data to be migrated.
• SAS Token — Shared access signature with read and list permissions for each Blob container.

AWS Cloud Infrastructure:

• SAS Token stored in Secrets Manager — The SAS token is securely stored so it is never exposed in plain text.
• S3 target bucket — The destination bucket where migrated data will land.
• IAM Role — Allows DataSync to read the SAS token from Secrets Manager and write to S3. Includes CloudWatch Logs permissions.
• DataSync (Enhanced mode) — Orchestrates the agentless migration directly from Azure Blob to S3.
• CloudWatch Logs — Captures DataSync transfer logs for monitoring and auditing.

Data flow

1. Generate SAS Token (Azure Portal)
↓
2. Store SAS Token in Secrets Manager
↓
3. DataSync assumes IAM Role & retrieves SAS Token
↓
4. DataSync connects to Azure Blob over HTTPS (Agentless)
↓
5. DataSync transfers objects to S3
↓
6. Verify integrity & log to CloudWatch

Tools

Code repository

Below is the single IAM policy to attach to the IAM Role that will be created by the user for the DataSync service. It combines the trust policy and permissions into one document.

Note: Replace <REGION>, <ACCOUNT_ID>, <SECRET_NAME>, and <TARGET_BUCKET_NAME_*> with your actual values.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowDataSyncAssumeRole",
      "Effect": "Allow",
      "Principal": {
        "Service": "datasync.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    },
    {
      "Sid": "AllowDataSyncSecretsManagerAccess",
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue",
        "secretsmanager:DescribeSecret"
      ],
      "Resource": "arn:aws:secretsmanager:<REGION>:<ACCOUNT_ID>:secret:<SECRET_NAME>*"
    },
    {
      "Sid": "AllowS3BucketAccess",
      "Effect": "Allow",
      "Action": [
        "s3:GetBucketLocation",
        "s3:ListBucket",
        "s3:ListBucketMultipartUploads"
      ],
      "Resource": [
        "arn:aws:s3:::<TARGET_BUCKET_NAME_1>",
        "arn:aws:s3:::<TARGET_BUCKET_NAME_2>"
      ]
    },
    {
      "Sid": "AllowS3ObjectAccess",
      "Effect": "Allow",
      "Action": [
        "s3:AbortMultipartUpload",
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:GetObjectTagging",
        "s3:GetObjectVersion",
        "s3:GetObjectVersionTagging",
        "s3:ListMultipartUploadParts",
        "s3:PutObject",
        "s3:PutObjectTagging"
      ],
      "Resource": [
        "arn:aws:s3:::<TARGET_BUCKET_NAME_1>/*",
        "arn:aws:s3:::<TARGET_BUCKET_NAME_2>/*"
      ]
    },
    {
      "Sid": "AllowCloudWatchLogsAccess",
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DescribeLogGroups",
        "logs:DescribeLogStreams"
      ],
      "Resource": "arn:aws:logs:<REGION>:<ACCOUNT_ID>:log-group:/aws/datasync:*"
    }
  ]
}

Best practices

• Least privilege IAM role access: The IAM role used by DataSync should have only the minimum permissions required — secretsmanager:GetSecretValue and secretsmanager:DescribeSecret scoped to the specific secret ARN, S3 actions scoped to the specific target buckets, and CloudWatch Logs scoped to the DataSync log group. Avoid using wildcards (*) for resource ARNs wherever possible.
• Generate the SAS token for a particular container: Do not generate a broad account-level SAS token when a container-level token is sufficient. Scope the token to the specific container being migrated to limit the blast radius if the token is compromised.
• Store the SAS token in Secrets Manager (not exposed in plain text): Never embed the SAS token directly in configuration files, environment variables, or source code. Store it in Secrets Manager and reference it from DataSync. This ensures the token is encrypted at rest, access is auditable via CloudTrail, and rotation can be managed centrally.
• Set appropriate SAS token expiry: Configure the SAS token expiration to cover the expected duration of the transfer with a reasonable buffer. Overly long-lived tokens increase risk; overly short tokens cause transfer failures.
• Enable CloudTrail logging: Ensure CloudTrail is enabled to audit all DataSync API calls, Secrets Manager access, and S3 operations for compliance and incident response.

Step-by-step implementation

Step 1 — Generate a SAS Token in Azure

Begin by identifying the specific Azure Blob container you wish to migrate. Navigate to the Azure Portal and open the storage account that contains your target container. From the container’s settings, you will generate a Shared Access Signature (SAS) token that grants DataSync the permissions it needs to read your data.You can generate the SAS token in two ways:

• Create SAS Token via Azure Portal — Navigate to the container in the Azure Portal, go to the “Shared access tokens” blade, select the required permissions (Read and List at minimum for a source transfer), set an appropriate start and expiry date/time, and click Generate. The portal will produce a SAS token string and a full SAS URL.
• (OR)
• Create SAS Token via Azure Storage Explorer — Right-click the container in Azure Storage Explorer, select “Get Shared Access Signature”, configure the permissions and expiry, and generate the token.

When configuring permissions, ensure you select at minimum Read and List for a source transfer.

Set the expiry time carefully — the SAS token must remain valid for the entire duration of your data transfer. To estimate transfer duration, use the formula from the DataSync Documentation: (DATA_SIZE × 8) / (CIRCUIT × NETWORK_UTILIZATION × 3600 × AVAILABLE_HOURS) = Number of days. For example, transferring 10 TB over a 1 Gbps connection at 80% utilization running 24 hours/day takes approximately 1.2 days, while 20 TB under the same conditions takes approximately 2.3 days. As a recommendation, add at least 30–50% buffer to your estimated transfer time to account for real-world conditions such as network fluctuations, storage performance variations, and potential retries. For a 10–20 TB migration over 1 Gbps, this means setting the SAS token expiry to approximately 2–4 days. For guidance on configuring a SAS expiration policy, see the Azure Blob Storage Documentation. If the token expires mid-transfer, DataSync will lose access to the container and you will encounter a “Failed to open directory” error, requiring you to generate a new token, update the location, and restart the task.

Copy only the SAS token portion (the string after the ? in the SAS URI). Do not copy the full URL — DataSync requires the token only, not the URL+Token combination.

Note: Generate the SAS token scoped to the specific container you intend to migrate. Do not reuse a token generated for one container across other containers and do not share token.

Step 2 — Store the SAS Token in Secrets Manager

Once you have the SAS token, it must be stored securely in Secrets Manager so that DataSync can retrieve it without the token ever being exposed in plain text. Navigate to the AWS Management Console and open the Secrets Manager service. Click “Store a new secret” and select “Other type of secret” as the secret type. Choose “Plaintext” and paste the SAS token value directly into the text field. Do not include the container URL — only the token string itself.Give the secret a descriptive name (for example, AzureBlobSAStoken) and complete the creation wizard. The following images show the secret configuration steps. Once the secret is created, note its ARN — you will need this when configuring the IAM role policy and when creating the DataSync source location.

Step 3 — Configure an IAM Role with the Required Permissions

DataSync requires an IAM role that it can assume to perform the migration. This role must grant DataSync three categories of access: the ability to retrieve the SAS token from Secrets Manager, the ability to read from and write to the target S3 bucket(s), and the ability to write transfer logs to CloudWatch Logs.To create this role, navigate to the IAM console in AWS. Create a new role and select “AWS Service” as the trusted entity type, then choose “DataSync” as the service that will use this role. This establishes the trust relationship that allows the DataSync service to assume the role. Next, attach a custom inline permissions policy to the role. The following images attached below show the role creation and custom inline policy attachment steps. The custom inline policy should follow the principle of least privilege and must be written in the policy editor section, as shown in the images below. The policy must include:

• Secrets Manager access — secretsmanager:GetSecretValue and secretsmanager:DescribeSecret scoped to the specific secret ARN where your SAS token is stored.
• S3 bucket-level access — s3:GetBucketLocation, s3:ListBucket, and s3:ListBucketMultipartUploads scoped to the specific target bucket ARN(s). If you are migrating to multiple buckets, list each bucket ARN individually rather than using wildcards.
• S3 object-level access — s3:AbortMultipartUpload, s3:DeleteObject, s3:GetObject, s3:GetObjectTagging, s3:GetObjectVersion, s3:GetObjectVersionTagging, s3:ListMultipartUploadParts, s3:PutObject, and s3:PutObjectTagging scoped to the target bucket ARN(s) with the /* suffix.
• CloudWatch Logs access — logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents, logs:DescribeLogGroups, and logs:DescribeLogStreams scoped to the DataSync log group ARN (/aws/datasync).

The complete IAM policy is provided in the Tools > Code repository section. Replace the placeholder values for Account ID, Region, and ARNs with your own.

Step 4 — Configure Azure Blob Storage as a DataSync Source Location

With the SAS token stored in Secrets Manager and the IAM role created, you can now configure the Azure Blob container as a source location in DataSync. Navigate to the DataSync console and go to Data transfer > Locations > Create location.Select Microsoft Azure Blob Storage as the location type. In the “Container URL” field, enter the full URL of your Azure Blob container in the format https://<storage-account>.blob.core.windows.net/<container-name>. If you only want to transfer data from a specific virtual directory within the container, specify the folder path in the “Folder” field (for example, /my/data/). Note that DataSync does not support using a SAS token to access only a specific folder — the folder path acts as a prefix filter, but the token must have access to the entire container.For the SAS token configuration, select “Use a Secrets Manager secret” rather than entering the token directly. This is the recommended approach as it keeps the token encrypted and auditable. Select the IAM role you created in Step 2 and provide the ARN of the Secrets Manager secret where the SAS token is stored.Since this is an agentless transfer to S3 using Enhanced mode, you do not need to select or configure any DataSync agents. Leave the agent configuration at its default (no agents).Review the configuration and create the location. The following image shows the completed source location configuration. Note the Location ARN that is returned.

Step 5 — Configure S3 as a DataSync Destination Location

Next, create the target location where DataSync will write the migrated data. Navigate to the DataSync console and go to Data transfer > Locations > Create location.Select S3 as the location type. Choose the target S3 bucket from the dropdown or enter its ARN. If you want the migrated data to land under a specific prefix within the bucket, specify it in the “Folder” field (for example, /migrated-data/). This is useful for organizing migrated data separately from existing objects in the bucket.For the IAM role, select the same role you created in Step 2. This role provides DataSync with the permissions to write objects, manage multipart uploads, and set object tags in the target bucket.Select the appropriate S3 storage class for the destination objects. The default is S3 Standard, but you can choose S3 Standard-IA, S3 One Zone-IA, S3 Glacier Instant Retrieval, or other classes depending on your access patterns and cost requirements.Review the configuration and create the location. The following image shows the completed destination location configuration. Note the Location ARN.

Step 6 — Create, Configure, and Run the Migration Task

With both source and destination locations configured, you can now create the DataSync task that ties them together and defines how the transfer should behave. Navigate to the DataSync console and go to Data transfer > Tasks > Create task.Select the Azure Blob source location and the S3 destination location you created in the previous epics. On the configuration page, set the following recommended options:

• Task mode — Select Enhanced mode. This is required for agentless transfers between Azure Blob and S3.
• Verify mode — Select “Only files transferred” to verify the integrity of objects moved in this execution. This is the recommended mode for most migrations as it balances thoroughness with performance.
• Overwrite mode — Select “Always” to ensure the destination always reflects the latest source data.
• Transfer mode — Select “All” for a full initial migration. Use “Changed only” for subsequent incremental syncs.
• Logging — Select “Transfer” log level to capture detailed per-object transfer logs in CloudWatch Logs. This is recommended for auditing and troubleshooting.

Optionally, configure bandwidth throttling if you need to limit the network impact of the transfer, and set up scheduling if you want the task to run on a recurring basis.Review all settings and create the task. The following image shows the task configuration with the recommended options. Once created, select the task and click “Start” to begin execution. Monitor the transfer progress by navigating to the task’s “History” tab, which shows the status of each execution, including bytes transferred, files transferred, duration, and any errors encountered.

Outcome

Once the DataSync task completes, you can confirm the migration status by checking the task execution history in the DataSync console. The “History” tab shows total bytes transferred, number of objects migrated, verification status, and any errors encountered. A successful execution with zero errors and “Verified” status confirms your data is intact in S3.

What does moving this data to S3 enable?

With your data now in S3, you unlock capabilities that aren’t natively available with Azure Blob Storage in the AWS ecosystem:

• Analytics at scale — Query your migrated data directly with Athena using standard SQL, or build data lakes with AWS Glue and AWS Lake Formation — no data movement required.
• Machine learning — Feed data directly into SageMaker for model training, or use Amazon Bedrock for generative AI applications with your proprietary data.
• Event-driven processing — Trigger AWS Lambda functions automatically when new objects land in S3, enabling real-time data pipelines without managing servers.
• Cost-optimized archival — Use S3 Lifecycle policies to automatically transition infrequently accessed data to S3 Glacier storage classes, reducing long-term storage costs by up to 95% compared to hot-tier pricing.
• Native integration — Access your data from any AWS compute service (Amazon EC2, Amazon EKS, Lambda) without cross-cloud latency or egress fees.
• Ongoing synchronization — The agentless DataSync pattern supports recurring scheduled transfers, allowing you to keep S3 continuously synchronized with Azure Blob Storage during a phased migration or for ongoing multi-cloud operations.

Clean Up

To avoid ongoing charges after completing the migration, remove the resources you created during this walkthrough:

1. Delete the DataSync task — In the DataSync console, select the task and choose Delete. This does not affect your transferred data.
2. Delete the source and destination locations — In the DataSync console, go to Locations and delete both the Azure Blob source location and the S3 destination location.
3. Delete the Secrets Manager secret — In the Secrets Manager console, select the secret containing the SAS token and choose Delete secret. By default, Secrets Manager schedules deletion after a 7-day recovery window.
4. Delete the IAM role — In the IAM console, delete the DataSync role you created if it is no longer needed for other tasks.
5. Revoke the SAS token in Azure — If the SAS token has not yet expired, regenerate the storage account keys in the Azure Portal to invalidate all account-level SAS tokens, or simply let container-level tokens expire naturally.
6. Delete the CloudWatch log group (optional) — In the CloudWatch console, delete the /aws/datasync log group if you no longer need the transfer logs.

Note: Deleting these resources does not affect the data already transferred to your S3 bucket.

Troubleshooting

Related resources

1. AWS DataSync
2. AWS DataSync Agentless
3. AWS DataSync Agent based migration
4. Azure Blob Storage SAS token
5. AWS Secrets Manager – Creating a secret in AWS
6. Amazon Simple Storage Service (S3)