How to do complete scan of DynamoDb with Python boto3

To perform a complete scan of a DynamoDB table using Boto3, you can use the scan method of the DynamoDB client object. Here’s an example code snippet that demonstrates how to perform a complete scan:

import boto3

# Initialize a Boto3 DynamoDB client
dynamodb = boto3.client('dynamodb')

# Specify the name of the table to scan
table_name = 'my-table'

# Initialize a scan operation with no filter expression or projection expression
scan_args = {
    'TableName': table_name
}

# Perform the scan and keep scanning until all items have been processed
items = []
while True:
    response = dynamodb.scan(**scan_args)
    items += response['Items']
    if 'LastEvaluatedKey' not in response:
        break
    scan_args['ExclusiveStartKey'] = response['LastEvaluatedKey']

# Process the items returned by the scan
for item in items:
    # Do something with each item
    print(item)

In this code snippet, we first initialize a DynamoDB client object using Boto3. We then specify the name of the table to scan and initialize a scan operation with no filter expression or projection expression.

We then perform the scan operation in a loop, keeping track of all the items returned by each scan. If the scan returns a LastEvaluatedKey, we use that key to start the next scan operation. We keep scanning until there are no more items to scan.

Finally, we process the items returned by the scan in the loop and do something with each item (in this case, we just print the item).

Let’s insert some data into a dynamodb table and then scan using the above code and print the data from the table.

Here’s an example Python code snippet that inserts 20 records into a DynamoDB table with two attributes, id and domain. The id attribute is incremented for each record added.

import boto3

# Initialize a Boto3 DynamoDB client
dynamodb = boto3.client('dynamodb')

# Specify the name of the table to insert into
table_name = 'my-table'

# Insert 20 records into the table
for i in range(1, 21):
    # Define the item to insert
    item = {
        'id': {'N': str(i)},
        'domain': {'S': 'example.com'}
    }

    # Insert the item into the table
    response = dynamodb.put_item(
        TableName=table_name,
        Item=item
    )

    # Print the response
    print(f"Inserted item {i}: {response}")

In this code snippet, we first initialize a DynamoDB client object using Boto3. We then specify the name of the table to insert into, which should already exist in your AWS account.

We then use a for loop to insert 20 records into the table. For each record, we define a dictionary item that contains the id attribute as an incrementing integer value and the domain attribute as a string. We then use the put_item method of the DynamoDB client to insert the item into the table.

Finally, we print the response from each insert operation, which should include the status of the operation and any other relevant information.

Here’s an example output of the code that scans the table having 2 attributes id and domain:

{'id': {'N': '1'}, 'domain': {'S': 'example1.com'}}
{'id': {'N': '2'}, 'domain': {'S': 'example2.com'}}
{'id': {'N': '3'}, 'domain': {'S': 'example3.com'}}
{'id': {'N': '4'}, 'domain': {'S': 'example4.com'}}
{'id': {'N': '5'}, 'domain': {'S': 'example5.com'}}
...

In this example output, each item in the table is represented by a dictionary with two keys: id and domain. The value associated with the id key is another dictionary with a single key-value pair, where the key is 'N' to indicate that the value is a number, and the value is the actual integer value of the id attribute for that item. Similarly, the value associated with the domain key is another dictionary with a single key-value pair, where the key is 'S' to indicate that the value is a string, and the value is the actual string value of the domain attribute for that item.

Related posts: