Logo
Home Study Material Contact Search
AI Chat FREE
  1. mathematics
  2. basic math
  3. using python boto3 check if a folder is in an s3 b

Using python boto3 Check if a folder is in an S3 bucket and if not in...

Free

60.1K

Verified Solution

Question

Basic Math

Using python boto3

Check if a folder is in an S3 bucket and if not in the bucketthen it creates a folder in the right directory

Answer & Explanation Solved by verified expert
4.1 Ratings (597 Votes)

import boto3
import os

s3_client = boto3.client('s3')

def download_dir(prefix, local, bucket, client=s3_client):
\"\"\"
params:
- prefix: pattern to match in s3
- local: local path to folder in which to place files
- bucket: s3 bucket with target contents
- client: initialized s3 client object
\"\"\"
keys = []
dirs = []
next_token = ''
base_kwargs = {
'Bucket':bucket,
'Prefix':prefix,
}
while next_token is not None:
kwargs = base_kwargs.copy()
if next_token != '':
kwargs.update({'ContinuationToken': next_token})
results = client.list_objects_v2(**kwargs)
contents = results.get('Contents')
for i in contents:
k = i.get('Key')
if k[-1] != '/':
keys.append(k)
else:
dirs.append(k)
next_token = results.get('NextContinuationToken')
for d in dirs:
dest_pathname = os.path.join(local, d)
if not os.path.exists(os.path.dirname(dest_pathname)):
os.makedirs(os.path.dirname(dest_pathname))
for k in keys:
dest_pathname = os.path.join(local, k)
if not os.path.exists(os.path.dirname(dest_pathname)):
os.makedirs(os.path.dirname(dest_pathname))
client.download_file(bucket, k, dest_pathname)
Get Answers to Unlimited Questions

Join us to gain access to millions of questions and expert answers. Enjoy exclusive benefits tailored just for you!

Membership Benefits:
  • Unlimited Question Access with detailed Answers
  • Zin AI - 3 Million Words
  • 10 Dall-E 3 Images
  • 20 Plot Generations
  • Conversation with Dialogue Memory
  • No Ads, Ever!
  • Access to Our Best AI Platform: Zin AI - Your personal assistant for all your inquiries!
Become a Member

Other questions asked by students

Q

Identify a major problem facing the world today and discuss how a team of cross-disciplinary specialists...

Medical Sciences
Q

Moana is a sailing from her home to a nearby island which is 215 miles north...

Physics
Q

1. Go to the Microsoft Corporate website (Official Home Page) and click on “Investors” at the...

Accounting
Q

A box of negligible mass containing n moles of an ideal gas of molar mass...

Physics
Q

5 12 23 7 14 AM Question 15 If QRST is a kite find QRS...

Geometry
Q

2 1 An important quality characteristic of water is the concentration of suspended solid material...

Basic Math
Q

HW4 1 Provide a detailed explanation step by step proof or solution to the given...

Calculus
Q

Expand log 642 completely Keep variables in the order that given in the original term...

Algebra
Q

Let the Universal Set S have 154 elements A and B are subsets of S...

Algebra
Q

The assignment is to create two amortization tables for notes receivable. You are...

Accounting
Q

The thickness of composite panels is an important quality characteristic. The specificities for the thickness...

Accounting
Q

Shanice bought the car for $28,430. The value of the car is predicted to depreciate...

Finance