aws-sdk for Ruby access folder within bucket

aws-sdk ruby s3 example
aws-sdk-s3 ruby
ruby s3 get object
ruby upload file to s3
aws s3
ruby aws-sdk credentials
aws ruby sdk pagination
ruby require aws sdk s3

I have a bucket on the Amazon S3 with the folder in it. I'm trying to access it the following way via aws-sdk gem:

s3 = AWS::S3.new(
    :access_key_id => "KEY",
    :secret_access_key => "SECRET"
)

bucket = s3.buckets["my_bucket/my_folder"]
bucket.do_stuff....

I am getting the following error in return:

The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.

Any ideas what I may be doing wrong?

probably the S3 bucket are trying to use is located outside the US-EAST (default location), so this should help you:

s3 = AWS::S3.new(
    :access_key_id => "KEY",
    :secret_access_key => "SECRET"
    :s3_endpoint => 's3-eu-west-1.amazonaws.com'
)

Choose your S3 enpdpoint from the following list:

US Standard *                           s3.amazonaws.com(default)
US West (Oregon) Region                 s3-us-west-2.amazonaws.com
US West (Northern California) Region    s3-us-west-1.amazonaws.com
EU (Ireland) Region                     s3-eu-west-1.amazonaws.com
Asia Pacific (Singapore) Region         s3-ap-southeast-1.amazonaws.com
Asia Pacific (Tokyo) Region             s3-ap-northeast-1.amazonaws.com
South America (Sao Paulo) Region        s3-sa-east-1.amazonaws.com

In terms of object access, the bucket name is my_bucket, but my_folder should be a part of object.

Creating and Using an Amazon S3 Bucket, Be sure the AWS credentials map to an AWS Identity and Access Management ( IAM) entity with access to the AWS actions and resources described in this� Install the AWS SDK for Ruby. For more information, see Installing the AWS SDK for Ruby. Set the AWS access credentials that the AWS SDK for Ruby will use to verify your access to AWS services and resources. For more information, see Configuring the AWS SDK for Ruby.

You need to configure your region specific endpoint for the bucket (where it was created). You can do this with:

AWS.config(:s3_endpoint => '...')
s3 = AWS::S3.new

or

s3 = AWS::S3.new(:s3_endpoint => '...')

You can avoid this in the future by using DNS comptible bucket names (also avoid dots in bucket names). If a bucket name is a valid subdomain, then you can address your bucket without configuring the region specific endpoint. Consider the following:

http:://bucket-name.s3.amazonaws.com/path/to/object.txt

Where the bucket is named "bucket-name" and the object key is "path/to/object.txt". This bucket could exist in any region, and yet you can access it using the "default" region. When the bucket name is not dns-compatible, then the url looks like:

http://s3.amazon.com/bucket/name/path/to/object.txt

In the example above, the bucket is "bucket/name", which is not dns compatible. It becomes part of the path, and now s3.amazon.com must be replaced with the region specific endpoint (if the bucket was not created in the classic region).

As someone else mentioned, paths should be part of the object key, not bucket name. This allows you to group objects by a common prefix. The '/' is used as a virtual folder (by convention only).

# print the key of every object with the given prefix
s3.buckets['bucket-name'].objects.with_prefix('path/to/').each do |object|
  puts object.key
end

Class: AWS::S3::Bucket — AWS SDK for Ruby, You are viewing documentation for version 1 of the AWS SDK for Ruby. Version 2 deletes all object versions in batches bucket.delete. You can Given a bucket you can access its objects, either by key or by enumeration. Returns a tree that allows you to expose the bucket contents like a directory structure. #clear! ⇒ nil. This example demonstrates how to use the AWS SDK for Ruby to: Set a predefined grant (also known as a canned ACL) for a bucket in Amazon S3. Add an object to the bucket. Set a canned ACL for an object in the bucket. Get the bucket’s current ACL. For the complete code for this example, see Complete Example .

has_attached_file :photo,
  storage: :s3,
  styles: { medium: "300x300>", thumb: "100x100>" },
  s3_credentials: "#{Rails.root}/config/aws.yml",
  bucket: "bucket-name",
  s3_host_name: "s3-ap-southeast-1.amazonaws.com",
  url: ":s3_domain_url",
  path: 'books/:id/photo/:style_:basename.:extension'

worked for me :)

Managing Amazon S3 Bucket and Object Access Permissions , methods provided by the AWS SDK for Ruby for Amazon S3. Then create an Aws::S3::Client object in the AWS Region where you want to create the bucket. The bucket name to which the upload was taking place. When using this API with an access point, you must direct requests to the access point hostname. The access point hostname takes the form AccessPointName-AccountId.s3-accesspoint.Region.amazonaws.com. When using this operation using an access point through the AWS SDKs, you provide the

The answer by Godsaur is essentially correct. However, it appears to be outdated, perhaps for SDK version 1?

This worked for me for version 2:

s3 = Aws::S3::Client.new(endpoint:'https://s3-ap-southeast-1.amazonaws.com')

See docs.

Class: AWS::S3::S3Object — AWS SDK for Ruby, s3 = AWS::S3.new obj = s3.buckets['my-bucket'].objects['key'] # no request made. You can also get objects by enumerating a objects in a bucket. Folder within an Amazon S3 bucket where you want AWS DMS to store the results of this assessment run. # service_access_role_arn ⇒ String ARN of a service role needed to start the assessment run.

In case anyone is looking for this, here is how it should work according to https://github.com/aws/aws-sdk-ruby

creds = JSON.load(File.read('secrets.json'))
Aws.config[:credentials] = Aws::Credentials.new(creds['AccessKeyId'], creds['SecretAccessKey'])
Aws.config[:region] = 'us-east-1'

client = Aws::S3::Client.new(
    region: Aws.config[:region],
    credentials: Aws.config[:credentials]
)

File.open('/local_directory/picture.jpg', 'rb') do |file|
  client.put_object(bucket: 'bucket', key: 'folder_inside_bucket/picture.jpg', body: file)
end

Class: Aws::S3::Client — AWS SDK for Ruby V2, Sets the permissions on an existing bucket using access control lists (ACL). An Amazon S3 bucket has no directory hierarchy such as you would find in a� You can have folders within folders, but not buckets within buckets. You can upload and copy objects directly into a folder. Folders can be created, deleted, and made public, but they cannot be renamed.

Class: Aws::S3::Bucket — AWS SDK for Ruby V3, Defined in: aws-sdk-resources/lib/aws-sdk-resources/services/s3/bucket.rb create a bucket, you must register with Amazon S3 and have a valid AWS Access Key ID prefix to make groups in the same way you\'d use a folder in a file system. Use an IAM policy to grant the user access to the folder and to specify which Amazon S3 actions the user can perform on the folder. For example, the following IAM policy allows a user to download objects from the folder awsexamplebucket/media using the Amazon S3 console.

Class: Aws::S3::Bucket — AWS SDK for Ruby V2, S3 Bucket and Object Access Permissions � Using a Amazon S3 Bucket to Host a Website. Warning Javascript is disabled or is unavailable in your browser. AWS SDK When you use the AWS SDKs to create a bucket, you first create a client and then use the client to send a request to create a bucket. If you don't specify a Region when you create a client or a bucket, Amazon S3 uses US East (N. Virginia), the

Amazon S3 Examples Using the AWS SDK for Ruby, If you need to copy to a bucket in a different region, use #copy_from. Copies this object to� This tutorial explains some basic file/folder operations in an AWS S3 bucket using AWS SDK for .NET (C#). First, we create a directory in S3, then upload a file to it, then we will list the content of the directory and finally delete the file and folder. We show these operations in both low-level and high-level APIs.

Comments
  • Possible repost stackoverflow.com/questions/6581502/…
  • @sgud it's not right, the link you provided is about AWS-S3 gem which has a different syntax - replace a class's constant
  • "my_folder" is a part of Object's name, not Bucket
  • @mikhailov this is indeed the case, it is part of objects name. Can you put it as an answer? cheers!
  • @Godsaur done, please check it out below
  • This worked for me: s3 = Aws::S3::Client.new(endpoint:'https://s3-ap-southeast-1.amazonaws.com')
  • s3 = Aws::S3::Client.new(region: 'eu-west-1') seems to be the latest version