How to parse the AWS S3 Path (s3://<bucket name>/<key>) using the AWSSDK.S3 in C# in order to get the bucket name & key

amazons3uri example
s3 get bucket from path
aws s3 get object url java
how to get full path of s3 bucket
amazons3uri python
aws s3 getobject example
parse s3 url scala
aws s3 read file

I have a s3 path => s3://[bucket name]/[key]

s3://bn-complete-dev-test/1234567890/renders/Irradiance_A.png 

and I need get the bucket_name and the key separately:

var s3PathParsed = parseS3Path("s3://bn-complete-dev-test/1234567890/renders/Irradiance_A.png");

s3PathParsed.BucketName == "bn-complete-dev-test"
s3PathParsed.Key == "1234567890/renders/Irradiance_A.png"

how to I could parse in the correct way using the AWS SDK?

1) I am parsing manually (using a regular expression) and work fine but i am not comfortable:

public class S3Path : IS3Path
{
    private const string _s3PathRegex = @"[s|S]3:\/\/(?<bucket>[^\/]*)\/(?<key>.*)";

    public S3Path(string s3Path)
    {
        Path = s3Path;

        var rx = new Regex(_s3PathRegex).Match(s3Path);

        if (!rx.Success || rx.Groups.Count != 3)
            throw new Exception($"the S3 Path '{s3Path}' is wrong.");

        BucketName = rx.Groups[1].Value;
        Key = rx.Groups[2].Value;
    }

    public string Path { get; }

    public string BucketName { get; }

    public string Key { get; }
}

2) I used the AmazonS3Uri from AWWSDK.S3:

string GetBucketNameFromS3Uri(string s3Uri)
{
    return new AmazonS3Uri(s3Uri).Bucket;            
}

I called the method:

GetBucketNameFromS3Uri("s3://sunsite-complete-dev-test/1234567890/renders/Irradiance_A.png");

and i have the following error:

System.ArgumentException: 'Invalid S3 URI - hostname does not appear to be a valid S3 endpoint'

3) Also I try

string GetBucketNameFromS3Uri(string s3Uri)
{
    return new AmazonS3Uri(new Uri(s3Uri)).Bucket;            
}

with the same error.

I created a new thread in AWS Forum with this issue: https://forums.aws.amazon.com/thread.jspa?threadID=304401

In Java, We can do something like

AmazonS3URI s3URI = new AmazonS3URI("s3://bucket/folder/object.csv");
S3Object s3Object = s3Client.getObject(s3URI.getBucket(), s3URI.getKey());

AmazonS3URI (AWS SDK for Java, AmazonS3URI extends Object. A URI wrapper that can parse out information about an S3 URI. Creates a new AmazonS3URI by parsing the given string. To learn about the AWS CLI commands specific to Amazon S3, you can visit the AWS CLI Command Reference S3 page.. Managing Files in S3. With AWS CLI, typical file management operations can be done like upload files to S3, download files from S3, delete objects in S3, and copy S3 objects to another S3 location.

Here is the scala version and usage of the regex.

val regex = "s3a://([^/]*)/(.*)".r
val regex(bucketName, key) = "s3a://my-bucket-name/myrootpath/mychildpath/file.json"

println(bucketName) // my-bucket-name
println(key)        // myrootpath/mychildpath/file.json

Class Aws\S3\S3UriParser, Extracts a region, bucket, key, and and if a URI is in path-style. Methods Summary. parse ( string|UriInterface $uri ). Parses a URL or S3 StreamWrapper Uri� Buckets created after September 30, 2020, will support only virtual hosted-style requests. Path-style requests will continue to be supported for buckets created on or before this date. For more information, see Amazon S3 Path Deprecation Plan – The Rest of the Story.

I believe that this regex will give you what you want:

s3:\/\/(?<bucket>[^\/]*)\/(?<key>.*)

The bucketname is the first part of the S3 path and the key is everything after the first forward slash.

Performing Operations on Amazon S3 Objects, How to list, upload, download, copy, rename, move or delete objects in an Amazon S3 bucket using the AWS SDK for Java. Create a request param object and pass in AWS S3 Bucket Name and File Location path (key) as shown below. For example if there is a bucket called example-bucket and there is a folder inside it called data then there is a file called data.json then you can construct getParams as following

The AWSSDK.S3 has not a path parser, we need parse manually. You could use the following class that work fine:

public class S3Path 
{
    private const string _s3PathRegex = @"[s|S]3:\/\/(?<bucket>[^\/]+)\/(?<key>.+)";

    public S3Path(string s3Path)
    {
        Path = s3Path;

        var rx = new Regex(_s3PathRegex).Match(s3Path);

        if (!rx.Success)
            throw new Exception($"the S3 Path '{s3Path}' is wrong.");

        BucketName = rx.Groups["bucket"].Value;
        Key = rx.Groups["key"].Value;
    }

    public string Path { get; }

    public string BucketName { get; }

    public string Key { get; }
}

I created a thread in AWS Forum to report the missing functionality.

AmazonS3Uri Class, Constructs a parser for the S3 URI specified as a string. Boolean. True if the URI contains the bucket in the path, false if it contains the bucket in the authority. aws s3 ls Copy from Local to Amazon S3. To copy files from local computer to Amazon S3 you can use the cp command. aws s3 cp filename s3:// bucket_name Copy Multiple Files to Amazon S3 aws s3 cp directory_path s3:// bucket_name--recursive Delete all files inside the bucket. You can use the recursive option to delete all files inside the bucket. aws s3 rm s3:// bucket_name--recursive Include/Exclude

You can use AmazonS3Uri:

var uri = new AmazonS3Uri(urlString);

var bucketName = uri.Bucket;
var key = uri.Key;

Get an object Using the AWS SDK for Java, printStackTrace(); } catch (SdkClientException e) { // Amazon S3 couldn't be contacted for a response, or the client // couldn't parse the response from Amazon � S3 object ARN(@s3_arn_of_file): It is the object ARN path in the S3 bucket RDS file path (@rds_file_path): It is an optional argument. We can specify a subfolder inside the D:\S3 directory. RDS automatically creates this folder for you. If we do not specify any folder, RDS stores files in the ‘D:\S3 folder

I Can't Connect to Amazon S3, To successfully connect to Amazon S3, make sure that you configure authentication and create a valid manifest file inside the bucket you are trying to access. $ aws s3 ls s3://bucket-name PRE path/ 2018-12-04 19:05:48 3 MyFile1.txt. The previous output shows that under the prefix path/ there exists one file named MyFile1.txt.

akofman/parse-aws-s3-url: Parse an S3 URL and returns a , Parse an S3 URL and returns a suitable object for using with AWS SDK methods. parts = parseS3Url('s3://my-bucket/path/to/my/file.png'); //=> parts = {Bucket:� Querying Amazon S3 access logs for requests using Amazon Athena. You can identify Amazon S3 requests with Amazon S3 access logs using Amazon Athena. Amazon S3 stores server access logs as objects in an S3 bucket.

parse-aws-s3-url, const parseS3Url = require('parse-aws-s3-url');. const parts = parseS3Url('s3://my -bucket/path/to/my/file.png');. //=> parts = {Bucket: 'my-bucket',� With Amazon S3 object lock, you can store objects in Amazon S3 using a write-once-read-many (WORM) model. You can use Amazon S3 object lock to prevent an object from being deleted or overwritten

Comments
  • There's obviously an issue with the S3 URI you are using. Can you update your question to include the actual URI? It should be something like https://bucket.s3-aws-region.amazonaws.com/key.
  • I need parse the S3 path (s3://<bucket_name>/<key>), not the URL to the object
  • Didn't know that helper object existed. Great idea.
  • I want to avoid the use of manual solutions like regular expressions because it is a manual method, I want to use the AWS SDK for S3
  • Unfortunately I don't think you're going to file a solution in the Amazon SDK as it seems to expect the http://bucket.s3.aws-region.amazonaws.com/Key format for its URIs
  • Yes, I agree. AWS SDK has not that functionality at the moment.
  • You can't have rx.Success == true and have rx.Groups.Count be anything other than 3. Otherwise it wouldn't match the regex. Also, the whole point of naming the groups was so you could use rx.Groups["Bucket"].Value and rx.Groups["Key"].Value.
  • What about docs.aws.amazon.com/sdkfornet/v3/apidocs/items/S3/TS3Uri.html ?