mesos-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From haosdent <haosd...@gmail.com>
Subject Re: Downloading s3 uris
Date Fri, 26 Feb 2016 16:50:04 GMT
So far have to install a hdfs client if you want to use "s3n://xxx". :-(

On Sat, Feb 27, 2016 at 12:39 AM, Abhishek Amralkar <
Abhishek.Amralkar@talentica.com> wrote:

> Agreed with @aaron, it will be too much manual work to generate S3 url
> everytime.
>
> Thanks
> Abhishek
>
> From: Aaron Carey <acarey@ilm.com>
> Reply-To: "user@mesos.apache.org" <user@mesos.apache.org>
> Date: Friday, 26 February 2016 at 8:31 AM
> To: "user@mesos.apache.org" <user@mesos.apache.org>
> Subject: RE: Downloading s3 uris
>
> I was trying to avoid generating urls for everything as this will
> complicate things a lot.
>
> Is there a straight forward way to get the fetcher to do it directly?
>
> ------------------------------
> *From:* haosdent [haosdent@gmail.com]
> *Sent:* 26 February 2016 16:27
> *To:* user
> *Subject:* Re: Downloading s3 uris
>
> I think still could pass AWSAccessKeyId if it is private?
> http://www.bucketexplorer.com/documentation/amazon-s3--how-to-generate-url-for-amazon-s3-files.html
>
> On Sat, Feb 27, 2016 at 12:25 AM, Abhishek Amralkar <
> Abhishek.Amralkar@talentica.com> wrote:
>
>> In that case do we need to keep bucket/files public?
>>
>> -Abhishek
>>
>> From: Zhitao Li <zhitao@uber.com>
>> Reply-To: "user@mesos.apache.org" <user@mesos.apache.org>
>> Date: Friday, 26 February 2016 at 8:23 AM
>> To: "user@mesos.apache.org" <user@mesos.apache.org>
>> Subject: Re: Downloading s3 uris
>>
>> Haven't directly used s3 download, but I think a workaround (if you don't
>> care ACL about the files) is to use http
>> <http://stackoverflow.com/questions/18239567/how-can-i-download-a-file-from-an-s3-bucket-with-wget>
url
>> instead.
>>
>> On Feb 26, 2016, at 8:17 AM, Aaron Carey <acarey@ilm.com> wrote:
>>
>> I'm attempting to fetch files from s3 uris in mesos, but we're not using
>> hdfs in our cluster... however I believe I need the client installed.
>>
>> Is it possible to just have the client running without a full hdfs setup?
>>
>> I haven't been able to find much information in the docs, could someone
>> point me in the right direction?
>>
>> Thanks!
>>
>> Aaron
>>
>>
>>
>
>
> --
> Best Regards,
> Haosdent Huang
>



-- 
Best Regards,
Haosdent Huang

Mime
View raw message