[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: GNU Parallel on Amazon Linux
From: |
Osborne, Darryl |
Subject: |
Re: GNU Parallel on Amazon Linux |
Date: |
Sat, 16 Dec 2017 00:16:33 +0000 |
User-agent: |
Microsoft-MacOutlook/f.27.0.171010 |
Scratch my comment about the issue of Parallel Digest. I just read the latest
issue from this morning. Thanks for including it.
Darryl S. Osborne
Storage Specialist | AWS Solutions Architecture
o: 469.283.5878
c: 206.817.2906
On 12/15/17, 5:50 PM, "Osborne, Darryl" <darrylo@amazon.com> wrote:
Darryl S. Osborne
Storage Specialist | AWS Solutions Architecture
o: 469.283.5878
c: 206.817.2906
On 12/15/17, 5:00 PM, "ole.tange@gmail.com on behalf of Ole Tange"
<ole.tange@gmail.com on behalf of ole@tange.dk> wrote:
On Fri, Dec 15, 2017 at 3:10 PM, Osborne, Darryl <darrylo@amazon.com>
wrote:
> First, thanks for developing and maintaining GNU Parallel. It’s an
essential
> tool and I recommend it in just about every customer call, talk, or
> presentation I give.
Happy to hear that.
I believe I linked to one of your presentations in the release notes
for 20170122:
* AWS re:Invent 2016: Deep Dive on Amazon Elastic File System (STG202)
slide 45
http://www.slideshare.net/AmazonWebServices/aws-reinvent-2016-deep-dive-on-amazon-elastic-file-system-stg202
and https://youtu.be/PlTuJx4VnGw?t=30m16s
>> Great! Thanks for including that.
> Second, I don’t know if you’re aware but parallel is now included in
the
> Amazon repository so AWS customers running Amazon Linux can easily
install
> it with a simple “sudo yum install parallel –y” command.
I was not aware of that.
>> This may be a good to mention during the next issues of Parallel
Digest
> One of my sample tutorial results sums it up well. In this example
> (attached) when I combined GNU parallel and standard Linux copy
commands
> like cp and cpio, I achieved a throughput increase of 690% and
transferred
> files 7.9 times faster when compared to using the copy commands alone.
Good to see. Do you know what the theoretical max is?
>>The maximum throughput you can drive an EFS file system per EC2
instance is 250 MB/s. 3 GB/s+ throughput is achieved scaling across multiple
instances (like a data center).
> Have you had a chance to play around with Amazon EFS? I’d be curious
to hear
> your feedback.
I have yet to have a job where I needed EFS.
/Ole