11 Feb
2015
11 Feb
'15
4 p.m.
Hi All,
There exist a threaded and (even MPI-parallel) 'cp' for large/parallel filesystems.
http://people.nas.nasa.gov/~kolano/projects/mutil.html
It is a bit of pain to build as it patches a particular version of GNU coreutils.
--
Grigory Shamov
Westgrid/ComputeCanada Site Lead
University of Manitoba
E2-588 EITC Building,
(204) 474-9625
21 Feb
21 Feb
6:47 a.m.
On 2015-02-11 Grigory Shamov wrote:
Hi All,
There exist a threaded and (even MPI-parallel) 'cp' for large/parallel filesystems.
What did it get your cp time down to from 90 days?
23 Feb
23 Feb
3:28 p.m.
Hi,
About 6 days it takes, using 2 nodes with 6 cp processes on each of them, using MPI-parallel version of cp and copying from one shared FS (Lustre) to another (NFS). Threaded version of cp (i.e. on one node, several threads) was also tried but the speedup was less than when copying form the two clients.
--
Grigory Shamov
Westgrid/ComputeCanada Site Lead
University of Manitoba
E2-588 EITC Building,
(204) 474-9625
On 15-02-21 12:47 AM, "Trevor Cordes" trevor@tecnopolis.ca wrote:
>On 2015-02-11 Grigory Shamov wrote:
>> Hi All,
>>
>> There exist a threaded and (even MPI-parallel) 'cp' for large/parallel
>> filesystems.
>>
>> http://people.nas.nasa.gov/~kolano/projects/mutil.html
>
>What did it get your cp time down to from 90 days?
>_______________________________________________
>Roundtable mailing list
>Roundtable@muug.mb.ca
>http://www.muug.mb.ca/mailman/listinfo/roundtable
3632
Age (days ago)
3644
Last active (days ago)
2 comments
2 participants
participants (2)
-
Grigory Shamov -
Trevor Cordes