Let me repeat: If you are not comfortable with Linux RETURN THE DEVICE AND GET YOUR MONEY BACK. Seriously,. If you don’t know what you are doing you can brick your device.
Having said that, the reason you are seeing this error is because the files have DOS line endings (CRLF) and should have Unix line endings instead.
I agree with Tony’Matthey’s exhortation – don’t screw around in the system’s internals unless you know what you’re doing.
But in this case, the fix is simple – convert the files to Unix text…
dos2unix dos.sh
Much better now, thanks!
(Forgot to do the simple EOL conversion to Unix in the free Notepad++ before saving).
Tony,
Thank you very much with the solution. It works for me. Now, MyBook Duo is way much faster
Hi Tony,
Thanks for all your help here.
I’m wondering how this problem improves or not with the new firmware? My observations are that the swap usage is higher again.
In particular I see when sorting the ‘top’ listing by virtual memory use, there are now at least 4 if not 8 apache processes running.
I’m continuing to use the mbld as a server for my iPhoto library, with mixed success before and after the update. I would like to improve the performance so that I can move my iTunes library there also, but that’s unthinkable right now with performance being as it is.
Did this thread have any bearing do you think on the development of the new firmware?
Thanks again,
-dene
Dene09 wrote:
In particular I see when sorting the ‘top’ listing by virtual memory use, there are now at least 4 if not 8 apache processes running.
That doesn’t matter that much. Swap USAGE isn’t the overall issue. It’s “Thrashing” that’s the issue. Thrashing is the colloquial term describing when a process is constantly moved from RAM (real memory) to SWAP (virtual memory) frequently.
An idle process sitting in SWAP space isn’t a problem at all – because it usually stays there and is not impacting performance.
I have recently purchased a 6 Tb Mybook Duo running Raid1 format. It takes an unreasonable time to transfer files. (I am running OS 10.8.2 with a wilrelss network through a 1 Tb Time Capsule.) For example, to transfer a 10 Gb file wirelessly between 2 Macs takes around 2 hours. To transfer the file to the Duo is listed as 14 days on the ‘Copy’ dialog. The problem seems to be with OS 10.8.x. If I transfer the file to another Mac running 10.7.x and then transfer to the Duo from that Mac, the transfer time is around the same as that transfer time Mac to Mac. It was suggested this has something to do with the encryption built into Mountain Lion.
I sent a message to WD on 7 Jan and received an acknowledgment but no other response yet. In frustration, I have sent another asking if they have an update available or planned that will solve the problem. My only alternatives seem to be to revert to Lion or to use the Duo as an anchor for a very small boat.
My other problem is the device never goes into standby mode. It just runs, and runs, and runs… I have the energy saver switch set to power down the disk when not in use but it makes no difference. This may be a separate problem or a part of the one above. Dunnow!
First of all, my advice would be to return the device to where you bought it and get your money back. Buy something else. This device is FUNDAMENTALLY flawed unless you take some fairly drastic steps to fix it yourself. Even then, the performance will only be “adequate” and never very good.
If that is not possible for you, then see my earlier posts where I describe a process (and include some shell scripts) to reconfigure the device to remove unneccessary processes which cause thrashing due to insufficient memory in the device. The basic performance problem you are seeing is due to over-committed memory on the device.
I would not disable the monitorio process and cron. moniyotio.sh does some essential maintenance tasks.
Thanks for the replies Tony & Myron. I read your work around and all the info on Nagle’s algorithm but I am reluctant to hack the system. The drive works OK from OS 10.7.x on my wife’s iMack but not from mine running OS 10.8.x . OK, it might be faste again on hers if i did the hack, but for my use the transfer rate is satisfactory. I feel that the problem lies in the relationship between the firmware on the hard drive and OS 10.8.x. When WD finally respond, I will make anther post to this formu. Hopefully this will be in the next few days. If WD/Apple are not addressing the problem then I will have no altrenative but to change the disk (I bouht it through WD store) for a couple of large firewire drives (don’t have Thunderbolt) and link them to a RAID 1 configuraton.
If you have any other ideas, i the meantime, please post. Much appreciated.
(PS Had to change my useername as the community website dd not recognise my old one any longer.)
''The tweaks should not affect any of that AFAIK.
My suggestion (seriously) is that unless you are very familiar with Linux and very comfortable making these kinds of changes, DON’T DO IT. Return the device and get your money back. There is a fundamental issue with it in that it was never given enough RAM to allow it to run all the services that WD starts. There is also no way (as far as I am aware anyway) to upgrade its internal RAM, so the only option you have is to reduce your memory footprint.
The device is fundamentally flawed out of the box.‘’
I wish somebody had written this on Amazon as a customer comment when I bought it … too late to return it now.
I was getting roughly 8 MB/sec transferring between two WD My Book Live 3TB models (non-Duo’s).
First, I checked the link speed between them. They’re both registered at full-duplex, 1gbps. They’re both sitting on the same switch. I transfer files between PC’s on this same switch at roughly 70 MB/sec. I’m referring to large media files (4GB average size). Nothing wrong on the network, or the WD’s connection to the network – that’s all great.
Then I found this post. I applied all of the settings in the dos.sh script.
Rebooted, and tested SCP between WD boxes again … same exact results, about 8 MB/sec.
Any other ideas to improve transfer speeds?
Thank you in advance!
RKM
Sorry to hear that WD has claimed another “victm”. My first advice would be to return the device and get your money back. The device is not capable of being used in any reasoable way as shipped from the factory.
If that is not possible then we should try systematically finding out what is wrong with your setup. The narrative below assumes some familiarity with Linux.
The first thing I would do is ssh into each box and run “top” while doing some test file transfers. You are looking to see if your device is running short of memory. The dos.sh script I wrote really only sets up the device in a way that makes is “less useless” for a typical DOS user. I am not sure how this would apply between devices. In particular, you will need to optimize the devices for the type of work you want to do with them.
If you only need to scp between them then pretty much all you need of the “file transfer” services is the sshd daemon. Examine the dos.sh and mac.sh scripts to understand what services can be safely disabled. That will free up some memory that can be used as cache by Linux and therefore help your inter-device transfer speeds. Maybe try disabling all services except those that you absolutley cannot live without and see how much swapping you get and correspondingly how much buffer memory becomes available.
As always, you run the risk of bricking the device if you make a mistake, so tread carefully.
Don’t give up hope, though. My own device reaches speeds of 40MB/sec after lots of tweaking and twiddling. The problem is that it does require some Linux knowledge to do really effectively. Let us know how you get on and hopefully the many talented members on this forum can help you where I am sure WD has failed.
This is the reply received from WD which was not particularly useful as i have tried the strategies listed here without success
"In regards of this case, if the performance between the computer and the drive is decreasing and the computer is out of space, this could be a problem when transferring files as the computer would not possess enough free virtual space in order to perform a desired action, although it needs to be taken in consideration that the transfer rate displayed by the drive is determined by the router speed.
In this case, to increase the data transfer rate it is usually recommended to work on a wired connection for all the device in order to work on a more stable network connection, when connected through the network connection depending on the connection used for Ethernet, the transfer rate can be increase potentially.
For this try connecting the computer to the same router in which the drive is connected and test the transfer rate when using a wired connection, also you can try and test by connecting the NAS drive directly to the computer and transfer the files. The transfer rate of the cable used for the My Book Live should increase between 10/100 Mbps which should make the data transfer easier for bigger files.
For more information on how to transfer the speed more than 10/100 Mbps you can click on the link below in order to read a documentation on our Knowledge base on how to increase the data transfer rate.
Data transfer: http://wdc.custhelp.com/app/answers/detail/a_id/5736/session/L3RpbWUvMTM1MjE0NjY4OS9zaWQvVTVtMXd4YWw%3D
In case the problem persist, try freeing up space from the computer to obtain more virtual memory.
If you have any further questions, please reply to this email and we will be happy to assist you further.
Sincerely,
Eduardo
Western Digital Service and Support
http://support.wdc.com
In the meantime I noted that a single file of around 1Gb transferred rapidly but a folder with 100 files aggregating around the same size slowed dramatically at around 50 Mb of transfer, So I compresssed the file and sent it and it went through quicly. I then compressed a13.5 Gb file compressed to just over 10 Gb, that contained 7500 individual photographs and it transferred in around 20 minutes. Only problem was that I cannot expand it on the drive, I have used the Apple compression utility but it gives an error message. I thought that I might try Stuffit and put a copy of Stuffit expander on the Duo. Not yet tried this. I will keep you posted.
So it seems that it is the number of files rather than the size that controls the transfer rate. At a guess, the Duo fiddles with the addresses of each file as it is received into a small buffer whose useful size is around 50 Mb. Not sure if this makes sense to the more technically talented members of the community, and I would appreciate some input.
An alternative is to simply compress and archive the compressed files, copying them back to the computer if I want to expand and use them, as needed
Tony_Mattheys wrote:
Sorry to hear that WD has claimed another “victm”. My first advice would be to return the device and get your money back. The device is not capable of being used in any reasoable way as shipped from the factory.
I couldn’t disagree more.
My Read / Write speeds, unmodified, are fine.
READ:
Write:
Tonyph12345: These are single files that you are transferring, not a foldre containing many files.Try 1000 files in a single folder and check the transfer. I have just transferred 102Gb in a single file and it took 4 hours. To transfer 1125 files (photos) in a folder of 2.8Gb takes several days. It seems the critical issue is the number of files rather than the size of the file. Is there a problem with writing addresses. That is causing the slowdown?
Hi mailmac2, i transfered most of my document data to the 6tb mbld, this contacted a program with 20.000 small files, around a hundred meg in size. It took around 2 hours. Took around 10 hours to tansfer 100gig in total, including several thusand photos, hundreds of avi movies.
My setup is the duo is on a 100 wired switch, which in turn plugs into a 1gig switch. Ive updated to latest firmware.
Ps, ive done none of the formentioned tweaks to improve performance yet. I have disabled apple time machine, and assigned a static ip.
Wow, the reply from WD was surprisingly unhelpful.
I can’t believe there are people transferring over wifi, then wondering why their transfers are slow, that’s level-1 support for you…
So here’s where I’m at. I noticed a bunch of people had good transfer speeds in Windows. Therefore, I tested copying to/from the WD boxes directly from Windows. I was able to get about 60MB/sec on average, on large 4+ GB files.
This made me think that the WD boxes just couldn’t do any sort of encrypted transfer (RSYNC, SCP, etc). With that in mind, I mounted a share using CIFS, and copied some files that way. Keep in mind, this is going directly from one WD 3TB to another WD 3TB box. No intermediary system.
Doing this I was able to get my throughput up to about 30+ MB/sec. Is this ideal? No. However, it is usable. It’s a bummer that I can transfer to Windows 200%+ faster than I can transfer WD box-to-box. Yet it’s enough to allow me to transfer all of my files from one WD box, to my new backup box, within a day or two.
Thanks again for your very helpful work/insight on this topic!
RKM
MalMac2 wrote:
Tonyph12345: These are single files that you are transferring, not a foldre containing many files.Try 1000 files in a single folder and check the transfer.
Fair enough! :)
Here’s copying a few folders of photos (1000+) totalling over 4.0 GB:
At the start:
…and when close to finishing (about 3.5 minutes later):
| Forums | News and Announcements | Register | Help | Forum Guidelines |
Copyright © 2021 Western Digital Technologies, Inc. All rights
reserved.
|
Trademarks
|
Privacy
|
Community Terms of Service
|
Site Terms of Use
|
Copyright
|
Contact WD
|
