WD My Cloud Data Recovery

My Cloud OS 3 Personal & Network Attached Storage My Cloud
1

Good Day to you all.

I recently had a break in to my home and burglars stole my entire desktop, along with 2 of my laptops as well as 3 USB 1TB hard drives. I’ve lost a lot of my files such as family pictures etc…Now my question to you is - Last year some time before I formatted my Desktop I copied all these files on to the My Cloud Storage. After I restored the Desktop I copied the files back to the PC and then deleted them from the WD My Cloud. Is there anyway these ‘DELETED FILES’ can be recovered from the WD My Cloud using any recovery software etc…IF you think its possible, What would the procedure be, GIVEN the type of connections and structure of the WD my Cloud. What would you suggest. Do you think its worth giving it a shot??

Regards
Ihsaan

2

First, do a forum search, using the magnifying glass icon upper right, for “recovery” or some variation on recovering data from the My Cloud. This question on how to recover deleted files from the My Cloud gets asked from time to time. There are a number of past discussions on the issue with no really good answer other than using any number of free and paid data recovery programs or sending the My Cloud off to a professional data recovery service (and paying them a good chunk of money to attempt recovery of deleted data).

There is no guarantee you’ll be able to recover anything that’s been deleted from the single bay/single drive My cloud units since they do not have a; “recycle bin”, “trash” or “undelete” option/feature.

Typically the chances of recovering any usable deleted file data goes down quickly over time as the hard drive is used and new data is written to the same location as the deleted file/data.

https://support.wdc.com/warranty/datarecovery.aspx

3

The best time to recover files on a disk is as soon as they were deleted. Once they are deleted and disk activity continues to the disk. The longer the acrivity occurs the less chance there is to recover the files.

4

Indeed. Recovery MAY still be possible though. Best solution is to remove the drive from the MyCloud enclosure, hook it to a computer, and attempt file system recovery with TestDisk, or something similar.

Testdisk is free, and knows how to deal with EXT4 file systems. If you are mostly interested in restoring lost pictures, photorec (companion of testdisk) is also worth a shot, because it actually trolls the whole disk looking for image data. It can detect and recover most image and video formats.

Removing the disk from the enclosure will void the warranty though. :frowning:

I really should see about making a static compile of Testdisk and Photorec for 64k pages, so that they can be run from USB sticks over SSH. That would fix a lot of these “How do I recover files?” support questions.

2 Likes
5

Ok, I have successfully statically compiled Testdisk and Photorec. They appear to be working.

I pushed them to my Google Drive here, along with the source tarball (for GPL compliance.) Since it is statically linked, if you wanted to compile it yourself you would need to get all the necessary library sources too, but meh. Not doing it. The sources are available from the usual places.

These were compiled for use on a gen2. They might also work on a gen1, I dont know for sure though.

https://drive.google.com/drive/folders/0B0EsuN03YHarLUlrbGV1cXdxSnM

To use:

  1. Copy the tiles Photorec and Testdisk to a USB stick, then plug it into the MyCloud on the back.
  2. Start an SSH session with the MyCloud
  3. at the ssh prompt:

df

this displays the information about mounted filesystems, including the USB stick you just plugged in. Make note of where the USB stick is mounted. We will call this location [USB_mount] from here on. It will be unique to your system.

cd /mnt/USB/[USB_mount]

This changes the directory to the USB stick.

ls -ll

This lists the files on the stick, and shows permissions. Make sure that Photorec and Testdisk are present, and that they have the e(X)ecute permission set for root user.

./testdisk

This starts testdisk from the local folder.

  1. Unless you want there to be logged diagnostic information about the session, choose no logging.

    823×509 59.6 KB

  2. Select /dev/sda as the drive to mess with.

  3. Select GPT partition type.

  4. Choose [Advanced] Filesystem Utils

  5. Use the arrow keys to highlight parition 2 (waaaay at the bottom), and use the left and right arrow keys to navigate to [List]. Press Enter

  6. A file system tree of the internal storage partition is shown. Items that have been deleted will appear in red. [I have one highlighted here] Use the arrow keys and the enter key to navigate around the filesystem.

  7. When you are ready to recover a file, either select it (see text at bottom of the testdisk screen for details) or highlight it with the cursor, then press ‘C’ or ‘c’ on the keyboard.

  8. It wants to know where to write the recovered file. It is very dumb to try writing to the volume you are recovering files from. Instead, recover to the USB stick. (remember the location we looked up with df earlier?)

  9. When it completes, you will have some nice verbose text telling you the copy completed successfully.

Just do that for all the files or folders that you want to recover. Some might not be recoverable, if you have written data to the drive. Those are the breaks. Some files recovered is better than no files recovered.

3 Likes
6

I downloaded your testdisk to my gen1 My Cloud. I then ran the following command.
./testdisk --help

It displayed the help instructions. So I guess it will work on a gen1 system.

7

Good to know!

Now we are armed and dangerous the next time somebody needs to recover data from their MyCloud without voiding their warranty by opening the box and removing the drive. :stuck_out_tongue:

8

Nice write up will test at some point on my Gen 1.

Gotta add one comment here for the non computer literate who read this, make sure to use a USB stick/drive that is larger than the contents you want to recover. Especially if attempting to recover large media files.

And to remind the non computer literate, as said above, its best to attempt file recovery immediately after the file was deleted. The longer one waits to attempt file recover the greater the chance of being unable to recover an intact file due to the drive data being overwritten with new data.

Edit to add:

The “testdisk” program does work on the first gen v4.x My Cloud however I couldn’t successfully restore any file I deleted earlier today. One restored video file ended up being a MP3 file deleted several weeks ago. Two other restored video files ended up being zero bytes.

The steps on the v4.x firmware My Clouds may be a bit different than listed above, at least it was for me. In my case I have a 1TB Seagate drive rather than a 4TB WD Red drive the first gen enclosure came with.

I selected the drive (/dev/sda):

661×418 60.1 KB

The data partition 4 is listed as Unknown. So I had to select Type and proceed through in order to get List to show on this screen as indicated in a subsequent screen capture:

661×418 71.4 KB

I selected EFI System:

661×418 75.6 KB

Then selected EXT4:

661×418 68.8 KB

Then I could select List to continue with the rest of the steps indicated above:

661×418 70.9 KB

Few other comments. The program is run by using the following command: ./testdisk
If the “./” isn’t before the word testdisk the program won’t run.

The recovered file(s) were in their respective directories/folders located in the recovery location under a folder called share.

2 Likes
9

Testdisk tries to look at filesystem metadata to determine what inodes a file previously had allocated.

If you are unable to recover a media file with testdisk, attempt recovery with photorec instead. It ignores the filesystem type, and reads raw sectors from the drive sequentially, then checks them to see if media file headers are detected. It then parses any media file header it finds to determine length, and attempts to recover the file that way. This only has a reasoable chance of success if the files were written sequentially though. High fragmentation will make photorec fail.

Despite the name, photorec can recover movie files (MPG, AVI, etc…) and music files (MP3, etc…). Because it does not read the filesystem to recover the data, it names the files using a sequentially populated numeric system on the output directory you choose. If one is unable to recover the file, the other may yet be able to succeed.

Testdisk’s primary purpose is to restore accidentally nuked partition tables. Filesystem recovery is an additional feature that is very useful. There are probably better tools for this, but test disk knows about a great many kinds of file system. On file system types that store a tremendous amount of metadata (like NTFS), file undeletion is much more reliable.

Another potentially useful tool that I may attempt to statically compile is ddrescue. It is much like dd, but is able to aggressively retry failed reads, and can often get a good read off a bad sector, just by being so perniciously persistent. It creates a disk image on another volume, or can be used to clone the damaged drive onto another. Unlike normal dd, ddrescue gives useful feedback about its progress, making it a superior tool in my opinion.

10

Thank you so much for all the help and info. I think I may be able to
navigate myself around the instructions you have provided. However I will
only be back home in 5 days time. So I will then attempt it and get back to
you with any further help I may require and with feedback. Just a note, I
haven’t really copied any new files onto the drive since deletion. But the
drive remains ON all the time. Deletion took place quite some time ago. At
least 7 or 8 months ago. However I can confirm that NO new files were
copied on to the drive. I hope the time period doesn’t play a factor.
However I will be in touch with you just before I start the process and
keep you updated.

Thank you again for all the info, efforts and help.

Regards

Ihsaan

11

Hello Wierd_w,

I have a problem when starting the testdisk. It always says:

Terminfo file is missing.
Extract all files and subdirectories before running the program.

Do you know how to solve this?

12

Researching the issue, it looks like Testdisk cannot find your terminal capabilities database. See this page for more details:

http://www.tldp.org/HOWTO/Text-Terminal-HOWTO-16.html

For reference, the singe bay myclouds put this file in /usr/local/modules/files/terminfo

You can point your session at the appropriate file using:

export TERMINFO=/usr/local/modules/files/terminfo

then running testdisk.

13 
export TERM=xterm

before you run testdisk,
If this work you can add that line to your /etc/profile

14

Thank you for the help. I found a solution…

15

Please explain your solution to assist others who may have the same issue/problem.

1 Like
16

What about extundelete: http://extundelete.sourceforge.net/
Could somebody compile it to work statically with My Cloud?

17

Short history about my data loss:
I bought a new external 4TB WD My Book drive to backup the current 4TB WD My Cloud… Of course I have an obsession about the drive sleep, and since I couldn’t manage to make My Book sleep, I decided to edit some system files in hope it would allow it to sleep normally (like My Cloud). That’s how I found file /etc/nas/wdAutoMount.conf and… I was stupid enough to set the “AUTO_MOUNT_MOUNT_DIR” to the same as “AUTO_MOUNT_SHARES_DIR”. I thought it would prevent double mount of the external drive. After long reboot all my files were gone. My cloud literally deleted 3.5TB of data because of the change in that config file. I was very resigned, especially because I just bought a drive to backup stuff and secure this data :frowning:

First I tried to recover with testdisk - It didn’t work, all files I could list were 0 in size. Then I disassembled My Cloud and put the disk to an external USB2.0 HDD reader… Then I used photorec on a Linux Mint virtual machine, but I noticed that this HDD case has a limit of 2TB :frowning: So I recovered only half of files. It took over 2 days with USB2.0. Restored files were sometimes damaged (depends if they were parted or not). I put the HDD back to My Cloud, and used photorec from this topic. Another 2 or 3 days of scanning and it found many partial files, with no names.

Then I read more about data recovery on ext4 partitions, and I found an amazing program called: extundelete: http://extundelete.sourceforge.net/
I didn’t want to install it by apt-get to not destroy My Cloud totally, so I managed to compile version 0.2.4 using gpl-source-wd_my_cloud-04.00.00-607 build-armhf-package.sh script (replaced the default version 0.2.0 source files to compile 0.2.4). I think it’s only compatible with firmware v04, because I selected 64k pagesize. Anyway, here’s how to recovery your data:

PLEASE NOTE THAT NOT ALL FILES CAN BE RECOVERED WITH THIS METHOD, HOWEVER I WAS VERY SATISFIED FROM RESULTS.

METHOD1:

  1. Don’t panick. Stress can only make it worse. Don’t write anything to My Cloud, disable all synchronization scripts, etc.
  2. Download folder “extundelete” from: Microsoft OneDrive - Access files anywhere. Create docs with free Office Online.
  3. Paste that folder to an external USB HDD.
  4. Connect the external USB HDD to My Cloud.
  5. Type in terminal/putty: “umount /dev/sda4” many times, until you see a message that /dev/sda4 is not mounted. So basically type this command 6-10 times.
  6. Type: “cd /nfs”. Type “ls”. You should see names of shares. Find the name of your external HDD and type: “cd NAME”, where NAME is the disk name (different for everyone). Type “cd extundelete”.
  7. Type: “date -d “Aug 1 00:00” +%s” (change to any date you want, before the date when your files got deleted). It’s recommended to not restore junk files which were intentionally deleted before and could have incorrect size / increase recovery time.
  8. A number like “1501538400” should be shown. Copy it somewhere, for example to notepad.
  9. Type the actual recovery command: “./extundelete2.4 --restore-all --after 1501538400 /dev/sda4 --log ExtundeleteFullRestoreLog.txt”.
  10. Done. All recovered files should be put to “RECOVERED_FILES”. They should have the orginal file path/name, size and work even if were partial.
  11. Type “umount /dev/sdb1” to safely dismout your external HDD, and check recovered files on your computer. Note that many files will be found in folder “RECOVERED_FILES/lost+found”

I recovered around 1.5TB of useful data with this method. The rest was just big laptop backup archives, etc, not really important.
With extundelete you can also recover a single file, for example command (please note that file names are only for joke):
./extundelete2.4 --after 1501538400 --restore-file “shares/Mona/MonaNudesLeak.zip” /dev/sda4 --log ExtundeleteFileRestoreLog.txt

or a single folder, for example:
./extundelete2.4 --after 1501538400 --restore-directory “shares/Mona/MonaNudes/HopeToLeak” /dev/sda4 --log ExtundeleteDirectoryRestoreLog.txt

However, I would recommend to do a --restore-all, because some files won’t have their orginal path and will be available in “lost+found” instead.

===============================
METHOD2:
Another good recovery program I found is ext4magic. I couldn’t compile it myself, but I found some version online, which somehow works on My Cloud.

  1. Download folder “ext4magic” from: https://onedrive.live.com/?id=D8D5E8AFD47D9E3!5880&cid=0D8D5E8AFD47D9E3
  2. Repeat steps 2-6 from METHOD1, but instead use “ext4magic” as the directory.
  3. To recover a single file type:
    ./ext4magic /dev/sda4 -r -f “shares/Mona/BigFile.zip” -d RECOVERED_FILES
    I managed to recover a 500GB file with this method, which extundelete was unable to recover. However, when I tried to restore all files with ext4magic - it restored only a few files. Maybe I used a wrong command, I don’t know:
    ./ext4magic /dev/sda4 -M -d RECOVERED_FILES

My restored 500GB file was not full, but was taking only 250GB on disk. Thankfuly it was an archive so I was able to recover half of files, which actually contained most important for me files. The rest was added later and was not important at all. I can say I had a big luck with it.

===============================
SUMMARY:
Recovering from ext4 is possible, but not perfect. You should make regular Safepoints backups by MyCloud website. I was/am just stupid to modify system files before creating my first backup. Lesson learnt. Over 1 week of life wasted for recovering.

If somebody could please compile newest extundelete and ext4magic version on My Cloud - it would be much better. I think the ext4magic version I found doesn’t work as good as it could.
Please tell me if there are any mistakes in this post, so i’ll fix them as soon as possible.

2 Likes
18

Hi. All great help on this forum. Much appreciated. I have successfully restored all my files from my Mycloud 6tb (~100hrs). I foolishly deleted one folder of pics. Of course everything BUT the deleted folder/files were restored. AND the parent folder was one of the last to be restored - hence the wait. Have also tried photorec, but nothing, just heap of elf files that when I amend to jpg, still don’t work. Have now tried the extundelete2.4 method. Didn’t work first up, but then changed sda4 to sda6 (assuming this will suit my 6tb drive). Didn’t work either, but did get the following message in the log. “Bad magic number in super-block when trying to open filesystem /dev/sda6”
Are there any more ideas? Thanks.

19

Hi Ged. Are you sure that /nfs shares are mounted to sda6? You can check if they’re mounted with command: “cat /proc/mounts”. Remember to unmount sda6 before running extundelete. I’m not sure if it would work but you can try to run the command with just “sda”. Also check if that partition actually has journals: partitioning - How can I check if a particular partition ext4 is journaled? - Ask Ubuntu
Sorry for not providing exact commands, but I’m currently writing on a phone.

20

I have obviously assumed wrong. I simply can’t seem to unmount the /dev/sda4. I keep getting “device is busy” - 10-20 times. Its seems I can force an unmount, but this may kill processes and/or lose data.