Hello everybody!

OK.. I've got a USB disk that worked fine.. then started to have some hissy fits.. wouldn't write during rsync ops...

So for some reason it seems that it was getting hotter than normal, although its been sitting in that spot for the last 6months or so???

Anyway... tl;dr

I put testdisk on a box... and it CAN FIND the partition that is there, that won't mount now... and it CAN SEE THE dirs in the partition so there is hope it
might recover the data...

BUT

I'd like something that walks thru this step by step, that I can review, think on, review , think on review some more...

I think I am just going to have Tarjay send me another 4TB drive and back up the original and then use this for a test to learn how to recover the partition
and put back to use at least to read stuff.... and possibly recover another drive that exhibits similar behavior. (Was stoped, unmounted, removed, and now wont remount..)

So any one got some GOOD for RISK AVERSE persons tutorials, FAQ's etc.. for testdisk?

I looked at their site and it seems more geared to HD experts vs. I just want my stuff back (nothing critical) person.

Thanks


Rick


... NOprah!
--- GoldED+/LNX 1.1.5-b20161221
* Origin: Vina's Talos Moon Base Alpha (1:135/377)

Rick Christian wrote to All <=-

Hello everybody!

OK.. I've got a USB disk that worked fine.. then started to have some hissy fits.. wouldn't write during rsync ops...

So for some reason it seems that it was getting hotter than normal, although its been sitting in that spot for the last 6months or so???

USB thumb drives, that get to hot ... can crush the onboard memory chips, and controller.

Anyway... tl;dr

I put testdisk on a box... and it CAN FIND the partition that is there, that won't mount now... and it CAN SEE THE dirs in the partition so
there is hope it might recover the data...

BUT

I'd like something that walks thru this step by step, that I can
review, think on, review , think on review some more...

I think I am just going to have Tarjay send me another 4TB drive and
back up the original and then use this for a test to learn how to
recover the partition and put back to use at least to read stuff....
and possibly recover another drive that exhibits similar behavior. (Was stoped, unmounted, removed, and now wont remount..)

So any one got some GOOD for RISK AVERSE persons tutorials, FAQ's etc.. for testdisk?

I looked at their site and it seems more geared to HD experts vs. I
just want my stuff back (nothing critical) person.

Hi Rick,

If your drive was part of a lvm volume group, you may be missing the other part of that volume that is preventing you from seeing all the data. IF not then what I attempt to write below may help.

I'm not sure there is a recovery. USB ssd type drives (non-spinner drives), have a write limit, eventually they just break.
Now I've not broke a internal or MR2 type ssd yet, but plenty of USB thumb drives, and micro ssd drives, and often times this is earlier than expected end of life so they are still under warranty.

If your computer is reporting a different size of the drive than manufacture specification, you could have a controller issue, and that could be solved by a controller swap. .. lots more steps for that recovery.

Here is what I usually do to test if the disk is broke, the easy way.

Caveat, must use linux, the drive wasn't part of an lvm volume group:

find the device of the disk. This is usually known when plugging it into the computer, then do the following:

dmesg |grep sd

Another easy way is to do the following:

sudo fdisk -lu

Both will produce some identifier about your drive. If it doesn't you may have another issue, but from your statements above those two command should work well enough.

Next ... now understand this is where you must be precise in your command syntax and typing.

NOTE!!! dd can wipe out all data on your drive if used incorrectly.
NOTE(2) dd can take a long long time to read a 4TB drive.

I'll explain what I'm doing and show you and example.
dd .. used to read the drive directly ignoring any partitions at all.
using dd I read the device from start to finish. If it can't read the entire device it usually gets stuck or errors out, and this happens with USB ssd type devices right at the memory chip boundaries. Lets say you have a 8 gig USB thumb drive, most often I've seen at the 4GB boundary the drive is broke, or the 2G boundary, ... etc.

from my command:

dmesg |grep sd

I see a device [sdb] Attached SCSI removable disk ..
I use the dd command as follows:

dd if=/dev/sdb of=/dev/null bs=512

eventually you'll get something like (i made up the numbers below):
11118188+1 records in
11118188+1 records out
5000000000 bytes (5.7 GB, ... ) copied ... 22 MB/s
If the size of 5.7 GB happens to be the size of your device, you probably have a good working drive, if not, you have a broken ssd and recovery is very difficult. Good working drives with ext2,3,4 formats can usually get data back with an: fsck

dd will be attempting to read your drive device from start to finish. If it finishes really early say after about 2GB when its a 8GB you see it, and it may error out.

If you use the dd command as follows:

dd if=/dev/sdb of=/dev/null bs=512 conv=noerror

and it runs for days and days, you've got it stuck on some spot that is broke on your drive, but .. the last time I've ever used dd on a USB thumb drive, I've never seen this, I've only seen it stop reading prior to reading the entire drive.

If your dd can read the entire device, then you can used fsck on the drive, and there are many options.
Also if dd can read the entire device, I usually use dd to make an exact duplicate of my drive, and operate on the image.

dd if=/dev/sdb of=./sdb.drive.img bs=512 conv=noerror,notrunc

Then I start operating on the image. If this is an lvm type setup, ... way more things to do,


For spinner drives .. there is a different process, you can use dd, but other methods work depending on the partitions and format type.



--- MultiMail/Linux v0.49
* Origin: Electronic Warfare BBS | telnet:\\bbs.ewbbs.net (1:227/201)

Hello Bbsing!

15 Aug 19 20:39, you wrote to me:

USB thumb drives, that get to hot ... can crush the onboard memory
chips, and controller.

Never had that happen with USB thumb drives... but heat is bad for any electronics period.

If your drive was part of a lvm volume group, you may be missing the

Nope.

Standard USB external HD (not a SSD! I don't use or trust that garbage)

other part of that volume that is preventing you from seeing all the
data. IF not then what I attempt to write below may help.

I can SEE THE PARTIOTION in test disk... it just comes up with the Message "Can't mount drive.... " if I plug it in and try to access it like normal use.

I'm not sure there is a recovery. USB ssd type drives (non-spinner drives), have a write limit, eventually they just break. Now I've not broke a internal or MR2 type ssd yet, but plenty of USB thumb drives,
and micro ssd drives, and often times this is earlier than expected
end of life so they are still under warranty.

This is why I don't use SSD unreliable and write limited.

If your computer is reporting a different size of the drive than

Just can't access the drive if I plug it in...

Here is what I usually do to test if the disk is broke, the easy way.

Caveat, must use linux,

Caveat, I don't use any thingBUT LINUX. OK, Z/vm, z/os, s3xx, :) ;) :) but other crap? nope. :)

the drive wasn't part of an lvm volume group:

Nope, don't use'm too risky.

find the device of the disk. This is usually known when plugging it
into the computer, then do the following:

You can see that below.

NOTE!!! dd can wipe out all data on your drive if used incorrectly. NOTE(2) dd can take a long long time to read a 4TB drive.

dd is too dangerous for use for exactly #1, and #2.

For spinner drives .. there is a different process, you can use dd,
but other methods work depending on the partitions and format type.

What info for real drives? ie:
9339387.713615] usb 1-3: USB disconnect, device number 6
[9339387.714581] sd 7:0:0:0: [sdb] Synchronizing SCSI cache
[9339387.714706] sd 7:0:0:0: [sdb] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[9339395.976052] usb 1-3: new high-speed USB device number 10 using ehci_hcd [9339396.186116] scsi8 : usb-storage 1-3:1.0
[9339397.184886] scsi 8:0:0:0: Direct-Access Seagate BUP BK 0109 PQ: 0 ANSI: 6
[9339397.186602] sd 8:0:0:0: Attached scsi generic sg1 type 0

I guess I am more interested in:

1) What acn be done via testdisk to use its internal backup/recovery options.... BEFORE I DO

2) Reset the info, likely the partition info, so that plugging it in and selecing to open in file manager or mounting it other ways works

I have HUGE RISK AVERSION. Meaning that measure 40x then review, measure 50x more, review.. backup backup backup.. try it...

So I'd like to use testdisk to suck out the data I can to say either another 4TB drive I attach to the box and then I can review what I've recovered and go from there.. then test out the steps to recover the disk.. ie: do the x,y,z in testdisk to reset it and then see if it is mountable...

I just can seem to find a GOOD STEP BY STEP EXPLAIN it in full for testdisk , that after I am at the point, below, do, x,y,z to reset things so the drive is mountable...

Here is some info from testdisk... I have a box with 7.xx testdisk, but where this golded is at is another box, and I quickly attached and ran this to see ... Which based on looking via the testdisk file browser the STUFF IS STILL THERE, the partition table is fubar'd and needs to be reset????

testdisk.log
:~$ cat testdisk.log |more


Wed Aug 14 19:37:32 2019
Command line: TestDisk

TestDisk 6.13, Data Recovery Utility, November 2011
Christophe GRENIER <grenier@cgsecurity.org>
http://www.cgsecurity.org
OS: Linux, kernel 3.2.0-79-generic-pae (#115-Ubuntu SMP Thu Mar 12 14:33:25 UTC 2015) i686
Compiler: GCC 4.6
Compilation date: 2012-02-05T07:16:54
ext2fs lib: 1.42, ntfs lib: 10:0:0, reiserfs lib: none, ewf lib: none
/dev/sda: LBA, DCO support
/dev/sda: size 156301488 sectors
/dev/sda: user_max 156301488 sectors
/dev/sda: dco 160086528 sectors
Warning: can't get size for Disk /dev/mapper/control - 0 B - CHS 1 1 1, sector size=512
Hard disk list
Disk /dev/sda - 80 GB / 74 GiB - CHS 9729 255 63, sector size=512 - Maxtor 6Y080L0, S/N:Y286W68E, FW:YAR41BW0
Disk /dev/sdb - 4000 GB / 3726 GiB - CHS 486401 255 63, sector size=512 - Seagate BUP BK, FW:0109

Partition table type (auto): EFI GPT
Disk /dev/sdb - 4000 GB / 3726 GiB - Seagate BUP BK
Partition table type: EFI GPT

Analyse Disk /dev/sdb - 4000 GB / 3726 GiB - CHS 486401 255 63
hdr_size=92
hdr_lba_self=1
hdr_lba_alt=7814037166 (expected 7814037166)
hdr_lba_start=34
hdr_lba_end=7814037133
hdr_lba_table=2
hdr_entries=128
hdr_entsz=128
check_part_gpt failed for partition
1 P MS Reserved 34 262177 262144 [Microsoft reserved partition]
Current partition structure:
No FAT, NTFS, ext2, JFS, Reiser, cramfs or XFS marker
1 P MS Reserved 34 262177 262144 [Microsoft reserved partition]
1 P MS Reserved 34 262177 262144 [Microsoft reserved partition]
2 P MS Data 264192 7814035455 7813771264 [Basic data partition] [Seagate Backup Plus Drive]

search_part()
Disk /dev/sdb - 4000 GB / 3726 GiB - CHS 486401 255 63
NTFS at 16/113/34
filesystem size 7813771264
sectors_per_cluster 8
mft_lcn 786432
mftmirr_lcn 2
clusters_per_mft_record -10
clusters_per_index_record 1
MS Data 264192 7814035455 7813771264 [Seagate Backup Plus Drive]
NTFS, 4000 GB / 3725 GiB
Search for partition aborted

Results
P MS Data 264192 7814035455 7813771264 [Seagate Backup Plus Drive]
NTFS, 4000 GB / 3725 GiB

interface_write()
1 P MS Data 264192 7814035455 7813771264 [Seagate Backup Plus Drive]
simulate write!

Analyse Disk /dev/sdb - 4000 GB / 3726 GiB - CHS 486401 255 63
hdr_size=92
hdr_lba_self=1
hdr_lba_alt=7814037166 (expected 7814037166)
hdr_lba_start=34
hdr_lba_end=7814037133
hdr_lba_table=2
hdr_entries=128
hdr_entsz=128
check_part_gpt failed for partition
1 P MS Reserved 34 262177 262144 [Microsoft reserved partition]
Current partition structure:
No FAT, NTFS, ext2, JFS, Reiser, cramfs or XFS marker
1 P MS Reserved 34 262177 262144 [Microsoft reserved partition]
1 P MS Reserved 34 262177 262144 [Microsoft reserved partition]
2 P MS Data 264192 7814035455 7813771264 [Basic data partition] [Seagate Backup Plus Drive]

search_part()
Disk /dev/sdb - 4000 GB / 3726 GiB - CHS 486401 255 63
NTFS at 16/113/34
filesystem size 7813771264
sectors_per_cluster 8
mft_lcn 786432
mftmirr_lcn 2
clusters_per_mft_record -10
clusters_per_index_record 1
MS Data 264192 7814035455 7813771264 [Seagate Backup Plus Drive]
NTFS, 4000 GB / 3725 GiB

Results
P MS Data 264192 7814035455 7813771264 [Seagate Backup Plus Drive]
NTFS, 4000 GB / 3725 GiB
ntfs_device_testdisk_io_ioctl() unimplemented
ntfs_device_testdisk_io_ioctl() unimplemented
NTFS filesystem need to be repaired.


dir_partition inode=5
P MS Data 264192 7814035455 7813771264 [Seagate Backup Plus Drive]
NTFS, 4000 GB / 3725 GiB
Directory /
5 dr-xr-xr-x 0 0 0 11-Aug-2019 09:01 .
5 dr-xr-xr-x 0 0 0 11-Aug-2019 09:01 ..
35 -r--r--r-- 0 0 386470 19-Sep-2018 04:00 .VolumeIcon.icns
36 -r--r--r-- 0 0 194586 19-Sep-2018 04:00 .VolumeIcon.ico
37 -r--r--r-- 0 0 4096 19-Sep-2018 04:00 ._
38 -r--r--r-- 0 0 33 19-Sep-2018 04:00 Autorun.inf
89 dr-xr-xr-x 0 0 0 12-Aug-2019 22:05 Grabbed_Photos
384686 dr-xr-xr-x 0 0 0 5-Jun-2019 07:19 PossImages
41 dr-xr-xr-x 0 0 0 17-Dec-2018 00:07 Seagate
86 dr-xr-xr-x 0 0 0 11-Aug-2019 10:33 seed
87 dr-xr-xr-x 0 0 0 11-Aug-2019 19:30 seed_imagesets
88 dr-xr-xr-x 0 0 0 12-Aug-2019 06:00 Smxxty
43 dr-xr-xr-x 0 0 0 17-Dec-2018 00:07 Start_Here_Mac.app
39 -r--r--r-- 0 0 18033016 19-Sep-2018 04:01 Start_Here_Win.exe
84 dr-xr-xr-x 0 0 0 17-Dec-2018 00:16 System Volume Information
1422010 dr-xr-xr-x 0 0 0 11-Aug-2019 09:02 testssh
40 -r--r--r-- 0 0 1172289 19-Sep-2018 04:00 Warranty.pdf

dir_partition inode=384686
P MS Data 264192 7814035455 7813771264 [Seagate Backup Plus Drive]
NTFS, 4000 GB / 3725 GiB
Directory /PossImages
384686 dr-xr-xr-x 0 0 0 5-Jun-2019 07:19 .
5 dr-xr-xr-x 0 0 0 11-Aug-2019 09:01 ..
893834 dr-xr-xr-x 0 0 0 12-Aug-2019 09:42 SxxyBacks
651926 dr-xr-xr-x 0 0 0 1-May-2019 11:38 SxxyBacks050119
384688 dr-xr-xr-x 0 0 0 5-Jun-2019 07:17 SxxBacks060519

dir_partition inode=893834
P MS Data 264192 7814035455 7813771264 [Seagate Backup Plus Drive]
NTFS, 4000 GB / 3725 GiB
ntfs_readdir

Rick


... NOprah!
--- GoldED+/LNX 1.1.5-b20161221
* Origin: Vina's Talos Moon Base Alpha (1:135/377)

Rick Christian wrote to Bbsing Bbs <=-

I'm not sure there is a recovery. USB ssd type drives (non-spinner drives), have a write limit, eventually they just break. Now I've not broke a internal or MR2 type ssd yet, but plenty of USB thumb drives,
and micro ssd drives, and often times this is earlier than expected
end of life so they are still under warranty.

This is why I don't use SSD unreliable and write limited.

Your information on SSD's is quite outdated. SSD's are *FAR* from
unreliable and the "write limited" part is not really a factor any
more. Even consumer-grade SSD's have an expected lifetime that is
measured in decades (of far heavier use than you would do).

NOTE!!! dd can wipe out all data on your drive if used incorrectly. NOTE(2) dd can take a long long time to read a 4TB drive.

dd is too dangerous for use for exactly #1, and #2.

Too dangerous? Well, you could get hit by a bus when you cross
the street, too.

I have HUGE RISK AVERSION. Meaning that measure 40x then review,
measure 50x more, review.. backup backup backup.. try it...

Maybe that's a little excessive?


... Internal Error: The system has been taken over by sheep at line 19960
=== MultiMail/Linux v0.52
--- SBBSecho 3.09-Linux
* Origin: Palantir * palantirbbs.ddns.net * Pensacola, FL * (1:123/115)

On 09-11-19 15:26, Dan Clough wrote to Rick Christian <=-

Your information on SSD's is quite outdated. SSD's are *FAR* from unreliable and the "write limited" part is not really a factor any
more. Even consumer-grade SSD's have an expected lifetime that is measured in decades (of far heavier use than you would do).

Yes, SSDs do have write limitations, but for all practical purposes, modern SSDs are likely to outlive the rest of the system they're installed in. I love them, and will happily use a SSD for the performance gains on the OS drive. A spinning HDD is still handy for bulk data storage though, because they're far cheaper in larger sizes.


... But the FACTS keep interfering with your theories!
=== MultiMail/Win v0.51
--- SBBSecho 3.03-Linux
* Origin: Freeway BBS Bendigo,Australia freeway.apana.org.au (3:633/410)

Hello Dan!

11 Sep 19 15:26, Dan Clough wrote to Rick Christian:

This is why I don't use SSD unreliable and write limited.

Your information on SSD's is quite outdated. SSD's are *FAR* from unreliable and the "write limited" part is not really a factor any
more. Even consumer-grade SSD's have an expected lifetime that is measured in decades (of far heavier use than you would do).

This heavily depends on the use case. I already managed to blow away like 10% of several SSD's expected lifetime in a rather simple home setup within only a couple of months. Write amplification and aggressive caching by webbrowsers like Firefox and Chrome make it possible.


Regards,
Gerrit

... 8:55PM up 149 days, 5:16, 8 users, load averages: 0.68, 0.50, 0.40

--- Msged/BSD 6.1.2
* Origin: All carefully conceived (2:240/12)

Hello Rick!

11 Sep 19, Rick Christian wrote to Bbsing Bbs:

Standard USB external HD (not a SSD! I don't use or trust that
garbage)

This is why I don't use SSD unreliable and write limited.

Reality Check: My SSD works fine, what is broken on your side?
HDD are unreliable and are write limited just like any other storage media too. They are sold with a MTBF with the warranty that they will fail.

In 20 years i lost two internal HDD, had two new HDD with many badblocks (replaced by warranty), one dead SSD and two sudden death USB Sticks.

Bashing one type of storage may calm your emotions, but it would not help you.

I can SEE THE PARTIOTION in test disk...

You don't. You see the data entry in the partition table which tells you where the real partition was.

it just comes up with the Message "Can't mount drive.... " if I plug
it in and try to access it like normal use.

If your computer is reporting a different size of the drive than

Just can't access the drive if I plug it in...

How could you see something with testdisk then?

NOTE!!! dd can wipe out all data on your drive if used
incorrectly. NOTE(2) dd can take a long long time to read a 4TB
drive.

dd is too dangerous for use for exactly #1, and #2.

If you are talking about too dangerous and risky you must use dd to reduce risk and danger. There is no way around.

If you are working with the original data there is a high risk that any rescue tool make things worse or destroy mandatory data.

At the moment you don't know what kind of damage you have. Is there a software failure in the data structure on the disk or do you have a real hardware failure that prevents some kind of data to be read.

The first thing is to make a mirror of the damaged device. dd_rescue is a useful Linux tool for that. If testdisk finds something, than some data can be read. dd_rescue copies all data that can be read to another disk and skips the unreadable data. It's suggested to keep the first mirror and make a working copy of that mirror for trouble shooting. A broken disk could loose more data with every usage. Creating additional mirrors from the original disk may result into additional data loss.

1) What acn be done via testdisk to use its internal backup/recovery options.... BEFORE I DO

It can recover the partition table, the partition directory, from a backup sector. It can't repair a broken filesystem i.e. if the filesystem can't be mounted.

2) Reset the info, likely the partition info, so that plugging it in
and selecing to open in file manager or mounting it other ways works

Can't work. A partition can hold one of dozens of different file systems which are mounted in different ways. Linux also does not need a partition, a filesystem can be placed directly on /dev/sdx.

I have HUGE RISK AVERSION. Meaning that measure 40x then review,
measure 50x more, review.. backup backup backup.. try it...

Hm, sounds like you overkilled your disk with too many backups.

So I'd like to use testdisk to suck out the data I can to say either another 4TB drive I attach to the box and then I can review what I've recovered and go from there.. then test out the steps to recover the disk..

Sounds good. Step one: Suck out data = use dd_rescue

ie: do the x,y,z in testdisk to reset it and then see if it is mountable...

I just can seem to find a GOOD STEP BY STEP EXPLAIN it in full for testdisk , that after I am at the point, below, do, x,y,z to reset
things so the drive is mountable...

There are too many if then else in the process to write them down in a step by step tutorial. Many things are useless and do not apply if a previous "else" matches.

testdisk is for partitions and testdisk tells you:

NTFS filesystem need to be repaired.

that your filesystem is broken. So testdisk is useless.

I don't know how reliable the linux ntfs repair modes are, if your mirror disk is ready you maybe could give an MS system a try.

Regards

Kai

--- GoldED+/LNX 1.1.4.7
* Origin: Monobox (2:240/77)

On Wed 11-Sep-2019 10:26 , Dan Clough@1:123/115.0 said to Rick Christian:

I have HUGE RISK AVERSION. Meaning that measure 40x then review, measure 50x more, review.. backup backup backup.. try it...

Maybe that's a little excessive?

In the old days they called a mouse a mice.
--- CNet/5
* Origin: 1:275/201.0 (1:275/201.30)