Unable to copy large amounts of data (solved, ish)

[ChrisH]

Check for bad blocks using SMART (only works over SATA & IDE, not USB). I found that AmigaOS seems unable to correctly handle some bad blocks (possibly Pending ones) that are handled fine under other OSes like Windows, generating error messages EXACTLY like those in the first post.

I was going to raise this issue on the OS4 mailing list since September last year, but TBH I have been too busy to find time to do properly, so I haven't done at all.

On the Amiga you can check for SMART errors by downloading smartmontools.lha, and then issuing a Shell command similar to this:
SmartCtl -a sii3114ide.device:0
(the device & unit number will probably be different)

[daveyw]

Check for bad blocks using SMART (only works over SATA & IDE, not USB).

Thanks Chris. Yes, I've run SmartCTL on my PATA drive, it reports all OK.

[ChrisH]

Check for bad blocks using SMART (only works over SATA & IDE, not USB).

it reports all OK.

Are you sure? What does it report in the RAW_VALUE column for the following rows:
Reallocated_Sector_Ct
Current_Pending_Sector
Offline_Uncorrectable
UDMA_CRC_Error_Count
Hardware_ECC_Recovered
Seek_Error_Rate
Raw_Read_Error_Rate

(Note that only the first few are related to bad blocks, and non-zero values for any of them is not necessarily bad.)

[daveyw]

Check for bad blocks using SMART (only works over SATA & IDE, not USB).

it reports all OK.

Are you sure? What does it report in the RAW_VALUE column for the following rows:
Reallocated_Sector_Ct
Current_Pending_Sector
Offline_Uncorrectable
UDMA_CRC_Error_Count
Hardware_ECC_Recovered
Seek_Error_Rate
Raw_Read_Error_Rate

(Note that only the first few are related to bad blocks, and non-zero values for any of them is not necessarily bad.)

Here's the full results:

Hidden Text - Click to Show :

smartctl version 5.33 [Cygwin AmigaOS4 cross compiler] Copyright (C) 2002-4 Bruce Allen
Home page is http://cvs.sourceforge.net/projects/smartmontools/

=== START OF INFORMATION SECTION ===
Device Model: WDC WD5000AAKB-00H8A0
Serial Number: WD-WCASYE075496
Firmware Version: 05.04E05
User Capacity: 500,107,862,016 bytes
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: 8
ATA Standard is: Exact ATA specification draft version not indicated
Local Time is: Wed Jul 24 06:42:45 2013 GMT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (11160) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 131) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 155 153 021 Pre-fail Always - 5250
4 Start_Stop_Count 0x0032 099 099 000 Old_age Always - 1132
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 100 253 000 Old_age Always - 0
9 Power_On_Hours 0x0032 086 086 000 Old_age Always - 10417
10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 099 099 000 Old_age Always - 1131
192 Power-Off_Retract_Count 0x0032 199 199 000 Old_age Always - 1129
193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 1132
194 Temperature_Celsius 0x0022 119 096 000 Old_age Always - 28
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 18
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Short offline Completed without error 00% 5 -

SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

[theamigaone]

I know its not many, but UDMA_CRC_ERROR_COUNT is at 18

Has it shown that SFS messege 18 times?

Btw what USB card are you using for the XE?

[ChrisH]

@daveyw
OK, that does indeed rule-out my bad-block theory (and makes it less likely that bad blocks were really to blame for my own problem).

Have you tried doing a FULL format of the problematic harddrive? (Assuming you are writing to it, not reading from.) For me it always stopped at exactly the same place, which would not be the case if it was just data transfer errors.

[Belxjander]

Would this be related to the SmartFileSystem 1.290 update issue as another thread on these forums?

I ran into that after several error lockups and have only just "repaired" a drive entirely by getting another one and rebuilding the filesystem after a quick format.

Would there at all be some means of adding specific anti-corruption features to the OS such as presented by the fsck tools on Linux
where a partition can have the entire metadata tree rebuilt in-memory before writing it back out to disk?

could a FileSystem startup in doing something like that for whatever "metadata block"s are required?

[ChrisH]

@daveyw
I just re-read all your posts, and it isn't clear to me whether this error only comes from the TARGET empty drive, or if it has sometimes come from the SOURCE drive. Specifically, when you were doing IDE to IDE, was it the source or target drive that had errors? (Your textual copy of the message doesn't indicate.)

Anyway, assuming it was always the target drive, please try doing a FULL format several times, making a note of the error location each time (to see if it changes).

P.S. Did you make any further progress with your RAM tests?

[daveyw]

I know its not many, but UDMA_CRC_ERROR_COUNT is at 18

Has it shown that SFS messege 18 times?

Btw what USB card are you using for the XE?

Not sure about how many times it would've appeared. It might be 18.

The USB card I'm using is this:

http://www.startech.com/Value-6-Port-US ... CI625USB2I

I reported elsewhere I couldn't get it to work, but I tried it again earlier this year and found it worked fine (at least, apart from this current problem, which I don't believe is connected to this card)

[daveyw]

@daveyw
Have you tried doing a FULL format of the problematic harddrive? (Assuming you are writing to it, not reading from.) For me it always stopped at exactly the same place, which would not be the case if it was just data transfer errors.

OK, I'll be honest, this is one of the things that really annoys me about owning an Amiga. You're telling me to do a full format. Elsewhere there are posts say never do a full format, only quick format. Which is it ??? (Expects 10 different answers to this question)