raw
# zpool status -v
  pool: serverh
 state: ONLINE
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
  scan: resilvered 0B in 02:02:41 with 0 errors on Sat Dec 17 17:29:15 2022
config:

        NAME                                      STATE     READ WRITE CKSUM
        serverh                                   ONLINE       0     0     0
          mirror-0                                ONLINE       0     0     0
            3c8b2333-7b7a-46f8-b515-d9163679a83b  ONLINE       0     0     4
            973f071f-ecf4-47da-a2c7-243d918cbbc7  ONLINE       0     0     4
        special
          mirror-3                                ONLINE       0     0     0
            84ea3d26-04                           ONLINE       0     0     0
            61b88409-04                           ONLINE       0     0     0
        logs
          mirror-2                                ONLINE       0     0     0
            84ea3d26-01                           ONLINE       0     0     0
            61b88409-01                           ONLINE       0     0     0
        cache
          84ea3d26-03                             ONLINE       0     0     0
          61b88409-03                             ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

        serverh/virt-windowsvm1:<0x1>

  pool: servers
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: scrub repaired 0B in 00:00:15 with 0 errors on Sun Dec 11 00:24:20 2022
config:

        NAME             STATE     READ WRITE CKSUM
        servers          ONLINE       0     0     0
          mirror-0       ONLINE       0     0     0
            84ea3d26-02  ONLINE       0     0     0
            61b88409-02  ONLINE       0     0     0

errors: No known data errors

# zfs get all | grep windows
serverh/virt-windowsvm0                         type                  volume                                   -
serverh/virt-windowsvm0                         creation              Fri Dec  2 11:48 2022                    -
serverh/virt-windowsvm0                         used                  24.2G                                    -
serverh/virt-windowsvm0                         available             354G                                     -
serverh/virt-windowsvm0                         referenced            19.8G                                    -
serverh/virt-windowsvm0                         compressratio         1.19x                                    -
serverh/virt-windowsvm0                         reservation           none                                     default
serverh/virt-windowsvm0                         volsize               40G                                      local
serverh/virt-windowsvm0                         volblocksize          8K                                       default
serverh/virt-windowsvm0                         checksum              on                                       default
serverh/virt-windowsvm0                         compression           lz4                                      inherited from serverh
serverh/virt-windowsvm0                         readonly              off                                      default
serverh/virt-windowsvm0                         createtxg             10406178                                 -
serverh/virt-windowsvm0                         copies                1                                        default
serverh/virt-windowsvm0                         refreservation        none                                     local
serverh/virt-windowsvm0                         guid                  11207487241737076084                     -
serverh/virt-windowsvm0                         primarycache          all                                      default
serverh/virt-windowsvm0                         secondarycache        all                                      default
serverh/virt-windowsvm0                         usedbysnapshots       4.41G                                    -
serverh/virt-windowsvm0                         usedbydataset         19.8G                                    -
serverh/virt-windowsvm0                         usedbychildren        0B                                       -
serverh/virt-windowsvm0                         usedbyrefreservation  0B                                       -
serverh/virt-windowsvm0                         logbias               latency                                  default
serverh/virt-windowsvm0                         objsetid              13322                                    -
serverh/virt-windowsvm0                         dedup                 off                                      default
serverh/virt-windowsvm0                         mlslabel              none                                     default
serverh/virt-windowsvm0                         sync                  always                                   local
serverh/virt-windowsvm0                         refcompressratio      1.20x                                    -
serverh/virt-windowsvm0                         written               13.8G                                    -
serverh/virt-windowsvm0                         logicalused           28.6G                                    -
serverh/virt-windowsvm0                         logicalreferenced     23.6G                                    -
serverh/virt-windowsvm0                         volmode               default                                  default
serverh/virt-windowsvm0                         snapshot_limit        none                                     default
serverh/virt-windowsvm0                         snapshot_count        none                                     default
serverh/virt-windowsvm0                         snapdev               hidden                                   default
serverh/virt-windowsvm0                         context               none                                     default
serverh/virt-windowsvm0                         fscontext             none                                     default
serverh/virt-windowsvm0                         defcontext            none                                     default
serverh/virt-windowsvm0                         rootcontext           none                                     default
serverh/virt-windowsvm0                         redundant_metadata    all                                      default
serverh/virt-windowsvm0                         encryption            aes-256-gcm                              -
serverh/virt-windowsvm0                         keylocation           none                                     default
serverh/virt-windowsvm0                         keyformat             raw                                      -
serverh/virt-windowsvm0                         pbkdf2iters           0                                        default
serverh/virt-windowsvm0                         encryptionroot        serverh                                  -
serverh/virt-windowsvm0                         keystatus             available                                -
serverh/virt-windowsvm0@20221203_fresh_install  type                  snapshot                                 -
serverh/virt-windowsvm0@20221203_fresh_install  creation              Sat Dec  3 19:34 2022                    -
serverh/virt-windowsvm0@20221203_fresh_install  used                  4.41G                                    -
serverh/virt-windowsvm0@20221203_fresh_install  referenced            10.4G                                    -
serverh/virt-windowsvm0@20221203_fresh_install  compressratio         1.14x                                    -
serverh/virt-windowsvm0@20221203_fresh_install  volsize               40G                                      local
serverh/virt-windowsvm0@20221203_fresh_install  createtxg             10439575                                 -
serverh/virt-windowsvm0@20221203_fresh_install  guid                  14956087430293257985                     -
serverh/virt-windowsvm0@20221203_fresh_install  primarycache          all                                      default
serverh/virt-windowsvm0@20221203_fresh_install  secondarycache        all                                      default
serverh/virt-windowsvm0@20221203_fresh_install  defer_destroy         off                                      -
serverh/virt-windowsvm0@20221203_fresh_install  userrefs              0                                        -
serverh/virt-windowsvm0@20221203_fresh_install  objsetid              46535                                    -
serverh/virt-windowsvm0@20221203_fresh_install  mlslabel              none                                     default
serverh/virt-windowsvm0@20221203_fresh_install  refcompressratio      1.14x                                    -
serverh/virt-windowsvm0@20221203_fresh_install  written               10.4G                                    -
serverh/virt-windowsvm0@20221203_fresh_install  logicalreferenced     11.8G                                    -
serverh/virt-windowsvm0@20221203_fresh_install  context               none                                     default
serverh/virt-windowsvm0@20221203_fresh_install  fscontext             none                                     default
serverh/virt-windowsvm0@20221203_fresh_install  defcontext            none                                     default
serverh/virt-windowsvm0@20221203_fresh_install  rootcontext           none                                     default
serverh/virt-windowsvm0@20221203_fresh_install  encryption            aes-256-gcm                              -
serverh/virt-windowsvm0@20221203_fresh_install  encryptionroot        serverh                                  -
serverh/virt-windowsvm0@20221203_fresh_install  keystatus             available                                -
serverh/virt-windowsvm1                         type                  volume                                   -
serverh/virt-windowsvm1                         creation              Sat Dec  3 19:41 2022                    -
serverh/virt-windowsvm1                         used                  183G                                     -
serverh/virt-windowsvm1                         available             354G                                     -
serverh/virt-windowsvm1                         referenced            183G                                     -
serverh/virt-windowsvm1                         compressratio         1.00x                                    -
serverh/virt-windowsvm1                         reservation           none                                     default
serverh/virt-windowsvm1                         volsize               400G                                     local
serverh/virt-windowsvm1                         volblocksize          8K                                       default
serverh/virt-windowsvm1                         checksum              on                                       default
serverh/virt-windowsvm1                         compression           lz4                                      inherited from serverh
serverh/virt-windowsvm1                         readonly              off                                      default
serverh/virt-windowsvm1                         createtxg             10439708                                 -
serverh/virt-windowsvm1                         copies                1                                        default
serverh/virt-windowsvm1                         refreservation        none                                     local
serverh/virt-windowsvm1                         guid                  2664709986551831935                      -
serverh/virt-windowsvm1                         primarycache          all                                      default
serverh/virt-windowsvm1                         secondarycache        all                                      default
serverh/virt-windowsvm1                         usedbysnapshots       0B                                       -
serverh/virt-windowsvm1                         usedbydataset         183G                                     -
serverh/virt-windowsvm1                         usedbychildren        0B                                       -
serverh/virt-windowsvm1                         usedbyrefreservation  0B                                       -
serverh/virt-windowsvm1                         logbias               latency                                  default
serverh/virt-windowsvm1                         objsetid              46423                                    -
serverh/virt-windowsvm1                         dedup                 off                                      default
serverh/virt-windowsvm1                         mlslabel              none                                     default
serverh/virt-windowsvm1                         sync                  always                                   local
serverh/virt-windowsvm1                         refcompressratio      1.00x                                    -
serverh/virt-windowsvm1                         written               183G                                     -
serverh/virt-windowsvm1                         logicalused           182G                                     -
serverh/virt-windowsvm1                         logicalreferenced     182G                                     -
serverh/virt-windowsvm1                         volmode               default                                  default
serverh/virt-windowsvm1                         snapshot_limit        none                                     default
serverh/virt-windowsvm1                         snapshot_count        none                                     default
serverh/virt-windowsvm1                         snapdev               hidden                                   default
serverh/virt-windowsvm1                         context               none                                     default
serverh/virt-windowsvm1                         fscontext             none                                     default
serverh/virt-windowsvm1                         defcontext            none                                     default
serverh/virt-windowsvm1                         rootcontext           none                                     default
serverh/virt-windowsvm1                         redundant_metadata    all                                      default
serverh/virt-windowsvm1                         encryption            aes-256-gcm                              -
serverh/virt-windowsvm1                         keylocation           none                                     default
serverh/virt-windowsvm1                         keyformat             raw                                      -
serverh/virt-windowsvm1                         pbkdf2iters           0                                        default
serverh/virt-windowsvm1                         encryptionroot        serverh                                  -
serverh/virt-windowsvm1                         keystatus             available                                -

# dmesg | grep 'Buffer I/O ' | tail
[624196.254744] Buffer I/O error on dev zd0, logical block 42448516, async page read
[624197.263609] Buffer I/O error on dev zd0, logical block 42448516, async page read
[624198.264699] Buffer I/O error on dev zd0, logical block 42448516, async page read
[624198.265712] Buffer I/O error on dev zd0, logical block 42448516, async page read
[624199.329293] Buffer I/O error on dev zd0, logical block 42448516, async page read
[624200.336659] Buffer I/O error on dev zd0, logical block 42448516, async page read
[624201.342334] Buffer I/O error on dev zd0, logical block 42448516, async page read
[624202.346839] Buffer I/O error on dev zd0, logical block 42448516, async page read
[708214.312688] Buffer I/O error on dev zd0, logical block 42025802, async page read
[708214.772361] Buffer I/O error on dev zd0, logical block 42025802, async page read

# zgrep 'Buffer I/O error on dev' syslog* | awk '{ print $15 }' | sort | uniq -c
     35 42025802,
      1 42025803,
     79 42052336,
      7 42052337,
    155 42345494,
     53 42369212,
      6 42369213,
     21 42448516,
      1 42448517,
      4 43413716,
     11 44048162,
      5 45643964,

# dpkg -l | grep -e linux -e zfs
ii  binutils-x86-64-linux-gnu            2.35.2-2                       amd64        GNU binary utilities, for x86-64-linux-gnu target
ii  console-setup-linux                  1.205                          all          Linux specific part of console-setup
ii  firmware-linux-free                  20200122-1                     all          Binary firmware for various drivers in the Linux kernel
ii  libnvpair3linux                      2.1.6-3~bpo11+1                amd64        Solaris name-value library for Linux
ii  libselinux1:amd64                    3.1-3                          amd64        SELinux runtime shared libraries
ii  libuutil3linux                       2.1.6-3~bpo11+1                amd64        Solaris userland utility library for Linux
ii  libvirt-daemon-driver-storage-zfs    7.0.0-3                        amd64        Virtualization daemon ZFS storage driver
ii  libzfs4linux                         2.1.6-3~bpo11+1                amd64        OpenZFS filesystem library for Linux - general support
ii  libzpool5linux                       2.1.6-3~bpo11+1                amd64        OpenZFS pool library for Linux
ii  linux-base                           4.6                            all          Linux image base package
ii  linux-compiler-gcc-10-x86            5.10.149-2                     amd64        Compiler for Linux on x86 (meta-package)
ii  linux-headers-5.10.0-19-amd64        5.10.149-2                     amd64        Header files for Linux 5.10.0-19-amd64
ii  linux-headers-5.10.0-19-common       5.10.149-2                     all          Common header files for Linux 5.10.0-19
ii  linux-headers-6.0.0-0.deb11.2-amd64  6.0.3-1~bpo11+1                amd64        Header files for Linux 6.0.0-0.deb11.2-amd64
ii  linux-headers-6.0.0-0.deb11.2-common 6.0.3-1~bpo11+1                all          Common header files for Linux 6.0.0-0.deb11.2
ii  linux-headers-amd64                  6.0.3-1~bpo11+1                amd64        Header files for Linux amd64 configuration (meta-package)
ii  linux-image-5.10.0-19-amd64          5.10.149-2                     amd64        Linux 5.10 for 64-bit PCs (signed)
ii  linux-image-6.0.0-0.deb11.2-amd64    6.0.3-1~bpo11+1                amd64        Linux 6.0 for 64-bit PCs (signed)
ii  linux-image-amd64                    6.0.3-1~bpo11+1                amd64        Linux for 64-bit PCs (meta-package)
ii  linux-kbuild-5.10                    5.10.149-2                     amd64        Kbuild infrastructure for Linux 5.10
ii  linux-kbuild-6.0                     6.0.3-1~bpo11+1                amd64        Kbuild infrastructure for Linux 6.0
ii  linux-libc-dev:amd64                 5.10.149-2                     amd64        Linux support headers for userspace development
ii  util-linux                           2.36.1-8+deb11u1               amd64        miscellaneous system utilities
ii  zfs-dkms                             2.1.6-3~bpo11+1                all          OpenZFS filesystem kernel modules for Linux
ii  zfsutils-linux                       2.1.6-3~bpo11+1                amd64        command-line tools to manage OpenZFS filesystems

# uname -a
Linux server.example.com 6.0.0-0.deb11.2-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.0.3-1~bpo11+1 (2022-10-29) x86_64 GNU/Linux

# smartctl -a /dev/sda
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-6.0.0-0.deb11.2-amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Toshiba 3.5" MG04ACA... Enterprise HDD
Device Model:     TOSHIBA MG04ACA400EY
Serial Number:    Y7MPK03FF7GB
LU WWN Device Id: 5 000039 82c302228
Firmware Version: FQ1B
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sat Dec 17 20:39:44 2022 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  120) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 473) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000b   100   100   050    Pre-fail  Always       -       0
  2 Throughput_Performance  0x0005   100   100   050    Pre-fail  Offline      -       0
  3 Spin_Up_Time            0x0027   100   100   001    Pre-fail  Always       -       7206
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       26
  5 Reallocated_Sector_Ct   0x0033   100   100   050    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000b   100   100   050    Pre-fail  Always       -       0
  8 Seek_Time_Performance   0x0005   100   100   050    Pre-fail  Offline      -       0
  9 Power_On_Hours          0x0032   007   007   000    Old_age   Always       -       37517
 10 Spin_Retry_Count        0x0033   100   100   030    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       25
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       1436322
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       11
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       257
194 Temperature_Celsius     0x0022   100   100   000    Old_age   Always       -       35 (Min/Max 23/51)
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   200   000    Old_age   Always       -       0
220 Disk_Shift              0x0002   100   100   000    Old_age   Always       -       0
222 Loaded_Hours            0x0032   007   007   000    Old_age   Always       -       37353
223 Load_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
224 Load_Friction           0x0022   100   100   000    Old_age   Always       -       0
226 Load-in_Time            0x0026   100   100   000    Old_age   Always       -       632
240 Head_Flying_Hours       0x0001   100   100   001    Pre-fail  Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%      9767         -
# 2  Short offline       Completed without error       00%      9766         -
# 3  Extended offline    Completed without error       00%      9764         -
# 4  Short offline       Completed without error       00%      9753         -
# 5  Extended offline    Completed without error       00%      9739         -
# 6  Short offline       Completed without error       00%      9728         -
# 7  Extended offline    Completed without error       00%      9713         -
# 8  Extended offline    Completed without error       00%      9694         -
# 9  Short offline       Completed without error       00%         1         -
#10  Short offline       Completed without error       00%         0         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

# smartctl -a /dev/sdb
smartctl 7.2 2020-12-30 r5155 [x86_64-linux-6.0.0-0.deb11.2-amd64] (local build)
Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Toshiba 3.5" MG04ACA... Enterprise HDD
Device Model:     TOSHIBA MG04ACA400EY
Serial Number:    Y7MTK02FF7GB
LU WWN Device Id: 5 000039 82c501b20
Firmware Version: FQ1B
User Capacity:    4,000,787,030,016 bytes [4.00 TB]
Sector Sizes:     512 bytes logical, 4096 bytes physical
Rotation Rate:    7200 rpm
Form Factor:      3.5 inches
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS (minor revision not indicated)
SATA Version is:  SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Sat Dec 17 20:39:45 2022 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (  120) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   2) minutes.
Extended self-test routine
recommended polling time:        ( 477) minutes.
SCT capabilities:              (0x003d) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000b   100   100   050    Pre-fail  Always       -       0
  2 Throughput_Performance  0x0005   100   100   050    Pre-fail  Offline      -       0
  3 Spin_Up_Time            0x0027   100   100   001    Pre-fail  Always       -       7015
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       26
  5 Reallocated_Sector_Ct   0x0033   100   100   050    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000b   100   100   050    Pre-fail  Always       -       0
  8 Seek_Time_Performance   0x0005   100   100   050    Pre-fail  Offline      -       0
  9 Power_On_Hours          0x0032   007   007   000    Old_age   Always       -       37555
 10 Spin_Retry_Count        0x0033   100   100   030    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       25
191 G-Sense_Error_Rate      0x0032   100   100   000    Old_age   Always       -       15220635
192 Power-Off_Retract_Count 0x0032   100   100   000    Old_age   Always       -       13
193 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       269
194 Temperature_Celsius     0x0022   100   100   000    Old_age   Always       -       32 (Min/Max 22/51)
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   200   253   000    Old_age   Always       -       0
220 Disk_Shift              0x0002   100   100   000    Old_age   Always       -       0
222 Loaded_Hours            0x0032   007   007   000    Old_age   Always       -       37359
223 Load_Retry_Count        0x0032   100   100   000    Old_age   Always       -       0
224 Load_Friction           0x0022   100   100   000    Old_age   Always       -       0
226 Load-in_Time            0x0026   100   100   000    Old_age   Always       -       639
240 Head_Flying_Hours       0x0001   100   100   001    Pre-fail  Offline      -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Short offline       Completed without error       00%      9805         -
# 2  Short offline       Completed without error       00%      9804         -
# 3  Extended offline    Completed without error       00%      9802         -
# 4  Short offline       Completed without error       00%      9791         -
# 5  Extended offline    Completed without error       00%      9735         -
# 6  Short offline       Completed without error       00%      9720         -
# 7  Extended offline    Completed without error       00%      9713         -
# 8  Extended offline    Completed without error       00%      9694         -
# 9  Short offline       Completed without error       00%         1         -
#10  Short offline       Completed without error       00%         0         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

# zpool history serverh
History for 'serverh':
2021-10-12.08:15:27 zpool create -o ashift=12 -O encryption=aes-256-gcm -O keylocation=file:///usr/share/zfs_key.wpool -O keyformat=raw -O acltype=posixacl -O compression=lz4 -O dnodesize=auto -O normalization=formD -O relatime=on -O xattr=sa -O mountpoint=none serverh /dev/disk/by-partuuid/3c8b2333-7b7a-46f8-b515-d9163679a83b
2021-10-12.08:15:56 zfs create -o mountpoint=/mnt/new/home serverh/home
2021-10-12.08:16:00 zfs create -o mountpoint=/mnt/new/var serverh/var
2021-10-12.08:16:05 zfs create -o mountpoint=/mnt/new/lxc serverh/lxc
2021-10-12.08:16:09 zfs create serverh/lxc/mail
2021-10-12.08:16:09 zfs create serverh/lxc/misc
2021-10-12.08:16:15 zfs create serverh/lxc/web
2021-10-14.21:45:59 zfs set mountpoint=/home serverh/home
2021-10-14.21:46:06 zfs set mountpoint=/var serverh/var
2021-10-14.21:47:37 zpool import -c /etc/zfs/zpool.cache -aN
2021-10-14.21:47:37 zfs load-key -a
2021-10-14.21:52:46 zpool import -c /etc/zfs/zpool.cache -aN
2021-10-14.21:52:46 zfs load-key -a
2021-10-14.21:58:16 zpool import -c /etc/zfs/zpool.cache -aN
2021-10-14.21:58:16 zfs load-key -a
2021-10-14.22:13:44 zpool import -c /etc/zfs/zpool.cache -aN
2021-10-14.22:13:44 zfs load-key -a
2021-10-14.22:20:37 zpool import -c /etc/zfs/zpool.cache -aN
2021-10-14.22:20:37 zfs load-key -a
2021-10-14.22:53:34 zpool import -c /etc/zfs/zpool.cache -aN
2021-10-14.22:53:34 zfs load-key -a
2021-10-14.22:54:54 zfs create -o mountpoint=/mnt/new/tmp serverh/tmp
2021-10-14.22:56:32 zfs set mountpoint=/tmp serverh/tmp
2021-10-14.22:58:09 zpool import -c /etc/zfs/zpool.cache -aN
2021-10-14.22:58:09 zfs load-key -a
2021-10-14.23:14:44 zpool add serverh log /dev/disk/by-partuuid/84ea3d26-01
2021-10-14.23:15:04 zpool add serverh cache /dev/disk/by-partuuid/84ea3d26-03
2021-10-14.23:43:35 zfs set mountpoint=/lxc/mail serverh/lxc/mail
2021-10-17.10:26:25 zfs set mountpoint=/lxc/web serverh/lxc/web
2021-10-17.11:51:03 zpool set autotrim=on serverh
2021-10-18.22:13:44 zfs set mountpoint=/lxc/misc serverh/lxc/misc
2021-10-18.22:24:23 zfs set mountpoint=/mnt/new/lxc/misc serverh/lxc/misc
2021-10-19.08:09:47 zfs create -o mountpoint=/mnt/new/lxc/experimental serverh/lxc/experimental
2021-10-19.21:31:57 zfs set mountpoint=/lxc/misc serverh/lxc/misc
2021-10-19.21:37:47 zfs create -o mountpoint=/lxc/misc/rootfs/var/lib/docker serverh/lxc/misc-docker
2021-10-19.21:58:48 zfs destroy serverh/lxc/misc-docker
2021-10-19.22:34:08 zfs set mountpoint=/lxc/experimental serverh/lxc/experimental
2021-10-19.22:39:34 zfs set mountpoint=/lxc serverh/lxc
2021-10-20.21:04:32 zpool attach serverh /dev/disk/by-partuuid/84ea3d26-01 /dev/disk/by-partuuid/61b88409-01
2021-10-20.21:50:34 zpool add serverh cache /dev/disk/by-partuuid/61b88409-03
2021-10-20.21:52:19 zpool attach serverh /dev/disk/by-partuuid/3c8b2333-7b7a-46f8-b515-d9163679a83b /dev/disk/by-partuuid/973f071f-ecf4-47da-a2c7-243d918cbbc7
2021-11-14.00:24:08 zpool scrub serverh
2021-12-12.00:24:09 zpool scrub serverh
2022-01-09.00:24:12 zpool scrub serverh
2022-02-13.00:24:12 zpool scrub serverh
2022-03-13.00:24:09 zpool scrub serverh
2022-04-10.00:24:04 zpool scrub serverh
2022-05-08.00:24:10 zpool scrub serverh
2022-06-12.00:24:10 zpool scrub serverh
2022-07-10.00:24:09 zpool scrub serverh
2022-08-14.00:24:06 zpool scrub serverh
2022-09-11.00:24:05 zpool scrub serverh
2022-10-09.00:24:10 zpool scrub serverh
2022-11-13.00:24:12 zpool scrub serverh
2022-12-01.21:44:25 zfs create -o mountpoint=/lxc/virt serverh/lxc/virt
2022-12-02.11:40:24 zfs create -o mountpoint=/var/lib/libvirt serverh/libvirt
2022-12-02.11:47:23 zfs create -V 41943040K serverh/windowsvm0
2022-12-02.11:48:02 zfs destroy serverh/windowsvm0
2022-12-02.11:48:12 zfs create -V 41943040K serverh/virt-windowsvm0
2022-12-03.19:34:35 zfs snap serverh/virt-windowsvm0@20221203_fresh_install
2022-12-03.19:41:21 zfs create -V 419430400K serverh/virt-windowsvm1
2022-12-05.10:13:28 zfs destroy serverh/lxc/virt
2022-12-06.17:45:02 zfs set sync=always serverh/virt-windowsvm0
2022-12-06.17:45:07 zfs set sync=always serverh/virt-windowsvm1
2022-12-11.00:24:04 zpool scrub serverh
2022-12-17.13:47:22 zfs rename serverh/virt-windowsvm0 serverh/virt-windowsvm0.old
2022-12-17.14:41:21 zfs rename serverh/virt-windowsvm0.old serverh/virt-windowsvm0
2022-12-17.14:42:50 zfs clone serverh/virt-windowsvm0@20221203_fresh_install serverh/virt-windowsvm0-fresh
2022-12-17.14:45:45 zfs set refreservation=none serverh/virt-windowsvm1
2022-12-17.14:46:19 zfs set refreservation=none serverh/virt-windowsvm0
2022-12-17.15:19:29 zpool detach serverh 84ea3d26-01
2022-12-17.15:20:54 zpool remove serverh 61b88409-01
2022-12-17.15:26:12 zpool add serverh log /dev/disk/by-partuuid/84ea3d26-01
2022-12-17.15:26:41 zpool attach serverh /dev/disk/by-partuuid/84ea3d26-01 /dev/disk/by-partuuid/61b88409-01
2022-12-17.15:31:36 zpool add serverh -o ashift=12 special mirror /dev/disk/by-partuuid/84ea3d26-04 /dev/disk/by-partuuid/61b88409-04
2022-12-17.17:38:25 zpool clear serverh
2022-12-17.20:14:57 zfs destroy serverh/virt-windowsvm0-fresh