TerraMaster Forum

Hi,

I'd like to report a firmware bug I root-caused on my TNAS (TOS, kernel 6.12.63+), affecting the small system RAID1 array (md9, the "core" array per /etc/tos/scripts/scripts, signature UTOSCORE-X86-S64).

Symptom: After a reboot, md9 sometimes comes up degraded with only one of its two mirror members (/proc/mdstat shows [2/1] [_U] instead of [2/2] [UU]), even though the missing disk's on-disk superblock is completely valid and up to date (mdadm --examine shows correct Events count and Array State: AA matching the pre-reboot fully-synced state). A manual mdadm --add /dev/md9 /dev/sdXX immediately re-adds it and resync completes in ~30 seconds, confirming the disk and data were fine all along — this is a boot-time assembly bug, not a disk failure. The larger data array (md0) has not shown this issue.

Root cause (traced by extracting and inspecting /boot/initrd.img read-only):

/init runs udevadm trigger && udevadm settle, then hands off to /etc/tos/scripts/mntcore, a plain shell script that assembles md9. That script:
1. Builds its candidate-disk list (RAIDHDD) via ls /dev/sd* once, near the top of the script — before udev is guaranteed to have finished enumerating every physical disk.
2. Force-assembles md9 with the first disk that passes a CheckSystem integrity check.
3. Only afterward calls loopCoreMember to hot-add the other candidate disk(s) from that same (possibly-incomplete) list, with just 10 fast retries and no sleep/backoff between them.

If the second physical disk hasn't appeared in /dev/sd* yet when that list is built (e.g. it enumerates a little later on that boot), it's never considered for re-add at all, and md9 boots permanently degraded until someone manually re-adds it.

By contrast, md0's assembly (done later, in userspace, via tnas-mntdata.service / /etc/tos/scripts/mntdata) explicitly names both member disks/partitions in its mdadm -A call instead of relying on a possibly-stale enumeration snapshot, so it doesn't hit this race — which matches what I've observed (only md9 recurs).

Suggested fix: either re-run/refresh the RAIDHDD disk enumeration immediately before the loopCoreMember retry loop (not just once near the top of the script), or add a short sleep/backoff between the 10 retries so a late-enumerating disk still gets picked up.

Workaround I'm using in the meantime (userspace only, doesn't touch the initrd): a systemd oneshot service that runs ~20s after every boot, checks both md0 and md9 via mdadm --detail, and re-adds any missing mirror member automatically. This fully masks the symptom but obviously doesn't fix the underlying race.

Also noting two smaller issues found while debugging, in case they're useful:
- mdadm: ARRAY line /dev/md9 has no identity information warning on every mdadm invocation — the mdadm.conf ARRAY lines for md9 appear to lack UUID/identity info.
- The shipped mdadm (v4.0, 2017) fails --grow/adding a device to an array with md: couldn't update array info. -22 on this kernel (6.12.63+) — its legacy SET_ARRAY_INFO ioctl usage seems incompatible; using the sysfs interface (/sys/block/mdX/md/raid_disks) worked as a workaround when I needed to actually grow an array from 1→2 devices.

Happy to provide full dmesg, mdstat, and mdadm --examine output from an affected boot if that helps triage. Let me know if you'd like me to open a formal support ticket instead of/in addition to this forum post.

Thanks!

Update (2026-09-05): Found a related side-effect from the same reboot/update cycle. After the TOS7 update that triggered the 2026-09-04 22:01 reboot (see above), SMACK started actively blocking outbound network access for systemd-managed processes that aren't part of the base TOS image. Base binaries (ssh, curl, etc.) carry a security.SMACK64EXEC=tos-32f0 xattr that transitions them to a fully-networked label on exec; anything installed later via apt (in my case, rclone) has no such xattr, so systemd spawns it with the default floor label "_", which has no rule granting access to net-32f0 — every DNS lookup and outbound connection then fails with permission denied (confirmed via smk_ipv4_check action=denied in the audit log).

Worked around it on my end (routing the affected job through a locally-scoped SSH loopback so it inherits a properly-labeled process), so no ask here — just flagging it in case this was an intentional hardening step in this TOS7 release and not previously documented, since it could silently break anyone's custom cron/systemd jobs that call non-stock binaries.

Thank you for your thorough testing and feedback. Please provide the complete dmesg, mdstat, and mdadm --examine output from the affected boot, and send it to our technical support email (support(at)terra-master.com). We will analyze it.