U12-322 shows very slow performance and fails any

CPU, memory, fan, process, services status, system log, tempreture
Locked
User avatar
las68
Posts: 35
Joined: 14 Oct 2022, 13:52

U12-322 shows very slow performance and fails any

Post by las68 »

U12-322
TOS 5.0.176-00227
2023-02-06_10-10-38.png
NAS fails any attempts to save files bigger than 3GB on it.
CPU shows high load with no any user connections. I've described it here on the forum three months ago, but did not get any fixes. Now even reboot won't help.

Screen Interface is not working. Clicking on icon cause nothing, Clicking on icon with right button shows thin white line on screen. WTF?
2023-02-06_10-23-55.jpg
Now we redirected our backups to another NAS.

Our experience with Terramaster for enterprise is negative. I would never recommend anyone to buy this for enterprise use. Expensive waste of money.
Software troubles has no fixes for a long time and getting even worse.
User avatar
TMSupport
TerraMaster Team
Posts: 2314
Joined: 13 Dec 2019, 15:15

Re: U12-322 shows very slow performance and fails any

Post by TMSupport »

1. NAS fails any attempts to save files bigger than 3GB on it.
How do you store files larger than 3gb?
2. CPU shows high load with no any user connections.
Please check which process is occupying high CPU.
3. Screen Interface is not working. Clicking on icon cause nothing, Clicking on icon with right button shows thin white line on screen.
The desktop may be loading slowly due to system resource overload.
Please try updating to TOS 5.1.17.
To contact our team, please send email to following addresses, remember to replace (at) with @
Technical team: support(at)terra-master.com (for technical support)
Service team: service(at)terra-master.com (for purchasing, return, replacement, RMA service)
User avatar
TMroy
TerraMaster Team
Posts: 2562
Joined: 10 Mar 2020, 14:04
China

Re: U12-322 shows very slow performance and fails any

Post by TMroy »

{L_BUTTON_AT}las68

Show from the screen shot, you created a bond for your networking, please make sure your router support link aggregation feature. You can delete the bond and verify if it is because of the bond issue.

One more thing, google Chrome and Safari browser are recommended browser, please change your browser and try again.

By the way, I know WTF is nice word, but do not do that again.
To contact our team, please send email to following addresses, remember to replace (at) with @:
Support team: support(at)terra-master.com (for technical support only)
Service team: service(at)terra-master.com (for purchasing, return, replacement, RMA service)
User avatar
las68
Posts: 35
Joined: 14 Oct 2022, 13:52

Re: U12-322 shows very slow performance and fails any

Post by las68 »

1. LAG is not related at all.

2.Further investigation show problems with BTRFS defragmentation. For unknown reason it started at backup time, and cause severe slow down and initiator refused back it up to destination. Especially longer files, small backups luckily goes through.

I believe Storage pool / Settings / Fast synchronization affects this. Now it set in Slow synchronization. And this problem won't show at least last two days.

3. It would be nice idea, to enable BTRFS defrag procedure to be scheduled at late night at 'Scheduled Tasks' with logged result in 'Logs'. Now I could not see this ability, or it is unclear

4. Still don't know why it requires for almost 48 hours to stay busy and non-functional. Hard reboot only makes things only worse since it gives additional sanity check to fs.

5. Systems work in business should much more reliable than now. I, as customer, should be assured the data are safe and complete. So beta version installation is a bad idea, it would never be approved by security on production site. When do you plan to release it ? Is there any more detailed description of fix patch (consequent 'fixed some issues',.. fixed some issues)?


P.S. WTF - World Taekwondo Foundation. And what do you mean? ))
User avatar
TMroy
TerraMaster Team
Posts: 2562
Joined: 10 Mar 2020, 14:04
China

Re: U12-322 shows very slow performance and fails any

Post by TMroy »

As the btrfs file system defragment was enabled by default in the last releases, so as soon as the system reboot and file system is mounted, the defragment auto start, which causes the drives busy and the system to run sluggishly.

The file system defragment will be disabled by default in the coming release. TOS 5.1 final version will be released in about one week, please wait.
To contact our team, please send email to following addresses, remember to replace (at) with @:
Support team: support(at)terra-master.com (for technical support only)
Service team: service(at)terra-master.com (for purchasing, return, replacement, RMA service)
User avatar
crisisacting
Posts: 253
Joined: 20 Jan 2022, 16:42

Re: U12-322 shows very slow performance and fails any

Post by crisisacting »

TMroy wrote: 07 Feb 2023, 23:35TOS 5.1 final version will be released in about one week, please wait.
Will that release be extended to all ×86/×64 models or maintain the same restrictions as the betas?
User avatar
las68
Posts: 35
Joined: 14 Oct 2022, 13:52

Re: U12-322 shows very slow performance and fails any

Post by las68 »

TMroy wrote: 07 Feb 2023, 23:35 As the btrfs file system defragment was enabled by default in the last releases, so as soon as the system reboot and file system is mounted, the defragment auto start, which causes the drives busy and the system to run sluggishly.

The file system defragment will be disabled by default in the coming release. TOS 5.1 final version will be released in about one week, please wait.
The problem is btrfs NEEDS defragmentation by design. Rare manual defrags or starting it at system reload will require too much time to bring RAID back to life (add additional time for sanity check if restart was hard). The bigger filesystem, the longer time needs to complete rebuild.

So the better way is to create a task for defrag in scheduler. In my environment for ~60TB array, it requires ~3-5 minutes if previous defrag was done a day ago. This require more time, when lot of files were created or deleted.

I added this in the scheduler:

btrfs filesystem defrag -v -r -f /Volume1 > /var/log/btrfs_defrag.log

I'll see how it works
User avatar
TMroy
TerraMaster Team
Posts: 2562
Joined: 10 Mar 2020, 14:04
China

Re: U12-322 shows very slow performance and fails any

Post by TMroy »

{L_BUTTON_AT}crisisacting

Yes, the final version will be available for all x.86 models.
To contact our team, please send email to following addresses, remember to replace (at) with @:
Support team: support(at)terra-master.com (for technical support only)
Service team: service(at)terra-master.com (for purchasing, return, replacement, RMA service)
User avatar
las68
Posts: 35
Joined: 14 Oct 2022, 13:52

Re: U12-322 shows very slow performance and fails any

Post by las68 »

I remember one strange QNAP problem (i.e TS459). RAID rebuild after change of failed disk was 2-3 days (4TB). And there were solution for that


#cat /proc/sys/dev/raid/speed_limit_max
200000
#cat /proc/sys/dev/raid/speed_limit_min
1000

We increase value of speed_limit_min to 50000 by issuing command:

echo 50000 >/proc/sys/dev/raid/speed_limit_min

Rebuild shortens to 3-4 hours, and rates were higher in
# watch cat /proc/mdstat

Terramaster soft RAID is very close to QNAP, but speed_limit_min is set up in 100К

# cat /proc/sys/dev/raid/speed_limit_max
200000
# cat /proc/sys/dev/raid/speed_limit_min
100000

The higher this value, the more the load on system by any job of raid rebuild or btrfs defragmentation.
speed_limit_min, I believe, should be lowered to 60000-70000.

This value should be carefully tuned. It depends on CPU performance, RAM and btrfs volume sizes
Locked