Free up space and delete files

[jollino]

Hi TMpraj, I migrated the files from Volume1 (from a user's home) to Volume3 (into a shared folder to which only that user has access) by simply dragging them from the TOS file manager as the admin user. The files now appear on Volume3 and not on Volume1, so they have moved successfully; even the file permissions were set properly.

[jollino]

Running du -ch /Volume1 confirms that it should only use about 14 TB even though df still reports 18, and the files I moved are 4.7 TB:

Code: Select all

master@Supernova:/# du -ch /Volume1 | tail -n2
14T     /Volume1
14T     total

master@Supernova:/# df -h | grep Volume
/dev/mapper/vg0-lv0   22T   19T  3.2T  86% /Volume1
/dev/mapper/vg1-lv1  7.3T  5.2T  2.1T  72% /Volume3

master@Supernova:/# du -ch /Volume3/ExtDump/ | tail -n2
4.7T    /Volume3/ExtDump/
4.7T    total
[/coda]

As you can see, the actual usage of Volume1 is 14 TB, but the OS still reports 19 TB as if I had never moved the ~5 TB.
(Volume3 has 5.2 TB used because I had already put some other data, there before moving the 4.7 TB from Volume1)

I'm stumped as I'm not sure if it depends on BTRFS, LVM or what else. Maybe I should just reboot? :-D

[jollino]

I noticed one thing. Even though I do not have snapshots enabled in TOS Backup, running btrfs subvolume show /Volume1 does show a bunch of them. They're all from before I updated to TOS6, and they also seem to be not accessible since there's no such directory inside /Volume1.

Code: Select all

master@Supernova:~# btrfs subvolume list -p /Volume1
ID 332 gen 19472 parent 5 top level 5 path @sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01
ID 333 gen 19467 parent 332 top level 332 path @sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/@apps
ID 334 gen 19468 parent 332 top level 332 path @sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/@cache
ID 335 gen 19469 parent 332 top level 332 path @sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/@desktop
ID 336 gen 19470 parent 332 top level 332 path @sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/@system
ID 337 gen 26144 parent 332 top level 332 path @sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/User
ID 338 gen 19472 parent 332 top level 332 path @sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/public
ID 339 gen 19482 parent 5 top level 5 path @sysSnapShoot/Snapshot/GMT+02-2024.09.30-01.00.01
ID 340 gen 19477 parent 339 top level 339 path @sysSnapShoot/Snapshot/GMT+02-2024.09.30-01.00.01/@apps
ID 341 gen 19478 parent 339 top level 339 path @sysSnapShoot/Snapshot/GMT+02-2024.09.30-01.00.01/@cache
ID 342 gen 19479 parent 339 top level 339 path @sysSnapShoot/Snapshot/GMT+02-2024.09.30-01.00.01/@desktop
ID 343 gen 19480 parent 339 top level 339 path @sysSnapShoot/Snapshot/GMT+02-2024.09.30-01.00.01/@system
ID 344 gen 26144 parent 339 top level 339 path @sysSnapShoot/Snapshot/GMT+02-2024.09.30-01.00.01/User
ID 345 gen 19482 parent 339 top level 339 path @sysSnapShoot/Snapshot/GMT+02-2024.09.30-01.00.01/public
...
ID 759 gen 22244 parent 5 top level 5 path @sysSnapShoot/Snapshot/GMT+01-2024.11.29-01.00.01
ID 760 gen 22239 parent 759 top level 759 path @sysSnapShoot/Snapshot/GMT+01-2024.11.29-01.00.01/@apps
ID 761 gen 22240 parent 759 top level 759 path @sysSnapShoot/Snapshot/GMT+01-2024.11.29-01.00.01/@cache
ID 762 gen 22241 parent 759 top level 759 path @sysSnapShoot/Snapshot/GMT+01-2024.11.29-01.00.01/@desktop
ID 763 gen 22242 parent 759 top level 759 path @sysSnapShoot/Snapshot/GMT+01-2024.11.29-01.00.01/@system
ID 764 gen 26144 parent 759 top level 759 path @sysSnapShoot/Snapshot/GMT+01-2024.11.29-01.00.01/User
ID 765 gen 22244 parent 759 top level 759 path @sysSnapShoot/Snapshot/GMT+01-2024.11.29-01.00.01/public
ID 766 gen 26123 parent 5 top level 5 path @
ID 767 gen 22691 parent 766 top level 766 path public
ID 768 gen 26144 parent 766 top level 766 path homes
ID 769 gen 22941 parent 766 top level 766 path @system
ID 770 gen 26144 parent 766 top level 766 path @apps
ID 772 gen 26123 parent 766 top level 766 path @desktop
ID 776 gen 26123 parent 766 top level 766 path @cache

But they're not accessible in any way:

Code: Select all

master@Supernova:~# btrfs subvolume show /Volume1/@sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01
ERROR: cannot find real path for '/Volume1/@sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01': No such file or directory

master@Supernova:~# btrfs subvolume delete /Volume1/@sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01
ERROR: Could not statfs: No such file or directory

master@Supernova:~# ls -ld /Volume1/@*
drwxr-xr-x 1 master master   42 Dec 18 13:32 /Volume1/@apps
drwxr-xr-x 1 master master   32 Dec 18 13:32 /Volume1/@cache
drwxr-xr-x 1 master master   12 Nov 29 10:59 /Volume1/@collect
drwxr-xr-x 1 master master  868 Dec 18 13:32 /Volume1/@desktop
drwxr-xr-x 1 master master   10 Jul 31 08:14 /Volume1/@system
drwxr-xr-x 1 master master 3520 Oct 31 13:22 /Volume1/@thumbnail
drwxr-xr-x 1 master master   36 Dec 16 16:04 /Volume1/@usb
drwxr-xr-x 1 master master  384 Nov 30 01:22 /Volume1/@videoframe
drwxrwxrwx 1 master master 2204 Dec 22 13:20 /Volume1/@zlog

I'm not sure if that's the problem, but running btrfs balance start -dusage=50 -musage=50 also didn't make a difference. Also, writing more data to Volume1 reduce the space further (I thought it might 'reuse' the pending blocks, but it doesn't).

Code: Select all

master@Supernova:~# btrfs filesystem df /Volume1
Data, single: total=18.59TiB, used=18.59TiB
System, DUP: total=32.00MiB, used=2.00MiB
Metadata, DUP: total=27.00GiB, used=25.99GiB
GlobalReserve, single: total=512.00MiB, used=0.00B

master@Supernova:~# df -h /Volume1
Filesystem           Size  Used Avail Use% Mounted on
/dev/mapper/vg0-lv0   22T   19T  3.2T  86% /Volume1

Any hints would be most welcome, as 5 TB of free space is a sizable amount... Thanks

[jollino]

Update, but still without a solution: I was able to delete a couple of those snapshots manually, by using `brtfs subvolume delete` directly on /var/subvols/8vEbTxkKvwa (deleting the sub-subvolumes individually first), but that didn't reclaim any space whatsoever despite a reboot.

[Gremlin]

You could try enabling 'File System Snapshot' under 'Backup' apps. Create a snapshot with the name "Snapshot" (which is the default iirc). With Snapshot enabled - but not run - check to see if there are any snapshots in the snapshot list. If there are delete them from within the app
If you work outside the app, snapshots are protected.

[jollino]

@Gremlin, I had already tried that, but it doesn't work. It still shows an empty list. I believe these were created by TOS5 (even though I had never enabled snapshots). For some reason they go from September 29 to October 27, and then there's one on November 29. I was able to delete the first two (September 29 and 30) by manually disabling the read-only flag, and then deleting the various sub-subvolumes (@apps, @cache, etc.).

I'm pretty sure that's the reason for my "lost" space, because if I go into e.g. GMT+02-2024.10.01-01.00.01/User/myuser I can find a full copy of the data that I had moved to the external volume. I'm just concerned that I may accidentally delete everything else as well, though I _think_ that the "working copy" is the one inside /var/subvols/8vEbTxkKvwa/@, and that /var/subvols/8vEbTxkKvwa/@sysSnapShoot only contains snapshots.

Why those were ever created remains a mystery; hopefully someone from TerraMaster can provide some guidance on how to sort it out, since there's no indication that those snapshots will get removed over time since the system appears not to be aware of them beyond the BRTFS layer.

[jollino]

A Christmas update in case anyone finds themselves in the same situation: it was indeed caused by those snapshots. After lots of research and deliberation, I was able to recover the missing free space by deleting ALL of those snapshots. This is how I did it:

1) Running "mount" showed the various mountpoints for Volume1, including the main one in /var/subvols (in my case /var/subvols/8vEbTxkKvwa):

Code: Select all

master@Supernova:~# mount
/dev/md9 on / type ext4 (rw,noatime,nodiratime,data=ordered)
...
/dev/mapper/vg0-lv0 on /var/subvols/8vEbTxkKvwa type btrfs (rw,noatime,space_cache=v2,subvolid=5,subvol=/)
/dev/mapper/vg0-lv0 on /Volume1 type btrfs (rw,noatime,space_cache=v2,subvolid=766,subvol=/@)
/dev/mapper/vg0-lv0 on /home type btrfs (rw,noatime,space_cache=v2,subvolid=768,subvol=/@/homes)
...

2) Inside /var/subvols/8vEbTxkKvwa I found two directories: one named @ and one named @sysSnapShoot (sic).
@ is the "live" version of data, the other one contains the snapshots. I checked this by confirming that the files I had moved to the external volume were still available inside the snapshots directories, but not inside the live one. I also tested it by adding a file to the live version (via SMB) and checking that it was not found inside any of the snapshots.

3) I got a list of subvolumes by running "btrfs subvolume list /Volume1". This gave me all the snapshots, plus the live version of the data at the end. Remember, the live data resides in "@".

Code: Select all

master@Supernova:~# btrfs subvolume list -p /Volume1
ID 332 gen 19472 parent 5 top level 5 path @sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01
ID 333 gen 19467 parent 332 top level 332 path @sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/@apps
ID 334 gen 19468 parent 332 top level 332 path @sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/@cache
ID 335 gen 19469 parent 332 top level 332 path @sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/@desktop
ID 336 gen 19470 parent 332 top level 332 path @sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/@system
ID 337 gen 26144 parent 332 top level 332 path @sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/User
ID 338 gen 19472 parent 332 top level 332 path @sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/public
ID 339 gen 19482 parent 5 top level 5 path @sysSnapShoot/Snapshot/GMT+02-2024.09.30-01.00.01
ID 340 gen 19477 parent 339 top level 339 path @sysSnapShoot/Snapshot/GMT+02-2024.09.30-01.00.01/@apps
ID 341 gen 19478 parent 339 top level 339 path @sysSnapShoot/Snapshot/GMT+02-2024.09.30-01.00.01/@cache
ID 342 gen 19479 parent 339 top level 339 path @sysSnapShoot/Snapshot/GMT+02-2024.09.30-01.00.01/@desktop
ID 343 gen 19480 parent 339 top level 339 path @sysSnapShoot/Snapshot/GMT+02-2024.09.30-01.00.01/@system
ID 344 gen 26144 parent 339 top level 339 path @sysSnapShoot/Snapshot/GMT+02-2024.09.30-01.00.01/User
ID 345 gen 19482 parent 339 top level 339 path @sysSnapShoot/Snapshot/GMT+02-2024.09.30-01.00.01/public
...
ID 759 gen 22244 parent 5 top level 5 path @sysSnapShoot/Snapshot/GMT+01-2024.11.29-01.00.01
ID 760 gen 22239 parent 759 top level 759 path @sysSnapShoot/Snapshot/GMT+01-2024.11.29-01.00.01/@apps
ID 761 gen 22240 parent 759 top level 759 path @sysSnapShoot/Snapshot/GMT+01-2024.11.29-01.00.01/@cache
ID 762 gen 22241 parent 759 top level 759 path @sysSnapShoot/Snapshot/GMT+01-2024.11.29-01.00.01/@desktop
ID 763 gen 22242 parent 759 top level 759 path @sysSnapShoot/Snapshot/GMT+01-2024.11.29-01.00.01/@system
ID 764 gen 26144 parent 759 top level 759 path @sysSnapShoot/Snapshot/GMT+01-2024.11.29-01.00.01/User
ID 765 gen 22244 parent 759 top level 759 path @sysSnapShoot/Snapshot/GMT+01-2024.11.29-01.00.01/public
ID 766 gen 26123 parent 5 top level 5 path @
ID 767 gen 22691 parent 766 top level 766 path public
ID 768 gen 26144 parent 766 top level 766 path homes
ID 769 gen 22941 parent 766 top level 766 path @system
ID 770 gen 26144 parent 766 top level 766 path @apps
ID 772 gen 26123 parent 766 top level 766 path @desktop
ID 776 gen 26123 parent 766 top level 766 path @cache

4) At this point there were two things to consider. The first is that all the snapshots subvolumes are set as read-only, and the also contain sub-subvolumes (those listed below each "main" one). You can confirm this by taking note of the ID of each of the main snapshots, and checking that the next few ones have that ID as their parent; also, more simply, the directory structure shows them nested.
So, for each one of those, I had run these commands:

Code: Select all

btrfs property set -t subvol /var/subvols/8vEbTxkKvwa/@sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01 ro false
btrfs subvolume delete /var/subvols/8vEbTxkKvwa/@sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/@apps
btrfs subvolume delete /var/subvols/8vEbTxkKvwa/@sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/@cache
btrfs subvolume delete /var/subvols/8vEbTxkKvwa/@sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/@desktop
btrfs subvolume delete /var/subvols/8vEbTxkKvwa/@sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/@system
btrfs subvolume delete /var/subvols/8vEbTxkKvwa/@sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/User
btrfs subvolume delete /var/subvols/8vEbTxkKvwa/@sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01/public
btrfs subvolume delete /var/subvols/8vEbTxkKvwa/@sysSnapShoot/Snapshot/GMT+02-2024.09.29-01.00.01

To avoid accidentally making a mess, and considering that most of the ones I had were in October, I wrote a little script to generate the commands for me to copy and paste.

Code: Select all

#!/usr/bin/env python3

base = "/var/subvols/8vEbTxkKvwa/@sysSnapShoot/Snapshot/GMT+02-2024.10.XYZ-01.00.01"
subs = ["@apps", "@cache", "@desktop", "@system", "User", "public"]

for i in range(1,28):
    dirname = base.replace("XYZ", str(i).rjust(2, "0"))
    print(f"btrfs property set -t subvol {dirname} ro false")
    for sub in subs:
        print(f"btrfs subvolume delete {dirname}/{sub}")
    print(f"btrfs subvolume delete {dirname}")
    print()

I still had one in November so I did that by hand, as I was quite terrified I had miscalculated something and was about to wipe everything. Thankfully I didn't.

6) After I deleted the last subvolume, nothing happened for a while. Then, a couple of minutes later, I noticed the free space increasing as I ran 'df' multiple times. I eventually went from this

Code: Select all

master@Supernova:~# df -h
Filesystem           Size  Used Avail Use% Mounted on
/dev/md9             7.5G  2.4G  4.7G  34% /
tmpfs                 50M   28K   50M   1% /dev/shm
tmpfs                1.6G  8.7M  1.6G   1% /run
tmpfs                5.0M     0  5.0M   0% /run/lock
tmpfs                512M  1.4M  511M   1% /tmp
/dev/mapper/vg0-lv0   22T   18T  4.7T  79% /var/subvols/8vEbTxkKvwa
/dev/mapper/vg0-lv0   22T   18T  4.7T  79% /Volume1
/dev/mapper/vg0-lv0   22T   18T  4.7T  79% /home
tmpfs                779M     0  779M   0% /run/user/0

to this

Code: Select all

master@Supernova:~# df -h
Filesystem           Size  Used Avail Use% Mounted on
/dev/md9             7.5G  2.4G  4.7G  34% /
tmpfs                 50M   28K   50M   1% /dev/shm
tmpfs                1.6G   21M  1.6G   2% /run
tmpfs                5.0M     0  5.0M   0% /run/lock
tmpfs                512M   16M  497M   4% /tmp
/dev/mapper/vg0-lv0   22T   14T  7.9T  65% /var/subvols/8vEbTxkKvwa
/dev/mapper/vg0-lv0   22T   14T  7.9T  65% /Volume1
/dev/mapper/vg0-lv0   22T   14T  7.9T  65% /home
tmpfs                779M     0  779M   0% /run/user/0

7) At this point, I ran "btrfs filesystem show /Volume1" and saw that for some reason it still said that the used space was 18 TB. I think this is just an internal counter, as 'df' is what matters for the OS .

Code: Select all

master@Supernova:~# btrfs filesystem show /Volume1
Label: 'TOS_VOL_20240722'  uuid: 9d2c5f6f-2269-4334-b117-cc89793eaa1a
	Total devices 1 FS bytes used 18.42TiB
	devid    1 size 21.80TiB used 18.66TiB path /dev/mapper/vg0-lv0

Still, wanting to get things clean, I ran the balance command: "btrfs balance start -v -dusage=50 /Volume1". This takes a while, but you can check its status in a different session by using the status subcommand, which tells you much is left. (I have no copy of that to show here, my bad).
After that — it took a while — the used space is correct everywhere:

Code: Select all

master@Supernova:~# btrfs filesystem show /Volume1
Label: 'TOS_VOL_20240722'  uuid: 9d2c5f6f-2269-4334-b117-cc89793eaa1a
	Total devices 1 FS bytes used 13.96TiB
	devid    1 size 21.80TiB used 18.36TiB path /dev/mapper/vg0-lv0

I hope this may be helpful to anyone else who has the same problem. It's definitely nerve-wracking, but as long as you don't touch the "@" directory or volume, your live data is fine. Why those snapshots were created in the first place remains a mystery, probably some bug with TOS5? Still, all's well that ends well.

Happy holidays!

[detritusuk]

I had an issue here. Similarly, I didnt have snapshots enabled on my unit. However, space wasn't being freed up when I was removing items. Not great if you are running out of space and concerned its very low. Turns out, items were being put into the recycle bin once they were deleted meaning it wasn't freeing up the space.

If you go to File Manager there may be a folder called #recycle under each share on your Terramaster NAS. If you select with a single left click and then right click there is an option for properties or empty recycle bin.

[RyanYang]

You may try the following solutions to resolve your issue:viewtopic.php?t=6999

[jollino]

In my case (solved 3 months ago, see viewtopic.php?p=41485#p41485) the problem was "hidden" Snapshots that had been created despite not showing up anywhere in TOS. The Recycle Bin described above seems unrelated