Error: "too many open files"

Discussion about USB Copy.
Locked
User avatar
DimensionAll
Posts: 4
Joined: 15 Jan 2023, 20:40

Error: "too many open files"

Post by DimensionAll »

USB Copy F4-423 to D2-310

Code: Select all

Backup failed! USB Copy_ mirror backup failed: backup file or folder failed: create file failed: open /Volume1/@usb/usbshare1/Data-02/02 Media/Books/Toyota Hilux Surf/Audio Amplifiers & Speakers/4 Inch Speakers/75 watts RMS/100 W RMS/Kicker 40CS654 Speaker Testing.mp4: too many open files
:o

So, I locate the file within the folder containing many (small) files and archive the folder as a .7z, then delete the offending folder. I restart USB Copy which happily proceeds further along until again it encounters another folder containing many small files and halts, giving the same error: too many open files.

Note: When I first tried USB Copy I was able to back up this same data in reverse without issue i.e., From D2-310 to F4-423.

Out of desperation, I ordered a 16GB DDR stick. Unfortunately, 16GB will most likely do nothing to resolve the situation, it's only utilising around 20% of my current 4GB anyhow.
User avatar
TMroy
TerraMaster Team
Posts: 2578
Joined: 10 Mar 2020, 14:04
China

Re: Error: "too many open files"

Post by TMroy »

Can you manually copy this folder which contains many small files from D2-310 to your TNAS through the File Manager?
To contact our team, please send email to following addresses, remember to replace (at) with @:
Support team: support(at)terra-master.com (for technical support only)
Service team: service(at)terra-master.com (for purchasing, return, replacement, RMA service)
User avatar
DimensionAll
Posts: 4
Joined: 15 Jan 2023, 20:40

Re: Error: "too many open files"

Post by DimensionAll »

Thank you for your help. Currently, I'm waiting on an external utility (Beyond Compare) to complete the task (~4 hrs remain). I'll report back File Manager result asap.
User avatar
DimensionAll
Posts: 4
Joined: 15 Jan 2023, 20:40

Re: Error: "too many open files"

Post by DimensionAll »

Before testing on File Manager I found that USB Copy was successful when I tested on an isolated branch containing the problem folder of many files. The problem seems to persist only when the entire folder tree structure is involved in the copy process.

When I isolate any problem branch and test this alone the problem does not occur, neither with USB Copy nor File Manager. In summary, it occurs when the entire drive is included (~16 TB)

To be certain, I currently have File Manager performing the task of copying the entire drive (~15 hours remain), so far without issue; by now USB Copy had failed several times over (before reaching at present 7.5% completion).
User avatar
TMroy
TerraMaster Team
Posts: 2578
Joined: 10 Mar 2020, 14:04
China

Re: Error: "too many open files"

Post by TMroy »

{L_BUTTON_AT}DimensionAll

Please check the log in USB copy, and find out the error message, we need to study it and verify the issue.
To contact our team, please send email to following addresses, remember to replace (at) with @:
Support team: support(at)terra-master.com (for technical support only)
Service team: service(at)terra-master.com (for purchasing, return, replacement, RMA service)
User avatar
DimensionAll
Posts: 4
Joined: 15 Jan 2023, 20:40

Re: Error: "too many open files"

Post by DimensionAll »

I no longer have that particular log "too many open files". However, the particular line showing the error is pasted (above). Instead I have another.

Apparently, there may also be conflicts where hidden folders are involved. I started a new USB copy task for a 4TB external drive. An error occurs within a few minutes (actually this is convenient). The log is below. The problem folder in this case is a hidden folder. (Note: Hidden folders were included to be copied, in the settings).

Error,2023-01-17 14:08:28,Creator-PC USB BlkB,BW,Backup failed! USB Copy_/Volume1/Creator-PC/usbshare4/Bak/invokeai/.venv/bin/python open file failed: open /Volume1/@usb/usbshare4/Bak/invokeai/.venv/bin/python: no such file or directory
Info,2023-01-17 14:08:28,Creator-PC USB BlkB,BW,Partially completed. USB Copy_Creator-PC USB BlkB
Error,2023-01-17 14:08:28,Creator-PC USB BlkB,BW,Backup failed! USB Copy_ mirror backup failed: backup file or folder failed: open file failed: open /Volume1/@usb/usbshare4/Bak/invokeai/.venv/bin/python: no such file or directory

I was going to mention that when I had (earlier) used the external utility 'Beyond Compare' there was also a conflict regarding hidden files (utility log: "no such file"), except I checked and located them manually (the error seems to be misinformed!). I was able to manually copy those hidden folders over (windows explorer) but decided not to mention the issue since I assumed it was an unrelated issue.
User avatar
TMroy
TerraMaster Team
Posts: 2578
Joined: 10 Mar 2020, 14:04
China

Re: Error: "too many open files"

Post by TMroy »

{L_BUTTON_AT}DimensionAll

Thank you for your report! we will verify it.
To contact our team, please send email to following addresses, remember to replace (at) with @:
Support team: support(at)terra-master.com (for technical support only)
Service team: service(at)terra-master.com (for purchasing, return, replacement, RMA service)
Locked