Simplify it down if you can. Forget NFS, SSHFS and syncthing as those are to complex and overkill at the moment. SMB is dead simple in a lot of ways and is hard to mess up.

I would start by figuring out how to get more drives in your desktop. SSDs can be put anywhere but HDDs need to be mounted. I would get two big disks and put them in raid 1 and then get a small disk for the boot drive. TrueNAS is still going to be the easiest route and TrueNAS scale is based on Debian under the hood although its designed to not be modified for reliability.

Forget the USB drives. I did that for a while and it is bad in so many ways. It will come back to haunt you even if the work fine for now.

For the desktop you always could upgrade the ram or CPU if that becomes the bottleneck. If the ram is 8gb or less you should upgrade it. Make sure you are getting ram that is fast enough to keep up with the CPU. You can find more information about your CPU and its ram speed online.

For your WiFi, it actually isn’t bad. You might be ok using it depending on what you are doing.

For the minipc I wouldn’t use it as a NAS. You can but it is probably going to be limiting.

In case your desktop doesn’t have enough Sata ports you could get a ePCI sata card. They are pretty cheap ($10) and will potentially work better if your hardware is limited.

2 internal hdds

As I understand the PC case is some sort of small form factor and the motherboard has more SATA ports to plug in disks.

If that’s the case then try to find old pc case in the dumpster or from a neighbour. It should have more space for disks.

Some PCs 15 years ago had incompatible layouts so be a little cautious but gratis is a fair price.

I’d say mostly because the client is fairly good and works about the way people expect it to work.

It sounds very much like a DropBox/Google Drive kind of use case and from a user perspective it does exactly that, and it’s not Linux-specific either. I use mine to share my KeePass database among other things. The app is available on just about any platform as well.

Yeah NextCloud is a joke in how complex it is, but you can hide it all away using their all in one Docker/Podman container. Still much easier than getting into bcachefs over usbip and other things I’ve seen in this thread.

Ultimately I don’t think there are many tools that can handle caching, downloads, going offline, reconcile differences when back online, in a friendly package. I looked and there’s a page on Oracle’s website about a CacheFS but that might be enterprise only, there’s catfs in Rust but it’s alpha, and can’t work without the backing filesystem for metadata.

Nextcloud AIO in docker is dead simple and has been reliable for me.

The sync client is capable of syncing the whole tree as remote pointers that didn’t take space until you access the file then it downloads it local. You can also set files to always be local.

Nextcloud has a desktop app that can sync folders.

It’s certainly possible, if not else than with the use of FUSE. But I did not see such a project yet that can do this.

If you’re on Linux I’d recommend using btrfs, or bcachefs with snapshots. It’s basically like time machine on MacOS. That way if you accidentally delete something you can still recover it.

The VPN is to give you access to your files from anywhere, since you don’t have the storage capacity on your laptop for all of them.

If you have an encrypted connection to home, laptop storage isn’t a concern.

As a benefit, this also solves the risk of losing files that are only on the laptop, by keeping the at home.

Yea, Syncthing has it’s moments (and uses - I keep hundreds of gigs between 5 phones and 5 laptops/desktops in sync with it).

Resilio does use the bittorrent protocol, but uses keys and authorization for shares. Give it a try, it may address your need to access files remotely. I use it to access my media (about 2TB) which clearly can’t be sync’d to my laptop (or phone). I can grab any file, at any time.

If there is sufficient RAM on the laptop, Linux will cache a lot of metadata in other cache layers without NFS-Cache.

ZeroTier allows for a mobile, LAN-like experience. If the laptop is at a café, the files can be accessed as if at home, within network performance limits.

NFS-Cache is a specific cache for NFS, and does not represent all caching that can be done of files over NFS. “Direct I/O” is also a specific thing, and should not be generalized in the meanings of “direct” and “I/O”.

Let’s skip those entirely for now as I cannot simply explain either. I doubt either will matter in your use case, but look back if performance lags.

One laptop accessing one NFS share will have good performance on a quite local network.

NFS is an old protocol that is robust and used frequently. NFSv3 is not encrypted. NFSv4 has support for encryption. (ZeroTier can handle the encryption.)

SSHFS is a pseudo file system layered over SSH. SSH handles encryption. SSHFS is maybe 15 years old and is aimed at convenience. SSH is largely aimed at moving streams of text between two points securely. Maybe it is faster now than it was.

NFS is generally the way network storage appliances are accessed on Linux. If you’re using a computer you know you’re going to be accessing files on in the long term it’s generally the way to go since it’s a simple, robust, high performance protocol that’s used by pros and amateurs alike. SSHFS is an abuse of the ssh protocol that allows you to mount a directory on any computer you can get an ssh connection to. You can think of it like VSCode remote editing, but it’ll work with any editor or other program.

You should be able to set up NFS with write caching, etc that will allow it to be more similar in performance to a local filesystem. Note that you may not want write caching specifically if you’re going to suddenly disconnect your laptop from the network without unmounting the share first. Your actual performance might not be the same, especially for large transfers, due to the throughput of your network and connection quality. In my general experience sshfs is kind of slow especially when accessing many different small files, and NFS is usually much faster.

You’re getting a lot of advice in this thread and it’s all pretty good, but not all of it seems to answer the problems you have in your order of priority or under your constraints. I’ll try to give an explanation of why I think my advice will do so then give it.

Getting off usb will speed up file access and increase the number of operations you can do from the laptop on your lan. Some stuff will still need to be copied over locally, normal people like us just can’t afford the kind of infrastructure that lets you do everything over the lan. For those things, rsync is perfectly good, and they’re most likely going to be enough of an edge case that it won’t be very often.

When you’re ready, and from your responses in this thread I’d say you are, a vpn doesn’t expose you to much security risk if any. There are caveats to that, but if you’re doing something like openvpn or wireguard it’s all encrypted and key based and basically ain’t nobody getting into it unless they were to get a key off an old computer you use and didn’t wipe before throwing out or something. That would solve your remote access bonus problem. No pressure and in your own time, of course.

You are me twenty years ago.

Cobbling together solutions from what’s available at the cost of the parts from the hardware store. Serial experiments lain but shot in the trailer park boys set. Hackers with the cast of my name is earl.

Don’t ever change.

So you want to kick the bad habit but don’t have enough physical space in your desktops case or enough sata ports! You have a bigger tower case but don’t know if it’ll really hold the drives you want.

The best bet is to transplant the motherboard and power supply from your sff desktop into the big case. If the big case has at least three 5 1/4 bays you can use a bracket to go from 3 big bays to 4 or 5 smaller 3.5” hdd bays. I’d recommend 4 instead of 5, more on that later.

If the big tower case has the little dangly 2x 3.5 bay cage hanging down from its cd cage, you can use four strips of sheet metal and a carpenters square (or the square corner of some copy paper) to make a column of hdd mounting space all the way to the floor of the chassis. Just remember to use vibration damping grommets.

Make sure when you’re filling your tower up with drives to put some fans blowing on em. Drives need to be kept cool for maximum life. Those 3 cd bay to 4 hdd bay adapter brackets are nice for that because they usually have a fan mount or one included.

Now you need sata (or maybe ide) ports to plug all these in. Someone else already said to use those cheap little sata expanders and those are great. I used an old cheap pc mounted in a salvaged case just like you might with four of em back in the day.

You’ll actually want to use the towers power supply if it has one and it works and matches the sff desktops connectors because it probably has more power capacity than the sff desktops supply. You may need some molex or sata splitters to get power to all your drives.

Consider mergerfs and snapraid once you have this wheezing Frankensteins monster operational.

Mergerfs displays all the drives as one big file system to the system and to users. So if drive one had /pics/dog.jpg and drive two had /pics/cat.jpg then the mergerfs of the two of them would have both pictures in the /pics directory when you open it.

Snapraid is like if raid5 (really zpools because you can have many parity devices) but it only runs once a day or whatever and is basically a snapshot.

Anyway sorry for the tangent. Post some pics of the tower case or its model number or whatever and I can give better advice about filling it with drives.

Zerotier + sshfs is something I use consistently in similar situations to yours - and yes, zerotier is similar to a vpn. Using it for a constant network connection makes it less critical to have everything mirrored locally. . . But I guess this doesn’t solve your speed issue.

I"m not an expert in rclone. I use it for connecting to various cloud drives and have occasionally used it for an alternative to sshfs. I"ve used vfs-cache for cloud syncs but not quite in the manner you are trying. I do see there is a vfs cache read-ahead option that might he|p? Agreed on the documentation, sometimes their forum helps.

Removed by mod

good luck. what’d you end up figuring out to access your files quicker?

There’s one thing I forgot to consider in my original reply and I’m sorry for that. With TrueNAS you’d probably have to copy your data off of the existing drives to somewhere else because they will have to be reformatted to create a ZFS pool. I don’t know if that is practical for you, so please don’t feel like the following is something you must do or anything.

I think you’re doing great. Sorry for the late reply. To answer your questions:

1. TrueNAS Scale is an OS that is built on top of Debian. Using TrueNAS makes set up simpler to set up, but you could implement what you want with a Debian install, but if you were to install TrueNAS, it would replace whatever existing OS you have installd.
2. Yes, TrueNAS would manage your filesystem. It can manage your hard drives for you. Its UI isn’t too hard to understand, and it can be accessed and managed through a web browser on your Laptop.
3. TrueNAS has some software packages in the form of docker containers, they are managed through the TrueNAS UI. You can browse them though their website here. My advice with these apps would be to set up your NAS with all of the drives in storage pools first before installing these. If theres something you want that isn’t supported, TrueNAS can also set up Virtual Machines, and you can use one of those to run those services, provided your CPU supports it and its turned on in the BIOS. If you go need to go down this route, you will have to set up a bridge network in TrueNAS in order to get the VM to communicate with TrueNAS over your network, but that’s not particularly hard or anything.
4. You will need to run TrueNAS on its own computer, yes. What I was suggesting was installing TrueNAS as the OS on your Desktop. Idk if thats practical for you or not, since doing so would need you to wipe everything on the boot drive of your Desktop, so idk if you have a place to copy any important data off of it to.

In terms of comprehension, yeah I think you’ve got it. I think a NAS system would handle your caching idea for you, if I’m understanding you correctly. Having a good file sharing setup over LAN, whether its using NFS, or Samba would allow you to mount a folder from your Desktop on your Laptop and access them.

For files that you want to have access to when not on your home network, you could set up a folder that Syncthing tracks on your Desktop, sync that with your Laptop, and then have access to them that way.