Oh oof. I thought removing that requirement would be the whole point of something named "FFmpeg-over-IP". Shared filesystem usually involves full trust between machines, bad network error handling, and setting things up by hand (different config on every distro)
One could write a small Python server in one day which receives a chunked POST request and transcodes the video on the fly.
The same server could also offer a download link for the transcoded video, and also receive URL parameters for the transcoding options. Or the transcoded video itself is returned after the subprocess finishes.
Something along the lines of:
Server
import asyncio
import subprocess
from aiohttp import web
async def transcode_video(request):
cmd = [
"ffmpeg",
"-i", "pipe:0",
"-f", "mp4",
"-preset", "fast",
"output_file.mp4"
]
process = await asyncio.create_subprocess_exec(
*cmd,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
)
async for chunk in request.content.iter_any():
process.stdin.write(chunk)
await process.stdin.drain()
await process.stdin.close()
await process.wait()
return web.Response(text="Video transcoded successfully!")
app = web.Application()
app.router.add_post('/upload', transcode_video)
if __name__ == '__main__':
web.run_app(app, port=8080)
This and the other solutions in this thread wont work with plex or the media servers. The way it works with the media servers is that there’s a variable number of outputs, and you just cant emulate that on the client side without a virtual filesystem or by some creative hacks that use multiple file descriptors (even then how do you handle dynamic?).
As for why the variable number of outputs exist in the first place, it’s because when you’re a media server, you dont want to wait until the full file is converted before serving it to the user. You want to start playing immediately so the output is chunked into multiple files
Ah unfortunately my use case is similar to yours: Use Windows desktop to transcode files stored on a Linux NAS. My files are ~100GB so encoding multiple files in parallel would waste a lot of space and unnecessarily burn write cycles
FWIW, you can run an smb server from within a docker container (on the linux side). I forget which one I used but it makes the setup painless and you can configure different auth strategies as well. Network errors (little bit of packet loss) are generally handled by the underlying OS, and in case of windows, it can use multiple network paths simultaneously to give you the aggregate bandwidth.
If you're talking about how today's open source NAS software has a button for enabling NFS/SMB on a directory:
1. I built my NAS long before software like that was common. Some of my custom stuff (Eg tiered storage, storing the first few seconds of every video on flash, etc) would be a pain to migrate.
2. Some of my Windows machines are untrusted. Unlike the NAS, they have internet access. I can't give them read access to the entire NAS, but I still want to use their GPUs and CPUs to run ffmpeg on arbitrary files on the NAS.
3. I could spend a day writing more code to move files in and out of a shared directory every time I need to run ffmpeg on them. But I was hoping "FFmpeg-over-IP" would let me run a remote ffmpeg on a local file. Like calling an RPC
Did you build your NAS in the 90’s or early 2000’s?
This has been available in Synology and QNAP devices for that long…
I used to own the tape library that is the Disney Vault. A common pattern for transcoding is to have a watched folder. Drop files you need transcoded in, get files you want in a diff directory when finished.
More like you’re not familiar enough with the video encoding performance of a typical NAS that is not in the thousand dollar range. Or what’s a NAS — it can be anything really.
Why would you transcode on the NAS?
Not what I’m suggesting at all; perf on most NAS is awful for that…
There’s a pretty well understood concept of what a NAS is; this isn’t a complicated philosophical problem.
A very common workflow in motion picture production is to use NAS for storage on a fast network, mount the SMB share, have a script/tool/app that monitors the ingest directory and writes to an output dir.
FWIW the key differentiator between a NAS and other types of network storage is the protocols they use.
If files is the main primitive, it’s a NAS; it’s its blocks it’s considered a SAN.
Sometimes SANs have NAS “heads” for clients that want file access or a block level storage device.
My five-year-old Synology DS418play has support for hardware realtime transcoding via its integrated low-power GPU. I think I paid about $600 for it. You can get a new DS423+ for about the same price.
What's the point of transcoding to HEVC? If your playback device already supports it, and the file is already HEVC, then there's no need to transcode. And there's no benefit I'm aware of in transcoding from an older format like H.264 to HEVC. AFAIK, every device that supports HEVC media also supports H.264 as well.
It would be helpful to understand your specific use cases in more detail, particularly for batch transcoding (my understanding is that this solution is not about stream transcoding).
Aight have you tried transcoding a normal file (today that means 4K resolution) to H264? It usually won't play because the standard limits resolution to 2048 pixels.
Mounting anything requires root even if you use FUSE.
There are ways to intercept writes without root and send them to another machine. Eg you could use LD_PRELOAD. But that's exactly the kind of pain in the ass that I was hoping a project named FFmpeg-over-IP would automate.
Interesting, the man page says "To allow mounting and unmounting by unprivileged users, fusermount3 needs to be installed set-uid root". Is this the default case on any distros?
User space limits potential security impacts, and a restricted VFS could be used to prevent clients from accessing anything that they shouldn't.
(Although I'm not even pretending to know whether or not this is a remotely good idea - my guess is that it isn't, but I'd like to know just how bad an idea it actually is.)
Oh oof. I thought removing that requirement would be the whole point of something named "FFmpeg-over-IP". Shared filesystem usually involves full trust between machines, bad network error handling, and setting things up by hand (different config on every distro)