One thing about this particular benchmark is that the file it produces is a little bit unusual even for large text files, because it is only contains one (massive) line. Most huge text files you open in text editors are things like multi-gigabyte logs and database dumps, which are composed of many relatively short lines. Text editors often bake that assumption into their data structures. A more representative test would probably be to use hexdump instead of base64 there.
Though I take the point: if a text editor advertises itself as being "fast", this is the kind of stuff you expect it to handle.
EDIT: ok, so i just tried the hexdump version. Vim and Emacs took a couple of seconds each (Emacs being slightly slower and chunkier when you navigate, Vim handles it without issue and is perfectly repsonsive). less opened it instantly (obviously, it's less!). ox has been going for a few minutes now, nothing happening.
It's not the benchmark's fault, ox is just slow for large files.
> the file it produces is a little bit unusual even for large text files, because it is only contains one (massive) line.
XML and JSON without pretty print. Minimized HTML with embedded resources. While not often gigabyte sized those seem to be enough to get most editors to hang and make editing the files a miserable experience.
Yes, true enough, it does happen occasionally, but it's pretty rare (and usually there are other ways to deal with it: using less or similar tools, or just running your massive single-line JSON thing through jq first). And my point was exactly what you're saying: many text editors choke on files like this, so it's not such a huge indictment of ox that it chokes on it as well.
However, ox clearly performs poorly even when your text files aren't massive single lines, so it doesn't really matter: ox basically can't handle files like this regardless of line distribution.
> and usually there are other ways to deal with it: using less or similar tools, or just running your massive single-line JSON thing through jq first
Older Unixen at least (SVR3-based) had a tool called bfs - big file scanner. Used it some, e.g. when, as a system engineer, I helped IT staff of one of our customers, a university processing exam results of tens of thousands of students from dozens of colleges affiliated to that univ. IIRC, its UI was something like a read-only ed (google "unix bfs command"). You used it to scan really large (for the time) files, e.g. data files (input/output) in large data processing environments, for purposes like checking if the input or output files anecdotally looked okay, no major noticeable garbage in them.
Haven't checked if it is present in modern Linuxes. Also don't remember if it could handle large files without newlines. Likely not, if based on ed. Didn't have such files to work on then.
As an emacs user, one of the things I really wish it had was robust support for arbitrary file sizes and shapes.
(It'd be fine if not all features worked on these files.)
Sure, most files are reasonably sized and shaped, but sometimes you do find yourself needing to e.g. dig into that single-line 1GB json file to try to debug something, and it's inefficient to have to think, before opening any file, "is more normal tool sufficient for this, or should I use something else?"
There's no fundamental reason we can't have one tool that does it all!
emacs -q or -Q is often quite fast. It's things like syntax highlighting, word wrapping, etc. which modes add which usually slow it down. Anything trying to run regexes over the buffer isn't going to scale well.
I think long lines are (were? [1] seems to be a new feature) a well known problem in emacs even without a custom config. There have been ways to work around this for awhile I think, but as far as I know emacs -Q isn't one of them.
oh that's neat, i didn't know about that! could help.
my ideal editor would never block, and use file-sized based approximations to allow scrolling to arbitrary locations w/o having to actually read the contents of the entire. things like syntax highlighting &c would be either strictly time-limited to not drop frames, or done in the background.
Whether there's a wrapping feature depends on the implementation. coreutils base64, yes. Other implementations, maybe not. macOS's /usr/bin/base64 for instance doesn't have the concept of wrapping. FreeBSD base64, judging from the manpage, doesn't do that either. https://www.freebsd.org/cgi/man.cgi?query=base64
Edit: Oops, posted some misinformation. The mistaken part, reproduced below:
> First, wrapping by default only occurs when writing to a tty. When stdout is redirected to a file, wrapping base64 output usually doesn't make sense, so wrapping doesn't happen.
Correction: coreutils base64 wraps regardless of whether stdout is a tty.
Though I take the point: if a text editor advertises itself as being "fast", this is the kind of stuff you expect it to handle.
EDIT: ok, so i just tried the hexdump version. Vim and Emacs took a couple of seconds each (Emacs being slightly slower and chunkier when you navigate, Vim handles it without issue and is perfectly repsonsive). less opened it instantly (obviously, it's less!). ox has been going for a few minutes now, nothing happening.
It's not the benchmark's fault, ox is just slow for large files.