guest wrote:
It would be really nice for bit-tornado to be able to work on Linux.
You're kidding, right?
Anyway.
Maoh wrote:
Considering how slow pre-allocation is, I think you could improve its performance by using ftruncate() call and not doing whatever you're doing now (which I presume is simply writing out a lot of junk to the files until they're of correct size). This is sadly Mac/Unix-only improvement, so Windows users wouldn't benefit from it.
...
Another note: You could probably "pre-allocate" with sparse files by seeking to the end of the file (outside the current file size) and writing a single garbage byte there. The file would still be sparse, but at least the system would "know" how large the file's going to get, and hopefully avoid file fragmentation with that.
The thing is, Linux, and I would guess most UNIX-like OS's (including Mac), handle things like sparse files transparently (and competently). With "normal" allocation -- which I think on Linux is the same as "sparse", and really not much at all -- what you describe in your last paragraph is exactly what happens. The first part. Sometimes. If / when it happens, you'll have, say, a 700MB file, empty but for a few MB at the end (or in the middle, actually) that only takes up a few MB on disk. If another few MB are written somewhere else, the filesystem allocates some space, not necessarily anywhere near the first, and writes the data there. The filesystem keeps track of what goes where and where nothing has been written, and the file remains small (on disk), but fragmented. So bittorrent's normal allocation can end up with chunks written all over the place, giving highly fragmented files. To avoid this, (I think) the prealloc method *has* to write some garbage the entire length of the file. I think. In any case, it probably writes a few MB at a time to prevent too much fragmentation.
For a sparse file, the system may "know" the full file size, but I don't think it can do much about it. In theory, it could put the 1MB-at-the-end-of-the-file at the end of a large unallocated space, and then reserve that space. But then if that space never gets written to, that space would be wasted, unless the fs had some half-allocated state that expires after a while or something else that is hard, or non-deterministic, or not useful in the general case (and not useful enough to be made a special case).
This is getting long.
But I'm done with that, and don't have too much else to say.
I don't know much about mmap (and more in theory than in practice), but if it's whole-file, and I think the Python one is, it could only be used (or be useful) with small files. ...er, maybe not. I guess in theory you could mmap the whole disk and let the kernel figure it out. The system mmap can start anywhere in the file, but it still seems like if the program is just filling up an array and writing it, mmap wouldn't be terribly useful. It might save a copy or two in memory on read, but that probably wouldn't make much difference. Maybe it could make some code cleaner / easier. I don't know.
The disk full issue is annoying; a few times I've let it run - er, hang - all night after forgetting or not noticing the disk was low on space. I doubt many programs handle it gracefully, though; I'd expect most to just crash, either with or without saving open files. BitTornado just seems to hang, though. I don't know why. It may just be stuck in a loop trying and failing to write to a file, but I don't think clearing up space gets it unstuck, so it might be stuck somewhere else. It would be nice if it woke up and continued when space became available, and at least upload in the meantime. ...that was the annoying part; seeding one thing and some other thing fills up the disk. *hang* ...at least it doesn't hang everything when it can't open (or write?) a file for some other reason. That got really annoying. And is why I switched back from the original client after trying it for a week.
Maybe disk space status could be in the client somewhere, so you'd have some warning (if you're paying attention). ...or at least have a bar to watch shrinking as stuff gets downloaded. whee.
(So much for not much left to say.
)
Login
FAQ
Search
