CNFS buffers bigger than 2 GB under Linux ?
jeff at mansukhani.net
Mon Mar 18 21:14:43 UTC 2002
I have a similar issue w/ files on Linux kernel 2.4. Perhaps someone can give
me some pointers. WHen I compiled INN 2.x w/ large file support it complained,
even though I believe that 2.4 kernel can support files sizes larger than 2GB.
dsr+inn at mail.lns.cornell.edu wrote:
> Russ Allbery <rra at stanford.edu> writes:
> > Antoine Delvaux <antoine.delvaux at belnet.be> writes:
> > >> Well, you can just use large disk files.
> > > Yes, but this will slow down innd I think, no ? That's why I wanted to
> > > use raw devices.
> > It may slow it down a bit, but we've not run bechmarks. I doubt it will
> > be particularly noticeable.
> I haven't benchmarked, but my guess would be that there should be very
> little slowdown with an extent based file system (e.g. SGI's xfs or
> Veritas VxFS). ufs with the default parameters probably is not well
> suited to one large file--in particular, I would expect the limits on
> the fraction of a cylinder group a file can allocate to lead to very
> poor layout of a single CNFS file the size of the filesystem, so some
> filesystem tuning would probably be needed to get decent performance
> from ufs. What this means for ext2fs I don't know--the "Design and
> Implimentation" paper doesn't say anything about ext2fs's
> allocation policy within a block group, and mke2fs doesn't offer much
> in the way of tuning options. It does look like ext2fs has
> traditional ufs inodes, so it will have the "double indirect inode"
> problem with large files; a large block size might help (this should
> mostly be a problem for random access (nnrpd), not sequential access
> (innd), if Linux has a halfway decent inode cache). I know even
> less about other Linux file systems...
> > INN imposes no limitations; the limitation is in the kernel. It's a
> > common limitation; I don't know of any operating system at present that
> > supports memory mapping raw devices over 2GB. (Tru64, probably, if it
> > supports mmap on raw devices at all, but I'm not sure it does.)
> I couldn't get it to mmap raw devices at all--we ended up with large
> files on AdvFS (an extent based filesystem).
>  http://web.mit.edu/tytso/www/linux/ext2intro.html
More information about the inn-workers