CNFS buffers bigger than 2 GB under Linux ?

Day Dreamer jeff at mansukhani.net
Mon Mar 18 21:14:43 UTC 2002


I have a similar issue w/ files on Linux kernel 2.4. Perhaps someone can give
me some pointers.  WHen I compiled INN 2.x w/ large file support it complained,
even though I believe that 2.4 kernel can support files sizes larger than 2GB.
Thanks.

Jeff

dsr+inn at mail.lns.cornell.edu wrote:

> Russ Allbery <rra at stanford.edu> writes:
> > Antoine Delvaux <antoine.delvaux at belnet.be> writes:
> > >> Well, you can just use large disk files.
> >
> > > Yes, but this will slow down innd I think, no ?  That's why I wanted to
> > > use raw devices.
> >
> > It may slow it down a bit, but we've not run bechmarks.  I doubt it will
> > be particularly noticeable.
>
> I haven't benchmarked, but my guess would be that there should be very
> little slowdown with an extent based file system (e.g.  SGI's xfs or
> Veritas VxFS).  ufs with the default parameters probably is not well
> suited to one large file--in particular, I would expect the limits on
> the fraction of a cylinder group a file can allocate to lead to very
> poor layout of a single CNFS file the size of the filesystem, so some
> filesystem tuning would probably be needed to get decent performance
> from ufs.  What this means for ext2fs I don't know--the "Design and
> Implimentation"[1] paper doesn't say anything about ext2fs's
> allocation policy within a block group, and mke2fs doesn't offer much
> in the way of tuning options.  It does look like ext2fs has
> traditional ufs inodes, so it will have the "double indirect inode"
> problem with large files; a large block size might help (this should
> mostly be a problem for random access (nnrpd), not sequential access
> (innd), if Linux has a halfway decent inode cache).  I know even
> less about other Linux file systems...
>
> > INN imposes no limitations; the limitation is in the kernel.  It's a
> > common limitation; I don't know of any operating system at present that
> > supports memory mapping raw devices over 2GB.  (Tru64, probably, if it
> > supports mmap on raw devices at all, but I'm not sure it does.)
>
> I couldn't get it to mmap raw devices at all--we ended up with large
> files on AdvFS (an extent based filesystem).
>
> -dan
>
> [1] http://web.mit.edu/tytso/www/linux/ext2intro.html



More information about the inn-workers mailing list