Re: Storing a lot of little files

Eugene wrote:

>Hello freebsd-current,
>
>  I need to store a lot (hundreds of millions) of very little files (from 8 bytes
>  to 50K) in my filesystem.
>  What's the best way to optimize it? Which newfs options can you
>  recommend me?
>  Can UFS2 optiomize storage of little files? For example, put some
>  files in one cluster - so don't let the 8 byte file to occupy a
>  whole cluster of about 16K?
>  Is it possible to allocate inodes dynamically (like in ReiserFS?),
>  bacause I'm not sure I can predict accuratly number of little files,
>  and inodes not to finish before disk get full.
>
>  Does FreeBSD support file systems except of UFS?
>  What about ReiserFS, XFS, JFS?
>  
>

I've never done 'hundreds of millions', but I've done about 10 
million..  I can say that you will definitely need to organize them in a 
heirarchical directory structure..  You'll want to look at the following 
options to newfs:

-g -h -i

and these options to tunefs: -e -f -s


Eric




-- 
------------------------------------------------------------------
Eric Anderson     Sr. Systems Administrator    Centaur Technology
Talk sense to a fool and he calls you foolish.
------------------------------------------------------------------