Feed problem

[Brennon Church]

Hello,

On Mon, Oct 16, 2000 at 11:42:38AM -0700, Joe St Sauver wrote:
> >There are two drives in the system, one 4gig boot drive, and a 50gig spool
> >drive.  The system is transit only, so retention isn't an issue.  Watching
> >a systat iostat, I see that the transfers per second go between 50 and 100,
> >and the MB/s is about 1.5 to 1.7.
> 
> That may be the problem right there... I'd plan on running multiple drives
> striped as a RAID0 stripe set. You also didn't mention the type of drive,
> but I assume ultra or ultra2 SCSI... a single drive solution will generally
> not be satisfactory.

I'd thought that as well.  In fact, I've tested this on another system that
is running on 7 drives.  They aren't raided at all, but with cycbuffs, that
shouldn't matter.  I get basically the same results.  Still showing the
sockets as mostly full.  And yes, the drives are ultra2 SCSI.

> 
> Also, what sort of network connection? I'd recommend fast ethernet or gig
> ethernet. I also assume that you are not capacity limited on the wide area
> side of things, true?

All 100M full-duplex ethernet.  

> 
> The other factor to keep in mind is that you may be hitting a bandwidth
> delay product limitation depending upon on how remote your peers are, how
> many feeds you have open to them, packet loss rates, etc. See: 
> http://www.psc.edu/networking/perf_tune.html

In this case, the feed is actually coming from a Cidera Skycache feed.  So
it's coming from an extremely local system (connected to the same switch
within our network).

> 
> >On a side note... What is about the average size for a full news feed
> >nowadays.  I've managed to get up to 152Gigs in a day, but I'm usually
> >hovering around 140 or so.
> 
> That sounds approximately correct to me, depending on what you carry and
> don't carry (individual groups may shift your feed number by 10% or more).

I carry anything I can get my hands on.

> For general trending information, see: 
> http://newsfeed.mesh.ad.jp/flow/index.html

Thanks, I'll check that out.

> 
> Regards,
> 
> Joe

-- 
Brennon Church
System Administrator
CTSnet Internet Services
mailto:ctsbac at cts.com
t 858.637.3600
f 858.637.3631