RtConfig has limited cache support

Ruediger Volk, Deutsche Telekom T-Com - TE141P1 rv at nic.dtag.de
Wed May 16 12:03:29 UTC 2007

Hi Glen,

thanks for your comments
  > On Tue, 2007-05-15 at 18:54 +0200, Hagen Boehm wrote:
  > > Currently I'm working on a fix, but it would be nice to know the reason
  > > for the limited cache support.
  > Hi Hagen,
  > I don't know the reason for the lack of a cache.
  > I do know that RtConfig is the wrong place to be caching. Caching
  > in RtConfig assumes that you are only going to run RtConfig once.
  > That seems a bit unlikely.
The name "cache file" for RtConfig/peval's option "-f" is probably a 
misnomer; the actual function is only loosely related to the commonly
known concept of caching; but it's the term used in the documentation
- and mislead you about the intended use.
The "cache files" (the programs actually process multiple -f options!)
can be used to provide definitions of RPSL objects (e.g. route-sets,
- that will have precedence over definitions of the same name from
  the database server
- can help for locally defined private data suplementing content available
  from the database server used (only a single one can be used with a single
  selection/precedence list of "sources")
- can help to avoid running/using/feeding a database server
  (which may require serious overhead in particular for local or temporary
  data - e.g. during testing and developement)

  > It would be nicer to have a caching server running on localhost.
  > Then if you are running RtConfig for each router in your network
  > you end up with less WHOIS traffic and a consistent configuration
  > across all of the routers.
I don't see how the current IRRtoolset would straight forward support
such an idea of caching.  Is there any RPSL database server implementation
that supports this operational model?
The database servers that I know support mirroring but not caching.

  > RtConfig is far too complex, and stripping complexity out of it
  > would make maintenance a lot simpler.
I certainly agree;  we are using actually only a pretty small subset of
the functionality:  slightly more than what is available through peval
+ AS path regular expressions.
However in my view the "-f cache file" option is pretty basic,
and I consider it an important requirement.

To get the required output formats (Cisco IOS, Cisco IOS-XR, Juniper JunOS,
and RPSL) we - or more precisely: Hagen - actually built our own frontend
for the peval/RtConfig code base...
With our RPSL output format we can actually retrieve and store data 
for caching - and could feed the production program run via the "-f cache-file"
options; though transparent automatic caching still would require more
additional work...

So we consider the failure of peval/RtConfig to use the "-f cache files"
when the access protocol for the RPSL database is the default "irrd"
a surprising problem.
Our analysis shows that the cache files are in fact read and stores into
internal data structures regardless of the database access protocol.
Different methods are used for evaluating RPSL expressions depending
on the database access protocol;  unfortunately only the methods for
"ripe" make use of the internal cache data structures.

For fixing this it would be helpful to know:
- was there support for using the cache file with the default
  access protocol "irrd" (=rawhoisd) at any time?
- or was the "-f cache file" option only added for other access protocols
  (such as "ripe")?
  (if so, why does the documentation of "-f" not mention any restrictions?)
- any information on for which version of the RA/IRRtoolSet the option
  was created/withdrawn?
- any additional information from previous developers on cache use with
  the default access protocol?

Best regards,

Ruediger Volk

Deutsche Telekom AG -- Internet Backbone Engineering

E-Mail:  rv at NIC.DTAG.DE

More information about the irrtoolset mailing list