Skip to end of metadata
Go to start of metadata

Install squid caching server to reduce build time

Building using mock will trigger downloads of many RPMs from CentOS and Fedora mirrors each time the chroot initialized, which is often enough. If you install a caching proxy server, then RPMs are only downloaded once then cached on your local network. This will speed build times up substantially after running through a complete build once.

I recommend these settings in file /etc/squid/squid.conf for optimal caching. You can place these settings at the top of the config file.

Then call

Finding a proper mirror site

yum is normally configured to selected the optimal site for downloading packages based on speed and on spreading the traffic equally among several mirror sites. When using a download proxy, you'll want to choose a single site for downloading packages that has the proper cache settings, is up reliably and is relatively quick. Be sure you understand why this is true otherwise you risk decreasing your build times and filling your disk cache quickly.

Here are the steps for selecting the proper mirror site.

1.) Check the CentOS mirror sites page for a single site that is in your area. Also consider reliability of the company or institution providing the mirror. Do some investigation on speed of the download speeds from the site.

2.) Test the mirror's cache settings

For a squid server : localhost
CentOS mirror site : http://mirrors.rit.edu/centos

You would run this command twice :

Where you'll see on the first run

Then on the second run

If you do not see HIT, select a different mirror.

3.) Follow steps 1 and 2 again to select a good fedora mirror fedora mirror list. You'll need to do this even if you're not planning for build on Fedora because the build system will require you to specify a fedora mirror url.
Example:

Building with a download proxy

To tell sipXecs build scripts to use your caching proxy, just add this configure switch when preparing to build sipXecs

You should verify caching is working by your second build by looking for the string TCP_MEM_HIT in /var/log/squid/access.log.

Clearing squid's cache.

If you switch mirror sites or you suspect the caching is causing build issues, you'll want to clear squid's cache. There is no easy way to clear a portion of the cache, but this set of commands will clear all of squid's cache.

Advanced: Using your download proxy for testing installations

You can also use your download proxy server to install systems faster. To use the proxy, edit /etc/yum.conf and add the setting proxy=http://your-squid-server:3128 to the [main] section.

Next you'll want to edit as many of the repository files as you can to use a fixed location for downloading. To do this, comment out the lines starting with mirrorlist= and uncomment out lines #baseurl. To automate editing of repos files to use base urls instead or mirror lists, this command should work.
sed -i -e 's/^mirrorlist/#mirrorlist/g' -e 's/^#baseurl/baseurl/g' /etc/yum.repos.d/*.repo

Labels
  • None