Skip to content

CreateRepo_SL

Jump to bottom
cstackpole edited this page May 19, 2013 · 7 revisions

Table of Contents

Web Server / YUM repo

A local repository is useful for many reasons, but the top two that most consider are: Speed and lower bandwidth. Where you place it on your network is dependant on many factors. It does not, and probably should not be on the frontend, but it should be accessible. This guide assumes that it is on the same subnet as the frontend but does not have access to the nodes directly.

Install

This is a simple minimal install of Scientific Linux 6.4 (SL64) from the install DVD's.

  • Use DVD as install source
  • Skip DVD check.
  • Next past the Logo
  • Language = English
  • Keyboard = US English
  • Basic Storage
  • Hostname = http.cluster.domain
  • Configure Network
    • Edit System eth0
    • Connect Automatically
    • IPv4 Settings
      • This is completely dependant on your network configuration. These instructions are here for those that need to edit the network manually.
    • Apply, close, and next.
  • Time Zone = America/Chicago
  • Set root password
  • Create custom layout for partitions.
    • Wipe all the partitions on the drive and make all new partitions primary partitions.
    • The information below is just a recommendation. This is a minimal install for a single purpose use: A local repository. Not a lot of space is required, still this should be a bare minimal amount of space so that the majority of the disk can be used for the repository. If you have the disk space or if you have to comply to a standard then feel free to make this bigger. As for swap space, the old adage used to be 1.5x the size of memory. These days my work servers have upto 512GB(!!) of memory. If your server HAS to use more than 4GB of swap than you have a _serious_ problem. More swap isn't the answer. Then again, all of these sizes are just recommendations based on personal experience. You know your needs better then I do so adjust accordingly.
      • / : 5GB (min) or 20GB (max). This should be the first primary partition.
      • swap : 2GB (min) or 4GB (max). This should be the second primary partition.
      • Remainder of disk to physical volume LVM. This should be the third primary partition. Why LVM? Repurposing use. It isn't a great idea to have a directory like /tmp, /home, or any other user-fillable volume on /. No one wants their server to freak out because of no space in /. Since there can only be 4 primary volumes and there are at least 3 more volumes we want to create then LVM helps solve the issue. LVM also allows an easy way to adjust volume sizes later on if it must happen.
        • /home : 10GB (min) or 50GB (max)
        • /tmp : 10GB (min) or 20GB (max)
        • /var : Remainder so long as the space is greater than 60GB.
  • Set boot loader password.
  • Minimal install.
  • Reboot

Copy Repo

This depends on how you get the packages to the network. Not everyone has access to the internet on their cluster, so there are two variations. Using the packages as provided by the DVD, and rsyncing from an upstream mirror.

Select the option that best fits your needs.

Copying off of the DVD

  • mkdir -p /var/www/http/scientificlinux/6.4/x86_64/os/
    • Change the above if you are working with i386
  • cp -r /media/cdrom/Packages /var/www/http/scientificlinux/6.4/x86_64/os/.
  • cd /media/cdrom/Packages
    • Change directories into whatever directory hold your rpms so we can install the HTTPd server. Replace $ARCH with either i686 or x86_64 depending on your architecture..
  • rpm -ivh httpd-2.*.$ARCH.rpm httpd-tools-*.$ARCH.rpm mailcap-2*.noarch.rpm apr-1*.$ARCH.rpm apr-util-ldap*.$ARCH.rpm apr-util-1*.$ARCH.rpm deltarpm*.$ARCH.rpm python-deltarpm*.$ARCH.rpm libxml2-python*.$ARCH.rpm createrepo*.noarch.rpm system-config-firewall-tui*.noarch.rpm
    • There should only be 11 packages installed. I did not list the full file name just in case your version has slightly different version numbers. You can run `ls` instead of `rpm -ivh` to verify that there are only 11. You can also just type the string and at the * hit the tab button for auto-complete.
  • Ensure the webserver starts on boot.
    • chkconfig httpd on
  • Open the firewall for HTTP
    • system-config-firewall-tui
    • Customize
    • Select "WWW (HTTP)" and close
    • 'OK' you have configured the firewall and 'Yes' you want to reload the configuration.
  • Start the webserver
    • service httpd start
    • Ignore the errors for now; we will fix them later.
  • Create your repo.
    • createrepo /var/www/html/scientificlinux

Syncing from an upstream repository

  • Install needed tools
    • yum install httpd rsync system-config-firewall-tui
  • Create a repository and sync from an upstream mirror.
    • mkdir -p /var/www/html/scientificlinux/6.4/
    • rsync -avSHlP --delete --exclude=archive --exclude=repoview rsync://rsync.scientificlinux.org/scientific/6.4/ /var/www/html/scientificlinux/6.4/
      • This command will copy /everything/ so if you know you don't need i386 or x86_64, then only copy that architecture. No need wasting their bandwidth. Also, some additional useful excludes are:
        • --exclude=SRPMS You may have a need for a few source RPMS, but do you really need them all?
        • --exclude=iso You probably do not need a local copy of every ISO they have.
  • Open the firewall for HTTP
    • system-config-firewall-tui
    • Customize
    • Select "WWW (HTTP)" and close
    • 'OK' you have configured the firewall and 'Yes' you want to reload the configuration.
  • Start the webserver
    • service httpd start
    • Ignore the errors for now; we will fix them later.
  • Ensure the webserver starts on next reboot
    • chkconfig httpd on
  • Edit crontab and add your rsync command to run just after midnight every night.
    • 00 01 * * * root /usr/bin/rsync -avkSH --delete --exclude=archive --exclude=repoview --exclude=SRPMS --exclude=iso rsync://rsync.scientificlinux.org/scientific/6.4/ /var/www/html/scientificlinux/6.4/

Update the yum conf files

  • Edit your yum config files. You need to update all the repos you have on your system. If it is a full mirror you need to check them all; if it is just the DVD you need to disable all but the main sl. Starting with SL6.4 the sl6x repo is enabled by default. This pulls updates from the next version of SL. If you have any need to stay close to RHEL or CentOS then disable it or just delete the file sl6x.repo.
    • An example of your base /etc/yum.repos.d/sl.repo file
  [sl]
  name=Scientific Linux $releasever - $basearch
  baseurl=http://http.cluster.domain/scientificlinux/$releasever/$basearch/os/
  enabled=1
  gpgcheck=1
  gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-sl file:///etc/pki/rpm-gpg/RPM-GPG-KEY-dawson
    • If your repo contains the "security" addition include this in your /etc/yum.repos.d/sl.repo file
  [sl-security]
  name=Scientific Linux Security $releasever - $basearch
  baseurl=http://http.cluster.domain/scientificlinux/$releasever/$basearch/updates/security/
  enabled=1
  gpgcheck=1
  gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-sl file:///etc/pki/rpm-gpg/RPM-GPG-KEY-dawson
    • If your repo contains the "fastbugs" addition include this in your /etc/yum.repos.d/sl-other.repo file
  [sl-fastbugs]
  name=Scientific Linux Fastbugs $releasever - $basearch
  baseurl=http://http.cluster.domain/scientificlinux/$releasever/$basearch/updates/fastbugs/
  enabled=1
  gpgcheck=1
  gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-sl file:///etc/pki/rpm-gpg/RPM-GPG-KEY-dawson
    • When it comes time, a local repo will be needed for custom built packages. The file cluster.repo should look something like the below.
  [Cluster_Repo]
  name=Cluster_Repo $basearch
  baseurl=http://http.cluster.domain/Cluster_Repo
  enabled=1
  gpgcheck=1
  gpgkey=http://http.cluster.domain/Cluster_Repo/RPM-GPG-KEY-your_user_name_or_project_name

Update permissions and verify the repo works

  • Update the SELinux permissions
    • restorecon -RFvv /var/www/
  • Check to ensure yum works and update your system.
    • yum clean all && yum update
    • You may need to reboot if the update upgraded your system to a new kernel.

Install from new repo

  • Install SSH client, acpid, and createrepo
    • yum install openssh-clients acpid createrepo

Setup Repo and Kickstart Folders

Now to set up the repo files for download from other machines.

  • Create the Repo directory
    • mkdir /var/www/html/repo_files
  • Copy the files
    • cp /etc/yum.repos.d/* /var/www/http/repo_files/.
  • Create the directory structure for your project repo. This is where all your in-house-built RPMS will reside.
    • mkdir -p /var/www/html/ClusterRepo/{i386,x86_64, noarch}
  • Create the Kickstart directory for the nodes.
    • mkdir /var/www/html/ks
  • Configure SELinux
    • restorecon -RFvv /var/www

Disable Autoupdate

  • Don't forget to disable yum auto update!
Clone this wiki locally