iRODS Release Notes
Complete history of iRODS Release Notes.
iRODS Upgrade Instructions
If you are upgrading to a new version of iRODS and want to keep your existing state information (the ICAT information on dataObjects, collections, resources, users, etc), run either of these scripts (they are equivalent):
./irodsupgrade or ./irodssetup --upgrade
and follow the prompts.
This runs in a similar manner as irodssetup described in Install (described below), but with fewer prompts and default settings for upgrading an existing iCAT.
Note that the ./irodsupgrade is run in the new iRODS directory that is created when you unpack the release tarball, the irodsx.y.tgz file.
Upgrading from iRODS x.y to the next version
- Note that iRODS Servers are usually backward compatible but iRODS Clients sometimes are not. See the particular Release Notes for the version you are installing.
- STEPS: Upgrade iRODS Servers first, and after this upgrade iRODS Clients.
- If a patch to the ICAT database is required when converting, the 'irodssetup --upgrade' (equivalent to 'irodsupgrade') script will warn and prompt you about this and explain what steps are needed.
- The new core.re file may contain new rules that are needed for the new release, so if you have your own core rules and so you can't simply use the released core.re, you'll need to compare yours with the new version and integrate the changes.
- For more information see the Release Notes for the particular release you are installing. There is also an INSTALL.txt file in the iRODS code distribution.
To upgrade more than one version step of iRODS, first upgrade from your current version to each version step in between. There may be iCAT changes that need to be installed for some steps.
- For more information see the INSTALL.txt file in the iRODS code distribution.
iRODS Install Overview
iRODS, the Integrated Rule-Oriented Data System, has a client-server architecture.
Data is stored on one or more iRODS Servers and accessed and managed by iRODS Clients.
The iRODS Metadata Catalog, or iCAT, runs on a database. The open source PostgreSQL db can be automatically installed as part iRODS installation script, or iRODS can use an existing database (PostgreSQL, MySQL, Oracle, but see PostgreSQL caveat in next sentence). In most cases problems are encountered when trying to use an existing PostgreSQL installation (perhaps due to an incompatibility between ODBC and PostgreSQL), so it is recommended that you opt to have 'irodssetup' install a new PostgreSQL installation.
Installation builds the iRODS Server, Client(s), and the iCAT Metadata Catalog (or optional subsets of these).
Installation is very straightforward, with a script that fully automates the procedure for Unix-based OSs including Mac OSX. For Windows information click here.
Skills: To install iRODS it's helpful to have a Unix background (compiling code, setting environment variables) or access to IT assistance with this experience.
Installing a basic iRODS system does not take long using the Installation Script:
- Full installation of a) iRODS Server, b) iCAT Metadata Catalog (including PostgreSQL database), c) iRODS Clients ~30 minutes
- iRODS Server and iRODS Clients (if iCAT is already available) ~10 minutes
- iRODS Clients ~3 minutes
For more information see the INSTALL.txt file in the iRODS code distribution.
- Unix-based operating systems: Linux, Solaris, Macintosh, AIX.
- All three iRODS components supported: iRODS Server, iCAT Metadata Catalog, Clients.
- Windows (Vista, XP).
To build iRODS (and the Postgresql database if you're using it) the Unix-based host needs a gcc (or cc) compiler and the standard include files and libraries.
Macintosh OSX: You need the development environment, which you can download from Apple at http://developer.apple.com.
Linux: On some distributions (e.g. Ubuntu) the compiler is installed by default, but you'll need to install the "build-essential" package for the include files (header files) and libraries: use the System pull-down menu, then Administration, and use "Synapic Package Manager" and mark "build-essential" for installation; it will ask for the distribution CD.
If the c++ compiler is not already, you'll need to install it.
On RedHat Linux (Red Hat 4.4.6-3) it was reported that the following installed c++:
yum install gcc gcc-cpp gcc-c++ kernel-devel kernel-headers ;yum groupinstall "Development Tools"
And when using Postgres/ODBC on that platform libtdl was needed:
yum install libtool-ltdl qt qt-devel
Windows: For Windows information click here.
The iCAT Platforms page lists the supported OSs and configurations for ICAT-enabled servers.
Quick Start iRODS Install Instructions
The install script (for Unix-based OS) is very comprehensive (for Unix-based OSs, click here for Windows information). For iRODS 1.0 and beyond all you do to install is run the 'irodssetup' script at the top level of the release.
• Unpack the tar distribution file (gzip -d irods2.0.tgz; tar xf irods2.0.tar), as a non-root user cd into the directory (iRODS).
• Run 'irodssetup' which will prompt for a few options and parameters (see below).
The same script handles all three types of installations:
a) The full iRODS Metadata Catalog (iCAT)-enabled server (i.e. all three components: iRODS Server, iCAT Metadata Catalog, Client (i-commands); b) Additional non-ICAT-enabled server(s) that use the existing iCAT; c) i-commands clients-only installations.
For iCAT-enabled servers, it supports PostgreSQL, MySQL, or Oracle databases. When using PostgreSQL the script can download and install PostgreSQL and ODBC (UnixODBC or Postgres ODBC). An existing installation of MySQL or Oracle can be used.
For all cases the script configures and builds all selected components and sets up the user environment. See INSTALL.TXT for more information.
Take a look at an iRODS Installation Example.
The Install Script prompts you for key iRODS configuration options.
Often the standard settings are sufficient, and Default values (if any) are shown in square brackets [ ] at each prompt. Press return to use the default, or enter a new value.
If you need more control you can choose to see additional prompts for advanced settings, where additional questions will be asked, see below.
iRODS consists of clients (e.g. i-commands, iRODS Web browser) with at least one iRODS Server.
One iRODS Server must include the iCAT Metadata catalog.
For the initial installation, you normally build the iRODS Server with the iCAT (an iCAT-Enabled Server or IES), along with the i-commands (Unix-like command line interface).
In the future, you may want to build another iRODS Server to support a storage resource on another computer. In this case you build the additional iRODS Server non-iCAT, and configure it with the existing iCAT-enabled Server host name (the additional servers connect to the iCAT-enabled Server for iCAT metadata operations).
If you already have an iCAT-enabled iRODS Server installed, you may skip building the iRODS Server and iCAT, and just build the i-commands command-line tools.
Your iRODS Zone
The initial iRODS installation builds an iRODS Zone, which is an iRODS Server with associated iCAT Metadata Catalog (an iCAT-enabled Server). Supporting distributed data collections, your iRODS Zone can be expanded to include additional non-iCAT enabled servers (which can reach hundreds and may be distributed worldwide) supported by the initial iCAT-enabled iRODS Server.
The build process will create a new iRODS administrator account for managing the iRODS system (unless one already exists in an iCAT-enabled Server).
Each iRODS Zone has a unique name, which appears at the beginning of collection names.
iRODS Zones can interoperate (that is, independent iRODS systems (iCAT Metadata Catalogs) can communicate) in what is called Zone Federation.
Database Options for iCAT Metadata Catalog
The iCAT Metadata Catalog uses a DBMS (database management system) to store complete state information about the system (data objects, collections, meta-data, user information, resource information, etc.) in a database. You have two choices:
1. Have the Install Script download and build a new PostgreSQL DBMS system.
2. Use an existing PostgreSQL, Oracle, or MySQL DBMS and database.
Multiple versions of PostgreSQL are available from their web site. We recommend the default, or newer, version.
Using an existing PostgreSQL instance usually fails, perhaps due to an incompatibility between ODBC and PostgreSQL. If you want to use PostgreSQL, we recommend having the irodssetup script download and build it (and ODBC) for you.
Future releases of iRODS iCAT may support additional database systems.
iRODS has a secure password system (challenge/response, no plain-text).
iRODS can also make use of the Grid Security Infrastructure (GSI).
In most cases, the iRODS password system is sufficient, but if you are using GSI for other applications you may want to include GSI in iRODS. Both clients and servers need to be built with GSI, and then users can select it by setting irodsAuthScheme=GSI in their .irodsEnv files (or still use the iRODS password system if they wish).
Installing Additional Clients
If you already have an iCAT-enabled Server and want to install additional i-command clients on different systems, follow these steps.
1. Use the iRODS Install Script (which handles all installation tasks -- clients, server, and iCAT).
2. You run irodssetup on the remote host where you want the client, and via the prompts tell it you want to build only the i-commands client (which causes the install script to ask many fewer questions and do a simple configure/build of just the i-commands client).
3. Then you need to set up your user environment as described at user environment.
The easiest is to copy your HOME/.irods/.irodsEnv file from your iCAT-enabled server to the remote client.
To see collections, use the 'ils' command.
For information on installing the iRODS iExplore Windows client click here, and additional client options are on the left navigation bar of the wiki home page.
Managing Multiple .irodsEnv files/Accessing Multiple Hosts
Sometimes you may need to access more than one iRODS Server, and each server requires different ienv information from a .irodsEnv file. There are multiple ways you can handle this, but here is one example of a manual process.
Example .irodsEnv 1:
localhost:iRODS jsmith$ ienv
NOTICE: Release Version = rods2.0.1, API Version = d
Example .irodsEnv 2:
localhost:iRODS jsmith$ ienv
NOTICE: Release Version = rods2.0.1, API Version = d
In your ~/.irods/ directory, store each file separately. In the example below, the .irodsEnv file will be the one created when you installed iRODS on this host. In this case, the .irodsEnv information is in #2 above; let's say that the information in #1 above was emailed to you. First, make a copy of your original .irodsEnv file with some kind of unique name added at the end. Then, create a second environment file from the information emailed to you (a quick cut and paste), again using the .irodsEnv name but with a different unique name added at the end.
localhost:~ jsmith$ cp ~/.irods/.irodsEnv ~/.irods/.irodsEnv_localhost
localhost:~ jsmith$ vi ~/.irods/.irodsEnv_yourgroup1
(Insert the text, write, and quit the vi file.)
Check to see your files are there:
localhost:.irods jsmith$ ls -l
rwx------ 23 jsmith staff 782 May 29 15:18 .
drwxr-xr-x+ 41 jsmith staff 1394 May 29 15:18 ..
-rw------- 1 jsmith staff 14 May 29 15:18 .irodsA
-rw-r--r-- 1 jsmith staff 494 May 29 15:17 .irodsEnv
-rw------- 1 jsmith staff 494 May 29 13:35 .irodsEnv_yourgroup1
-rw-r--r-- 1 jsmith staff 492 May 29 13:35 .irodsEnv_localhost
Now check to make sure the correct ienv information is in each file:
localhost:~ jsmith$ more .irodsEnv_yourgroup1
localhost:~ jsmith$ more .irodsEnv_localhost
Since the current .irodsEnv contains the information now copied into .irodsEnv_localhost, and you want to access the host for .irodsEnv_yourgroup1, then at the command line type:
localhost:~ jsmith$ iexit
localhost:~ jsmith$ cp ~/.irods/.irodsEnv_yourgroup1 .irodsEnv
localhost:~ jsmith$ iinit
You will be prompted for your iRODS password. Enter it, and you should be now be able to access both the iRODS server and iCAT for host server.yourgroup1.org. To check, at the command line type:
localhost:~ jsmith$ ienv
localhost:~ jsmith$ ils
(Remember: "ils" checks your iCAT access.)
To switch to localhost, substitute '.irodsEnv_localhost' for .irodsEnv_yourgroup1 in the steps above.
Common Installation Issues
Postgres on Mac OS X
Starting with iRODS 2.5 a new installation will download PostgreSQL 9.x.x, whose shared memory requirements may exceed the default settings on OS X.
Depending on your configuration you will see this error during setup:
Step 4 of 4: Setting up Postgres... Initializing database... [...] creating template1 database in /Users/test/Postgres/pgsql/data/base/1 ... FATAL: could not create shared memory segment: Cannot allocate memory [...] Abort. Please re-run this script when the problem is fixed.
Postgres' documentation provides useful insight into the problem.
So far on OS X the amount of shared memory is set by default to 4 MB regardless of your hardware. This limit is considered rather low and safe to increase on any Mac that is reasonably new.
In the example below we are going to make it 4 times the default value:
To check your current settings:
sysctl -a | grep kern.sysv.shm
You should see:
kern.sysv.shmmax: 4194304 kern.sysv.shmmin: 1 kern.sysv.shmmni: 32 kern.sysv.shmseg: 8 kern.sysv.shmall: 1024
The new settings must be written to /etc/sysctl.conf
If that file isn't there, create a new one (sudo vi/emacs) with the following lines:
kern.sysv.shmmax=16777216 kern.sysv.shmmin=1 kern.sysv.shmmni=128 kern.sysv.shmseg=32 kern.sysv.shmall=4096
You will need to restart your system for the update to take effect. Then type 'sysctl -a | grep kern.sysv.shm' again and you should see:
kern.sysv.shmmax: 16777216 kern.sysv.shmmin: 1 kern.sysv.shmmni: 128 kern.sysv.shmseg: 32 kern.sysv.shmall: 4096
You can now run the irodssetup script. Crossing fingers...
iRODS configuration setup ---------------------------------------------------------------- Use the existing iRODS configuration without changes [yes]? Start iRODS build [yes]? Build and configure ------------------- [...] Step 6 of 6: Configuring iRODS user and starting server... Updating iRODS user's ~/.irods/.irodsEnv... Starting iRODS server... Opening iRODS connection... Creating default resource... Testing resource... Done!