Install R Packages in a Server or Cluster Environment

  1. Install R Packages in a Server or Cluster Environment
Advertisements

Compiling R-3.4.2 on CentOS 6 with GNU

If you are compiling CentOS 6, you will notice that the R source will not compile without a updated version of zlib, pcre, bzip2. libcurl. There is a very good site on how to compile R on Red Hat Linux

Step 1: Compile R-3.4.2

Download R-3.4.2 from The R Project for Statistical Computing

$ tar -zxvf R-3.4.2.tar.gz
$ ./configure --prefix=/usr/local/RH6_apps/R-3.4.2

During configuring…. error surfaced.

checking for zlib.h... yes
checking if zlib version >= 1.2.5... no
checking whether zlib support suffices... configure: error: zlib library and headers are required

Step 2: Compile zlib 1.2.11

a. In your .bashrc

export CFLAGS="-I/usr/local/zlib-1.2.11/include"
export LDFLAGS="-L/usr/local/zlib-1.2.11/lib"

b. Compile zlib-1.2.11

$ wget http://zlib.net/zlib-1.2.11.tar.gz
$ tar -zxvf zlib-1.2.11
$ cd zlib-1.2.11
$ configure --prefix=/usr/local/zlib-1.2.11

Step 3: Compile bzip2
a. Download the bzip2

$ wget http://www.bzip.org/1.0.6/bzip2-1.0.6.tar.gz
$ tar xzvf bzip2-1.0.6.tar.gz
$ cd bzip2-1.0.6

b. Compile bzip2

$ make -f Makefile-libbz2_so
$ make clean
$ make -n install PREFIX=/usr/local/R-3.4.2
$ make install PREFIX=/usr/local/R-3.4.2

Step 4: Compile liblzma or xz

$ wget http://tukaani.org/xz/xz-5.2.3.tar.gz --no-check-certificate
$ tar xzvf xz-5.2.3.tar.gz
$ cd xz-5.2.3
$ ./configure --prefix=/usr/local/xz-5.2.3
$ make -j8
$ make install

Step 5: Compile pcre-8.40

$ wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.40.tar.gz
$ tar xzvf pcre-8.40.tar.gz
$ ./configure --prefix=/usr/local/pcre-8.40
$ make -j 8
$ make install

Step 6: Compile libcurl-7.47.1

$ wget  https://curl.haxx.se/download/curl-7.47.1.tar.gz --no-check-certificate
$ tar xzvf curl-7.47.1.tar.gz
$ cd curl-7.47.1
$ ./configure --prefix=/usr/local/curl-7.47.1
$ make -j 8
$ make install

Step 7: Update CFLAGS and LDFLAGS

export CFLAGS="-I/usr/local/zlib-1.2.11/include -I/usr/local/bzip2-1.0.6/include -I/usr/local/xz-5.2.3/include -I/usr/local/pcre-8.40/include -I/usr/local/curl-7.47.1/include/curl"
export LDFLAGS="-L/usr/local/zlib-1.2.11/lib -L/usr/local/bzip2-1.0.6/lib -L/usr/local/xz-5.2.3/lib -L/usr/local/pcre-8.40/lib -L/usr/local/curl-7.47.1/lib"

Step 8: Compile R-3.4.2

$ ./configure --enable-utf8 --prefix=/usr/local/R-3.4.2
$ make -j 8
$ make install

References:

  1. Building R-devel on RedHat Linux 6

Rectifying Corrupted Keytab file on Centrify Client

Do the required Centrify Log Collection. Do take a look at Collecting logs from Centrify Client

If you notice the logs

============

Oct  5 20:19:06 lsf-login2 adclient[16400]: DEBUG <fd:27 PAMVerifyPassword > base.osutil Module=Kerberos : while getting service credentials: Decrypt integrity check failed (reference base/aduser.cpp:1629 rc: -1765328353) Oct  5 20:19:06 lsf-login2 adclient[16400]: DEBUG <fd:27 PAMVerifyPassword > base.aduser Unable to verify user’s credentials: while getting service credentials: Decrypt integrity check failed Oct  5 20:19:06 lsf-login2 adclient[16400]: DIAG  <fd:27 PAMVerifyPassword > daemon.ipcclient validate password caught exception: while getting service credentials: Decrypt integrity check failed Oct  5 20:19:06 lsf-login2 adclient[16400]: WARN  <fd:27 PAMVerifyPassword > audit User ‘user1’ not authenticated: while getting service credentials: Decrypt integrity check failed Oct  5 20:19:06 lsf-login2 adclient[16400]: DEBUG <fd:27 PAMVerifyPassword > daemon.ipcclient2 doPAMVerifyPassword: user ‘jliu024’ not OK: -1765328353 (Kerberos) Oct  5 20:19:06 lsf-login2 adclient[16400]: DEBUG <fd:27 PAMVerifyPassword > daemon.ipcclient2 request ‘PAMVerifyPassword’ complete =======================

To resolve the issue, do the following

# adkeytab -r -u User1

And try to restart adclient again as root

# /usr/share/centrifydc/bin/centrifydc restart

Collecting logs from Centrify Client

On the Centrify Unix server, as root or sudo, please run the following commands:

1. Turn on Centrify debug:

# /usr/share/centrifydc/bin/addebug on
# /usr/share/centrifydc/bin/addebug clear

2. Reproduce the issue, attempt to log-in.

3. Collect adinfo:

# adinfo -t

4. Turn Off Debug

# /usr/share/centrifydc/bin/addebug off

5. Run adquery and dzinfo for the selected Users:

# adquery user -A UserID1 > /tmp/adquery.txt
# dzinfo UserID1 > /tmp/dzinfo.txt

6. Send the Logs to Centrify Support

/var/centrify/tmp/adinfo_support.tar.gz
/tmp/adquery.txt
/tmp/dzinfo.txt

Developing a Linux Kernel Module using GPUDirect RDMA

Taken from Developing a Linux Kernel Module using GPUDirect RDMA

1.0 Overview

GPUDirect RDMA is a technology introduced in Kepler-class GPUs and CUDA 5.0 that enables a direct path for data exchange between the GPU and a third-party peer device using standard features of PCI Express. Examples of third-party devices are: network interfaces, video acquisition devices, storage adapters.

GPUDirect RDMA is available on both Tesla and Quadro GPUs.

A number of limitations can apply, the most important being that the two devices must share the same upstream PCI Express root complex. Some of the limitations depend on the platform used and could be lifted in current/future products.

A few straightforward changes must be made to device drivers to enable this functionality with a wide range of hardware devices. This document introduces the technology and describes the steps necessary to enable an GPUDirect RDMA connection to NVIDIA GPUs on Linux.

 

1.1. How GPUDirect RDMA Works

When setting up GPUDirect RDMA communication between two peers, all physical addresses are the same from the PCI Express devices’ point of view. Within this physical address space are linear windows called PCI BARs. Each device has six BAR registers at most, so it can have up to six active 32bit BAR regions. 64bit BARs consume two BAR registers. The PCI Express device issues reads and writes to a peer device’s BAR addresses in the same way that they are issued to system memory.

Traditionally, resources like BAR windows are mapped to user or kernel address space using the CPU’s MMU as memory mapped I/O (MMIO) addresses. However, because current operating systems don’t have sufficient mechanisms for exchanging MMIO regions between drivers, the NVIDIA kernel driver exports functions to perform the necessary address translations and mappings.

To add GPUDirect RDMA support to a device driver, a small amount of address mapping code within the kernel driver must be modified. This code typically resides near existing calls to get_user_pages().

The APIs and control flow involved with GPUDirect RDMA are very similar to those used with standard DMA transfers.

References:

Read more at: http://docs.nvidia.com/cuda/gpudirect-rdma/index.html

Remove Mapping of Null User to Windows User or Group in NetAPP

If you need to remove the mapping of null users to windows user or group in NetApp, it can be done with the following steps

Step 1: Set the privilege level to advanced:

# set -privilege advanced

Step 2: Configure the restrict anonymous setting:

# vserver cifs options modify -vserver vserver_name -restrict-anonymous {no-restriction|no-enumeration|no-access}

Step 3: Verify that the option is set to the desired value:

vserver cifs options show -vserver vserver_name

Step 4: Return to the admin privilege level:

set -privilege admin