Experimental PC

Extra stuff for all the hardware in blanket & pouffe, the ARRO-SAM PC & OSAM

In December 2007 I upgraded blanket, the PC used to control the ARRO-SAM experiment, to the current version of openSuSE (10.3 when this page was first written, now 11.0). This PC had been running Redhat 9 for a number of years and due to heavy use as an experimental PC and reasonable stability, it had not been updated. Redhat 9 runs a 2.4 kernel, so there were some extra things to do with the drivers and compiling c-scan.

Addition (April 2008): Rob upgraded pouffe in the O-SAM lab, pretty much the same as blanket accept for ISA PI stages

The information contained here will also be useful if you want to install any of these bits of hardware on any other Linux PC.

Pre-install

Scenario 1: inherited PC with working hardware, updating Linux

Make a list of all the hardware you have (or likely to have) attached to the PC
From this list, work out which programs/utilities you use
- Most of these will already be installed in /usr/local/bin/
Work out how to recompile (and re-install) all these programs
- Most will have their source code in /home/scan/source/hardware/
Save all the configuration files you think you might need
- Most of these will be in /etc/

Example: updating blanket from RdeHat 9 to SuSE 10.3:

I tarballed /etc/ and /usr/local/bin/ and stuck the files in my home directory where they would be recoverable after new install. The aim was to retain the old root partition of the old (Redhat 9) system, to enable it to be mounted by the new (SuSE 10.3) system under /old_redhat_9_root/ but since installation involved resizing the old partition (always potentially problematic) it's a good idea to backup anything you think you might need. /etc/ for configuration files (including our own, for the stages, the SLM etc) and /usr/local/bin/ just as a reminder of the extra stuff (mainly our own) that we installed.

In the event, the old root partition wasn't wiped, and I was able to grab the files from there (/old_redhat_9_root). After the install, I copied the files I'd actually needed to /home/share/backup_system_files/blanket/

Scenario 2: Adding new hardware to an already up-to-date PC

Use this wiki, find the instructions for the relevant hardware
Any extra configuration files you need (e.g. /etc/stages/IW_stage_origin) - copy off a PC that already has this hardware installed (or you may find a copy in /home/share/backup_system_files/<name of PC> )

Install

I had to resize the old partition as it was taking up all the disk (except for the swap partition). I resized the old partition to 20GB (it had 15GB of stuff in it) and made a new 30GB root partition. I also extended the swap to 4GB.

Other than that I just did a standard install, then performed the standard local configuration stuff as with any other PC.

The resizing of the old partition was successful and I was able to copy the files I needed directly from /old_redhat_9_root/etc/

Scope (and AFG) utilities

These are the standard command line utilities to grab traces, save setups etc

vxi11

(Used by Agilent and Tek utilities, and a low-level diagnostic tool in its own right)

cd /home/scan/source/hardware/vxi11/
make clean; make; sudo make install
Installs:
- vxi11_cmd

Agilent scope

cd /home/scan/source/hardware/agilent_scope/
make clean; make; sudo make install
Installs:
- agetwf
- agilent_load_setup
- agilent_save_setup

Tek scope/AFG

cd /home/scan/source/hardware/tek/
make clean; make; sudo make install
Installs:
- tgetwf
- tek_load_setup
- tek_save_setup
- tek_afg_upload_arb

LeCroy scope

VXI11-compatible LeCroy scopes

Anything bought in 2010 onwards...

cd /home/scan/source/hardware/lecroy/
make clean; make; sudo make install
Installs:
lgetwf

VCIP-only LeCroy scopes

The old (non-Windows) LT564L scope, and Teti's old Waverunner 6050 (the one that runs Windows 2000).

cd /home/scan/source/hardware/ethernet_scope/
make clean; make; sudo make install
Installs:
- eth_cmd
- /etc/lecroy_tcp/ (empty directory)
cd /home/scan/source/egetwf/
make clean; make; sudo make install
Installs:
- egetwf

PI PCI stage driver

cd /home/scan/source/dev/pi_pci/driver_2.6/library/
make clean; make; sudo make install
Installs:
- pi_save_pos
- pi_recall_pos
- (pi_user library)
- /etc/pi_stage/stage_database/

cd /home/scan/source/hardware/pi_pci_stage/
make clean; make; sudo make install
Installs:
- set_stages_safe
- manual_pci_stage (manual_stage)
- /etc/pi_stage/position_um if it doesn't already exist (default stage position file, all axes at 0)
- /etc/pi_stage/installed_stages - this is the default list of installed stages, which you may need to edit if yours are different:

axis 0  m-531.dd
axis 1  m-531.dd
axis 2  m-605.2dd
axis 3  m-605.2dd

cd /home/scan/source/dev/pi_pci/driver_2.6/driver/
make
sudo ./install_pi_stage_recall
Installs:
- PI PCI stage device driver

Note: if you see something like the following after the last "make"....:

make -C /lib/modules/2.6.27.45-0.1-default/build M=/home/scan/source/dev/pi_pci/driver_2.6/driver modules
make[1]: Entering directory `/usr/src/linux-2.6.27.45-0.1-obj/x86_64/default'
make[1]: *** No rule to make target `modules'.  Stop.
make[1]: Leaving directory `/usr/src/linux-2.6.27.45-0.1-obj/x86_64/default'
make: *** [default] Error 2

... it means that you don't have the kernel-source package installed. To fix this:

sudo zypper install kernel-source

then...

cd /home/scan/source/dev/pi_pci/driver_2.6/driver/
make
sudo ./install_pi_stage_recall

PI ISA stage

cd /home/scan/source/hardware/stages
make -f Makefile.osam clean; make -f Makefile.osam;
sudo make -f Makefile.osam install
Installs:
- manual_stage
- /etc/stages/PI_stage_origin if it doesn't already exist (default stage position file, all axes at 0)

Inchworm and IDAC stages

cd /home/scan/source/hardware/IW_driver
sudo make install
Installs:
- /etc/stages/IW_stage_origin if it doesn't already exist (default stage position file, both axes at 0)
cd /home/scan/source/hardware/IDAC_driver
sudo make install
Installs:
- /etc/stages/IDAC_stage_origin if it doesn't already exist (default stage position file, both axes at 0)

BNS SLM

cp -av /old_redhat_9_root/etc/slm /etc/

cd /home/scan/source/dev/BNS/driver_2.6/
make

To install use

sudo ./install_slm_mtrr

OR

sudo ./install_for_boot

The latter install the kernel driver in the system and will load it automatically at boot time for you. If you want it load right now without rebooting use the previous version as well - matt

Installs:
- BNS SLM device driver

cd /home/scan/source/dev/BNS/utils/
make clean; make; sudo make install
Installs:
- bns_change_frame (change_frame_bns)
- bns_laser_ab
- bns_power
- bns_power_off
- bns_power_on
- bns_upload_frame (upload1_bns)

cd /home/scan/source/hardware/bns_slm/manual_bns/
make clean; make; sudo make install
Installs:
- manual_bns

cd /home/scan/source/hardware/bns_slm/set_bns/
make clean; make; sudo make install
Installs:
- set_bns

FFTW libraries

These are needed by the higher order correction algorithm when using the SLM in c-scan. I use the float version of the libraries (rather than the standard double version) for speed, and so an extra option must be set before compiling.

cd /home/scan/source/fftw-3.0.1_blanket/
make distclean; ./configure --enable-float; make; sudo make install
Installs:
- (FFTW float libraries)

Comedi

comedi kernel driver compilation

The "current release" version of the comedi is 0.7.76, however this will not compile on modern kernels (it's more than 2 years old). The current comedi policy seems to be one of continuous code development, they seem to have given up on producing "releases." We need to create a directory on the local disk (there is a lot to compile) and download a "cvs snapshot".

cd /usr/local/src/
sudo mkdir comedi
sudo chown <your_username>: comedi
cd comedi

The following two lines are from the comedi download page: http://www.comedi.org/download.html

cvs -d :pserver:anonymous@cvs.comedi.org:/cvs/comedi login
cvs -d :pserver:anonymous@cvs.comedi.org:/cvs/comedi co comedi
cd comedi
./autogen.sh

The "./configure" script needs some extra arguments to work (ref Roger). "./configure -h" to see all options. These might change in the future.

For 32-bit (tested by Steve):

./configure --with-linuxsrcdir=/usr/src/linux

For 64-bit (untested):

./configure --with-linuxsrcdir=/usr/src/linux --with-rpm-target=x86_64

make
sudo make install
sudo /sbin/depmod -a
sudo make dev

Installs:
- (comedi libraries)
- (device libraries)
- (comedi devices in /dev/)

sudo echo 'KERNEL=="comedi*", MODE="0666"' > /etc/udev/rules.d/53-comedi.rules
- Sets the permissions of the DAQ to world-readable-and-writable during boot

Specific instructions for the Amplicon PCI230

(See also Roger's Troubleshooting Tips below for other cards)

sudo echo 'install amplc_pci230 PATH=/sbin:/usr/sbin:/usr/local/sbin:$PATH; comedi_config /dev/comedi0 amplc_pci230' >> /etc/modprobe.conf.local
- Means that when you use modprobe to load the amplc_pci230 module, it automatically runs comedi_config too.
- Unfortunately, udev ignores this, so we have to...
sudo echo '/sbin/modprobe amplc_pci230' >> /etc/init.d/boot.local
sudo echo '/usr/local/sbin/comedi_config /dev/comedi0 amplc_pci230' >> /etc/init.d/boot.local
- Makes the PCI230 modules get loaded automatically at boot time

Troubleshooting tips

Installing comedi onto different machines may require a slight modification to the approach above.

Change the name of driver (amplc_pci230 above) depending on the DAQ card used (e.g. most PCI National instruments DAQ cards use ni_pcimio). A complete list can be viewed on comedi's hardware list.
Some drivers may not compile using make. If this happens, put #if 0 and #endif at the top and bottom of the driver's code respectively. The drivers are located in ./comedi/drivers/, adv_pci1710.c and adv_pci_dio.c have been known to cause problems.
Once installed, if the modules do not load into the kernal using modprobe (e.g. if comedi_test returns cannot find comedi0 or drivers), insmod may be required. The following needs to be placed in and at the end of /etc/init.d/boot.local (these are for National Instruments DAQ, change drivers as required)
- /sbin/insmod /lib/modules/comedi/comedi.ko
- /sbin/insmod /lib/modules/comedi/drivers/8255.ko
- /sbin/insmod /lib/modules/comedi/drivers/ni_pcimio.ko
- /usr/local/sbin/comedi_config /dev/comedi0 ni_pcimio (if this is not already there)

Dealing with kernel updates

Updates to the kernel mean that the comedi drivers need to be reinstalled. As the kernel updates are usually fairly minor, the comedi drivers usually don't need recompiling. It's annoying having to deal with this, so this solution helps to remove some of that annoyance. When you've installed comedi run the following as root:

cp -r /lib/modules/$(uname -r)/comedi /lib/modules/

This will copy your current comedi installation to /lib/modules/comedi.

Now add the following to /etc/init.d/boot.local (*before* any of the commands to load the comedi modules):

KERNEL=$(uname -r)

if [ ! -d /lib/modules/${KERNEL}/comedi/ ]; then
        cp -r /lib/modules/comedi /lib/modules/${KERNEL}/
        depmod -a
fi

This will check for the presence of the comedi directory in the modules directory for the current kernel and copy it over if it doesn't exist. depmod -a updates the module dependencies.

It is important that if you install a newer version of comedi that you remove /lib/modules/comedi and replace it with your updated version. Also note that some kernel updates may be incompatible with your compiled comedi drivers and so you'll have to recompile and reinstall using the instructions above.

comedilib

comedilib should now be installed automatically through Roger's openSUSE repository. Updates will also become available through the normal update procedure. If this doesn't work for any reason, see Roger or use the old method:

cd /home/scan/source/hardware/comedi/comedilib-0.8.1/
make distclean; ./configure; make; sudo make install

Installs:

libcomedi
comedi_test
/usr/local/sbin/comedi_config

Comedi Settings for the NI PCI-6251

If you want to use the PFI port as a digital output, you must do some extra configuration. This is possible using the demo tools in comedilib (which could also be copied across to your own code of course). So, to change channel 14 of subdevice 7 (ie. pin 1 on the connector) to be a digital output do the following in the comedilib/demo directory:

./choose_routing -s7 -c14 16 # NI_PFI_OUTPUT_DO == 16
./dio -s7 -c14 1 # now its an output
./outp -s7 -c14 1 # test the output (or 0)

All these can be set up at boot. Putting /usr/local/bin/choose_rooting -s7 -c** 16 for all/any channels in /etc/rc.d/boot.local and copying (as root) choose_rooting from the demo folder in (compiled?) comedilib to /usr/local/bin/.

Comedi on openSUSE 11.4, Kernel 2.6.37.6-0.5-desktop

I did a clean install of openSUSE 11.4 64bit on Settee. I had a few problems, so I thought I'd write down how I finally managed to get it working. This was June 2011, on Settee (Samuel).

After running ~share/suse_hacks/<suse version>/std_suse_hacks_<location>, make sure to run the install_eee_packages too. This will install comedilib automatically for you, amongst other things.

I tried to use the kernel tree to install comedi. From /usr/src/linux/ type: sudo make menuconfig, and choose to install modules for comedi and the relevant drivers. Comedi may be found in the tree under Device Drivers >> Staging Drivers >> Data acquisition support (comedi). Use / for search if so desired. Use sudo make modules; sudo make install to install. Unfortunately, this method didn't work for me, and the modules for other devices like network cards got overwritten, so I had to rescue the system.

I thus downloaded the latest snapshot from the comedi download page, untarred it to /usr/local/src/comedi, and used ./autogen.sh then ./configure --with-linuxdir=/usr/src/linux. It correctly detected the 64bit arch. Run make then sudo make install to install. Use the directions above to add the relevant lines to /etc/init.d/boot.local. Since I use the NI PCI6251, the lines in my boot.local file are :

KERNEL=$(uname -r)
if [ ! -d /lib/modules/${KERNEL}/comedi/ ]; then
 cp -r /lib/modules/comedi /lib/modules/${KERNEL}/
 depmod -a
fi

/usr/local/sbin/comedi_config /dev/comedi0 ni_pcimio
choose_routing -s7 -c0 16 # NI_PFI_OUTPUT_DO == 16
choose_routing -s7 -c1 16 # NI_PFI_OUTPUT_DO == 16
etc etc
choose_routing -s7 -c14 16 # NI_PFI_OUTPUT_DO == 16
choose_routing -s7 -c15 16 # NI_PFI_OUTPUT_DO == 16

choose_routing is available from comedilib/demo/.libs/. Copy using sudo to the /usr/local/bin/

Hopefully that helps.

c-scan

I had to make a once-only change to blanket's c_scan Makefile, due to going from a 2.4 kernel to a 2.6 kernel (and subsequent change of BNS device driver).

cd /home/scan/source/hardware/scanner/
make clean; make; sudo make install
sudo cp ~sds/bin/cscan /usr/local/bin/
Installs:
- c_scan
- c_scan man pages
- cscan: this performs a "touch" on the con-file that you're running with c_scan, so you can see the time at which you started the scan, and - by the time stamp on the data file - the time at which you finished the scan.

Jai CCD camera

sudo cp ~sds/bin/ccd* /usr/local/bin/
Installs:
- ccd_set_serial
- ccd0
- ccd1
- ..
- ccd7 (different shutter speeds)

Andor iXon camera

You can find the latest driver here: /home/scan/source/hardware/andor_camera/andor you need to copy this directory to a local disk to build and install the driver and libraries because the andor supplied script borks if it isn't run as root and the permissions on the nfs disks prevent root running the script directly.

The current driver is v2.82 install in December 2008 by Matt. This is broken for kernel versions 2.6.23-2.6.27 and will cause hard kernel lock ups is you install it on these kernels. Under newer kernels it may not compile because of an error in src/driver/fault.c - just just #define NOPFN_SIGBUS -1 in the preamble of fault.c to fix this. We are waiting for a revised driver.

Nanoscope lab / Andor iXon camera

The Andor camera setup we use uses a client / server model to overcome some of the shortcomings of the Andor driver / libraries. All the code can be found in /home/scan/source/hardware/andor_camera/camera. You need to make the camera_driver and the camera_utils all application code should use and compile in the camera_client code. Before you use anything the server needs to be running (camera_server). Placing the following line in boot.local AFTER the Andor code will ensure it runs at boot time:

echo Installing the camera_server
su matt -c /usr/local/bin/camera_server &

Remember the meminfo thing! reserve at least 4Mb of RAM for the driver by adding this to the boot options in the grub boot script (menu.1st) mem=1000M where 1000 is the size of the RAM in Mb - 4 (maybe 5?) as in:

    kernel /boot/vmlinuz-2.6.25.18-0.2-default root=/dev/disk/by-id/scsi-SATA_ST340014A_3JX7N160-part2 resume=/dev/sda1 splash=silent showopts mem=1000M vga=0x31a

The current driver 2.82-30003 on kernel 2.6.27.xx appears to be working, however, Andor seem to be ironing out a few bugs still so this could be volatile - if you have a working system I suggest leaving it alone.

2.78 works on pre 2.6.25 (or thereabouts) but not after because it causes hard kernel locks, it also requires a little hacking to compile on these newer kernels but since it crashes this is a little pointless. 2.82-30002 compiles and doesn't crash but returns scrambled image data. 2.82-30003 compiles on 2.6.27 kernels but not older ones, it doesn't crash and the image data seems ok.