Scientific Computing
Science, Technology, Engineering, Arts & Math
Teaching & Learning Math & Science with Technology
Graphing Calculators, Numerical Methods, Computer Algebra Systems & Linux Clusters
Why Shadowfax? Our cluster is so fast that we named it after the Lord Of The Rings character!
Website once known as CalcPage.tripod.com (1988 – 2008)
First thing we did this week was to observe a moment of silence. Yes, Colossus, our ssh server, has finally died after running flawlessly 24x7 for 12 years non-stop (the hdd melted and ground to a halt)! Kudos goto Slackware as I was running version 3.0 on Colossus all 12 years. Talk about set it and forget it! Sorry, Colossus, it's time to go, we don't use Intel Pentium IIs anymore....
Next on the agenda, was to install a new ssh server to replace Colossus. We will make the new server run as a ssh and an ftp server. His name shall be Guardian. We have another new server we will setup as a mirror site. Her name shall be Caprica.
So, we installed 32bit Ubuntu 10.04 Desktop edition on Guardian, our new quad-core Intel Xeon. Yes, it's 32bit - I know, I was shocked too. Well, that's a good thing, we can make this a Teacher Station as well as a server and not have to downgrade Shadowfax after all. We'll just need a video switch between the WimpDoze Teacher Station and the Linux Teacher Station! OK, so we don't really need 2 Linux Teacher Stations, but one is never enough!
We then had to figure out which of the 2 Ethernet cards was eth0 and which of the 4 Ethernet drops were live on the internet. We crashed and burned at this stage as we ran into a little SNAFU. We found eth0, but we couldn't find a live drop. We have 2 new intranet drops for the Linux LAN and 2 new internet drops for the 2 new servers to be accessible off campus. However, we could not access the internet. BTW, we also need the internet to finish installing openssh, vsftp and jdk!
As usual our network is FUBAR. Either the ethernet card's MAC address is blocked, or we didn't get any internet drops installed by the tech department after all. IDK why we have 2 ethernet cards per server and 4 drops when the old servers did fine with just one card and one drop each - KISS! I guess we better RTFM b4 the next meeting....
Well, "That's All Folks," for this meeting. We'll tackle the new server again next meeting in 2 weeks. Check our new Facebook Group, CIS(theta), for our next event! BTW, this 1 hour meeting every 2 weeks is mandatory.
Here's a YouTube showing sample game play for my latest favorite RTS (for iPod Touch) for no apparent reason other than we're all gamers and not lamers, right? Enjoy!
I welcomed our new CIS(theta) team today! We will meet every other Wednesday. Our task is to learn about cluster computing hardware in general. Investigate the current state of cluster computing software and find a problem worthy of running on our cluster.
First, let's get some vocabulary straight: CIS(theta) = Computing Independent Study class CIS(team) = DavidG, HerbertK, JoshG, JayW, RyanH (aka Geek Squad) Shadowfax = 25 node, 64bit dual-core AMD Athlon 2GHz, gigE cluster
How about some history: CIS(team) 0: We've done a bit of cluster programming over the years as a final project in AP Computer Science after AP Week all the way back to clusterKNOPPIX, i386/486s and slow ethernet! Actually, about 25 years ago I helped JamesM with a robotics project for the Westinghouse Talent Search. So, that's when the bug bit me to get into all this research with my students! I think the computing independent study started about 20 years ago when JoeB wanted to learn about Mathematica and Vector Calculus.
CIS(team) I: ChrisR, FrankK, NathanielR successfully installed openMOSIX on 25 Pentium IVs over fastE and created several Mandelbrot plots using C++ and fork(). We followed the advice from this blog, http://nullprogram.com (search there for mosix, octave, c++ or fractal). Take a look at the fractal prints on my deviantart site, http://cistheta2007.deviantart.com/gallery
CIS(team) II: MarcA, MitchellW were the first to use the current cluster hardware. They installed the Quantian Linux DVD and used bash scripts to scatter/gather povray jobs over publicly authenticated ssh. Take a look at the ray tracings on my zazzle site: http://www.zazzle.com/cistheta2008/gifts
CIS(team) III: ArthurD, DevinB, JeremyA, StevenB were the first to attempt using the current cluster hardware in 64bit mode. We tried Fedora, CentOS and even Rocks Clusters and OSCAR to no avail. I think we barely got HelloCluster.cc running on openMPI.
CIS(team) IV: DavidG, HerbertK, JayW, JoshG, RyanH will try 64bit mode again on Ubuntu Linux 10.04 that I have already installed on all nodes (see my post on this, http://shadowfaxrant.blogspot.com/2010/06/so-many-linux-distros-so-little-time.html). You may want to try pelicanHPC too. This is a 32bit live Linux CD. With this CD you are supposed to be able to reboot the whole room as an MPI cluster in minutes without installing a thing! What's nice about this solution is that it includes openMPI and has a lot of demos. We had this working on the new hardware last summer but sometimes its hard to use as its based on PXE and there's some DHCP server on our 10.5.*.* subnet that interferes with booting the nodes. However, another benefit of this CD is that it includes Octave (like MATLAB/SAGE) and mpiTB (an interface for Octave to work with MPI)! Hence your online research assignment listed above. Another CD solution with MPI is BCCD. An older DVD with openMOSIX is QUANTIAN.
So, our first meeting consisted of discussing all the above. Also, I mentioned that the CIS(theta) team will act as my personal Geek Squad and help me install hardware, firmware and software in our PC Classroom (since I am still recuperating from surprise hernia surgery). So, we had 2 "field trips."
Firld Trip 1: We went to the display case with some fractal and ray tracing prints we wanted to swap out. We also went to Mrs. Murthy's new office to help decorate it with some prints. We saved some nice big prints for the PC Classroom. We are getting 6 new marker boards in there which will be installed all around the room. I will hang a big print above each one. So, my decorative work here is done!
Firld Trip 2: I also showed the current CIS(theta) team around to all the servers centauri: ftp server in the book room colossus: ssh server in the business office guardian and caprica: new servers in the PC Classroom
So, the Geek Squad will also be helping me to install these new servers. The new servers will eventually replace the old ones. These new servers each have 4x72GB RAID drives and have 64bit Intel Xeon quadcores! Maybe we should make a cluster out of these? Google Cloud Computing using Ubuntu and Cloud Computing for HPC.
Wow, we have our work cut out for us. But, I think we are going to have a lot of fun this year!
OK, nobody said it would be easy! Whoever said "instant super computer, just add water" when referring to setting up an openMPI cluster over a LAN of 25 multi core 64-bit nodes was nuts. Oh, yeah, that was me...
How about a little bit of history? CIS(theta), the Computing Independent Study class, has been around in one incarnation or another for over a decade:
2009-2010 SHADOWFAX openMPI cluster with dual core 64-bit AMD 2 GHz Athlons (work in progress). We are trying to emulate pelicanHPC (http://idea.uab.es/mcreel/ParallelKnoppix) on a Linux Partition. We based this cluster on the Fedora 11 CD 64-bit Linux distribution and gigaE (gigabit Ethernet).
2008-2009 CISTHETA public key authentication cluster with dual core 64-bit AMD 2 GHz Athlons (used bash scripts to scatter/gather jobs on new architecture). We did some nice ray tracings using povray (http://www.zazzle.com/cistheta2008). We based this cluster on the KNOPPIX DVD 32-bit Linux distribution and fastE (100 megabit Ethernet).
1997-2007 CENTAURI openMOSIX cluster with various Pentium class processors (modified load balancing Linux Kernel that automatically sent jobs to other nodes whenever a node was overworked). We also dabbled with Beowulf, PVM and MPI. We based this cluster primarily on cluster KNOPPIX (http://clusterknoppix.sw.be), QUANTIAN (http://www.quantian.org) and BCCD (http://bccd.net) and basE (10 megabit Ethernet). For the last few years, we installed the QUANTIAN DVD to a Linux Partition and used a dual boot lab (with WimpDoze). We did some nice fractals with the fork() method of g++ (http://cistheta2007.deviantart.com).
The main difference between CISTHETA and SHADOWFAX is twofold. Last year, CISTHETA did not implement any clustering environment like MPI. We simply broke up our jobs into slices and sent and recieved results using bash scripts. Another main difference was that we set up public key authenticated ssh clients such that you could login as "cluster" on any node, and be automagically logged into all nodes. In fact, you could ssh to any node with or without the userid (cluster) and you did not need a passwd. Lets call that "static public key authenticated ssh." The static method is set up once and used many times. This year we are using something I'll call "dynamic public key authenticated ssh," such that the RSA keys are generated on the fly. Under both schemes, the user could start a parallel job from any node making that node the master node and control any number of other nodes as workers.
What's frustrating about all this, is that most of the clustering software we've tried seem to work great "right out of the box" on any one multi core node. In other words, SMP works by default, but extending the parallel jobs over a cluster of such nodes is a production! These clustering packages are as follows:
We've gotten openMPI's Torque severs to work with gcc. However, the Linux distro we based our new cluster on this year was the Fedora 11 CD 64-bit version. I think we may need to start over with the Fedora 12 DVD 64-bit Linux distribution so as to include g++ as well. We may all need to re implement "static public key authenticated ssh" which worked very well last year....
